Compare commits

...

654 Commits

Author SHA1 Message Date
Dan ca434e546f feat(ai-assistant): introduce test task tools and enhance AIChat integration
- Added new tools for listing testable tasks, generating test payloads, and running test tasks, improving the AI assistant's capabilities for testing workflows.
- Enhanced AIChatMessages and AIChatProvider to support the new test task functionalities, including handling generated payloads and navigating to test runs.
- Updated tool schemas to include new test-related operations, ensuring a seamless user experience when interacting with testing features.
- Improved the AI assistant's guidelines to reflect the new testing capabilities, providing clearer instructions for users.

These changes aim to enrich the testing experience within the AI assistant, making it easier for users to manage and execute test tasks effectively.
2026-06-04 10:21:53 +01:00
Dan 43ad5fcd2f feat(ai-assistant): enhance navigation and span detail tools
- Updated the navigateToPage tool to support deep-linking to specific runs and spans, improving user navigation capabilities.
- Introduced a new getSpanDetails tool for retrieving detailed information about individual spans within a run, aiding in error investigation.
- Enhanced AI assistant guidelines to reflect new navigation and inspection capabilities, ensuring users can effectively query run and span details.
- Added spanId and runId parameters to relevant schemas and functions for better context handling.

These changes aim to provide a more robust and informative experience for users interacting with the AI assistant, particularly in navigating and diagnosing run-related issues.
2026-06-03 17:35:42 +01:00
Dan 13e63db468 feat(ai-assistant): add ai-sparkle-hover animation and enhance AskAI and AIChatHeader components 2026-06-03 17:02:12 +01:00
Dan dfcadcaaf5 feat(ai-assistant): enhance AIChatMessages and introduce new tool result components
- Updated AIChatMessages to support structured output rendering for various tool results, improving user interaction.
- Added ToolOutput component to handle different tool outputs, including FailureSummaryCard, FilterChips, and MiniTable for better data presentation.
- Enhanced AIChatPanel to pass onSendMessage prop for improved message handling.
- Introduced AIChatToolResults for better organization of tool result components.

These changes aim to provide a richer and more informative user experience when interacting with the AI assistant.
2026-06-03 16:03:45 +01:00
Dan 76360ea261 feat(ai-assistant): enhance multi-step request handling and tool loading 2026-06-03 15:05:05 +01:00
Dan d6fa0d64ff feat(ai-assistant): enhance chat message handling, history and tool ux
- Updated AIChatMessages component to improve auto-navigation logic, ensuring navigation only occurs during active streaming.
- Introduced ToolResultCard for displaying tool outputs with collapsible details, enhancing user interaction.
- Added tool labels for better UI feedback on tool actions.
- Refined AIChatPanel to manage chat state more effectively, including resuming previous chats.
- Improved AIChatProvider to handle chat switching and state management with better reliability.

These changes aim to provide a more intuitive and responsive experience for users interacting with the AI assistant.
2026-06-03 14:52:55 +01:00
Dan f63b7c67a3 feat(ai-assistant): implement auto-navigation for dynamic tools in chat messages 2026-06-03 13:55:08 +01:00
Dan 830b830154 feat(ai-assistant): refine session management and error handling 2026-06-03 12:34:37 +01:00
Dan 0772ab4106 feat(ai-assistant): enhance analytics tools with ClickHouse integration 2026-06-03 11:20:04 +01:00
Dan 0e204a2a9a feat(ai-assistant): implement run tools 2026-06-03 07:42:33 +01:00
Dan e209bac792 feat(ai-assistant): implement error classification and retrieval tools 2026-06-03 07:41:59 +01:00
Dan 669d396f7c feat(ai-assistant): add scaffolding for expanding AI assistant capabilities with V1B tools and enhancements
- Introduced new tools for managing runs, errors, and analytics, including:
  - `listRuns`, `getRunDetails`, `getRunLogs`, `getRunGraph`, `applyRunFilters`, `queryRuns`
  - `listErrors`, `getErrorDetails`, `findSimilarErrors`, `classifyFailure`
  - `summarizeCurrentView`, `aggregateRuns`, `correlateRunsWithDeploy`
- Updated AIChatToolCall component with new tool labels for improved user feedback.
- Enhanced the AI assistant's system prompt to include capabilities for inspecting runs and errors.
- Added new utility functions for handling ClickHouse queries and summarizing run data.

These changes aim to provide a more comprehensive and user-friendly experience for users interacting with the AI assistant, enabling better insights into runs and error management.
2026-06-03 07:35:57 +01:00
Dan 36628d6bd3 feat(ai-assistant): enhance chat UI with improved input handling and message display
- Updated AskAI component to hide when the chat drawer is open.
- Modified AIChatContextBanner for better styling.
- Refactored AIChatHeader to include scroll handling and improved chat history display.
- Enhanced AIChatInput to support auto-growing text area and focus management.
- Improved AIChatMessages to handle scrolling and display messages more effectively.
- Added error handling and retry functionality in AIChatPanel.
- Introduced new suggested prompts with animation effects.

These changes aim to provide a more seamless and user-friendly chat experience in the AI assistant interface.
2026-06-02 20:19:20 +01:00
Dan f35a328e8a feat(ai-assistant): add chat panel UI and wire into dashboard nav
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-02 18:30:06 +01:00
Dan 3e42603e03 feat(ai-assistant): dashboard agent task, tools, and session routes
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-02 18:30:00 +01:00
Dan 94af0df33e feat(ai-assistant): add AiChat/AiChatSession persistence tables
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-02 18:29:53 +01:00
Matt Aitken b23ec26d16 chore: vouch ConProgramming (#3804)
Vouches GitHub user
[`ConProgramming`](https://github.com/ConProgramming) (Conner) as an
outside contributor by adding them to `.github/VOUCHED.td`, so their PRs
aren't auto-closed by the vouch check.

Done as a direct edit rather than via the issue flow because they have
no open Vouch Request issue to comment `vouch` on.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-02 10:48:00 +01:00
Dan cd252801eb feat: dashboard agent - package upgrades (#3793)
1. in webapp folder update ai-sdk to 6.x.x
2. update vitest to 4.xx
2026-06-02 10:46:34 +01:00
Eric Allam 4c4ed22e82 docs(ai-chat): document the chat.agent tools option (#3791)
## Summary

Documents the new `tools` option on `chat.agent` (companion to #3790).

Adds a dedicated [Tools](/ai-chat/tools) guide: the three places tools
show up (config, `toStreamTextOptions`, `streamText`), why declaring
them on the config matters for `toModelOutput` across turns, static vs
per-turn tools, the typed `run()` payload,
`InferChatUIMessageFromTools`, the relationship to skills, and the
manual `convertToModelMessages` path for `customAgent` loops.

Threads the option through the rest of the guide: the reference tables,
a happy-path section on the backend page, the types page, and the HITL /
skills / tool-result-auditing patterns. Corrects the sub-agents guide,
where the `toModelOutput` compression was implied to work across turns
but silently degraded from turn 2 without config tools.

Also unstacks the three callouts that were piled under the
`chat.agent()` header on the backend page, and adds a changelog entry.
2026-06-02 09:59:42 +01:00
github-actions[bot] e0681d2394 chore: release v4.5.0-rc.4 (#3788)
🚀 Publish Trigger.dev Docker / units (push) Failing after 3s
🚀 Publish Trigger.dev Docker / typecheck (push) Failing after 19s
🚀 Publish Trigger.dev Docker / publish-webapp (push) Has been skipped
🚀 Publish Trigger.dev Docker / publish-worker (push) Has been skipped
🚀 Publish Trigger.dev Docker / publish-worker-v4 (push) Has been skipped
🧭 Helm Chart Release / lint-and-test (push) Has been cancelled
🧭 Helm Chart Release / release (push) Has been cancelled
## Summary
1 new feature, 5 improvements.

## Highlights

- Mollifier `mutateSnapshot` now enforces a tag cap: an `append_tags`
patch carrying `maxTags` returns `"limit_exceeded"` (writing nothing)
when the deduped tag count would exceed the limit, so a buffered run
can't accumulate more tags via the tags API than the trigger validator
allows at creation.
([#3756](https://github.com/triggerdotdev/trigger.dev/pull/3756))

## Improvements
- Mollifier buffer extensions: idempotency dedup, an atomic
`mutateSnapshot` API, metadata CAS, claim primitives, and a
`MollifierSnapshot` type. The buffer's Redis client now reconnects with
jittered backoff so a fleet of clients doesn't stampede Redis in
lockstep after a blip.
([#3752](https://github.com/triggerdotdev/trigger.dev/pull/3752))
- Add `onTerminalFailure` callback to `MollifierDrainerOptions` so the
customer's run lands a SYSTEM_FAILURE PG row even when the drainer
exhausts `maxAttempts` on a retryable PG error. Previously,
retryable-error exhaustion called `buffer.fail()` directly, which
atomically marks FAILED + DELs the entry hash with no PG write — silent
data loss when PG was unreachable across the full retry budget. The
callback fires before `buffer.fail()` on any terminal path (`cause:
"non-retryable"` or `"max-attempts-exhausted"`); throwing a retryable
error from the callback causes the drainer to requeue rather than fail.
([#3754](https://github.com/triggerdotdev/trigger.dev/pull/3754))
- Bump `@s2-dev/streamstore` to `0.22.10` to fix a
`TASK_RUN_UNCAUGHT_EXCEPTION` ("Invalid state: Unable to enqueue") when
a `chat.agent` turn is aborted mid-stream.
([#3792](https://github.com/triggerdotdev/trigger.dev/pull/3792))
- Coerce numeric `concurrencyKey` values to string at the API boundary
across `tasks.trigger`, `tasks.batchTrigger`, and the Phase-2 streaming
batch endpoint.
([#3789](https://github.com/triggerdotdev/trigger.dev/pull/3789))
- Add a `tools` option to `chat.agent`. Declaring your tools here
threads them into the SDK's internal `convertToModelMessages`, so each
tool's `toModelOutput` is re-applied when prior-turn history is
re-converted.
([#3790](https://github.com/triggerdotdev/trigger.dev/pull/3790))

<details>
<summary>Raw changeset output</summary>

⚠️⚠️⚠️⚠️⚠️⚠️

`main` is currently in **pre mode** so this branch has prereleases
rather than normal releases. If you want to exit prereleases, run
`changeset pre exit` on `main`.

⚠️⚠️⚠️⚠️⚠️⚠️

# Releases
## @trigger.dev/redis-worker@4.5.0-rc.4

### Minor Changes

- Mollifier buffer extensions: idempotency dedup, an atomic
`mutateSnapshot` API, metadata CAS, claim primitives, and a
`MollifierSnapshot` type. The buffer's Redis client now reconnects with
jittered backoff so a fleet of clients doesn't stampede Redis in
lockstep after a blip.
([#3752](https://github.com/triggerdotdev/trigger.dev/pull/3752))
- Add `onTerminalFailure` callback to `MollifierDrainerOptions` so the
customer's run lands a SYSTEM_FAILURE PG row even when the drainer
exhausts `maxAttempts` on a retryable PG error. Previously,
retryable-error exhaustion called `buffer.fail()` directly, which
atomically marks FAILED + DELs the entry hash with no PG write — silent
data loss when PG was unreachable across the full retry budget. The
callback fires before `buffer.fail()` on any terminal path (`cause:
"non-retryable"` or `"max-attempts-exhausted"`); throwing a retryable
error from the callback causes the drainer to requeue rather than fail.
([#3754](https://github.com/triggerdotdev/trigger.dev/pull/3754))

### Patch Changes

- Pipeline the per-entry `HGETALL` fetches in
`MollifierBuffer.listEntriesForEnv`. The previous serial implementation
issued one Redis round-trip per runId returned by `LRANGE`, which
dominated stale-sweep wall-time at any meaningful backlog (at the
sweep's default maxCount=1000, this is ~1000 RTTs per env per pass).
Behaviour is unchanged — entries are still skipped when the entry hash
has been torn down by a concurrent drainer ack/fail between the LRANGE
and the HGETALL.
([#3752](https://github.com/triggerdotdev/trigger.dev/pull/3752))
- Mollifier `mutateSnapshot` now enforces a tag cap: an `append_tags`
patch carrying `maxTags` returns `"limit_exceeded"` (writing nothing)
when the deduped tag count would exceed the limit, so a buffered run
can't accumulate more tags via the tags API than the trigger validator
allows at creation.
([#3756](https://github.com/triggerdotdev/trigger.dev/pull/3756))
-   Updated dependencies:
    -   `@trigger.dev/core@4.5.0-rc.4`

## @trigger.dev/build@4.5.0-rc.4

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.5.0-rc.4`

## trigger.dev@4.5.0-rc.4

### Patch Changes

- Bump `@s2-dev/streamstore` to `0.22.10` to fix a
`TASK_RUN_UNCAUGHT_EXCEPTION` ("Invalid state: Unable to enqueue") when
a `chat.agent` turn is aborted mid-stream.
([#3792](https://github.com/triggerdotdev/trigger.dev/pull/3792))
-   Updated dependencies:
    -   `@trigger.dev/core@4.5.0-rc.4`
    -   `@trigger.dev/build@4.5.0-rc.4`
    -   `@trigger.dev/schema-to-json@4.5.0-rc.4`

## @trigger.dev/core@4.5.0-rc.4

### Patch Changes

- Coerce numeric `concurrencyKey` values to string at the API boundary
across `tasks.trigger`, `tasks.batchTrigger`, and the Phase-2 streaming
batch endpoint.
([#3789](https://github.com/triggerdotdev/trigger.dev/pull/3789))
- Bump `@s2-dev/streamstore` to `0.22.10` to fix a
`TASK_RUN_UNCAUGHT_EXCEPTION` ("Invalid state: Unable to enqueue") when
a `chat.agent` turn is aborted mid-stream.
([#3792](https://github.com/triggerdotdev/trigger.dev/pull/3792))

## @trigger.dev/plugins@4.5.0-rc.4

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.5.0-rc.4`

## @trigger.dev/python@4.5.0-rc.4

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/sdk@4.5.0-rc.4`
    -   `@trigger.dev/core@4.5.0-rc.4`
    -   `@trigger.dev/build@4.5.0-rc.4`

## @trigger.dev/react-hooks@4.5.0-rc.4

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.5.0-rc.4`

## @trigger.dev/rsc@4.5.0-rc.4

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.5.0-rc.4`

## @trigger.dev/schema-to-json@4.5.0-rc.4

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.5.0-rc.4`

## @trigger.dev/sdk@4.5.0-rc.4

### Patch Changes

- Add a `tools` option to `chat.agent`. Declaring your tools here
threads them into the SDK's internal `convertToModelMessages`, so each
tool's `toModelOutput` is re-applied when prior-turn history is
re-converted.
([#3790](https://github.com/triggerdotdev/trigger.dev/pull/3790))

    ```ts
    chat.agent({
      tools: { readFile, search },
      run: async ({ messages, tools, signal }) =>
        streamText({ model, messages, tools, abortSignal: signal }),
    });
    ```

Also exports `InferChatUIMessageFromTools<typeof tools>` to derive the
chat `UIMessage` type (typed tool parts) directly from a tool set.

-   Updated dependencies:
    -   `@trigger.dev/core@4.5.0-rc.4`

</details>

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-06-02 09:35:14 +01:00
nicktrn e35f5742d4 fix(webapp): upgrade posthog-node to v5, drop axios + stale override (#3801)
Follow-up to #3796, which bumped the slack-client axios paths but left
posthog-node's transitive `axios@1.15.1` in place.

`posthog-node` 4.17.1 → 5.35.6. v5 drops the axios dependency entirely
(it's now fetch-based via `@posthog/core`), so posthog's old axios path
disappears. With #3796 already on main (webapp + d3 references on
`@slack/web-api@7.16.0`), nothing else pins the old line, so the
now-dead `axios@>=1.0.0 <1.15.0` override is removed and axios resolves
to a single patched `1.16.1` repo-wide. This closes the remaining axios
advisories.

Compat: the webapp's usage in `telemetry.server.ts` (`new PostHog(key, {
host })`, `.identify`, `.groupIdentify`, `.capture`) is all object-form
API that v5 preserves; `pnpm run typecheck --filter webapp` passes.

Node: posthog-node v5 requires Node `^20.20.0 || >=22.22.0`. We run
20.20.0 in dev (`.nvmrc`), CI, and the published Docker image
(`node:20.20-bullseye-slim`), so we're compliant.
2026-06-02 10:33:44 +02:00
nicktrn 952139d492 fix(webapp): bump @slack/web-api to 7.16.0 for patched axios (#3796)
Bumps `@slack/web-api` 7.9.1 → 7.16.0 in the webapp and the two
`references` examples (d3-chat, d3-openai-agents). 7.16.0 depends on
`axios@^1.16.0`, so every slack-client axios path resolves to 1.16.1
instead of 1.15.1.

This clears the slack and references axios paths. `posthog-node`'s
transitive axios still resolves the older line - that's handled in a
follow-up that upgrades posthog-node to v5 (which drops the axios
dependency entirely and lets us retire the now-stale axios override).
The dependabot axios advisories fully close once both land.
2026-06-02 08:18:28 +01:00
Eric Allam 2dd9f37b40 fix(core,cli): stop chat.agent uncaught exception on mid-stream abort (#3792)
## Summary

A `chat.agent` turn that gets aborted mid-stream (stop generation, idle
suspend, cancellation) could surface a `TASK_RUN_UNCAUGHT_EXCEPTION` —
`TypeError: Invalid state: Unable to enqueue` — in the dashboard. The
run kept working, but the exceptions were loud and confusing.

## Root cause

The realtime stream writer batches chunks through
`@s2-dev/streamstore`'s `BatchTransform`, which holds them for a short
linger window before flushing. When the turn's abort signal fires while
a record is still buffered, the stream's writable is aborted and the
transform's readable controller is closed — but the pending linger
`setTimeout` still fires and calls `controller.enqueue()` on the dead
controller, throwing from a timer callback where nothing can catch it.

Fixed upstream in `@s2-dev/streamstore@0.22.10`, which wraps the linger
flush in a try/catch that discards the closed-controller error. This
bumps the dependency across core, the CLI, and the webapp, and adds a
regression test against the real `BatchTransform`.

Verified end-to-end: a mid-stream stop that previously failed the run
with `TASK_RUN_UNCAUGHT_EXCEPTION` now leaves the run healthy.
2026-06-01 17:12:48 +00:00
Eric Allam e9e2ec1cfc fix(sdk): re-apply tool toModelOutput across chat.agent turns (#3790)
## Summary

`chat.agent` now takes a `tools` option. Until now tools only went to
`streamText` inside `run()`, so the SDK had no tools when it
re-converted the persisted `UIMessage` history at the start of each
turn. Any tool with a `toModelOutput` (raw image bytes into an image
content part, or a sub-agent transcript compressed to a summary) had its
transform applied on turn 1 and skipped from turn 2 onward, so the raw
output got JSON-stringified back into the prompt and the model lost the
transformed view.

Declaring `tools` on the config threads them into that conversion, so
`toModelOutput` runs on every turn. The resolved set is handed back,
typed, on the `run()` payload as `tools`:

```ts
const tools = { searchDocs, renderChart };

export const myChat = chat.agent({
  tools,
  run: async ({ messages, tools, signal }) =>
    streamText({ ...chat.toStreamTextOptions({ tools }), messages, abortSignal: signal }),
});
```

`tools` also accepts a per-turn function for tools that depend on the
user or a feature flag. Only `inputSchema` and `toModelOutput` are read
during conversion, never `execute`. Also exports
`InferChatUIMessageFromTools<typeof tools>` to derive the chat
`UIMessage` type from a tool set. No behavior change for agents that
don't declare `tools`.
2026-06-01 17:26:27 +01:00
Daniel Sutton e21b68cc5f feat(webapp): dashboard parity for mollifier-buffered runs (#3757)
## Summary

Dashboard surfaces handle buffered runs by falling back to the mollifier
snapshot:

- Run detail, span detail, streams view (`_app.../runs.\$runParam`,
`resources.../spans.\$spanParam`, `resources.../streams.\$streamKey`).
- Redirect routes (`@.runs.\$runParam`, `runs.\$runParam`,
`projects.v3.\$projectRef.runs.\$runParam`).
- Action routes — cancel / replay / idempotency-reset / debug — under
`resources.taskruns/...` and `resources.../idempotencyKey.reset`.
- Logs download.
- Realtime subscription route + per-run resource
(`realtime.v1.runs.\$runId`, `resources.../realtime.v1.*`).
- `CancelRunDialog` gains an `onCancelSubmitted` callback so submit
isn't raced by the Radix `DialogClose` wrapper.

Stacked on the mutations PR.

## Test plan

- [x] \`pnpm run typecheck --filter webapp\` passes
- [x] \`pnpm run test --filter webapp
test/mollifierRealtimeRunResource.test.ts\` passes
- [x] \`pnpm run test --filter webapp
test/mollifierRealtimeRunResourceBuffer.test.ts\` passes
- [x] \`pnpm run test --filter webapp
test/mollifierRealtimeSubscription.test.ts\` passes
- [x] Manual smoke: trigger a buffered run, open it in the dashboard,
replay/cancel from the UI

---

## Ship-gate follow-up fixes

- **Auto-redirect to root span on direct nav** — loader sets `?span=`
from root span (PG) or buffered snapshot spanId before 302'ing, so
bookmark/share-link/direct-nav doesn't leave the panel collapsed.
- **RunPresenter switches from `findFirstOrThrow` to `findFirst` + typed
`RunNotInPgError`** — kills the per-poll `PrismaClient error` log spam
for buffered runs without changing the route-loader's fallback flow.
- **Span detail panel renders for buffered runs** — `SpanPresenter.call`
now falls back to `findRunByIdWithMollifierFallback` +
`buildSyntheticSpanRun` instead of returning undefined and triggering
the "Event not found" toast loop.
- **Logs download for buffered runs returns a gzipped placeholder line**
— replaces the 404 with a content-encoded line explaining the run is
queued. Same org-membership gate as the PG path.
- **Admin Debug-Run button hidden for buffered runs + SpanRun circular
type alias broken** (squashed) — buttons gate on a new `isBuffered` flag
on the synthetic SpanRun. Required grounding SpanRun in
`SpanPresenter.getRun` to break a circular type alias TS no longer
tolerates once `isBuffered` is a literal field on the shape.
- **Replay action requires user auth + org-membership** (🚩 Devin
finding) — `action` was unauthenticated and the PG `findFirst` had no
org filter, so any caller with a valid `runParam` could replay any run.
Buffered fallback inherited the same gap. Fixed to mirror the cancel
route.

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-01 16:50:31 +01:00
Daniel Sutton e1950778e2 feat(webapp): mollifier API mutations on buffered runs (#3756)
## Summary

Cancel, replay, reschedule, metadata, tags, and idempotency-key-reset
now succeed against a run that's still in the mollifier buffer.
Mutations are applied to the buffered snapshot via Lua CAS; the drainer
carries the mutation forward when it replays.

Primitives added:

- `mutateWithFallback` — PG-first / buffer-fallback resolver with
bounded-wait safety net for entries that transition mid-mutation.
- `applyMetadataMutation` — buffered metadata PUT mirroring the PG-side
retry loop with CAS atomicity.
- `resolveRunForMutation` — discriminated-union resolver used by route
`findResource` so the route builder's pre-action 404 check sees buffered
runs.

Routes wired (whole files, no GET/POST splits):
- `api.v2.runs.\$runParam.cancel.ts`
- `api.v1.runs.\$runParam.replay.ts`
- `api.v1.runs.\$runParam.reschedule.ts`
- `api.v1.runs.\$runId.metadata.ts`
- `api.v1.runs.\$runId.tags.ts`
- `resetIdempotencyKey.server.ts`

Stacked on the reads PR.

## Test plan

- [x] \`pnpm run typecheck --filter webapp\` passes
- [x] \`pnpm run test --filter webapp
test/mollifierMutateWithFallback.test.ts\` passes
- [x] \`pnpm run test --filter webapp
test/mollifierApplyMetadataMutation.test.ts\` passes
- [x] \`pnpm run test --filter webapp
test/mollifierResolveRunForMutation.test.ts\` passes

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-01 16:24:10 +01:00
Daniel Sutton a1dc3c5ae1 feat(webapp): mollifier API GET read-fallback — synthetic primitives + route wiring (#3755)
## Summary

Synthesise QUEUED/FAILED responses from the mollifier buffer when a
TaskRun row hasn't landed in Postgres yet. Wires the synthesis into:

- `ApiRetrieveRunPresenter`
- v1 trace GET route
- v1 spans GET route
- attempts route gains a GET loader (fixes pre-existing Remix "no
loader" 400)

The `readFallback` infra itself lives on the trigger PR (consumed by
`IdempotencyKeyConcern`); this PR adds the route-level
synthetic-rendering primitives.

Stacked on the replay PR.

## Test plan

- [x] \`pnpm run typecheck --filter webapp\` passes
- [x] \`pnpm run test --filter webapp
test/mollifierSyntheticRedirectInfo.test.ts\` passes
- [x] \`pnpm run test --filter webapp
test/mollifierSyntheticSpanRun.test.ts\` passes

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-01 14:53:58 +01:00
Daniel Sutton 4745754a7a feat(webapp,run-engine): mollifier drainer replay + stale sweep + cancelled-run engine API (#3754)
## Summary

The replay side of the mollifier:

- `DrainerHandler`: reads buffered snapshots and replays them through
`engine.trigger` to materialise PG rows.
- `RunEngine.createCancelledRun`: new public method the handler uses to
write CANCELED rows directly from snapshots (bypass queue + waitpoint,
emit `runCancelled`). Tolerates the cjson empty-table tags edge case
found during validation.
- Drainer fairness: org → env rotation so a heavy env doesn't starve
light ones in the same org.
- Stale-entry sweep + telemetry + alertable gauge so a stuck/offline
drainer surfaces in alerts.

Both the drainer and sweep default-off; nothing fires unless flagged on
(`TRIGGER_MOLLIFIER_DRAINER_ENABLED`,
`TRIGGER_MOLLIFIER_STALE_SWEEP_ENABLED`).

Stacked on the trigger-time decisions PR.

## Test plan

- [x] \`pnpm run typecheck --filter webapp\` passes
- [x] \`pnpm run test --filter webapp
test/mollifierDrainerHandler.test.ts\` passes
- [x] \`pnpm run test --filter webapp test/mollifierStaleSweep.test.ts\`
passes
- [x] \`pnpm run test --filter @internal/run-engine
src/engine/tests/createCancelledRun.test.ts\` passes
- [x] \`pnpm run test --filter @trigger.dev/redis-worker
packages/redis-worker/src/mollifier/drainer.test.ts\` passes

---

## Ship-gate follow-up fix

**Drainer writes SYSTEM_FAILURE on max-attempts exhaustion.** Adds an
`onTerminalFailure` callback on `MollifierDrainerOptions` so the
customer's run lands a SYSTEM_FAILURE PG row even when the drainer
exhausts `MAX_ATTEMPTS` on a retryable PG error (previously
`buffer.fail()` was called with no row written → silent data loss). The
callback runs before `buffer.fail()` on every terminal path
(non-retryable AND max-attempts-exhausted), and re-throwing a retryable
error from the callback causes the drainer to requeue rather than fail.

Bumps `@trigger.dev/redis-worker` to a **minor** changeset (additive
option + new exported types). Includes 5 unit tests covering both
terminal causes plus the requeue-on-retryable-callback-failure path and
no-callback back-compat.

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-01 13:20:05 +01:00
Daniel Sutton 20a676d2ed fix(core,webapp): coerce numeric concurrencyKey to string (#3789)
## Summary

`concurrencyKey` validation accepted only `z.string().optional()` on the
single-trigger and V2/V3 batch endpoints, and the Phase-2 streaming
NDJSON endpoint accepted `z.record(z.unknown()).optional()` for the
entire `options` field. Callers passing `concurrencyKey: someNumericId`
(e.g. `payload.userId`) either failed schema validation on the first two
paths or sailed through on Phase-2 and then failed downstream at
`prisma.taskRun.create` with `Argument concurrencyKey: Expected String
or Null, provided Int`.

The schema now accepts `string | number` for `concurrencyKey` and
stringifies on the way in, across all three paths. The Phase-2 NDJSON
`options` is tightened to reuse the strict
`BatchTriggerTaskItem.options` shape so it validates identically to the
V2/V3 batch endpoints.

A defensive `typeof === "number"` coercion at the `engine.trigger` call
site in `RunEngineTriggerTaskService` covers in-flight Redis-stored
batch items enqueued before the schema fix — those items are rebuilt
from a `Record<string, unknown>` shape that bypasses the new schema and
would otherwise continue failing for up to their TTL.

## Test plan

- [x] `packages/core/src/v3/schemas/batchItemNDJSON.test.ts` — unit
tests covering numeric→string coercion, string passthrough, no-options,
and rejection of non-string/non-number shapes across
`TriggerTaskRequestBody`, `BatchTriggerTaskItem`, and `BatchItemNDJSON`.
- [x] `apps/webapp/test/engine/triggerTask.test.ts` — `containerTest`
simulating the in-flight Redis batch-item shape (numeric
`concurrencyKey` via `Record<string, unknown>`), verifies the run is
created with `concurrencyKey: "51262"`. Without the worker coercion, the
test reproduces the production stack at `prisma.taskRun.create`.
- [x] `pnpm run typecheck --filter webapp` clean.
- [x] `pnpm run build --filter @trigger.dev/core --filter
@trigger.dev/sdk --filter trigger.dev` clean.

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-01 12:34:29 +01:00
Daniel Sutton 577f35eebe feat(webapp): mollifier trigger-time decisions — mollify, claim, read fallback (#3753)
## Summary

The trigger hot path's mollifier integration:

- `mollifyTrigger`: when the gate trips, write the engine.trigger
snapshot to the buffer and return a synthesised QUEUED response.
Postgres write is deferred to drainer-replay (next PR in the stack).
- Pre-gate idempotency-key claim: same-key triggers serialise through
Redis so a burst lands in PG / buffer exactly once.
- Read-fallback extensions: `findRunByIdWithMollifierFallback` for the
trigger-time idempotency lookup that must see buffered runs.
- Gate bypasses: `debounce`, `oneTimeUseToken`,
`parentTaskRunId`/`triggerAndWait` skip the mollify path entirely.
- `triggerTask` + `IdempotencyKeyConcern` wired to the above.

All behaviour gated by the master `TRIGGER_MOLLIFIER_ENABLED` switch;
off-state hot path is unchanged (the gate is not even consulted).

Stacked on the buffer extensions PR.

## Test plan

- [x] \`pnpm run typecheck --filter webapp\` passes
- [x] \`pnpm run test --filter webapp test/mollifierMollify.test.ts\`
passes
- [x] \`pnpm run test --filter webapp
test/mollifierIdempotencyClaim.test.ts\` passes
- [x] \`pnpm run test --filter webapp
test/mollifierReadFallback.test.ts\` passes
- [x] \`pnpm run test --filter webapp test/mollifierGate.test.ts\`
passes
- [x] \`pnpm run test --filter webapp test/engine/triggerTask.test.ts\`
passes

---

## Ship-gate follow-up fixes

- **Batch items bypass the mollifier gate** — fixes
`BatchTaskRunItem_taskRunId_fkey` FK violation on batch triggers when
the gate trips. End-state is a drainer-side `BatchTaskRunItem`
create-on-materialise; batch traffic passes through the gate until that
lands.
- **IdempotencyKeyConcern honours buffered-run TTL on expiry** —
buffered path now clears expired idempotency claims (read-side) and
resets the buffer's `mollifier:idempotency:*` SETNX binding (write-side)
so a re-trigger past the customer's TTL lands as a fresh run instead of
echoing the stale buffered runId.

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-01 12:00:24 +01:00
Daniel Sutton 3924aa4adb feat(redis-worker,webapp): mollifier buffer extensions + snapshot type (#3752)
## Summary

Buffer-side data layer used by the rest of the mollifier phase-3 stack.

- `buffer.ts` gains entry inspection (`getEntry`), idempotency lookup
(`lookupIdempotency`), in-place snapshot mutation (`mutateSnapshot`),
and dwell tracking. All atomic via Lua.
- `mollifierSnapshot.server.ts`: shared `MollifierSnapshot` type plus
(de)serialise helpers.
- Drops the entry-TTL config and its env var. The drainer is the
recovery mechanism; an entry that survives the drainer should surface as
a stale-sweep alert, not silently TTL away.

Adds methods to the buffer interface; nothing consumes them yet.
Subsequent PRs in the stack wire trigger-time mollify, read-fallback,
and mutation paths against this surface.

## Test plan

- [x] \`pnpm run typecheck --filter webapp\` passes
- [x] \`pnpm run test --filter @trigger.dev/redis-worker
packages/redis-worker/src/mollifier/buffer.test.ts\` passes

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-01 11:42:55 +01:00
Saadi Myftija 181d9ba541 feat: preview environment dispatch workflow (#3786)
Publishes a `repository_dispatch` `preview-deploy` event that enables
preview deployments. Opt-in only currently.
2026-06-01 11:57:40 +02:00
github-actions[bot] 8f066ac389 chore: release v4.5.0-rc.3 (#3763)
🚀 Publish Trigger.dev Docker / typecheck (push) Failing after 0s
🚀 Publish Trigger.dev Docker / publish-webapp (push) Has been skipped
🚀 Publish Trigger.dev Docker / publish-worker (push) Has been skipped
🚀 Publish Trigger.dev Docker / publish-worker-v4 (push) Has been skipped
🚀 Publish Trigger.dev Docker / units (push) Failing after 0s
🧭 Helm Chart Release / lint-and-test (push) Has been cancelled
🧭 Helm Chart Release / release (push) Has been cancelled
## Summary
1 improvement, 2 bug fixes.

## Improvements
- Retry `TASK_MIDDLEWARE_ERROR` under the task's retry policy instead of
failing the run on the first attempt. The error was already classified
as retryable by `shouldRetryError`, but `shouldLookupRetrySettings` did
not include it, so the retry flow fell through to `fail_run`. Fixes
#3231. ([#3676](https://github.com/triggerdotdev/trigger.dev/pull/3676))

## Bug fixes
- Fix `TypeError` in `unflattenAttributes` when the input attribute map
contains conflicting dotted key paths (e.g. both `a.b` set to a scalar
and `a.b.c` set to a value). The path-walk loop now applies
last-write-wins when a prior key wrote a primitive, null, or array at an
intermediate slot, matching the existing precedent in
`AttributeFlattener.addAttribute`. Callers no longer crash when handed
malformed external attribute inputs.
([#3762](https://github.com/triggerdotdev/trigger.dev/pull/3762))
- Fix external trace context leaking across runs on warm-started workers
with `processKeepAlive` enabled. Every subsequent run's attempt span was
being exported with the first run's `traceId` and `parentSpanId`,
breaking causal-chain navigation in external APM tools. Runs without an
external trace context are unaffected.
([#3768](https://github.com/triggerdotdev/trigger.dev/pull/3768))

<details>
<summary>Raw changeset output</summary>

⚠️⚠️⚠️⚠️⚠️⚠️

`main` is currently in **pre mode** so this branch has prereleases
rather than normal releases. If you want to exit prereleases, run
`changeset pre exit` on `main`.

⚠️⚠️⚠️⚠️⚠️⚠️

# Releases
## @trigger.dev/build@4.5.0-rc.3

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.5.0-rc.3`

## trigger.dev@4.5.0-rc.3

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.5.0-rc.3`
    -   `@trigger.dev/build@4.5.0-rc.3`
    -   `@trigger.dev/schema-to-json@4.5.0-rc.3`

## @trigger.dev/core@4.5.0-rc.3

### Patch Changes

- Retry `TASK_MIDDLEWARE_ERROR` under the task's retry policy instead of
failing the run on the first attempt. The error was already classified
as retryable by `shouldRetryError`, but `shouldLookupRetrySettings` did
not include it, so the retry flow fell through to `fail_run`. Fixes
#3231. ([#3676](https://github.com/triggerdotdev/trigger.dev/pull/3676))
- Fix `TypeError` in `unflattenAttributes` when the input attribute map
contains conflicting dotted key paths (e.g. both `a.b` set to a scalar
and `a.b.c` set to a value). The path-walk loop now applies
last-write-wins when a prior key wrote a primitive, null, or array at an
intermediate slot, matching the existing precedent in
`AttributeFlattener.addAttribute`. Callers no longer crash when handed
malformed external attribute inputs.
([#3762](https://github.com/triggerdotdev/trigger.dev/pull/3762))
- Fix external trace context leaking across runs on warm-started workers
with `processKeepAlive` enabled. Every subsequent run's attempt span was
being exported with the first run's `traceId` and `parentSpanId`,
breaking causal-chain navigation in external APM tools. Runs without an
external trace context are unaffected.
([#3768](https://github.com/triggerdotdev/trigger.dev/pull/3768))

## @trigger.dev/plugins@4.5.0-rc.3

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.5.0-rc.3`

## @trigger.dev/python@4.5.0-rc.3

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.5.0-rc.3`
    -   `@trigger.dev/build@4.5.0-rc.3`
    -   `@trigger.dev/sdk@4.5.0-rc.3`

## @trigger.dev/react-hooks@4.5.0-rc.3

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.5.0-rc.3`

## @trigger.dev/redis-worker@4.5.0-rc.3

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.5.0-rc.3`

## @trigger.dev/rsc@4.5.0-rc.3

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.5.0-rc.3`

## @trigger.dev/schema-to-json@4.5.0-rc.3

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.5.0-rc.3`

## @trigger.dev/sdk@4.5.0-rc.3

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.5.0-rc.3`

</details>

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-06-01 10:34:45 +01:00
Eric Allam 9211032733 chore(database): drop unused TaskRun status composite index (#3743)
## Summary

Drops the `TaskRun_status_runtimeEnvironmentId_createdAt_id_idx` index
from the `TaskRun` table. After #3742 gated the legacy
`WAITING_FOR_DEPLOY` drain to V1-engine workers only, this index sees
zero scans on both writer and reader replicas. Removing it cuts index
maintenance on every `TaskRun` INSERT/UPDATE.

## Why

The index existed to support `WHERE status = X AND runtimeEnvironmentId
= Y` queries from `ExecuteTasksWaitingForDeployService`, which is
V1-only and no longer triggered on V2 deployments. A code grep across
`apps/webapp` and `internal-packages/run-engine` confirmed no V2
production query uses this access pattern — every other `status:` filter
on `TaskRun` is paired with `id`/`friendlyId`/`parentSpanId` and uses a
different index.

Dropping it also unlocks HOT updates on the dequeue path. The dequeue
`UPDATE` modifies `status` (`QUEUED` -> `DEQUEUED`), and `status` is the
leading column of this index — its presence blocked HOT eligibility for
every `TaskRun` UPDATE. With the index gone, dequeue UPDATEs can become
HOT, reducing WAL bytes and removing the B-tree page contention on this
index's right-edge leaves.

Uses `DROP INDEX CONCURRENTLY` to avoid blocking writes during the drop.

## Sequencing

Should only ship once #3742 has soaked long enough to confirm the index
is genuinely cold (24h+ of zero scans on `pg_stat_user_indexes`).
2026-06-01 09:37:37 +01:00
Saadi Myftija 9cb6fd17c3 fix(webapp): idempotent DeploymentBackgroundWorker creation (#3772)
Make `POST /api/v1/deployments/:deploymentId/background-workers`
idempotent so client-side retries no longer collide on the
`BackgroundWorker` `(project, env, version)` unique index. Helps make
deployments more resilient against the class of indexing failures that
surfaces in the dashboard as "Indexing timed out", e.g. during transient
database issues.
2026-05-29 10:03:55 +02:00
nicktrn c043c4a6ad fix(core): external trace context leaks across warm-started runs (#3768)
On warm-started workers with `processKeepAlive` enabled, every run's
attempt span was exported with the first run's `traceId` and
`parentSpanId`. `ExternalSpanExporterWrapper` and
`ExternalLogRecordExporterWrapper` captured `externalTraceContext` at
`TracingSDK` construction, and the SDK is memoized for the worker's
lifetime - so the per-run reset of
`StandardTraceContextManager.traceContext` never reached the wrappers.

Reported by a customer running v4.4.x: 33 distinct runs on the same
host/pid showed up in their APM as siblings of one parent span.

Fix: the wrappers now read external context live from the trace context
manager per export. Runs without an external trace context fall through
to the unchanged `externalTraceId` fallback - no behaviour change for
them.

Regression test in
`packages/core/test/externalSpanExporterWrapper.test.ts` asserts that
reassigning the manager between exports produces correctly-parented
spans.
2026-05-28 10:43:39 +00:00
Matt Aitken 816986d44e fix(webapp): treat Phase 2 batch-stream retries as idempotent (TRI-9944) (#3766)
Returns sealed:true when a fast-completing batchTrigger races
the stream finalisation, instead of throwing 422/BatchTriggerError.
2026-05-28 09:28:18 +00:00
Iss 5083d161b2 docs: adds relevant env vars to self hosting docs (#3148) 2026-05-27 12:25:18 -04:00
Iss 8399aa2052 fix(core): retry TASK_MIDDLEWARE_ERROR under the task's retry policy (#3676)
Retries `TASK_MIDDLEWARE_ERROR` under the task's retry policy.
`shouldRetryError` already classed it as retryable, but
`shouldLookupRetrySettings` did not, so the run fell through to
`fail_run` on attempt 1 instead of using the task's `retry` config.
Fixes #3231.
2026-05-27 17:17:50 +01:00
Iss 7324a3307f chore(webapp): rename support tier to "Priority" on plan cards and li… (#3692) 2026-05-27 12:17:40 -04:00
Iss df96a937db docs: troubleshoot "Stream is being deleted" during long waits (#3704) 2026-05-27 12:17:21 -04:00
Matt Aitken 90bbbd1c44 fix(webapp): recover from ClickHouse JSON parse failures on out-of-range integers (#3759)
## Summary

Second class of poisoned-row failure in the runs replication path. PR
#3708 plugged lone UTF-16 surrogates; this one handles bare JSON integer
literals outside ClickHouse's `Int64`..`UInt64` range. Recovery stays
purely reactive — the existing `sanitizeRows` walker just gains an extra
branch, so the hot replication path pays nothing on healthy rows.

Fixes the still-firing customer-facing symptom from
[TRI-9755](https://linear.app/triggerdotdev/issue/TRI-9755):
`scan-social-profiles` runs continued to be stranded in `EXECUTING` on
the Tasks page after #3708 deployed. CloudWatch showed `Dropped batch —
ClickHouse JSON parse error but sanitizer found nothing to fix` firing
**8/8 times** since the previous deploy (zero successful sanitizations).
Root cause: upstream JS Number precision loss on a 21-digit Google Plus
ID (`117039831458782873093` → `117039831458782870000`) — the
precision-lossy value still serialises as a bare integer that exceeds
`UInt64.MAX`, which ClickHouse rejects with `INCORRECT_DATA`.

## How the bug ships

The customer task emits an output containing a Poshmark profile's
`spec_format`:

```json
{"key":"gp_id","proper_key":"Gp Id","value":117039831458782870000,"type":"int"}
```

That value is `1.17e20` — comfortably above `UInt64.MAX` (`1.84e19`) but
comfortably below `1e21`. `Number.prototype.toString` only switches to
exponential form at `|value| >= 1e21`, so `JSON.stringify` emits the
bare token `117039831458782870000` and the ClickHouse
`JSON(max_dynamic_paths)` column fails with:

```
Code: 117. DB::Exception: Cannot parse JSON object here: {…}: (while reading the value of key output): (at row 1)
: While executing ParallelParsingBlockInputFormat. (INCORRECT_DATA) (version 25.12.x)
```

Same error verbatim as prod. The same number quoted
(`"117039831458782870000"`) inserts fine — ClickHouse's dynamic JSON
column accepts a `String` subtype on the same path.

## What changed

`apps/webapp/app/v3/eventRepository/sanitizeRowsOnParseError.server.ts`:

- New private `isUnsafeJsonInteger(value)` helper — true iff `value` is
a finite integer-valued JS Number where `|value| < 1e21` (so
`JSON.stringify` emits integer form, not exponent) **and** `value` falls
outside `[Int64.MIN, UInt64.MAX]`.
- `sanitizeUnknownInPlace` gains a number-branch: when the predicate
holds, replace the Number with `String(value)`. The downstream JSON
column dynamic-types the path as String for that row — fine, since the
value was already precision-lossy upstream (no JS Number above 2^53 is
numerically meaningful anyway).
- Float-valued numbers, large floats (>= 1e21), NaN and Infinity are
left alone — `JSON.stringify` emits them with exponents or as `null`,
both of which ClickHouse accepts.

`apps/webapp/test/sanitizeRowsOnParseError.test.ts`: four new unit tests
+ an extension to `sanitizeRows` covering surrogate + integer fixes
counted together across rows. The unit suite now covers:

- Positive value above `UInt64.MAX` (`117039831458782870000` — the
actual prod value)
- Negative value below `Int64.MIN`
- Boundary values pass through (`42`, `Number.MAX_SAFE_INTEGER`, `2^63`)
- Non-integer numbers untouched (floats, `1e25`, NaN, Infinity)
- The actual `scan-social-profiles` nested shape — finds the offending
`gp_id` deep inside
`output.data.profiles[].spec_format[].platform_variables[].value`

`.server-changes/runs-replication-bigint-recovery.md` — release notes
entry.

## Why reactive, not pre-flight

`#prepareJson` runs millions of times per day on the replication hot
path. Walking every JSON tree to look for oversized integers would add
bounded-but-real CPU on every healthy row. `sanitizeRows` only fires
after a ClickHouse parse-error rejection, which is a few times a day
platform-wide. Extending it costs effectively zero on healthy traffic
and gains us recovery on the rare poisoned row.

## Verification

- Reproduced 1:1 in a throwaway Docker
`clickhouse/clickhouse-server:25.12.11.4` (closest available to the prod
`25.12.1.1579` build). Pre-sanitize JSON fails with the exact prod
error; post-sanitize JSON inserts cleanly and the row is readable with
`gp_id` stored as a String subtype.
- `pnpm --filter webapp exec vitest run
test/sanitizeRowsOnParseError.test.ts` — 22/22 passing (18 existing + 4
new).
- `pnpm run typecheck --filter webapp` — clean.

## Test plan

- [x] `pnpm run typecheck --filter webapp`
- [x] Unit tests pass against new + existing cases
- [x] End-to-end Docker ClickHouse repro confirms recovery
- [ ] Post-deploy: confirm `Sanitizing batch after ClickHouse JSON parse
error` warns fire instead of `Dropped batch …` errors when
`scan-social-profiles` outputs trip CH again
- [ ] Post-deploy: confirm `permanentlyDroppedBatches` counter stops
climbing in
`/stp/trigger-app-prod/ecs/replication/service-container/process-logs`

## What this does NOT do

- Doesn't backfill the ~120k+ existing stranded `EXECUTING` rows in
production. Same as #3708 — that needs a reconciliation/backfill sweep
(separate ticket — TRI-9755 fix #3).
- Doesn't address the upstream root cause (the customer task emitting a
JS-Number-precision-lossy big int). That's a customer-task concern; our
replication path needs to be robust to whatever shape arrives.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-27 09:00:53 -07:00
Oskar Otwinowski 5ba1b32d53 fix(core): handle conflicting dotted paths in unflattenAttributes (#3762) 2026-05-27 16:18:50 +02:00
Eric Allam 37eeaa3690 fix(webapp): skip V1 WAITING_FOR_DEPLOY drain on V2 promotes (#3742)
## Summary

Stops the legacy V1 `WAITING_FOR_DEPLOY` drain from running on every V2
deployment promotion. The drain queries `TaskRun` by
`status='WAITING_FOR_DEPLOY'`, which only V1-engine runs ever have — V2
runs use `PENDING_VERSION` and are handled out of band. Calling the
drain on V2 promotes produced empty queries against the status index and
unnecessary reader-DB load.

## Fix

Two layers:

1. Gate the enqueue at the call site in `ChangeCurrentDeploymentService`
so it only fires when the deployment's worker is on engine V1.
2. Add a `LEGACY_RUN_ENGINE_WAITING_FOR_DEPLOY_DISABLED` env var
(default `0`). When set to `1`, the service returns immediately from
`call()` — neuters any jobs already sitting in the worker queue from
before the deploy lands.

V1 customers see no change; V2 promotes no longer trigger the drain.
2026-05-25 07:10:13 +01:00
Matt Aitken 596a9bb90c fix(webapp): retain sessions-replication singleton import via globalThis assignment (#3738)
Linear: [TRI-9864](https://linear.app/triggerdotdev/issue/TRI-9864)
(Urgent)
Production incident:
[TRI-9863](https://linear.app/triggerdotdev/issue/TRI-9863) (mitigated
by image revert in cloud#910)

## Bug

`apps/webapp/package.json` declares `"sideEffects": false`. PR #3333
(`71d98b4e`) replaced the previous real method-call retention idiom at
the two `sessionsReplicationInstance` import sites with:

```ts
import { sessionsReplicationInstance } from "...";
void sessionsReplicationInstance;
```

esbuild treats `void <identifier>;` as a pure expression statement under
`sideEffects: false` and **tree-shakes the entire import** — including
the `singleton(...)` call inside `sessionsReplicationInstance.server.ts`
which is the only thing that fires
`initializeSessionsReplicationInstance()`. The sessions→ClickHouse
logical replication worker never starts, the slot is unconsumed, lag
grows.

### How it manifested in production

cloud#907's image bump rolled the `SessionReplicationService` ECS task
on prod at 14:32 UTC. The new container's startup log emitted `🗃️ Runs
replication service enabled` but **not** `🗃️ Sessions replication
service enabled` or `🗃️ Sessions replication service started`.
CloudWatch `OldestReplicationSlotLag` grew at ~220 MB/min and the `High
replication lag` alarm fired at 14:37 UTC. Prod was reverted to the
previous image (cloud#910) to stop the bleed.

### Verification

`grep` of the built bundle `apps/webapp/build/index.js` (built from
`c0365d36`):

- **3** occurrences of `Runs replication` / `runsReplicationInstance`
strings 
- **0** occurrences of `Sessions replication` /
`sessionsReplicationInstance` / `SessionsReplicationService` 

The runs path survives tree-shaking because `adminWorker.server.ts` and
`admin.api.v1.runs-replication.*` routes have real method calls
(`.start()`, `.teardown()`, `.backfill()`) — observable uses the
tree-shaker must preserve. The sessions singleton has no real callers,
only the `void` no-ops, hence its complete elimination from the bundle.

## Fix

Replace `void sessionsReplicationInstance;` with an assignment to
`globalThis`, an unambiguous observable side effect the bundler cannot
eliminate:

```ts
(globalThis as Record<string, unknown>).__sessionsReplicationInstance =
  sessionsReplicationInstance;
```

Applied at both call sites: `apps/webapp/app/entry.server.tsx` and
`apps/webapp/app/v3/services/adminWorker.server.ts`.

Surrounding comments updated to document the bundler interaction so the
next maintainer doesn't reintroduce `void`.

## Out of scope (follow-ups)

- **Robustness improvement**: change `apps/webapp/package.json` from
`"sideEffects": false` to an allowlist that includes
`*Instance.server.ts` files. Prevents the same regression shape via any
future `*Instance` singleton.
- **Build-time check**: add a `grep` post-build step in `publish.yml`
requiring `"Sessions replication"` to appear in
`apps/webapp/build/index.js`. Catches this exact regression at CI time.

## Test plan

- [x] `pnpm run typecheck --filter webapp` clean
- [ ] After merge + publish: confirm new image's
`SessionReplicationService` container logs `🗃️ Sessions replication
service enabled` and `🗃️ Sessions replication service started` at
startup
- [ ] After re-deploying to prod: confirm `OldestReplicationSlotLag`
stops growing and drains
2026-05-24 16:21:15 +01:00
Matt Aitken eefb96c87d fix(webapp): recover from ClickHouse JSON parse failures in runs replication (#3708)
## Summary

On a ClickHouse `Cannot parse JSON object` rejection,
`RunsReplicationService` now sanitizes lone UTF-16 surrogates across the
failing batch via the existing `sanitizeRows` helper and retries once.
If the sanitizer found nothing or the retry also fails, the batch is
dropped loudly with a counter increment, so the surrounding
`#insertWithRetry` layer doesn't spin three more times on a
deterministic failure. Non-parse errors propagate unchanged.

Mirrors the pattern from #3659 (for `ClickhouseEventRepository`) — same
root cause (lone UTF-16 surrogates in user-provided JSON), same recovery
shape, **reusing the same shared helpers** (`sanitizeRows`,
`isClickHouseJsonParseError`, `parseRowNumberFromError`).

Fixes the customer-facing symptom from
[TRI-9755](https://linear.app/triggerdotdev/issue/TRI-9755): a single
row's poisoned `output` JSON used to take down the
`COMPLETED_SUCCESSFULLY` UPDATE events for its 50+ batch-mates,
stranding them in `EXECUTING` in ClickHouse forever and inflating
"Running" counts on the Tasks page. Confirmed in production this is
ongoing — ~120k stale rows accumulated in a single 5-hour burst on
2026-05-18; smaller continuous leak before and after.

## What changed

`apps/webapp/app/services/runsReplicationService.server.ts`:

- Imports the three helpers from
`~/v3/eventRepository/sanitizeRowsOnParseError.server` (no duplication;
no move).
- New private `#insertWithJsonParseRecovery<T>(rows, doInsert,
contextLabel, attempt)` — generic over `TaskRunInsertArray[]` and
`PayloadInsertArray[]`, structurally identical to
`ClickhouseEventRepository.#insertWithJsonParseRecovery`. Try → on parse
error sanitize the whole batch (the `at row N` hint is logged but not
used to slice — semantics under `input_format_parallel_parsing` aren't
stable) → retry once → drop with loud log if sanitizer found nothing OR
retry still fails.
- `#insertTaskRunInserts` and `#insertPayloadInserts` extract a
`doInsert` closure and hand it to the wrapper. Existing error logging,
span recording, and `recordSpanError` are preserved inside the closure.
- New `private _permanentlyDroppedBatches = 0` counter with a public
getter, for ops dashboards and tests (matches the events-repo
convention). One shared counter for both insert sites — granularity
comes from the `contextLabel` (`task_runs_v2` /
`raw_task_runs_payload_v1`) on every log line.

`.server-changes/runs-replication-utf16-recovery.md` — release notes
entry.

## Why no new tests

The shared helpers already have full unit + real-ClickHouse contract
coverage from #3659
(`apps/webapp/test/sanitizeRowsOnParseError.test.ts`,
`apps/webapp/test/otlpUtf16Sanitization.integration.test.ts`). The new
wrapper is a line-for-line structural port. Adding a parallel
integration test would require synthesizing bad data that *escapes* the
preemptive `detectBadJsonStrings` check in `#prepareJson` but still
trips ClickHouse — non-trivial without hand-crafted fixtures and
wouldn't cover any new logic.

## What this does NOT do

- Doesn't touch the ~120k existing stale `EXECUTING` rows in production.
That needs a reconciliation/backfill sweep (separate ticket — TRI-9755
fix #3).
- Doesn't sanitize the `error` column path
(`runsReplicationService.server.ts:932 const errorData = { data:
run.error };`). Reactive recovery will catch it if it ever poisons a
batch, but feeding it through `#prepareJson` like `output` is a cheap
follow-up.

## Test plan

- [x] `pnpm run typecheck --filter webapp` — clean
- [ ] Post-deploy: confirm `permanentlyDroppedBatches` counter stays at
zero (or near-zero) in
`/stp/trigger-app-prod/ecs/replication/service-container/process-logs`,
and watch for `Sanitizing batch after ClickHouse JSON parse error` warns
to confirm recovery is firing on real traffic
- [ ] Post-deploy: confirm the rate of new
"EXECUTING-but-actually-COMPLETED" zombies in ClickHouse flattens
(current rate ≈ tens-to-hundreds per hour platform-wide)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-23 20:20:09 +01:00
Eric Allam 9f64bf404b docs(ai-chat): slim-wire HITL continuations + field-level merge contract (#3721)
## Summary

Updates the AI chat docs to match the slim-wire + field-level merge
behavior shipped in #3719 and the precise `.in/append` cap +
CORS-readable 413 shipped in #3720. No behavior changes here — code is
correct in `main`; the docs were lagging on three patterns customers
copy out of the page.

## What changed

- **`hydrateMessages` examples upsert by id** (in `lifecycle-hooks.mdx`,
`patterns/database-persistence.mdx`, and
`patterns/persistence-and-replay.mdx`). The previous
`stored.push(newMsg)` pattern duplicated the assistant id on HITL
continuations and caused the LLM to receive a tool call with no
`arguments`. The new examples include the rationale inline.
- **`onValidateMessages` example filters to user messages**
(`lifecycle-hooks.mdx`). The previous example called
`validateUIMessages({ messages, tools })` directly, which now throws on
HITL slim wires (the AI SDK schema requires `input` on resolved tool
parts). New example shows the filter pattern, with a Warning callout
explaining why.
- **Merge contract description updated** (`lifecycle-hooks.mdx`). The
old wording said incoming messages are "auto-merged" / "replaced"; the
new description explains the actual field-level overlay (state advances
only).
- **Approval-responded wire example slimmed** (`client-protocol.mdx`).
Shows the minimum shape the agent reads — `state` + `approval` (or
`output` / `errorText` for HITL). Notes that the built-in transports
ship this slim shape by default and that fuller shapes are still
accepted.
- **`/in/append` 413 row and FAQ updated** (`client-protocol.mdx`,
`patterns/trusted-edge-signals.mdx`). Reflects the new precise S2 cap
and the CORS-readable 413.
- **New changelog entry** at the top of `changelog.mdx` covering all of
the above.

The historical `## 512 KiB ceiling removed` entry further down the
changelog is left as-is (it's a snapshot of the prior transition), and
the v4.5 upgrade-guide section is skipped — the merge contract is
backwards compatible.

## Test plan

- Mintlify dev preview renders cleanly with no broken anchors
- Linked references resolve (`/ai-chat/lifecycle-hooks#hydratemessages`,
`/ai-chat/lifecycle-hooks#onvalidatemessages`,
`/ai-chat/patterns/database-persistence#alternative-hydratemessages`,
`/ai-chat/client-protocol#step-3-send-messages-stops-and-actions`,
`/ai-chat/patterns/large-payloads`)
2026-05-23 17:27:15 +01:00
github-actions[bot] d34014ddcc chore: release v4.5.0-rc.2 (#3702)
🚀 Publish Trigger.dev Docker / typecheck (push) Failing after 0s
🚀 Publish Trigger.dev Docker / publish-webapp (push) Has been skipped
🚀 Publish Trigger.dev Docker / publish-worker (push) Has been skipped
🚀 Publish Trigger.dev Docker / publish-worker-v4 (push) Has been skipped
🚀 Publish Trigger.dev Docker / units (push) Failing after 4s
## Summary
3 improvements, 1 bug fix.

## Improvements
- The per-turn merge now overlays the wire copy's tool-part state
advancement onto the agent's existing chain — `state` + the matching
resolution field (`output` / `errorText` / `approval`) come from the
wire, everything else (text, reasoning, tool `input`, provider metadata)
stays whatever the snapshot or `hydrateMessages` returned. Previously a
full-message replace overwrote those fields with whatever the client
shipped, so a slimmed wire copy landed a tool call with no `arguments`
on the next LLM call. Covers `output-available` / `output-error` (HITL
`addToolOutput`) and `approval-responded` / `output-denied` (approval
flow).
- `TriggerChatTransport.sendMessages` and `AgentChat.sendRaw` now slim
assistant messages that carry advanced tool parts. The wire payload is
just `{ id, role, parts: [<state + resolution field>] }` for
`submit-message` continuations; everything else passes through.
Reasoning blobs and full tool inputs no longer ride the wire on every
`addToolOutput` / `addToolApproveResponse`, so continuation payloads
stay well under the `.in/append` cap on long agent loops.
- Add `TriggerClient` for running multiple SDK clients side-by-side,
each with its own auth, preview branch, and baseURL. Useful when a
single process needs to trigger tasks or read runs across multiple
projects, environments, or preview branches without mutating shared
global state.
([#3683](https://github.com/triggerdotdev/trigger.dev/pull/3683))

## Bug fixes
- Fix `chat.agent` HITL continuations on reasoning-heavy turns. Two
changes that work together:
([#3719](https://github.com/triggerdotdev/trigger.dev/pull/3719))

<details>
<summary>Raw changeset output</summary>

⚠️⚠️⚠️⚠️⚠️⚠️

`main` is currently in **pre mode** so this branch has prereleases
rather than normal releases. If you want to exit prereleases, run
`changeset pre exit` on `main`.

⚠️⚠️⚠️⚠️⚠️⚠️

# Releases
## @trigger.dev/build@4.5.0-rc.2

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.5.0-rc.2`

## trigger.dev@4.5.0-rc.2

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/build@4.5.0-rc.2`
    -   `@trigger.dev/core@4.5.0-rc.2`
    -   `@trigger.dev/schema-to-json@4.5.0-rc.2`

## @trigger.dev/plugins@4.5.0-rc.2

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.5.0-rc.2`

## @trigger.dev/python@4.5.0-rc.2

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/sdk@4.5.0-rc.2`
    -   `@trigger.dev/build@4.5.0-rc.2`
    -   `@trigger.dev/core@4.5.0-rc.2`

## @trigger.dev/react-hooks@4.5.0-rc.2

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.5.0-rc.2`

## @trigger.dev/redis-worker@4.5.0-rc.2

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.5.0-rc.2`

## @trigger.dev/rsc@4.5.0-rc.2

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.5.0-rc.2`

## @trigger.dev/schema-to-json@4.5.0-rc.2

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.5.0-rc.2`

## @trigger.dev/sdk@4.5.0-rc.2

### Patch Changes

- Fix `chat.agent` HITL continuations on reasoning-heavy turns. Two
changes that work together:
([#3719](https://github.com/triggerdotdev/trigger.dev/pull/3719))

- The per-turn merge now overlays the wire copy's tool-part state
advancement onto the agent's existing chain — `state` + the matching
resolution field (`output` / `errorText` / `approval`) come from the
wire, everything else (text, reasoning, tool `input`, provider metadata)
stays whatever the snapshot or `hydrateMessages` returned. Previously a
full-message replace overwrote those fields with whatever the client
shipped, so a slimmed wire copy landed a tool call with no `arguments`
on the next LLM call. Covers `output-available` / `output-error` (HITL
`addToolOutput`) and `approval-responded` / `output-denied` (approval
flow).
- `TriggerChatTransport.sendMessages` and `AgentChat.sendRaw` now slim
assistant messages that carry advanced tool parts. The wire payload is
just `{ id, role, parts: [<state + resolution field>] }` for
`submit-message` continuations; everything else passes through.
Reasoning blobs and full tool inputs no longer ride the wire on every
`addToolOutput` / `addToolApproveResponse`, so continuation payloads
stay well under the `.in/append` cap on long agent loops.

Note: `onValidateMessages` receives the slim wire on HITL turns. If you
call `validateUIMessages` from `ai` against the full `messages` array it
will reject the slim assistant; filter to user messages (or skip on HITL
turns) — see the updated docstring on `onValidateMessages` for the
recommended pattern.

For `hydrateMessages` hooks that persist the chain, this release also
adds a small helper to the `@trigger.dev/sdk/ai` surface:

    ```ts
    import { chat, upsertIncomingMessage } from "@trigger.dev/sdk/ai";

    chat.agent({
hydrateMessages: async ({ chatId, trigger, incomingMessages }) => {
const record = await db.chat.findUnique({ where: { id: chatId } });
        const stored = record?.messages ?? [];
if (upsertIncomingMessage(stored, { trigger, incomingMessages })) {
await db.chat.update({ where: { id: chatId }, data: { messages: stored }
});
        }
        return stored;
      },
    });
    ```

It pushes fresh user messages by id, no-ops on HITL continuations (the
incoming shares an id with the existing assistant — the runtime overlays
the new tool-state advance), and skips on non-`submit-message` triggers.
Returns `true` if it mutated `stored` so the caller knows whether to
persist.

Net effect: `chat.addToolOutput(...)` /
`chat.addToolApproveResponse(...)` on multi-step reasoning agents
(OpenAI Responses with `store: false`, Anthropic extended thinking,
etc.) no longer blows the cap and no longer corrupts the LLM input.

- Add `TriggerClient` for running multiple SDK clients side-by-side,
each with its own auth, preview branch, and baseURL. Useful when a
single process needs to trigger tasks or read runs across multiple
projects, environments, or preview branches without mutating shared
global state.
([#3683](https://github.com/triggerdotdev/trigger.dev/pull/3683))

    ```ts
    import { TriggerClient } from "@trigger.dev/sdk";

const prod = new TriggerClient({ accessToken:
process.env.TRIGGER_PROD_KEY });
    const preview = new TriggerClient({
      accessToken: process.env.TRIGGER_PREVIEW_KEY,
      previewBranch: "signup-flow",
    });

    await prod.tasks.trigger("send-email", payload);
    await preview.runs.list({ status: ["COMPLETED"] });
    ```

-   Updated dependencies:
    -   `@trigger.dev/core@4.5.0-rc.2`

## @trigger.dev/core@4.5.0-rc.2

</details>

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-05-23 17:15:33 +01:00
Eric Allam 75679c7518 fix(sdk): chat HITL continuations no longer break the next LLM call (#3719)
## Summary

Multi-step reasoning agents with HITL tools (OpenAI Responses with
`store: false`, Anthropic extended thinking, etc.) failed on
`chat.addToolOutput(...)` continuations — either the wire payload blew
the `.in/append` cap (reasoning blobs + tool inputs routinely > 512
KiB), or app-side slimming workarounds got overwritten server-side and
the next LLM call landed a tool call with no `arguments`. Both modes are
fixed.

## Design

The per-turn merge in `chat.agent` now overlays only the tool-part state
advances (`output-available` / `output-error` / `approval-responded` /
`output-denied`) from the wire copy onto the hydrated/snapshot chain.
Previously it replaced the entire message, which dropped `input`,
reasoning, and text from the LLM's view whenever the wire was slim.

In parallel, `TriggerChatTransport.sendMessages` and `AgentChat.sendRaw`
now slim the assistant message themselves on `submit-message`
continuations: ship `{ id, role, parts: [<resolved tool part only>] }`,
everything else reconstructed server-side from `hydrateMessages` or the
durable snapshot. Continuation payloads drop from 600 KiB – 1 MiB to ~1
KiB.

`references/ai-chat` `aiChatHydrated.hydrateMessages` now upserts by id
instead of pushing. With slim continuations, a blind push duplicates the
assistant id in the returned chain — the merge updates the first match,
the slim duplicate goes straight to `toModelMessages` with no `input`,
and the LLM 4xx's. This is the canonical pattern customers should mirror
in their own hydrate implementations.

## Test plan

- 11 new tests (slim helper unit + slim+merge integration for HITL,
approval, default no-hydrate branch)
- Full SDK suite: 239 tests pass across 19 files
- End-to-end sweep against `references/ai-chat`: 19 customer-side smoke
tests green; HITL wire bodies confirmed at ~1 KiB (was 600 KiB+); no
provider 4xx errors across OpenAI Responses or Anthropic
2026-05-23 17:09:39 +01:00
Eric Allam c0365d36fb fix(webapp): precise S2 record cap + CORS 413 on session append (#3720)
## Summary

Two improvements to session `.in/append`:

- Oversize-body 413 responses now carry CORS headers, so browser fetches
see a readable status instead of an opaque `TypeError: Failed to fetch`.
App-side retry-on-disconnect loops no longer spin forever on a
permanently-rejected payload.
- The per-record cap is now computed precisely against S2's actual
ceiling instead of the conservative 512 KiB floor. Legitimate ~600-900
KiB tool outputs (search results, file content) now succeed;
pathological all-quote content that would double under JSON escape still
rejects cleanly.

## Design

S2 enforces a per-record metered size of `8 + 2*H + Σ(header name +
value) + body ≤ 1048576` bytes. With no record headers (our case), the
budget reduces to `body ≤ 1048568`. Verified empirically against cloud
S2 — append succeeds at metered=1048576 and 422s at 1048577 with `record
must have metered size less than 1 MiB`.

The old `MAX_APPEND_BODY_BYTES = 512 KiB` was derived by assuming
worst-case JSON escape doubling (every byte becomes `\"` or `\\`),
giving `(1 MiB - overhead) / 2`. Safe, but rejects ~half the legitimate
input space.

The new flow:

1. Pre-cap the HTTP body at 1 MiB (DoS guard against reading arbitrary
garbage before we can compute the wrap).
2. After reading, `S2RealtimeStreams.#appendPartByName` computes
`Buffer.byteLength(JSON.stringify({data: part, id: partId}), "utf8") +
8` and throws `S2RecordTooLargeError` (a `ServiceValidationError` with
status 413) if it would exceed S2's ceiling. The route's existing error
branch maps the throw to a 413 with a descriptive message.

The 413 CORS fix is a single-line change in `apiBuilder.server.ts` —
`wrapResponse` was being skipped on the body-too-large branch; every
other error branch wraps; the 413 was the exception.

## Test plan

- Empirically verified against cloud S2 with a boundary scan across
`[1048568, 1048569, ..., 1048576]` and across H ∈ {0, 1×5 hdr bytes,
1×14 hdr bytes} — the formula matches exactly
- Browser-side fetch on a 700 KiB POST now resolves with a readable
`status: 413` (no `TypeError: Failed to fetch`)
- A 900 KiB ASCII tool output now passes (would have 413'd at 512 KiB
pre-fix)
2026-05-23 16:08:10 +01:00
Matt Aitken 2fcc484a2b fix(webapp): route OrganizationDataStoresRegistry writes through the writer prisma (#3722)
## Bug

The `OrganizationDataStoresRegistry` singleton in
`apps/webapp/app/services/dataStores/organizationDataStoresRegistryInstance.server.ts`
was constructed with `$replica`. That client was then used by both the
polling read path *and* by `addDataStore` / `updateDataStore` /
`deleteDataStore` (and their backing `SecretStore.setSecret` upserts).
The write methods route through the read replica, which Postgres rejects
with **error code 25006**:

```
Invalid prisma.secretStore.upsert() invocation:
ConnectorError(ConnectorError { user_facing_error: None, kind: QueryError(PostgresError {
  code: "25006",
  message: "cannot execute INSERT in a read-only transaction",
  ...
}), transient: false })
```

User-visible symptom: the admin `/admin/data-stores` "Add data store"
form returns a 400 with this error wrapped, so no
`OrganizationDataStore` row can ever be created via the UI.

The read path (`loadFromDatabase` polling + `SecretStore.getSecret`) is
unaffected because `findMany` + secret read are read-only.

## Fix

Change the registry constructor to take both a writer and a replica:

```ts
constructor(writer: PrismaClient, replica: PrismaClient | PrismaReplicaClient)
```

- `loadFromDatabase()` keeps using `_replica` (and its
`SecretStore.getSecret` calls) — these are background cache-fillers, not
on user-latency-sensitive paths.
- `addDataStore` / `updateDataStore` / `deleteDataStore` (and their
`SecretStore.setSecret` / `deleteSecret` calls) now use `_writer`.

`organizationDataStoresRegistryInstance.server.ts` passes `(prisma,
$replica)` from `~/db.server`. Test sites that constructed with
`(prisma)` now pass `(prisma, prisma)` — the testcontainer exposes a
single client, so the writer/replica split collapses to one connection.

## Files

-
`apps/webapp/app/services/dataStores/organizationDataStoresRegistry.server.ts`
— constructor + read/write split
-
`apps/webapp/app/services/dataStores/organizationDataStoresRegistryInstance.server.ts`
— pass `prisma` alongside `$replica`
- `apps/webapp/test/organizationDataStoresRegistry.test.ts` — 14 call
sites bumped
- `apps/webapp/test/clickhouseFactory.test.ts` — 5 call sites bumped

## Test plan

- [x] Existing `organizationDataStoresRegistry.test.ts` +
`clickhouseFactory.test.ts` still pass (constructor sites updated;
behavior unchanged for tests).
- [ ] After deploy to test cloud, retry `/admin/data-stores` "Add data
store" form for the HIPAA org — should now succeed and the row should
appear.
- [ ] Verify the registry's polling reload picks up the new row within
`ORGANIZATION_DATA_STORES_RELOAD_INTERVAL_MS` (60s default) and the
factory starts routing to the org-scoped instance.
2026-05-23 12:00:31 +01:00
Eric Allam 61ca40b4b1 perf(run-engine,webapp): look up PENDING_VERSION runs via ClickHouse (#3707)
## Summary

When a background worker registers, the engine resolves runs that were
queued before the worker was ready (status `PENDING_VERSION`). That
lookup used to scan a Postgres status index on `TaskRun`. Move it to
ClickHouse: query candidate run ids from `task_runs_v2`, then refetch
the actual rows from Postgres by primary key with a `status =
'PENDING_VERSION'` guard for idempotency.

## Design

The lookup is a pluggable interface on the run engine
(`PendingVersionRunIdLookup`). The webapp wires a ClickHouse-backed
implementation through the org-scoped `clickhouseFactory` using a new
`"engine"` client type, configured by `RUN_ENGINE_CLICKHOUSE_*` env
vars. The URL falls back to `CLICKHOUSE_URL` when unset, so self-hosted
deployments don't need new config to keep working.

When the lookup returns no candidates, one bounded retry is scheduled
~5s later to cover ClickHouse replication lag against `task_runs_v2`.
The Postgres status guard on both the candidate refetch and the inner
`updateMany` prevents double-promotion when a retry races with a
concurrent deploy.

Tests cover three existing PENDING_VERSION cases via a small
Postgres-backed test adapter; new ClickHouse-backed integration tests
will follow.
2026-05-22 17:18:41 +01:00
Eric Allam 0d4891a5f2 perf(database): drop unused TaskRun(scheduleId, createdAt) index (#3706)
## Summary

Drops the unused composite Postgres index
`TaskRun_scheduleId_createdAt_idx`. The schedule list view reads from
ClickHouse, so this index served no Prisma query while still being
maintained on every `TaskRun` INSERT/UPDATE. Removing it reduces write
amplification on the primary database.

Sibling to the prior drop of `TaskRun_scheduleId_idx` and the earlier
removal of the `TaskRun.scheduleId` foreign key — all stemming from
migrating schedule-aware reads to ClickHouse.

## Verification

- Sampled `pg_stat_user_indexes` for `TaskRun` over multiple hours —
zero scans against this index.
- Grepped the codebase for any Prisma query filtering
`TaskRun.scheduleId` — none found. All schedule-aware listing routes
through `clickhouseRunsRepository`.
2026-05-22 16:34:00 +01:00
Saadi Myftija 2cbfcfac3d chore(webapp): make HTTP keep-alive timeout configurable (#3705)
Make the Express server's `keepAliveTimeout` configurable via
`HTTP_KEEPALIVE_TIMEOUT_MS`. Default preserved at 65000 ms — no behavior
change if unset.
2026-05-22 16:28:08 +01:00
nicktrn ddad9700d7 fix(supervisor): compat shim for COMPUTE checkpoint type (#3703)
Workloads bundled with CLI versions before v4.4.4 use a strict zod enum
for `checkpoint.type` that only allows DOCKER and KUBERNETES. When a
customer's runs are routed via the compute path, those old runners
receive `type: "COMPUTE"` on `/snapshots/since/...` and `/dequeue`
responses and fail validation - blocking silent migration of existing
deployments.

The workload never reads the field - only validates the shape. Rewriting
COMPUTE -> KUBERNETES on the way out lets older runners keep parsing
while the database and internal services keep the real value. Limited to
the two workload-facing endpoints whose response includes a checkpoint;
`/continue`, `/attempts/start`, `/attempts/complete` all return shapes
without one.

Followup to #3114.
2026-05-22 15:41:30 +01:00
Daniel Sutton 1015876b98 feat(webapp): user-based Sentry attribution with tenant tags (#3678)
## Summary

Stamp every Sentry event with the signed-in user and the tenant (org /
project / env) the request belongs to, so "Users Impacted" counts
distinct humans and events become filterable per tenant.

**Design after review (current):**

- `user.id = real user cuid` (from `requireUser`). "Users Impacted"
counts humans, not tenants.
- Tenant context (org / project / env slugs, IDs, env type) moves
entirely onto tags: `org_slug`, `project_slug`, `env_slug`, `org_id`,
`project_id`, `project_ref`, `environment_id`, `env_type`, plus
`impersonating` when set.
- Backed by an `AsyncLocalStorage` scope established at the HTTP entry.
Each entry point fills what it knows; loaders enrich the same scope with
what they already have.

**Zero new database queries.** The middleware does a regex match only.
Dashboard loaders that already query Prisma gain a couple of extra
selected columns; nothing new round-trips.

## How it's wired

- **Express middleware (`tenantContextResolver.server.ts`)** — parses
the URL with a regex and always opens an ALS scope. Populates whatever
subset of slugs is present: `/orgs/:o` → just `orgSlug`;
`/orgs/:o/projects/:p` adds `projectSlug`; the full triple adds
`envSlug`. Non-tenant paths get an empty scope so loaders can still
enrich.
- **`_app/route.tsx`** — already calls `requireUser`. Adds
`tenantContext.enrich({ userId: user.id })` for every authenticated
dashboard request. No new query.
- **Env layout loader (`_app.orgs.$o.projects.$p.env.$e/route.tsx`)** —
its existing `prisma.project.findFirst` gains two columns in `select`
(`externalRef`, `organization.id`). After it picks an env, calls
`tenantContext.enrich({ orgId, projectId, projectRef, envId, envType
})`. Same query, +2 columns.
- **API path (`apiBuilder.server.ts`)** — wraps every handler in
`tenantContext.run(tenantContextFromAuthEnvironment(authenticationResult.environment),
…)`. The mapper pulls `userId` from `env.orgMember?.userId` (already
selected by `authIncludeBase` — no schema change). Covers
`createLoaderApiRoute`, `createActionApiRoute`, and
`createMultiMethodApiRoute`.
- **Event processor (`sentryTenantContext.server.ts`)** — registered in
`entry.server.tsx` so it lives in the Remix bundle and shares the same
`tenantContext` ALS instance as the middleware and loaders. Stamps
whatever's present; nothing forced.

## Example events from local verification

| URL | `user.id` | Tags |
|-----|-----------|------|
| `/orgs/:o/projects/:p/env/:e/...` | real user cuid | `org_slug`,
`project_slug`, `env_slug`, `org_id`, `project_id`, `project_ref`,
`environment_id`, `env_type` |
| `/orgs/:o/settings` (non-env-scoped) | real user cuid | `org_slug`
only |
| API request with `orgMember` | `orgMember.userId` | full tenant set |
| API request without `orgMember` | (unset) | full tenant set |

## Trade-offs

1. On env-scoped pages, errors that fire before the env layout loader's
enrich callback runs get slugs + `user.id` but not the tenant IDs /
`env_type`. Realistic errors deep in async work get the full set. (Same
race as before, narrower window now that slugs/`user.id` are populated
up-front by the middleware and `_app` enrich.)
2. API requests where the environment has no `orgMember` get tenant tags
but no `user.id`. Those events still show in the issue but don't
contribute to "Users Impacted".

## Out of scope (deferred)

Background workers (`redis-worker`, `schedule-engine`) and socket
handlers. Those entry points don't set `tenantContext.run` yet — their
events ship without tenant attribution until each is wired in a
follow-up.

## Tests

31 unit tests across 4 files. New tests notably cover:

- `parseTenantPath`: org-only, org+project, and full-triple URL
variants.
- `tenantContext.enrich`: in-place patch, no-op outside `run()`,
concurrent-scope isolation, empty-scope + enrich pattern (for non-tenant
pages).
- `tenantContextFromAuthEnvironment`: with and without `orgMember` —
verifies the API path's `user.id` mapping.
- `addTenantContextToEvent`: empty scope, userId-only, slugs-only, full
enrichment, conditional tag emission, preservation of prior `event.user`
fields.

## Test plan

- [ ] `pnpm run typecheck --filter webapp`
- [ ] `pnpm run test --filter webapp -- test/tenantContext.test.ts
test/sentryTenantContext.test.ts test/tenantContextResolver.test.ts
test/tenantContextFromAuthEnvironment.test.ts`
- [ ] Local manual: with `SENTRY_DSN` set, hit a dashboard URL and an
API route, confirm the captured events carry `user.id` + the expected
tag set in Sentry.
- [ ] After ship: confirm "Users Impacted" on a real Sentry issue
reflects distinct users (not tenants).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-22 15:10:41 +01:00
Matt Aitken 71d98b4e6b Support for org-scoped ClickHouse (#3333)
Added `OrganizationDataStore` which allows orgs to have data stored in
specific separate services.

For now this is just used for ClickHouse. When using ClickHouse we get a
client for the factory and pass in the org id.

Particular care has to be made with two hot-insert paths:
1. RunReplicationService
2. OTLPExporter

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
2026-05-22 14:43:03 +01:00
Eric Allam 832cf7220b feat(sdk,core): add TriggerClient for per-instance SDK configuration (#3683)
## Summary

`new TriggerClient({...})` exposes the management API (tasks, runs,
schedules, envvars, batch, queues, deployments, prompts, auth) as an
explicit instance with its own auth, preview branch, and baseURL.
Multiple clients can coexist in one process without mutating shared
global state — useful when a single service triggers across multiple
projects, environments, or preview branches.

```ts
import { TriggerClient } from "@trigger.dev/sdk";

const prod = new TriggerClient({ accessToken: process.env.TRIGGER_PROD_KEY });
const preview = new TriggerClient({
  accessToken: process.env.TRIGGER_PREVIEW_KEY,
  previewBranch: "signup-flow",
});

await prod.tasks.trigger("send-email", payload);
await preview.runs.list({ status: ["COMPLETED"] });
```

The existing global `configure()` API keeps working unchanged.

## Design

Instance methods enter an `AsyncLocalStorage`-backed scope (`sdkScope`)
before delegating to the existing module-level functions. The four
"pollution" points that previously read globals now consult the scope
first:

- `apiClientManager.{baseURL, accessToken, branchName}` and
`clientOrThrow` — identity fields are scope-only when scoped; `baseURL`
still falls back to `TRIGGER_API_URL` because plumbing (where the API
lives) is not identity.
- `taskContext.{ctx, worker, isWarmStart, isInsideTask}` — masked inside
an isolated scope so a `client.tasks.trigger(...)` from inside a task
doesn't leak the parent's `parentRunId` / `lockToVersion` / `isTest`
into a trigger that hits a different project.
- Inline `getEnvVar("TRIGGER_VERSION")` reads in `shared.ts` go through
a `scopedEnvVar` helper that returns `undefined` inside an isolated
scope.

The `TriggerClient` class itself is a thin wrapper that captures the
scope in its constructor and proxies each namespace method to enter that
scope before calling the existing impl. Generic inference (e.g.
`client.tasks.trigger<typeof t>(...)`) is preserved via `Pick<typeof ns,
keyof curatedSubset>` typings.

Two correctness fixes uncovered along the way are folded in:

- `apiClientManager.setGlobalAPIClientConfiguration` no longer silently
no-ops on the second call. `configure()` now actually overrides as users
expect (this is the root cause behind some "I changed the config but
nothing happened" reports).
- `apiClientManager.runWithConfig` (and therefore `auth.withAuth`) is
now backed by `sdkScope.withScope` instead of "mutate the global and
restore in finally". Two parallel `withAuth` calls with different
configs no longer stomp each other.

Surface curation: instance namespaces drop methods that don't make sense
per-instance — `batch.*AndWait` (runtime-dependent), `schedules.task` /
`schedules.timezones` (definition-time / stateless), `prompts.define`
(definition-time), `auth.configure` / `auth.withAuth` (global-only).

## Test plan

- [x] 9 runtime unit tests in `triggerClient.test.ts` cover: required
accessToken, instance auth + branch headers, no env fallback for
identity fields, no leakage between global and instance, four parallel
calls across two clients stay isolated, taskContext masking +
`inheritContext: true` override, `configure()` second-call override,
parallel `auth.withAuth` isolation.
- [x] 10 type-level assertions in `triggerClient.types.test.ts` using
`expectTypeOf` + `@ts-expect-error` lock in generic inference, return
type passthrough, overload preservation, and curated-surface drift.
- [x] Full SDK suite (219 tests) and core suite (530 tests) pass.
- [x] Webapp typecheck clean.
- [x] End-to-end smoke test against local webapp and a
freshly-provisioned cloud project — six concurrent multi-client triggers
all returned 200 with run IDs, headers per-client as expected.
- [ ] Reviewer: run `references/multi-client` per its `README.md` to
reproduce the smoke test locally.

## Try it

`references/multi-client` is a new reference workspace that exercises
this end-to-end:

- `src/trigger/echo.ts` — trivial target task
- `src/trigger/fanOut.ts` — opens two `TriggerClient`s from inside a
task, fires `echo` through each in parallel
- `src/external/main.ts` — external Node script with two clients
triggering `echo` sequentially and concurrently; logs every outgoing
request's `authorization` + `x-trigger-branch`
- `src/external/isolation.ts` — interleaves global `configure()` and an
instance call, asserts the captured fetch sequence shows no leakage
either way
2026-05-22 12:29:42 +01:00
nicktrn 18d7144e74 ci: daily dependabot critical-severity slack alerts (#3701)
Sibling to the weekly summary, focused on critical alerts only. Pings
Slack daily while any critical alerts are open; skips the post entirely
when zero, so no daily "all clear" noise.

- Daily 08:00 UTC cron + `workflow_dispatch` with `severity` input
(default `critical`, override to `high`/`medium`/`low` for manual
checks)
- Reuses the existing `dependabot-summary` environment (token, channel,
bot)
- Alerts link at the end is severity-filtered
2026-05-22 10:32:08 +01:00
Eric Allam c0b9fdfce9 docs(ai-chat): clarify lastEventId is sessionId-keyed across run boundaries (#3700)
## Summary

Two docs edits that close a footgun customers persisting transport state
can hit. Clearing `lastEventId` on `chat.endRun()` looks intuitive — the
Run ended, the cursor must be stale — but the cursor is sessionId-keyed,
not runId-keyed. Clearing it forces the next `sendMessages` to subscribe
from `seq_num=0`, which may hit the prior turn's still-durable
`turn-complete` record and close the SSE empty before the new Run's
chunks arrive.

Spells out the invariant in the frontend transport persistence table and
adds a Warning in the `chat.endRun()` reference.

## Test plan

- [x] Mintlify preview renders
- [x] No callout stacking
2026-05-22 10:26:06 +01:00
Eric Allam c80b85e2cf docs(ai-chat): atomic onTurnComplete writes + Anthropic prose (#3693)
## Summary

Three post-merge fixes for the AI Agents docs (#3226), all caught by
review after merge.

## Fixes

- **`onTurnComplete` examples now use `db.$transaction`** — both the
Database persistence "Complete example" and the Lifecycle hooks
reference example were doing two separate `await` calls
(`db.chat.update` then `db.chatSession.upsert`). That's the exact
non-atomic pattern the warning earlier on the persistence page calls out
as : a refresh between the two writes reads a stale `lastEventId` and
duplicates the assistant message on resume. Both examples now use the
recommended atomic form.

- **Background injection self-review prose aligned with the code** — the
prose said "gpt-4o-mini" but the example above it had been swapped to
`claude-haiku-4-5`. The Anthropic-sweep script only touched code blocks;
this prose line wasn't picked up.

## Test plan

- [x] Both updated examples use `db.$transaction([...])`
- [x] Prose matches the model used in the code block
- [ ] Mintlify deployment passes
2026-05-21 17:04:13 +01:00
Eric Allam 80bb600cd5 docs(ai-chat): AI Agents documentation for v4.5 (#3226)
## Summary

Lands the full AI Agents documentation surface alongside the v4.5
release candidate of `@trigger.dev/sdk`. Covers `chat.agent` end to end
— defining agents, lifecycle hooks, the frontend transport, sub-agents,
recovery from cancel/crash/OOM, AI Prompts integration — and the
Sessions primitive that backs it.

## Coverage

- **Conceptual**: Overview, Quick Start, How it works.
- **Building agents**: Backend (`chat.agent` / `chat.createSession` /
raw primitives), Lifecycle hooks, Frontend transport, Server-side
`AgentChat`, Sessions reference, `chat.local` state primitive,
TypeScript types.
- **Features**: AI Prompts integration, Fast starts (Preload + Head
Start), Compaction, Pending Messages (steering), Background Injection
(`chat.inject` + `chat.defer`), Actions (undo / regenerate / edit),
Error handling.
- **Patterns (13)**: Sub-agents, Branching conversations, Code sandbox,
Database persistence, Persistence and replay, HITL, Tool result
auditing, Large payloads, Agent skills, OOM resilience, Recovery boot,
Trusted edge signals, Version upgrades.
- **Reference**: API Reference, Client Protocol (wire format), Testing
harness (`mockChatAgent`), MCP server tools, Upgrade guide, Changelog.

## Structure changes

- Top-level nav: AI → **Agents**, with sub-groups for *Building agents /
Features / Patterns / Reference*.
- New RC banner snippet on every page links to the supported AI SDK
versions table on the API Reference.
- All examples use Anthropic with `stopWhen: stepCountIs(15)`.

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 16:50:59 +01:00
github-actions[bot] 0b5b817a49 chore: release v4.5.0-rc.1 (#3691)
🚀 Publish Trigger.dev Docker / typecheck (push) Failing after 0s
🚀 Publish Trigger.dev Docker / publish-webapp (push) Has been skipped
🚀 Publish Trigger.dev Docker / publish-worker (push) Has been skipped
🚀 Publish Trigger.dev Docker / publish-worker-v4 (push) Has been skipped
🚀 Publish Trigger.dev Docker / units (push) Failing after 0s
🧭 Helm Chart Release / lint-and-test (push) Has been cancelled
🧭 Helm Chart Release / release (push) Has been cancelled
## Summary
2 bug fixes.

## Bug fixes
- Fix `chat.agent` skills silently missing in `trigger dev` for projects
whose task files read `process.env` at module top level (e.g. a
third-party SDK client initialized at import). Skill folders now bundle
into `.trigger/skills/` reliably regardless of which env vars are set
when the CLI launches.
([#3690](https://github.com/triggerdotdev/trigger.dev/pull/3690))
- Fix `COULD_NOT_FIND_EXECUTOR` when a task's definition is loaded via
`await import(...)` from inside another task's `run()`. The runtime
workers now register such tasks with a sentinel file context, and the
catalog logs a one-time warning per task id.
([#3688](https://github.com/triggerdotdev/trigger.dev/pull/3688))

<details>
<summary>Raw changeset output</summary>

⚠️⚠️⚠️⚠️⚠️⚠️

`main` is currently in **pre mode** so this branch has prereleases
rather than normal releases. If you want to exit prereleases, run
`changeset pre exit` on `main`.

⚠️⚠️⚠️⚠️⚠️⚠️

# Releases
## @trigger.dev/build@4.5.0-rc.1

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.5.0-rc.1`

## trigger.dev@4.5.0-rc.1

### Patch Changes

- Fix `chat.agent` skills silently missing in `trigger dev` for projects
whose task files read `process.env` at module top level (e.g. a
third-party SDK client initialized at import). Skill folders now bundle
into `.trigger/skills/` reliably regardless of which env vars are set
when the CLI launches.
([#3690](https://github.com/triggerdotdev/trigger.dev/pull/3690))
- Fix `COULD_NOT_FIND_EXECUTOR` when a task's definition is loaded via
`await import(...)` from inside another task's `run()`. The runtime
workers now register such tasks with a sentinel file context, and the
catalog logs a one-time warning per task id.
([#3688](https://github.com/triggerdotdev/trigger.dev/pull/3688))
-   Updated dependencies:
    -   `@trigger.dev/core@4.5.0-rc.1`
    -   `@trigger.dev/build@4.5.0-rc.1`
    -   `@trigger.dev/schema-to-json@4.5.0-rc.1`

## @trigger.dev/core@4.5.0-rc.1

### Patch Changes

- Fix `COULD_NOT_FIND_EXECUTOR` when a task's definition is loaded via
`await import(...)` from inside another task's `run()`. The runtime
workers now register such tasks with a sentinel file context, and the
catalog logs a one-time warning per task id.
([#3688](https://github.com/triggerdotdev/trigger.dev/pull/3688))

## @trigger.dev/plugins@4.5.0-rc.1

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.5.0-rc.1`

## @trigger.dev/python@4.5.0-rc.1

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.5.0-rc.1`
    -   `@trigger.dev/build@4.5.0-rc.1`
    -   `@trigger.dev/sdk@4.5.0-rc.1`

## @trigger.dev/react-hooks@4.5.0-rc.1

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.5.0-rc.1`

## @trigger.dev/redis-worker@4.5.0-rc.1

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.5.0-rc.1`

## @trigger.dev/rsc@4.5.0-rc.1

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.5.0-rc.1`

## @trigger.dev/schema-to-json@4.5.0-rc.1

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.5.0-rc.1`

## @trigger.dev/sdk@4.5.0-rc.1

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.5.0-rc.1`

</details>

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-05-21 16:40:21 +01:00
Eric Allam 5ddb81ac8d fix(cli): stop chat.agent skills silently disappearing from trigger dev (#3690)
## Summary

`trigger.dev dev` was silently dropping registered `chat.agent` skills
for any project whose task files read `process.env` at module top level
— e.g. a third-party SDK client initialized at import. The agent would
boot fine, but `skill.local()` failed at runtime with `ENOENT` because
the skill folder was never copied into `.trigger/skills/`.

## Design

The CLI ran two indexer passes in dev: the worker's own indexer (with
the full env it eventually executes tasks in), and a separate
skill-discovery indexer with only the CLI process's env. Top-level reads
of vars like `TRIGGER_API_URL` imported cleanly in the worker pass and
threw in the skill pass — the latter caught the error, warned, and
skipped skill copying. Failure was silent enough that `skill.local()`
only surfaced it at task runtime.

The skill registry is already part of the worker manifest. This PR drops
the duplicate pass and copies skill folders from that manifest after the
worker initializes. One indexer instead of two; a bad `SKILL.md` now
surfaces as a startup error instead of silently disappearing skills.

Deploy is unaffected — its skill discovery uses the project's
environment variables (fetched via the API, which fills in
`TRIGGER_API_URL` etc.), so the dev failure mode doesn't reach there.

## Test plan

- [x] New `references/agent-skills` reference project with
`skills.define` + a task that calls `skill.local()` and runs a bundled
script
- [x] On `main`, adding a top-level
`process.env.TRIGGER_API_URL!.includes(...)` read in any task file
reproduces the symptom: warning at dev startup, no `.trigger/skills/`
folder, `skill.local()` fails with ENOENT
- [x] On this branch, same project boots clean and `skill.local()` works
end-to-end
- [x] Deploy still works end-to-end with the new reference project
2026-05-21 16:30:24 +01:00
nicktrn dbf9b4e7e5 fix(core,cli): register tasks loaded via dynamic import during run (#3688)
On a warm worker process, a task whose `task()` definition is loaded via
`await import(...)` from inside another task's `run()` could end up
permanently missing from the catalog: the `task()` call fired with no
`_currentFileContext` set, `registerTaskMetadata` silently returned, and
Node's ESM module cache then blocked the worker's setContext + re-import
recovery from ever firing the call again. Subsequent runs of that task
on the same warm process failed with `COULD_NOT_FIND_EXECUTOR` until the
process hit `maxExecutionsPerProcess` and exited.

All five of these had to coincide on the same worker for the bug to
surface:

1. `processKeepAlive` enabled (so catalog state survives across runs).
2. A `run()` function (or lifecycle hook) does `await import(...)`.
3. The import's transitive static graph reaches a `task()` /
`schemaTask()` call.
4. The task containing the dynamic import is the **first** task to run
on a given warm worker process — so the dropped `task()` calls fire on
this process for the first time, are silently dropped, and Node's module
cache locks the wrong outcome in.
5. A subsequent run for one of the dropped task ids lands on the same
warm worker before it recycles.

The runtime workers now set a sentinel file context (`<no-context>`)
around the `executor.execute(...)` call, so `task()` invocations firing
during a run register normally. The catalog detects the sentinel and
emits a one-time `console.warn` per task id so the pattern stays visible
without spamming. The indexer never sets this context, so deploy-time
behavior is unchanged.

Repro is `references/hello-world/src/trigger/dynamicImportRepro*.ts`.
Verified end-to-end against a deployed image with firestarter
warm-starts on: pre-fix saw `COULD_NOT_FIND_EXECUTOR` on children that
landed on the parent-poisoned worker; post-fix all 23/23 runs succeeded
and the warning surfaces in the parent's run trace.
2026-05-21 16:21:49 +01:00
github-actions[bot] acfba02409 chore: release v4.5.0-rc.0 (#3563)
🚀 Publish Trigger.dev Docker / typecheck (push) Failing after 4s
🚀 Publish Trigger.dev Docker / units (push) Failing after 0s
🚀 Publish Trigger.dev Docker / publish-webapp (push) Has been skipped
🚀 Publish Trigger.dev Docker / publish-worker (push) Has been skipped
🚀 Publish Trigger.dev Docker / publish-worker-v4 (push) Has been skipped
🧭 Helm Chart Release / lint-and-test (push) Has been cancelled
🧭 Helm Chart Release / release (push) Has been cancelled
## Summary
44 improvements, 1 bug fix.

## Improvements
- **AI Prompts** — define prompt templates as code alongside your tasks,
version them on deploy, and override the text or model from the
dashboard without redeploying. Prompts integrate with the Vercel AI SDK
via `toAISDKTelemetry()` (links every generation span back to the
prompt) and with `chat.agent` via `chat.prompt.set()` +
`chat.toStreamTextOptions()`.
([#3629](https://github.com/triggerdotdev/trigger.dev/pull/3629))
- **Code-defined, deploy-versioned templates** — define with
`prompts.define({ id, model, config, variables, content })`. Every
deploy creates a new version visible in the dashboard. Mustache-style
placeholders (`{{var}}`, `{{#cond}}...{{/cond}}`) with Zod / ArkType /
Valibot-typed variables.
- **Dashboard overrides** — change a prompt's text or model from the
dashboard without redeploying. Overrides take priority over the deployed
"current" version and are environment-scoped (dev / staging / production
independent).
- **Resolve API** — `prompt.resolve(vars, { version?, label? })` returns
the compiled `text`, resolved `model`, `version`, and labels. Standalone
`prompts.resolve<typeof handle>(slug, vars)` for cross-file resolution
with full type inference on slug and variable shape.
- **AI SDK integration** — spread `resolved.toAISDKTelemetry({ ...extra
})` into any `generateText` / `streamText` call and every generation
span links to the prompt in the dashboard alongside its input variables,
model, tokens, and cost.
- **`chat.agent` integration** — `chat.prompt.set(resolved)` stores the
resolved prompt run-scoped; `chat.toStreamTextOptions({ registry })`
pulls `system`, `model` (resolved via the AI SDK provider registry),
`temperature` / `maxTokens` / etc., and telemetry into a single spread
for `streamText`.
- **Management SDK** — `prompts.list()`, `prompts.versions(slug)`,
`prompts.promote(slug, version)`, `prompts.createOverride(slug, body)`,
`prompts.updateOverride(slug, body)`, `prompts.removeOverride(slug)`,
`prompts.reactivateOverride(slug, version)`.
- **Dashboard** — prompts list with per-prompt usage sparklines;
per-prompt detail with Template / Details / Versions / Generations /
Metrics tabs. AI generation spans get a custom inspector showing the
linked prompt's metadata, input variables, and template content
alongside model, tokens, cost, and the message thread.
- Adds `onBoot` to `chat.agent` — a lifecycle hook that fires once per
worker process picking up the chat. Runs for the initial run, preloaded
runs, AND reactive continuation runs (post-cancel, crash, `endRun`,
`requestUpgrade`, OOM retry), before any other hook. Use it to
initialize `chat.local`, open per-process resources, or re-hydrate state
from your DB on continuation — anywhere the SAME run picking up after
suspend/resume isn't enough.
([#3543](https://github.com/triggerdotdev/trigger.dev/pull/3543))
- **AI SDK `useChat` integration** — a custom
[`ChatTransport`](https://sdk.vercel.ai/docs/ai-sdk-ui/transport)
(`useTriggerChatTransport`) plugs straight into Vercel AI SDK's
`useChat` hook. Text streaming, tool calls, reasoning, and `data-*`
parts all work natively over Trigger.dev's realtime streams. No custom
API routes needed.
- **First-turn fast path (`chat.headStart`)** — opt-in handler that runs
the first turn's `streamText` step in your warm server process while the
agent run boots in parallel, cutting cold-start TTFC by roughly half
(measured 2801ms → 1218ms on `claude-sonnet-4-6`). The agent owns step
2+ (tool execution, persistence, hooks) so heavy deps stay where they
belong. Web Fetch handler works natively in Next.js, Hono, SvelteKit,
Remix, Workers, etc.; bridge to Express/Fastify/Koa via
`chat.toNodeListener`. New `@trigger.dev/sdk/chat-server` subpath.
- **Multi-turn durability via Sessions** — every chat is backed by a
durable Session that outlives any individual run. Conversations resume
across page refreshes, idle timeout, crashes, and deploys; `resume:
true` reconnects via `lastEventId` so clients only see new chunks.
`sessions.list` enumerates chats for inbox-style UIs.
- **Auto-accumulated history, delta-only wire** — the backend
accumulates the full conversation across turns; clients only ship the
new message each turn. Long chats never hit the 512 KiB body cap.
Register `hydrateMessages` to be the source of truth yourself.
- **Lifecycle hooks** — `onPreload`, `onChatStart`,
`onValidateMessages`, `hydrateMessages`, `onTurnStart`,
`onBeforeTurnComplete`, `onTurnComplete`, `onChatSuspend`,
`onChatResume` — for persistence, validation, and post-turn work.
- **Stop generation** — client-driven `transport.stopGeneration(chatId)`
aborts mid-stream; the run stays alive for the next message, partial
response is captured, and aborted parts (stuck `partial-call` tools,
in-progress reasoning) are auto-cleaned.
- **Tool approvals (HITL)** — tools with `needsApproval: true` pause
until the user approves or denies via `addToolApprovalResponse`. The
runtime reconciles the updated assistant message by ID and continues
`streamText`.
- **Steering and background injection** — `pendingMessages` injects user
messages between tool-call steps so users can steer the agent
mid-execution; `chat.inject()` + `chat.defer()` adds context from
background work (self-review, RAG, safety checks) between turns.
- **Actions** — non-turn frontend commands (undo, rollback, regenerate,
edit) sent via `transport.sendAction`. Fire `hydrateMessages` +
`onAction` only — no turn hooks, no `run()`. `onAction` can return a
`StreamTextResult` for a model response, or `void` for side-effect-only.
- **Typed state primitives** — `chat.local<T>` for per-run state
accessible from hooks, `run()`, tools, and subtasks (auto-serialized
through `ai.toolExecute`); `chat.store` for typed shared data between
agent and client; `chat.history` for reading and mutating the message
chain; `clientDataSchema` for typed `clientData` in every hook.
- **`chat.toStreamTextOptions()`** — one spread into `streamText` wires
up versioned system [Prompts](https://trigger.dev/docs/ai/prompts),
model resolution, telemetry metadata, compaction, steering, and
background injection.
- **Multi-tab coordination** — `multiTab: true` + `useMultiTabChat`
prevents duplicate sends and syncs state across browser tabs via
`BroadcastChannel`. Non-active tabs go read-only with live updates.
- **Network resilience** — built-in indefinite retry with bounded
backoff, reconnect on `online` / tab refocus / bfcache restore,
`Last-Event-ID` mid-stream resume. No app code needed.
- **Sessions** — a durable, run-aware stream channel keyed on a stable
`externalId`. A Session is the unit of state that owns a multi-run
conversation: messages flow through `.in`, responses through `.out`,
both survive run boundaries. Sessions back the new `chat.agent` runtime,
and you can build on them directly for any pattern that needs durable
bi-directional streaming across runs.
([#3542](https://github.com/triggerdotdev/trigger.dev/pull/3542))
- Add `ai.toolExecute(task)` so you can wire a Trigger subtask in as the
`execute` handler of an AI SDK `tool()` while defining `description` and
`inputSchema` yourself — useful when you want full control over the tool
surface and just need Trigger's subtask machinery for the body.
([#3546](https://github.com/triggerdotdev/trigger.dev/pull/3546))
- Type `chat.createStartSessionAction` against your chat agent so
`clientData` is typed end-to-end on the first turn:
([#3684](https://github.com/triggerdotdev/trigger.dev/pull/3684))
- Add `region` to the runs list / retrieve API: filter runs by region
(`runs.list({ region: "..." })` / `filter[region]=<masterQueue>`) and
read each run's executing region from the new `region` field on the
response.
([#3612](https://github.com/triggerdotdev/trigger.dev/pull/3612))
- Add `TRIGGER_BUILD_SKIP_REWRITE_TIMESTAMP=1` escape hatch for local
self-hosted builds whose buildx driver doesn't support
`rewrite-timestamp` alongside push (e.g. orbstack's default `docker`
driver).
([#3618](https://github.com/triggerdotdev/trigger.dev/pull/3618))
- Reject overlong `idempotencyKey` values at the API boundary so they no
longer trip an internal size limit on the underlying unique index and
surface as a generic 500. Inputs are capped at 2048 characters — well
above what `idempotencyKeys.create()` produces (a 64-character hash) and
above any realistic raw key. Applies to `tasks.trigger`,
`tasks.batchTrigger`, `batch.create` (Phase 1 streaming batches),
`wait.createToken`, `wait.forDuration`, and the input/session stream
waitpoint endpoints. Over-limit requests now return a structured 400
instead.
([#3560](https://github.com/triggerdotdev/trigger.dev/pull/3560))
- **AI SDK `useChat` integration** — a custom
[`ChatTransport`](https://sdk.vercel.ai/docs/ai-sdk-ui/transport)
(`useTriggerChatTransport`) plugs straight into Vercel AI SDK's
`useChat` hook. Text streaming, tool calls, reasoning, and `data-*`
parts all work natively over Trigger.dev's realtime streams. No custom
API routes needed.
- **First-turn fast path (`chat.headStart`)** — opt-in handler that runs
the first turn's `streamText` step in your warm server process while the
agent run boots in parallel, cutting cold-start TTFC by roughly half
(measured 2801ms → 1218ms on `claude-sonnet-4-6`). The agent owns step
2+ (tool execution, persistence, hooks) so heavy deps stay where they
belong. Web Fetch handler works natively in Next.js, Hono, SvelteKit,
Remix, Workers, etc.; bridge to Express/Fastify/Koa via
`chat.toNodeListener`. New `@trigger.dev/sdk/chat-server` subpath.
- **Multi-turn durability via Sessions** — every chat is backed by a
durable Session that outlives any individual run. Conversations resume
across page refreshes, idle timeout, crashes, and deploys; `resume:
true` reconnects via `lastEventId` so clients only see new chunks.
`sessions.list` enumerates chats for inbox-style UIs.
- **Auto-accumulated history, delta-only wire** — the backend
accumulates the full conversation across turns; clients only ship the
new message each turn. Long chats never hit the 512 KiB body cap.
Register `hydrateMessages` to be the source of truth yourself.
- **Lifecycle hooks** — `onPreload`, `onChatStart`,
`onValidateMessages`, `hydrateMessages`, `onTurnStart`,
`onBeforeTurnComplete`, `onTurnComplete`, `onChatSuspend`,
`onChatResume` — for persistence, validation, and post-turn work.
- **Stop generation** — client-driven `transport.stopGeneration(chatId)`
aborts mid-stream; the run stays alive for the next message, partial
response is captured, and aborted parts (stuck `partial-call` tools,
in-progress reasoning) are auto-cleaned.
- **Tool approvals (HITL)** — tools with `needsApproval: true` pause
until the user approves or denies via `addToolApprovalResponse`. The
runtime reconciles the updated assistant message by ID and continues
`streamText`.
- **Steering and background injection** — `pendingMessages` injects user
messages between tool-call steps so users can steer the agent
mid-execution; `chat.inject()` + `chat.defer()` adds context from
background work (self-review, RAG, safety checks) between turns.
- **Actions** — non-turn frontend commands (undo, rollback, regenerate,
edit) sent via `transport.sendAction`. Fire `hydrateMessages` +
`onAction` only — no turn hooks, no `run()`. `onAction` can return a
`StreamTextResult` for a model response, or `void` for side-effect-only.
- **Typed state primitives** — `chat.local<T>` for per-run state
accessible from hooks, `run()`, tools, and subtasks (auto-serialized
through `ai.toolExecute`); `chat.store` for typed shared data between
agent and client; `chat.history` for reading and mutating the message
chain; `clientDataSchema` for typed `clientData` in every hook.
- **`chat.toStreamTextOptions()`** — one spread into `streamText` wires
up versioned system [Prompts](https://trigger.dev/docs/ai/prompts),
model resolution, telemetry metadata, compaction, steering, and
background injection.
- **Multi-tab coordination** — `multiTab: true` + `useMultiTabChat`
prevents duplicate sends and syncs state across browser tabs via
`BroadcastChannel`. Non-active tabs go read-only with live updates.
- **Network resilience** — built-in indefinite retry with bounded
backoff, reconnect on `online` / tab refocus / bfcache restore,
`Last-Event-ID` mid-stream resume. No app code needed.
- Retry `TASK_PROCESS_SIGSEGV` task crashes under the user's retry
policy instead of failing the run on the first segfault. SIGSEGV in Node
tasks is frequently non-deterministic (native addon races, JIT/GC
interaction, near-OOM in native code, host issues), so retrying on a
fresh process often succeeds. The retry is gated by the task's existing
`retry` config + `maxAttempts` — same path `TASK_PROCESS_SIGTERM` and
uncaught exceptions already use — so tasks without a retry policy still
fail fast.
([#3552](https://github.com/triggerdotdev/trigger.dev/pull/3552))
- The public interfaces for a plugin system. Initially consolidated
authentication and authorization interfaces.
([#3499](https://github.com/triggerdotdev/trigger.dev/pull/3499))
- Add MollifierBuffer and MollifierDrainer primitives for trigger burst
smoothing.
([#3614](https://github.com/triggerdotdev/trigger.dev/pull/3614))

## Bug fixes
- Fix `LocalsKey<T>` type incompatibility across dual-package builds.
The phantom value-type brand no longer uses a module-level `unique
symbol`, so a single TypeScript compilation that resolves the type from
both the ESM and CJS outputs (which can happen under certain pnpm
hoisting layouts) no longer sees two structurally-incompatible variants
of the same type.
([#3626](https://github.com/triggerdotdev/trigger.dev/pull/3626))

<details>
<summary>Raw changeset output</summary>

⚠️⚠️⚠️⚠️⚠️⚠️

`main` is currently in **pre mode** so this branch has prereleases
rather than normal releases. If you want to exit prereleases, run
`changeset pre exit` on `main`.

⚠️⚠️⚠️⚠️⚠️⚠️

# Releases
## @trigger.dev/sdk@4.5.0-rc.0

### Minor Changes

- **AI Prompts** — define prompt templates as code alongside your tasks,
version them on deploy, and override the text or model from the
dashboard without redeploying. Prompts integrate with the Vercel AI SDK
via `toAISDKTelemetry()` (links every generation span back to the
prompt) and with `chat.agent` via `chat.prompt.set()` +
`chat.toStreamTextOptions()`.
([#3629](https://github.com/triggerdotdev/trigger.dev/pull/3629))

    ```ts
    import { prompts } from "@trigger.dev/sdk";
    import { generateText } from "ai";
    import { openai } from "@ai-sdk/openai";
    import { z } from "zod";

    export const supportPrompt = prompts.define({
      id: "customer-support",
      model: "gpt-4o",
      config: { temperature: 0.7 },
      variables: z.object({
        customerName: z.string(),
        plan: z.string(),
        issue: z.string(),
      }),
      content: `You are a support agent for Acme.

    Customer: {{customerName}} ({{plan}} plan)
    Issue: {{issue}}`,
    });

    const resolved = await supportPrompt.resolve({
      customerName: "Alice",
      plan: "Pro",
      issue: "Can't access billing",
    });

    const result = await generateText({
      model: openai(resolved.model ?? "gpt-4o"),
      system: resolved.text,
      prompt: "Can't access billing",
      ...resolved.toAISDKTelemetry(),
    });
    ```

    **What you get:**

- **Code-defined, deploy-versioned templates** — define with
`prompts.define({ id, model, config, variables, content })`. Every
deploy creates a new version visible in the dashboard. Mustache-style
placeholders (`{{var}}`, `{{#cond}}...{{/cond}}`) with Zod / ArkType /
Valibot-typed variables.
- **Dashboard overrides** — change a prompt's text or model from the
dashboard without redeploying. Overrides take priority over the deployed
"current" version and are environment-scoped (dev / staging / production
independent).
- **Resolve API** — `prompt.resolve(vars, { version?, label? })` returns
the compiled `text`, resolved `model`, `version`, and labels. Standalone
`prompts.resolve<typeof handle>(slug, vars)` for cross-file resolution
with full type inference on slug and variable shape.
- **AI SDK integration** — spread `resolved.toAISDKTelemetry({ ...extra
})` into any `generateText` / `streamText` call and every generation
span links to the prompt in the dashboard alongside its input variables,
model, tokens, and cost.
- **`chat.agent` integration** — `chat.prompt.set(resolved)` stores the
resolved prompt run-scoped; `chat.toStreamTextOptions({ registry })`
pulls `system`, `model` (resolved via the AI SDK provider registry),
`temperature` / `maxTokens` / etc., and telemetry into a single spread
for `streamText`.
- **Management SDK** — `prompts.list()`, `prompts.versions(slug)`,
`prompts.promote(slug, version)`, `prompts.createOverride(slug, body)`,
`prompts.updateOverride(slug, body)`, `prompts.removeOverride(slug)`,
`prompts.reactivateOverride(slug, version)`.
- **Dashboard** — prompts list with per-prompt usage sparklines;
per-prompt detail with Template / Details / Versions / Generations /
Metrics tabs. AI generation spans get a custom inspector showing the
linked prompt's metadata, input variables, and template content
alongside model, tokens, cost, and the message thread.

See [/docs/ai/prompts](https://trigger.dev/docs/ai/prompts) for the full
reference — template syntax, version resolution order, override
workflow, and type utilities (`PromptHandle`, `PromptIdentifier`,
`PromptVariables`).

- Adds `onBoot` to `chat.agent` — a lifecycle hook that fires once per
worker process picking up the chat. Runs for the initial run, preloaded
runs, AND reactive continuation runs (post-cancel, crash, `endRun`,
`requestUpgrade`, OOM retry), before any other hook. Use it to
initialize `chat.local`, open per-process resources, or re-hydrate state
from your DB on continuation — anywhere the SAME run picking up after
suspend/resume isn't enough.
([#3543](https://github.com/triggerdotdev/trigger.dev/pull/3543))

    ```ts
const userContext = chat.local<{ name: string; plan: string }>({ id:
"userContext" });

    export const myChat = chat.agent({
      id: "my-chat",
      onBoot: async ({ clientData, continuation }) => {
const user = await db.user.findUnique({ where: { id: clientData.userId }
});
        userContext.init({ name: user.name, plan: user.plan });
      },
      run: async ({ messages, signal }) =>
streamText({ model: openai("gpt-4o"), messages, abortSignal: signal }),
    });
    ```

Use `onBoot` (not `onChatStart`) for state setup that must run every
time a worker picks up the chat — `onChatStart` fires once per chat and
won't run on continuation, leaving `chat.local` uninitialized when
`run()` tries to use it.

- **AI Agents** — run AI SDK chat completions as durable Trigger.dev
agents instead of fragile API routes. Define an agent in one function,
point `useChat` at it from React, and the conversation survives page
refreshes, network blips, and process restarts.
([#3543](https://github.com/triggerdotdev/trigger.dev/pull/3543))

    ```ts
    import { chat } from "@trigger.dev/sdk/ai";
    import { streamText } from "ai";
    import { openai } from "@ai-sdk/openai";

    export const myChat = chat.agent({
      id: "my-chat",
      run: async ({ messages, signal }) =>
streamText({ model: openai("gpt-4o"), messages, abortSignal: signal }),
    });
    ```

    ```tsx
    import { useChat } from "@ai-sdk/react";
import { useTriggerChatTransport } from "@trigger.dev/sdk/chat/react";

const transport = useTriggerChatTransport({ task: "my-chat",
accessToken, startSession });
    const { messages, sendMessage } = useChat({ transport });
    ```

    **What you get:**

- **AI SDK `useChat` integration** — a custom
[`ChatTransport`](https://sdk.vercel.ai/docs/ai-sdk-ui/transport)
(`useTriggerChatTransport`) plugs straight into Vercel AI SDK's
`useChat` hook. Text streaming, tool calls, reasoning, and `data-*`
parts all work natively over Trigger.dev's realtime streams. No custom
API routes needed.
- **First-turn fast path (`chat.headStart`)** — opt-in handler that runs
the first turn's `streamText` step in your warm server process while the
agent run boots in parallel, cutting cold-start TTFC by roughly half
(measured 2801ms → 1218ms on `claude-sonnet-4-6`). The agent owns step
2+ (tool execution, persistence, hooks) so heavy deps stay where they
belong. Web Fetch handler works natively in Next.js, Hono, SvelteKit,
Remix, Workers, etc.; bridge to Express/Fastify/Koa via
`chat.toNodeListener`. New `@trigger.dev/sdk/chat-server` subpath.
- **Multi-turn durability via Sessions** — every chat is backed by a
durable Session that outlives any individual run. Conversations resume
across page refreshes, idle timeout, crashes, and deploys; `resume:
true` reconnects via `lastEventId` so clients only see new chunks.
`sessions.list` enumerates chats for inbox-style UIs.
- **Auto-accumulated history, delta-only wire** — the backend
accumulates the full conversation across turns; clients only ship the
new message each turn. Long chats never hit the 512 KiB body cap.
Register `hydrateMessages` to be the source of truth yourself.
- **Lifecycle hooks** — `onPreload`, `onChatStart`,
`onValidateMessages`, `hydrateMessages`, `onTurnStart`,
`onBeforeTurnComplete`, `onTurnComplete`, `onChatSuspend`,
`onChatResume` — for persistence, validation, and post-turn work.
- **Stop generation** — client-driven `transport.stopGeneration(chatId)`
aborts mid-stream; the run stays alive for the next message, partial
response is captured, and aborted parts (stuck `partial-call` tools,
in-progress reasoning) are auto-cleaned.
- **Tool approvals (HITL)** — tools with `needsApproval: true` pause
until the user approves or denies via `addToolApprovalResponse`. The
runtime reconciles the updated assistant message by ID and continues
`streamText`.
- **Steering and background injection** — `pendingMessages` injects user
messages between tool-call steps so users can steer the agent
mid-execution; `chat.inject()` + `chat.defer()` adds context from
background work (self-review, RAG, safety checks) between turns.
- **Actions** — non-turn frontend commands (undo, rollback, regenerate,
edit) sent via `transport.sendAction`. Fire `hydrateMessages` +
`onAction` only — no turn hooks, no `run()`. `onAction` can return a
`StreamTextResult` for a model response, or `void` for side-effect-only.
- **Typed state primitives** — `chat.local<T>` for per-run state
accessible from hooks, `run()`, tools, and subtasks (auto-serialized
through `ai.toolExecute`); `chat.store` for typed shared data between
agent and client; `chat.history` for reading and mutating the message
chain; `clientDataSchema` for typed `clientData` in every hook.
- **`chat.toStreamTextOptions()`** — one spread into `streamText` wires
up versioned system [Prompts](https://trigger.dev/docs/ai/prompts),
model resolution, telemetry metadata, compaction, steering, and
background injection.
- **Multi-tab coordination** — `multiTab: true` + `useMultiTabChat`
prevents duplicate sends and syncs state across browser tabs via
`BroadcastChannel`. Non-active tabs go read-only with live updates.
- **Network resilience** — built-in indefinite retry with bounded
backoff, reconnect on `online` / tab refocus / bfcache restore,
`Last-Event-ID` mid-stream resume. No app code needed.

See [/docs/ai-chat](https://trigger.dev/docs/ai-chat/overview) for the
full surface — quick start, three backend approaches (`chat.agent`,
`chat.createSession`, raw task), persistence and code-sandbox patterns,
type-level guides, and API reference.

- Add read primitives to `chat.history` for HITL flows:
`getPendingToolCalls()`, `getResolvedToolCalls()`,
`extractNewToolResults(message)`, `getChain()`, and
`findMessage(messageId)`. These lift the accumulator-walking logic that
customers building human-in-the-loop tools were re-implementing into the
SDK. ([#3543](https://github.com/triggerdotdev/trigger.dev/pull/3543))

Use `getPendingToolCalls()` to gate fresh user turns while a tool call
is awaiting an answer. Use `extractNewToolResults(message)` to dedup
tool results when persisting to your own store — the helper returns only
the parts whose `toolCallId` is not already resolved on the chain.

    ```ts
    const pending = chat.history.getPendingToolCalls();
    if (pending.length > 0) {
      // an addToolOutput is expected before a new user message
    }

    onTurnComplete: async ({ responseMessage }) => {
const newResults = chat.history.extractNewToolResults(responseMessage);
      for (const r of newResults) {
await db.toolResults.upsert({ id: r.toolCallId, output: r.output,
errorText: r.errorText });
      }
    };
    ```

- **Sessions** — a durable, run-aware stream channel keyed on a stable
`externalId`. A Session is the unit of state that owns a multi-run
conversation: messages flow through `.in`, responses through `.out`,
both survive run boundaries. Sessions back the new `chat.agent` runtime,
and you can build on them directly for any pattern that needs durable
bi-directional streaming across runs.
([#3542](https://github.com/triggerdotdev/trigger.dev/pull/3542))

    ```ts
    import { sessions, tasks } from "@trigger.dev/sdk";

    // Trigger a task and subscribe to its session output in one call
const { runId, stream } = await tasks.triggerAndSubscribe("my-task",
payload, {
      externalId: "user-456",
    });

    for await (const chunk of stream) {
      // ...
    }

// Enumerate existing sessions (powers inbox-style UIs without a
separate index)
for await (const s of sessions.list({ type: "chat.agent", tag:
"user:user-456" })) {
      console.log(s.id, s.externalId, s.createdAt, s.closedAt);
    }
    ```

See [/docs/ai-chat/overview](https://trigger.dev/docs/ai-chat/overview)
for the full surface — Sessions powers the durable, resumable chat
runtime described there.

### Patch Changes

- Add Agent Skills for `chat.agent`. Drop a folder with a `SKILL.md` and
any helper scripts/references next to your task code, register it with
`skills.define({ id, path })`, and the CLI bundles it into the deploy
image automatically — no `trigger.config.ts` changes. The agent gets a
one-line summary in its system prompt and discovers full instructions on
demand via `loadSkill`, with `bash` and `readFile` tools scoped
per-skill (path-traversal guards, output caps, abort-signal
propagation).
([#3543](https://github.com/triggerdotdev/trigger.dev/pull/3543))

    ```ts
const pdfSkill = skills.define({ id: "pdf-extract", path:
"./skills/pdf-extract" });

    chat.skills.set([await pdfSkill.local()]);
    ```

Built on the [AI SDK cookbook
pattern](https://ai-sdk.dev/cookbook/guides/agent-skills) — portable
across providers. SDK + CLI only for now; dashboard-editable `SKILL.md`
text is on the roadmap.

- Add `ai.toolExecute(task)` so you can wire a Trigger subtask in as the
`execute` handler of an AI SDK `tool()` while defining `description` and
`inputSchema` yourself — useful when you want full control over the tool
surface and just need Trigger's subtask machinery for the body.
([#3546](https://github.com/triggerdotdev/trigger.dev/pull/3546))

    ```ts
    const myTool = tool({
      description: "...",
      inputSchema: z.object({ ... }),
      execute: ai.toolExecute(mySubtask),
    });
    ```

`ai.tool(task)` (`toolFromTask`) keeps doing the all-in-one wrap and now
aligns its return type with AI SDK's `ToolSet`. Minimum `ai` peer raised
to `^6.0.116` to avoid cross-version `ToolSet` mismatches in monorepos.

- Stamp `gen_ai.conversation.id` (the chat id) on every span and metric
emitted from inside a `chat.task` or `chat.agent` run. Lets you filter
dashboard spans, runs, and metrics by the chat conversation that
produced them — independent of the run boundary, so multi-run chats
correlate cleanly. No code changes required on the user side.
([#3543](https://github.com/triggerdotdev/trigger.dev/pull/3543))

- Type `chat.createStartSessionAction` against your chat agent so
`clientData` is typed end-to-end on the first turn:
([#3684](https://github.com/triggerdotdev/trigger.dev/pull/3684))

    ```ts
    import { chat } from "@trigger.dev/sdk/ai";
    import type { myChat } from "@/trigger/chat";

export const startChatSession = chat.createStartSessionAction<typeof
myChat>("my-chat");

// In the browser, threaded from the transport's typed startSession
callback:
    const transport = useTriggerChatTransport<typeof myChat>({
      task: "my-chat",
startSession: ({ chatId, clientData }) => startChatSession({ chatId,
clientData }),
      // ...
    });
    ```

`ChatStartSessionParams` gains a typed `clientData` field — folded into
the first run's `payload.metadata` so `onPreload` / `onChatStart` see
the same shape per-turn `metadata` carries via the transport. The opaque
session-level `metadata` field is unchanged.

- Unit-test `chat.agent` definitions offline with `mockChatAgent` from
`@trigger.dev/sdk/ai/test`. Drives a real agent's turn loop in-process —
no network, no task runtime — so you can send messages, actions, and
stop signals via driver methods, inspect captured output chunks, and
verify hooks fire. Pairs with `MockLanguageModelV3` from `ai/test` for
model mocking. `setupLocals` lets you pre-seed `locals` (DB clients,
service stubs) before `run()` starts.
([#3543](https://github.com/triggerdotdev/trigger.dev/pull/3543))

The broader `runInMockTaskContext` harness it's built on lives at
`@trigger.dev/core/v3/test` — useful for unit-testing any task code, not
just chat.

- Add `region` to the runs list / retrieve API: filter runs by region
(`runs.list({ region: "..." })` / `filter[region]=<masterQueue>`) and
read each run's executing region from the new `region` field on the
response.
([#3612](https://github.com/triggerdotdev/trigger.dev/pull/3612))

-   Updated dependencies:
    -   `@trigger.dev/core@4.5.0-rc.0`

## @trigger.dev/build@4.5.0-rc.0

### Patch Changes

- Add Agent Skills for `chat.agent`. Drop a folder with a `SKILL.md` and
any helper scripts/references next to your task code, register it with
`skills.define({ id, path })`, and the CLI bundles it into the deploy
image automatically — no `trigger.config.ts` changes. The agent gets a
one-line summary in its system prompt and discovers full instructions on
demand via `loadSkill`, with `bash` and `readFile` tools scoped
per-skill (path-traversal guards, output caps, abort-signal
propagation).
([#3543](https://github.com/triggerdotdev/trigger.dev/pull/3543))

    ```ts
const pdfSkill = skills.define({ id: "pdf-extract", path:
"./skills/pdf-extract" });

    chat.skills.set([await pdfSkill.local()]);
    ```

Built on the [AI SDK cookbook
pattern](https://ai-sdk.dev/cookbook/guides/agent-skills) — portable
across providers. SDK + CLI only for now; dashboard-editable `SKILL.md`
text is on the roadmap.

-   Updated dependencies:
    -   `@trigger.dev/core@4.5.0-rc.0`

## trigger.dev@4.5.0-rc.0

### Patch Changes

- Add Agent Skills for `chat.agent`. Drop a folder with a `SKILL.md` and
any helper scripts/references next to your task code, register it with
`skills.define({ id, path })`, and the CLI bundles it into the deploy
image automatically — no `trigger.config.ts` changes. The agent gets a
one-line summary in its system prompt and discovers full instructions on
demand via `loadSkill`, with `bash` and `readFile` tools scoped
per-skill (path-traversal guards, output caps, abort-signal
propagation).
([#3543](https://github.com/triggerdotdev/trigger.dev/pull/3543))

    ```ts
const pdfSkill = skills.define({ id: "pdf-extract", path:
"./skills/pdf-extract" });

    chat.skills.set([await pdfSkill.local()]);
    ```

Built on the [AI SDK cookbook
pattern](https://ai-sdk.dev/cookbook/guides/agent-skills) — portable
across providers. SDK + CLI only for now; dashboard-editable `SKILL.md`
text is on the roadmap.

- Add `TRIGGER_BUILD_SKIP_REWRITE_TIMESTAMP=1` escape hatch for local
self-hosted builds whose buildx driver doesn't support
`rewrite-timestamp` alongside push (e.g. orbstack's default `docker`
driver).
([#3618](https://github.com/triggerdotdev/trigger.dev/pull/3618))

- The CLI MCP server's agent-chat tools (`start_agent_chat`,
`send_agent_message`, `close_agent_chat`) now run on the new Sessions
primitive, so AI assistants driving a `chat.agent` get the same
idempotent-by-`chatId`, durable-across-runs behavior the browser
transport gets. Required PAT scopes go from `write:inputStreams` to
`read:sessions` + `write:sessions`.
([#3546](https://github.com/triggerdotdev/trigger.dev/pull/3546))

- MCP `list_runs` tool: add a `region` filter input and surface each
run's executing region in the formatted summary.
([#3612](https://github.com/triggerdotdev/trigger.dev/pull/3612))

-   Updated dependencies:
    -   `@trigger.dev/core@4.5.0-rc.0`
    -   `@trigger.dev/build@4.5.0-rc.0`
    -   `@trigger.dev/schema-to-json@4.5.0-rc.0`

## @trigger.dev/core@4.5.0-rc.0

### Patch Changes

- Add Agent Skills for `chat.agent`. Drop a folder with a `SKILL.md` and
any helper scripts/references next to your task code, register it with
`skills.define({ id, path })`, and the CLI bundles it into the deploy
image automatically — no `trigger.config.ts` changes. The agent gets a
one-line summary in its system prompt and discovers full instructions on
demand via `loadSkill`, with `bash` and `readFile` tools scoped
per-skill (path-traversal guards, output caps, abort-signal
propagation).
([#3543](https://github.com/triggerdotdev/trigger.dev/pull/3543))

    ```ts
const pdfSkill = skills.define({ id: "pdf-extract", path:
"./skills/pdf-extract" });

    chat.skills.set([await pdfSkill.local()]);
    ```

Built on the [AI SDK cookbook
pattern](https://ai-sdk.dev/cookbook/guides/agent-skills) — portable
across providers. SDK + CLI only for now; dashboard-editable `SKILL.md`
text is on the roadmap.

- Reject overlong `idempotencyKey` values at the API boundary so they no
longer trip an internal size limit on the underlying unique index and
surface as a generic 500. Inputs are capped at 2048 characters — well
above what `idempotencyKeys.create()` produces (a 64-character hash) and
above any realistic raw key. Applies to `tasks.trigger`,
`tasks.batchTrigger`, `batch.create` (Phase 1 streaming batches),
`wait.createToken`, `wait.forDuration`, and the input/session stream
waitpoint endpoints. Over-limit requests now return a structured 400
instead.
([#3560](https://github.com/triggerdotdev/trigger.dev/pull/3560))

- **AI Agents** — run AI SDK chat completions as durable Trigger.dev
agents instead of fragile API routes. Define an agent in one function,
point `useChat` at it from React, and the conversation survives page
refreshes, network blips, and process restarts.
([#3543](https://github.com/triggerdotdev/trigger.dev/pull/3543))

    ```ts
    import { chat } from "@trigger.dev/sdk/ai";
    import { streamText } from "ai";
    import { openai } from "@ai-sdk/openai";

    export const myChat = chat.agent({
      id: "my-chat",
      run: async ({ messages, signal }) =>
streamText({ model: openai("gpt-4o"), messages, abortSignal: signal }),
    });
    ```

    ```tsx
    import { useChat } from "@ai-sdk/react";
import { useTriggerChatTransport } from "@trigger.dev/sdk/chat/react";

const transport = useTriggerChatTransport({ task: "my-chat",
accessToken, startSession });
    const { messages, sendMessage } = useChat({ transport });
    ```

    **What you get:**

- **AI SDK `useChat` integration** — a custom
[`ChatTransport`](https://sdk.vercel.ai/docs/ai-sdk-ui/transport)
(`useTriggerChatTransport`) plugs straight into Vercel AI SDK's
`useChat` hook. Text streaming, tool calls, reasoning, and `data-*`
parts all work natively over Trigger.dev's realtime streams. No custom
API routes needed.
- **First-turn fast path (`chat.headStart`)** — opt-in handler that runs
the first turn's `streamText` step in your warm server process while the
agent run boots in parallel, cutting cold-start TTFC by roughly half
(measured 2801ms → 1218ms on `claude-sonnet-4-6`). The agent owns step
2+ (tool execution, persistence, hooks) so heavy deps stay where they
belong. Web Fetch handler works natively in Next.js, Hono, SvelteKit,
Remix, Workers, etc.; bridge to Express/Fastify/Koa via
`chat.toNodeListener`. New `@trigger.dev/sdk/chat-server` subpath.
- **Multi-turn durability via Sessions** — every chat is backed by a
durable Session that outlives any individual run. Conversations resume
across page refreshes, idle timeout, crashes, and deploys; `resume:
true` reconnects via `lastEventId` so clients only see new chunks.
`sessions.list` enumerates chats for inbox-style UIs.
- **Auto-accumulated history, delta-only wire** — the backend
accumulates the full conversation across turns; clients only ship the
new message each turn. Long chats never hit the 512 KiB body cap.
Register `hydrateMessages` to be the source of truth yourself.
- **Lifecycle hooks** — `onPreload`, `onChatStart`,
`onValidateMessages`, `hydrateMessages`, `onTurnStart`,
`onBeforeTurnComplete`, `onTurnComplete`, `onChatSuspend`,
`onChatResume` — for persistence, validation, and post-turn work.
- **Stop generation** — client-driven `transport.stopGeneration(chatId)`
aborts mid-stream; the run stays alive for the next message, partial
response is captured, and aborted parts (stuck `partial-call` tools,
in-progress reasoning) are auto-cleaned.
- **Tool approvals (HITL)** — tools with `needsApproval: true` pause
until the user approves or denies via `addToolApprovalResponse`. The
runtime reconciles the updated assistant message by ID and continues
`streamText`.
- **Steering and background injection** — `pendingMessages` injects user
messages between tool-call steps so users can steer the agent
mid-execution; `chat.inject()` + `chat.defer()` adds context from
background work (self-review, RAG, safety checks) between turns.
- **Actions** — non-turn frontend commands (undo, rollback, regenerate,
edit) sent via `transport.sendAction`. Fire `hydrateMessages` +
`onAction` only — no turn hooks, no `run()`. `onAction` can return a
`StreamTextResult` for a model response, or `void` for side-effect-only.
- **Typed state primitives** — `chat.local<T>` for per-run state
accessible from hooks, `run()`, tools, and subtasks (auto-serialized
through `ai.toolExecute`); `chat.store` for typed shared data between
agent and client; `chat.history` for reading and mutating the message
chain; `clientDataSchema` for typed `clientData` in every hook.
- **`chat.toStreamTextOptions()`** — one spread into `streamText` wires
up versioned system [Prompts](https://trigger.dev/docs/ai/prompts),
model resolution, telemetry metadata, compaction, steering, and
background injection.
- **Multi-tab coordination** — `multiTab: true` + `useMultiTabChat`
prevents duplicate sends and syncs state across browser tabs via
`BroadcastChannel`. Non-active tabs go read-only with live updates.
- **Network resilience** — built-in indefinite retry with bounded
backoff, reconnect on `online` / tab refocus / bfcache restore,
`Last-Event-ID` mid-stream resume. No app code needed.

See [/docs/ai-chat](https://trigger.dev/docs/ai-chat/overview) for the
full surface — quick start, three backend approaches (`chat.agent`,
`chat.createSession`, raw task), persistence and code-sandbox patterns,
type-level guides, and API reference.

- Stamp `gen_ai.conversation.id` (the chat id) on every span and metric
emitted from inside a `chat.task` or `chat.agent` run. Lets you filter
dashboard spans, runs, and metrics by the chat conversation that
produced them — independent of the run boundary, so multi-run chats
correlate cleanly. No code changes required on the user side.
([#3543](https://github.com/triggerdotdev/trigger.dev/pull/3543))

- Fix `LocalsKey<T>` type incompatibility across dual-package builds.
The phantom value-type brand no longer uses a module-level `unique
symbol`, so a single TypeScript compilation that resolves the type from
both the ESM and CJS outputs (which can happen under certain pnpm
hoisting layouts) no longer sees two structurally-incompatible variants
of the same type.
([#3626](https://github.com/triggerdotdev/trigger.dev/pull/3626))

- Unit-test `chat.agent` definitions offline with `mockChatAgent` from
`@trigger.dev/sdk/ai/test`. Drives a real agent's turn loop in-process —
no network, no task runtime — so you can send messages, actions, and
stop signals via driver methods, inspect captured output chunks, and
verify hooks fire. Pairs with `MockLanguageModelV3` from `ai/test` for
model mocking. `setupLocals` lets you pre-seed `locals` (DB clients,
service stubs) before `run()` starts.
([#3543](https://github.com/triggerdotdev/trigger.dev/pull/3543))

The broader `runInMockTaskContext` harness it's built on lives at
`@trigger.dev/core/v3/test` — useful for unit-testing any task code, not
just chat.

- Retry `TASK_PROCESS_SIGSEGV` task crashes under the user's retry
policy instead of failing the run on the first segfault. SIGSEGV in Node
tasks is frequently non-deterministic (native addon races, JIT/GC
interaction, near-OOM in native code, host issues), so retrying on a
fresh process often succeeds. The retry is gated by the task's existing
`retry` config + `maxAttempts` — same path `TASK_PROCESS_SIGTERM` and
uncaught exceptions already use — so tasks without a retry policy still
fail fast.
([#3552](https://github.com/triggerdotdev/trigger.dev/pull/3552))

- Add `region` to the runs list / retrieve API: filter runs by region
(`runs.list({ region: "..." })` / `filter[region]=<masterQueue>`) and
read each run's executing region from the new `region` field on the
response.
([#3612](https://github.com/triggerdotdev/trigger.dev/pull/3612))

- **Sessions** — a durable, run-aware stream channel keyed on a stable
`externalId`. A Session is the unit of state that owns a multi-run
conversation: messages flow through `.in`, responses through `.out`,
both survive run boundaries. Sessions back the new `chat.agent` runtime,
and you can build on them directly for any pattern that needs durable
bi-directional streaming across runs.
([#3542](https://github.com/triggerdotdev/trigger.dev/pull/3542))

    ```ts
    import { sessions, tasks } from "@trigger.dev/sdk";

    // Trigger a task and subscribe to its session output in one call
const { runId, stream } = await tasks.triggerAndSubscribe("my-task",
payload, {
      externalId: "user-456",
    });

    for await (const chunk of stream) {
      // ...
    }

// Enumerate existing sessions (powers inbox-style UIs without a
separate index)
for await (const s of sessions.list({ type: "chat.agent", tag:
"user:user-456" })) {
      console.log(s.id, s.externalId, s.createdAt, s.closedAt);
    }
    ```

See [/docs/ai-chat/overview](https://trigger.dev/docs/ai-chat/overview)
for the full surface — Sessions powers the durable, resumable chat
runtime described there.

## @trigger.dev/plugins@4.5.0-rc.0

### Patch Changes

- The public interfaces for a plugin system. Initially consolidated
authentication and authorization interfaces.
([#3499](https://github.com/triggerdotdev/trigger.dev/pull/3499))
-   Updated dependencies:
    -   `@trigger.dev/core@4.5.0-rc.0`

## @trigger.dev/python@4.5.0-rc.0

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/sdk@4.5.0-rc.0`
    -   `@trigger.dev/core@4.5.0-rc.0`
    -   `@trigger.dev/build@4.5.0-rc.0`

## @trigger.dev/react-hooks@4.5.0-rc.0

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.5.0-rc.0`

## @trigger.dev/redis-worker@4.5.0-rc.0

### Patch Changes

- Add MollifierBuffer and MollifierDrainer primitives for trigger burst
smoothing.
([#3614](https://github.com/triggerdotdev/trigger.dev/pull/3614))

MollifierBuffer (`accept`, `pop`, `ack`, `requeue`, `fail`,
`evaluateTrip`) is a per-env FIFO over Redis with atomic Lua transitions
for status tracking. `evaluateTrip` is a sliding-window trip evaluator
the webapp gate uses to detect per-env trigger bursts.

MollifierDrainer pops entries through a polling loop with a
user-supplied handler. The loop survives transient Redis errors via
capped exponential backoff (up to 5s), and per-env pop failures don't
poison the rest of the batch — one env's blip is logged and counted as
failed for that tick. Rotation is two-level: orgs at the top, envs
within each org. The buffer maintains `mollifier:orgs` and
`mollifier:org-envs:${orgId}` atomically with per-env queues, so the
drainer walks orgs → envs directly without an in-memory cache. The
`maxOrgsPerTick` option (default 500) caps how many orgs are scheduled
per tick; for each picked org, one env is popped (rotating round-robin
within the org). An org with N envs gets the same per-tick scheduling
slot as an org with 1 env, so tenant-level drainage throughput is
determined by org count rather than env count.

-   Updated dependencies:
    -   `@trigger.dev/core@4.5.0-rc.0`

## @trigger.dev/rsc@4.5.0-rc.0

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.5.0-rc.0`

## @trigger.dev/schema-to-json@4.5.0-rc.0

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.5.0-rc.0`

</details>

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-05-21 14:39:02 +01:00
Eric Allam 422f9f0bb2 ci: unblock changesets release PRs (#3687)
## Summary

Two CI workflows were blocking the v4.5.0-rc.0 release PR (#3563) and
would block every future changeset release PR.

### 1. `changesets-pr.yml` — self-report `All PR Checks`

The changesets bot pushes commits authored by `GITHUB_TOKEN`. By GitHub
design, `GITHUB_TOKEN`-authored pushes can't trigger downstream
workflows (loop-prevention). That means `pr_checks.yml` never fires on
release-PR commits, leaving the required `All PR Checks` status
permanently `Expected — Waiting for status to be reported`. The PR can't
merge.

The fix: after `changesets/action` creates the PR, post a `success`
check with the exact `All PR Checks` context onto the PR's head SHA.
GitHub's required-check evaluation is satisfied by any check with the
right context name — the source doesn't have to be `pr_checks.yml`.

**Why this is safe:** the release PR only mechanically bumps
`package.json`, `pnpm-lock.yaml`, and `CHANGELOG.md` from changesets
that were already on `main` (and already ran full CI when they merged).
If a human ever pushes a commit to `changeset-release/main`,
`pr_checks.yml` fires on that push (real user, not `GITHUB_TOKEN`) and
posts its own `All PR Checks` status — last write wins for the same
context on the same SHA, so the human-push result overrides the
auto-success.

### 2. `vouch-check-pr.yml` — exempt `github-actions[bot]`

The `require-draft` job auto-closes any non-draft PR whose author is not
a `MEMBER`/`OWNER`/`COLLABORATOR`, with an explicit allowlist for
`devin-ai-integration[bot]` and `dependabot[bot]`. The changesets bot
publishes as `github-actions[bot]` with `author_association:
CONTRIBUTOR`, so every release PR was getting auto-closed on open with a
"please re-open as draft" comment. Add `github-actions[bot]` to the
exemption list.

## Test plan
- [ ] After merge, the next changeset bot push to
`changeset-release/main` should post `All PR Checks: success` on the
release PR's head SHA, and the PR should not get auto-closed by `Vouch -
Check PR`.
- [ ] Confirm `pr_checks.yml` still fires + gates normal
(human-authored) PRs to `main`.
2026-05-21 14:22:52 +01:00
Eric Allam 89d085a496 fix(references): repair ai-chat typecheck against current wire shape (#3685)
## Summary

Pre-existing typecheck errors in `references/ai-chat` against the
current SDK shape. Unblocks `pnpm exec tsc --noEmit` in the reference
project.

## What changed

Three categories of fixes inside `references/ai-chat`. No SDK changes.

### 1. `payload.messages` → `payload.message`

The wire payload is now delta-only — one new message per trigger,
optional. Old code in two raw-task files reads `payload.messages`
(plural array) which no longer exists.

```ts
// before
const messages = await conversation.addIncoming(currentPayload.messages, ...);

// after
const messages = await conversation.addIncoming(
  currentPayload.message ? [currentPayload.message] : [],
  ...
);
```

Same fix to the `chat.messages.on` handler, reading `msg.message`
(singular) instead of `msg.messages[length - 1]`.

### 2. `clientData` non-null assertion in `cf-trust-test`

`ChatTurnContext.clientData` is typed as `?: TClientData` on
`onTurnStart` / `run` event objects even when the agent declares a
`clientDataSchema`. The runtime validates against the schema before the
hook fires, so it's structurally non-null — but TypeScript can't know
that. Non-null assert for now.

Follow-up worth filing: narrow `ChatTurnContext.clientData` to
non-optional when the agent has a `clientDataSchema`. Same friction the
docs friction-test subagent flagged.

### 3. `stress-emit.parseConfig` retyped against `ModelMessage[]`

The `run` callback hands `messages: ModelMessage[]`, not `UIMessage[]`.
Update `parseConfig` to accept `ModelMessage[]` and pull text from
`content` (string or array-of-parts).

## Test plan

- [x] `pnpm exec tsc --noEmit` in `references/ai-chat` passes (was 8
errors, now 0)
2026-05-21 12:41:18 +01:00
Eric Allam 9ff410bfa4 feat(sdk): type chat.createStartSessionAction against your chat agent (#3684)
## Summary

Type `chat.createStartSessionAction` against the chat agent so
`clientData` is typed end-to-end on the first turn. Closes the gap where
`useTriggerChatTransport`'s `startSession` callback already hands you a
typed `clientData` (via the transport generic) but the server-side
action couldn't accept it without untyped routing through the `metadata`
field.

## Design

`ChatStartSessionParams` gains a typed `clientData` field via the new
generic:

```ts
export type ChatStartSessionParams<TChat extends AnyTask = AnyTask> = {
  chatId: string;
  clientData?: InferChatClientData<TChat>;
  triggerConfig?: Partial<SessionTriggerConfig>;
  metadata?: Record<string, unknown>;
};

function createChatStartSessionAction<TChat extends AnyTask = AnyTask>(
  taskId: string,
  options?: CreateChatStartSessionActionOptions
): (params: ChatStartSessionParams<TChat>) => Promise<ChatStartSessionResult>
```

When provided, `clientData` is folded into the first run's
`triggerConfig.basePayload.metadata`, so `onPreload` / `onChatStart` see
the same shape per-turn `metadata` carries via the transport. The opaque
session-level `metadata` field stays exactly as before — it lands on the
Session row, not the run payload.

## Usage

```ts
// actions.ts
import { chat } from "@trigger.dev/sdk/ai";
import type { myChat } from "@/trigger/chat";

export const startChatSession = chat.createStartSessionAction<typeof myChat>("my-chat");
```

```tsx
// Chat.tsx
const transport = useTriggerChatTransport<typeof myChat>({
  task: "my-chat",
  accessToken: ({ chatId }) => mintChatAccessToken(chatId),
  startSession: ({ chatId, clientData }) =>
    startChatSession({ chatId, clientData }),
});
```

## Test plan

- [x] `pnpm run build --filter @trigger.dev/sdk` passes
- [ ] Verify a `chat.agent` with `clientDataSchema` reads the typed
clientData from `onPreload` payload metadata on the first turn
2026-05-21 11:51:35 +01:00
Eric Allam d343727021 fix(webapp,sdk): keep chat.agent snapshots on one object store (#3679)
(`OBJECT_STORE_BASE_URL`) and a named protocol provider
(`OBJECT_STORE_DEFAULT_PROTOCOL=s3`), chat.agent session snapshot writes
landed in the named provider but reads fell through to the default — so
the recovery boot couldn't find the snapshot it had just written.

After a mid-stream cancel, the missing snapshot triggered a fallback
replay path that dropped the user's follow-up message, leaving the chat
stuck in `submitted` indefinitely.

Fix:
- New `/api/v1/sessions/:id/snapshot-url` route handles PUT + GET
  symmetrically — both prefix unprefixed keys with
  `OBJECT_STORE_DEFAULT_PROTOCOL` so they always round-trip through the
  same store.
- `Session.chatSnapshotStoragePath` persists the resolved URI on first
  write so future protocol changes don't strand existing snapshots.
  Reads prefer the stored URI and fall back to the computed default for
  pre-column sessions.
- SDK calls `createChatSnapshotUploadUrl` / `getChatSnapshotUrl`; the
  generic v1/v2 packets endpoints are unchanged.

## Test plan
- [x] Configure local with two providers (R2 default + MinIO `s3` named)
      and `OBJECT_STORE_DEFAULT_PROTOCOL=s3`.
- [x] Reproduce hang: send a message, cancel mid-stream, send another —
      without the fix it hangs in `submitted`; with the fix it streams.
- [x] Snapshot lands in the `s3`-protocol bucket and
      `Session.chatSnapshotStoragePath` is set after first write.
- [x] SDK unit tests pass; webapp typecheck passes.
2026-05-20 20:22:59 +01:00
Eric Allam aec7e0a93d perf(webapp): index EnvironmentVariableValue.environmentId (#3675)
Env-var lookups via `GET
/api/v1/projects/:projectRef/envvars/:slug/:name` run a Prisma
`findMany` on `EnvironmentVariableValue` filtered by `environmentId` +
`isSecret`. The only existing indexes are the primary key and a unique
on `(variableId, environmentId)`, so `environmentId` is never the
leading column — the planner falls back to a Parallel Seq Scan over the
whole table to find what is, in practice, a handful of rows per
environment.

Two changes:

- Add a btree index on `EnvironmentVariableValue(environmentId)` so the
planner switches to an index scan. The composite `(variableId,
environmentId)` unique stays in place; the new index is purely additive.
- Route the `findMany` inside `getEnvironmentWithRedactedSecrets`
through the read replica via a new `replicaClient` constructor param on
the repository (defaulting to `$replica`, mirroring how `prismaClient`
defaults to `prisma`). Writes and read-after-write methods stay on the
primary.

## Test plan

- [ ] `pnpm run typecheck --filter webapp`
- [ ] Confirm `EXPLAIN` plan flips from Parallel Seq Scan to an index
scan
- [ ] Existing env-var route tests still pass
2026-05-20 13:43:44 +01:00
Daniel Sutton 6b46a34c46 fix(webapp): return 404 instead of 500 for missing env/project/schedule loaders (#3663)
## Summary

- Dashboard loaders for runs / sessions / batches / schedule-detail
threw bare `Error("X not found")` when a slug didn't resolve. Remix
surfaces this as a 500 and Sentry captures it via auto-instrumentation,
producing ongoing noise from real users following stale preview-branch
or deleted-resource links (the URLs in those Sentry events all carry
`?_data=routes/...`, i.e. client-side revalidation, not full-page
navigation).
- Added a `throwNotFound(statusText)` helper in
`app/utils/httpErrors.ts` that throws a Response with status 404,
matching the established pattern in sibling routes (agents, alerts,
bulk-actions, etc.).
- Migrated 5 loader sites to `throwNotFound` (4× "Environment not
found", 1× "Schedule not found").
- Migrated 1 loader site (`runs._index` project branch) to
`redirectWithErrorMessage("/", request, "Project not found")` to match
the pre-existing convention used by every other dashboard route's
project-not-found branch.
- Intentionally **not** touched: bare `throw new Error("X not found")`
inside `resources.*` action routes (sit inside try/catch blocks that
already redirect with a flash message), the invariant assertion in
`vercel.connect.tsx`, and the admin config check in
`admin.api.v1.runs-replication.backfill.ts`.

## Where the fix is visible

Normal browser navigation to these URLs doesn't reach the buggy loaders
— the parent env-layout
(`_app.orgs.$organizationSlug.projects.$projectParam.env.$envParam/route.tsx`)
already filters missing envs/projects and redirects/404s before the
child loader runs. The bug fires exclusively when Remix calls a single
child loader via `?_data=routes/...`, which happens during client-side
navigation or `useRevalidator`. That matches every Sentry event URL.

## Test plan

- [x] Unit test for the new helper —
`apps/webapp/test/httpErrors.test.ts`
- [x] `pnpm run typecheck --filter webapp` clean
- [x] Manual verification via Playwright on `main` vs this branch (6
cases): main returns 500 for each defective `_data` URL; branch returns
404 or 204 + `X-Remix-Redirect` as designed
- [x] Verified user-visible 404 catch boundary on `schedules/<missing>`
(the one case reachable via normal nav)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-20 09:46:44 +01:00
Eric Allam f91b96efb7 feat(sdk,core): preserve chat.agent context after cancel / OOM / crash (#3671) 2026-05-20 07:27:05 +01:00
Eric Allam 12d2125148 fix(sdk,core,build): SDK hardening pass (#3670)
## Summary

Five hardening fixes across `@trigger.dev/sdk`, `@trigger.dev/core`, and
`@trigger.dev/build`.

- `tasks.triggerAndSubscribe` now forwards caller `requestOptions`
(custom API keys, per-request overrides) to the underlying
`apiClient.triggerTask` call instead of silently dropping them.
- `SSEStreamSubscription` no longer retries permanent client errors
forever. The default `nonRetryableStatuses` widens from `[404, 410]` to
`[400, 404, 409, 410, 422]`, so a malformed session-stream request fails
fast instead of busy-looping under bounded backoff.
- Session writer falls back to manually wiring the caller's
`AbortSignal` on Node 18, where `AbortSignal.any` is unavailable.
Caller-driven cancellation now propagates on every supported runtime.
- `TriggerChatTransport` throws immediately when a `chat.handover`
response is missing `X-Trigger-Chat-Access-Token`, instead of silently
downgrading every subsequent turn back to the handover path. `dispose()`
aborts every active `session.out` subscription before tearing the
coordinator down, so unmount/navigation no longer leaves SSE readers in
flight.
- Removed the experimental `@trigger.dev/build/extensions/secureExec`
build extension. It will return alongside the sandbox feature it was
built to support.

## Test plan

- [ ] `pnpm run build --filter @trigger.dev/sdk --filter
@trigger.dev/core --filter @trigger.dev/build`
- [ ] `pnpm --filter @trigger.dev/sdk test --run` (183 tests, including
chat / chat-server / sessions / handover)
- [ ] `pnpm --filter @trigger.dev/core test --run`
- [ ] Manually trigger a `chat.handover` whose response strips
`X-Trigger-Chat-Access-Token`, and confirm the transport throws
synchronously rather than degrading.
- [ ] Unmount a chat UI mid-stream and confirm the active `session.out`
SSE connection closes immediately.
2026-05-19 16:10:32 +01:00
Iss 204a766bb1 feat(webapp): expose is_warm_start in TRQL runs schema (#3667)
Add is_warm_start to TRQL runs schema so warm vs cold start data is
queryable
2026-05-19 09:05:31 -04:00
Eric Allam 436b7a9ea1 fix(webapp): fold S2 token scope into access-token cache key (#3668)
## Summary

The S2 access-token cache key was `${basin}:${streamPrefix}` — purely
server-derived but blind to the **scope/ops list** hardcoded one method
away. When the ops list changes in code (e.g. #3644 added `trim` so
`chat.agent`'s per-turn trim chain can issue `AppendRecord.trim()`),
pre-deploy tokens still in cache get returned to SDK callers for up to
the token's TTL (24h default), surfacing as `Operation not permitted`
403s on any op outside the old scope.

## Fix

Lift the ops list to a module constant and fold its sorted-join
fingerprint into the cache key:

```ts
const S2_TOKEN_OPS = ["append", "create-stream", "trim"] as const;
const S2_TOKEN_OPS_FINGERPRINT = [...S2_TOKEN_OPS].sort().join(",");

// in getS2AccessToken
const cacheKey = `${this.basin}:${this.streamPrefix}:${S2_TOKEN_OPS_FINGERPRINT}`;

// in s2IssueAccessToken
scope: { /* ... */ ops: [...S2_TOKEN_OPS], /* ... */ }
```

The fingerprint is derived from the single source of truth, so any
future scope change auto-invalidates without anyone remembering to bump
a literal version. The Unkey L1 (in-memory LRU) and L2 (Redis) layers
share the same key derivation, so both reset together on the next deploy
with no manual cache busting.

## Test plan

- [ ] `pnpm run typecheck --filter webapp`
- [ ] Run a multi-turn `chat.agent` chat via `references/ai-chat` and
confirm no `chat.agent: trim failed; will retry next turn` warn span
fires across turn-completes.
2026-05-19 13:33:03 +01:00
Oskar Otwinowski 2fbac48e0d feat(webapp): prompt to clear TRIGGER_VERSION on disabling Vercel atomic deployments (#3666)
- Ask user if they want to remove TRIGGER_VERSION when they disable
atomic deployments, and explain what is the situation if they leave it
as it is
- Install TRIGGER_SECRET keys as sensitive values in Vercel
<img width="1136" height="714" alt="image"
src="https://github.com/user-attachments/assets/a7351da1-5b2a-44e5-acdd-d30c9359f3ed"
/>
<img width="1136" height="714" alt="image"
src="https://github.com/user-attachments/assets/e773ede2-74cb-438e-811c-338f678d2f7d"
/>
<img width="1136" height="714" alt="image"
src="https://github.com/user-attachments/assets/c7b235a8-e06d-48d3-ac28-c5c9aacc6069"
/>
2026-05-19 13:53:51 +02:00
Eric Allam e825409f0e ci(release): exit changeset pre mode before snapshot prerelease (#3665)
## Summary

The prerelease (snapshot) path of the release workflow fails immediately
whenever `main` carries an active `.changeset/pre.json` (i.e. during an
in-progress RC cycle, like the current v4 RC):

```
🦋 error Snapshot release is not allowed in pre mode
🦋 To resolve this exit the pre mode by running `changeset pre exit`
```

This blocks `chat-prerelease` snapshots from main even though the
snapshots are unrelated to the RC cycle.

Adds a conditional `changeset pre exit` step right before `Snapshot
version` in the prerelease job. The job runs on a checkout with
`persist-credentials: false`, so the `pre.json` deletion stays on the
runner's working tree — main's persisted pre-mode state is untouched,
and v4 RC publishes keep working normally.

## Test plan

- [ ] Re-run the `🦋 Changesets Release` workflow with `type=prerelease`,
`ref=main`, `prerelease_tag=chat-prerelease` and confirm it gets past
the snapshot step and publishes.
- [ ] Confirm `.changeset/pre.json` on `main` is unchanged after the
run.
2026-05-19 10:11:44 +01:00
Daniel Sutton 2f261e5e69 fix(webapp): catch loader/action throws before Remix serializes them (#3664)
## Summary

Companion to #3536, which patched routes that already had a leaking
`catch (e) { return json({error: e.message}, 500) }`. That pattern can't
reach routes which have no catch in the first place — when those throw,
Remix's default error path serializes `error.message` into the response
body, and the SDK then wraps the leaked string as `TriggerApiError`.

Across 28 raw api.v1 loaders/actions plus one dashboard polling
endpoint, each handler now:

- Wraps its body in `try { ... } catch (error) { ... }`.
- Re-throws `Response` instances so auth helpers' `throw json(...)` /
`throw redirect(...)` pass through unchanged.
- Logs non-Response errors via `logger.error` so server-side visibility
is preserved.
- Returns a generic body — `{"error": "Internal Server Error"}` 500 for
raw API routes, or `{ changelogs: [] }` 200 for the polling widget
(degrade silently across transient blips; the consumer hook already
coped with empty payloads).

For six routes where #3536 left an inner try/catch covering only a
service call (`alertChannels`, `batches.results`,
`deployments.finalize`, `deployments.background-workers`,
`deployments.promote`, `projects.background-workers`): an outer
try/catch is added so auth/parsing failures are also sanitized. Inner
typed-error handling (`ServiceValidationError` → 422 with message, etc.)
is preserved exactly.

For two routes whose existing catch returned 400 + `error.message`
(`api.v1.authorization-code`, `api.v1.orgs.\$orgParam.projects` action):
the body is sanitized to a generic per-route string. **Status code stays
400** — clients that key on the 4xx/5xx distinction (and the SDK's
no-retry-on-4xx behavior) are unaffected.

## Test plan

- [x] \`pnpm run typecheck --filter webapp\`
- [x] Per-route synthetic-throw probe: inject \`throw new
Error("SYNTHETIC ...")\` at the top of each catch'd try, curl the route
with a dummy bearer, confirm the response body is the generic shape and
that the synthetic message lands server-side via \`logger.error\`. 29
routes verified.
- [x] Real-P1001 probe on the envvars loader: \`docker stop database\`
mid-flight, confirm response is generic 500 (not the leaked Prisma
message).
- [x] Sampled legitimate 4xx/2xx paths across each pattern variant
(naked-wrap, partial-expanded, 400-preserved) to confirm the wraps don't
interfere with normal control flow.
2026-05-19 09:32:35 +01:00
nicktrn 5dacab0c72 fix: validate email format on magic link login (#3660)
Reject non-email strings at the magic link form instead of accepting any
string and proceeding through rate-limit / authenticator steps.
2026-05-18 16:18:58 +00:00
Oskar Otwinowski 02d61afc1c fix(webapp): sanitize OTel attributes on ClickHouse JSON parse rejection (#3659)
Before fix:
<img width="1264" height="987" alt="image"
src="https://github.com/user-attachments/assets/24b8b85c-b89f-4109-9004-8d6af61d2849"
/>
After fix:
<img width="1264" height="987" alt="image"
src="https://github.com/user-attachments/assets/89bbc587-c50a-45ab-b203-dbe91028e918"
/>
2026-05-18 17:42:31 +02:00
Eric Allam 427d9e078a feat(sdk): functional baseURL and fetch override on chat transports (#3655)
## Summary

`TriggerChatTransport`, `AgentChat`, and `chat.createStartSessionAction`
now accept a string-or-function `baseURL` so callers can route per
endpoint — e.g. `.in/append` through a trusted edge proxy while keeping
`.out` SSE direct. The same surfaces add a `fetch` override for header
injection, custom retries, or proxy rewrites that go beyond URL routing.
SSE GETs are covered too via a new `fetchClient` option on
`SSEStreamSubscription`.

```ts
// TriggerChatTransport / AgentChat — endpoints: "in" | "out"
baseURL: ({ endpoint }) =>
  endpoint === "out" ? DIRECT : PROXY,

fetch: (url, init, ctx) => {
  init.headers = new Headers(init.headers);
  init.headers.set("traceparent", currentTraceparent());
  return globalThis.fetch(url, init);
},

// chat.createStartSessionAction — endpoints: "sessions" | "auth"
chat.createStartSessionAction("my-agent", {
  baseURL: ({ endpoint }) => (endpoint === "sessions" ? PROXY : DIRECT),
});
```

`streamBaseURL` on `TriggerChatTransport` is kept as a backwards-compat
alias and continues to win for the `"out"` endpoint when set.
Plain-string `baseURL` still applies to every endpoint, matching prior
behavior.
2026-05-18 16:42:18 +01:00
Eric Allam 8b98e21b4b fix(sdk,core): cache realtime-stream credentials per slot with refresh on writer failure (#3658)
## Summary

Hot-loop writers — `streams.writer` / `streams.pipe` on the run-scoped
side, `chat.response.write` / `chat.stream.*` on the session side — were
issuing a fresh `PUT` to mint S2 credentials for every chunk. On run
streams, each PUT also pushed the streamId onto
`TaskRun.realtimeStreams`,
so a chat-agent turn writing N chunks produced N PUTs and N duplicate
array pushes against the same row.

The SDK now caches the initialize response per cache slot: `(runId,
key)`
for run streams, the session id for session streams. First call PUTs as
before; subsequent calls reuse the cached promise. Hot-loop writers do
one PUT per slot for the lifetime of the cache.

S2 access tokens have a 1-day TTL. If a writer's `wait()` rejects (auth
error, expired token, network blip), the cache evicts the matching slot
so the next call re-PUTs and mints fresh credentials, identity-checked
so a concurrent caller's fresh promise isn't accidentally cleared.

## chat.agent guardrail

`streams.pipe / writer / append / read` called inside a `chat.agent` run
now logs a one-time warning pointing at `chat.response.write` /
`chat.stream.*` — `streams.*` is run-scoped and isn't visible on the
chat session. The ai-chat docs are updated to drop the old guidance
toward run-scoped streams.
2026-05-18 16:41:58 +01:00
Daniel Sutton 906d5fafb6 feat(mollifier): trigger burst smoothing — Phase 1 (monitoring) (#3614)
## Summary
- Introduce the Mollifier: a Redis-backed buffer for `trigger()` API
calls during traffic spikes, with a per-env trip evaluator and a drainer
ack-loop.
- Phase 1 is dual-write monitoring — every mollified trigger is buffered
to Redis AND continues to `engine.trigger`. No customer-facing behaviour
change.
- Telemetry events: `mollifier.would_mollify`, `mollifier.buffered`,
`mollifier.drained`, plus the `mollifier.decisions` counter.
  - Gated behind a feature flag (default off).

  ## Test plan
  - [x] `pnpm run test --filter @trigger.dev/redis-worker`
  - [x] `pnpm run test --filter webapp -- mollifier`
  - [x] Manual: with flag off, no behaviour change vs main
- [x] Manual: with flag on + threshold lowered, observe
`mollifier.buffered` + `mollifier.drained` log pairs with matching
`runId`

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-18 14:58:38 +01:00
Eric Allam 6c9f1f197e chore: parameterize docker host ports and wire s2-lite by default (#3642)
## Summary

Two papercuts new contributors hit running this repo locally:

1. Fresh clones default to v1 (Redis-only) realtime streams, so Sessions
and `chat.agent` error with `"S2 configuration is missing"`, even though
the `s2` service is already in `docker/docker-compose.yml` and pre-seeds
a `trigger-local` basin. Wire `REALTIME_STREAMS_S2_*` to it in
`.env.example` so the new-contributor flow just works. (Also drop the s2
healthcheck: the image is distroless, so the `wget` check always reports
unhealthy.)

2. Two clones can't both run `pnpm run docker` because ports, project
name, and container names are all hardcoded. Parameterize every host
port as `${VAR:-default}`, drive the project name via
`COMPOSE_PROJECT_NAME` (with a top-level `name:` field as the default),
prefix container names with `${CONTAINER_PREFIX:-}`, and pass
`--env-file .env` so compose reads the same root `.env` the webapp does.
The "Running multiple instances side by side" block in `.env.example`
lists every overridable knob.

Also split the optional services (`electric-shard-1`, `ch-ui`,
`toxiproxy`, `nginx-h2`, `otel-collector`, `prometheus`, `grafana`) into
`docker-compose.extras.yml` behind a new `pnpm run docker:full` script.
The core stack keeps everything the webapp actually needs to boot:
postgres, redis, electric, minio, clickhouse + migrator, s2-lite.

Defaults match every previous hardcoded value, so existing setups keep
working without touching `.env`.

## Test plan

- [x] `pnpm run docker` on a clean clone brings up the core services on
the standard ports under the `triggerdotdev-docker` project name.
- [x] Setting `COMPOSE_PROJECT_NAME=triggerdotdev-docker-alt` + the
`*_HOST_PORT` overrides in `.env` brings up a second stack alongside the
default one with no port or container-name clashes.
- [x] Webapp boots cleanly against the default `.env.example` values;
`/healthcheck` returns 200, no S2 errors.
- [x] s2-lite basin `trigger-local` accepts an append + read via the
same REST endpoints the webapp uses.
- [x] `pnpm run docker:full` brings up the optional services alongside
the core ones in the same project.
2026-05-18 09:28:58 +00:00
Eric Allam 82853debea feat(webapp,core,sdk,cli): bound session.out via per-turn trim (#3644)
## Summary

Long-running chat agents were filling `session.out` forever — every
`chat.agent` turn appended to the same S2 stream with no trim, and the
Sessions dashboard re-streamed the entire history from `seq_num=0` on
every page load. After this change the agent appends an S2 `trim`
command record after each `trigger:turn-complete`, pointing back at the
previous turn-complete's seq_num. `session.out` stays roughly one turn
long at steady state, regardless of session age.

`trigger:turn-complete` and `trigger:upgrade-required` move from
`chunk.type`-shaped data records into header-form control records under
a uniform `trigger-control` namespace. Built-in transports
(`TriggerChatTransport`, `AgentChat`, the dashboard's `AgentView`)
handle the new shape transparently. Custom transports need a one-line
filter on the `trigger-control` header — see the rewritten "Records on
session.out" section in the client-protocol docs.

The Sessions detail page in the dashboard fetches the agent's per-turn
S3 snapshot via a presigned URL and seeds the transcript view, then
SSE-tails from the snapshot's `lastOutEventId`. Bandwidth and
time-to-first-render scale with unread turns instead of session
lifetime.

Resume contract is now explicit: single-turn-boundary resume always
works (the prior turn-complete is still on the stream), the S2 trim is
eventually consistent over 10-60s, and multi-turn-away resume falls back
to a snapshot reload.
2026-05-18 10:21:56 +01:00
Eric Allam f88d4018cc fix(webapp): dedupe realtimeStreams array push on stream create (#3653)
## Summary

The PUT handler at `/realtime/v1/streams/:runId/:target/:streamId` ran
`taskRun.update({ realtimeStreams: { push: streamId } })` on every call,
even when the `streamId` was already present. SDK call patterns that
re-initialize the same stream key on every chunk produce a per-write row
UPDATE, duplicate entries pile up in the array, and the row-lock + TOAST
rewrite cost grows unbounded on long-running stream sessions.

## Fix

Mirror the sibling append handler: read the array first and only push
when the `streamId` isn't already present. Identical behavior for
first-time stream creation; repeat creates short-circuit to a single
indexed read. The dashboard's per-run stream listing keeps working
because the first create still records the entry.

## Test plan

- [ ] A fresh PUT for a new `(run, streamId)` adds the entry to the
array
- [ ] A repeat PUT for the same pair leaves the array unchanged
- [ ] 404 is returned when the run doesn't exist; 400 when the run is
completed
2026-05-18 10:19:20 +01:00
Daniel Sutton 9623e88b05 fix(webapp): collapse Prisma P1001 errors into a single Sentry issue (#3632)
## Summary

- Adds a `beforeSend` rule in `apps/webapp/sentry.server.ts` that
collapses Prisma `P1001` ("Can't reach database server") errors into a
single Sentry issue regardless of which call site threw, by setting
`event.fingerprint = ["prisma-p1001-db-unreachable"]` and tagging
`db_unreachable:true`.
- Matches both `err.code === "P1001"` (Prisma's `KnownRequestError` when
a connection drops mid-query) and `err.errorCode === "P1001"`
(`InitializationError` when the client fails to connect at startup).
- Implemented as a small extensible `FINGERPRINT_RULES` table so further
fan-out errors can be added with one entry.

## Verification

End-to-end verified locally with `debug: true` on the SDK:
- Real Prisma `P1001` thrown from a loader (DB stopped mid-request) is
captured by Sentry's Remix auto-instrumentation
- `beforeSend` fires with `originalException.code === "P1001"`, rule
matches
- `event.fingerprint = ["prisma-p1001-db-unreachable"]` and
`tags.db_unreachable = "true"` applied
- Event lands in Sentry under the new fingerprint

## Test plan

- [ ] Deploy to staging; confirm P1001 events appear under a single
`prisma-p1001-db-unreachable` issue rather than fanning out
- [ ] Confirm `db_unreachable:true` tag is filterable in Sentry
- [ ] Verify non-P1001 errors are unaffected (event passes through
`beforeSend` untouched)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-18 09:37:35 +01:00
Eric Allam 55fa2d4967 fix(cli): TRIGGER_BUILD_SKIP_REWRITE_TIMESTAMP escape hatch for local self-hosted builds (#3618)
## Summary

Local self-hosted deploys (`trigger deploy --local-build --push
--builder orbstack` or any other buildx setup using the **docker**
driver) fail at the push step with:

```
ERROR: failed to build: failed to solve:
  exporter option "rewrite-timestamp" conflicts with "unpack"
```

The docker driver auto-enables `unpack=true` when pushing, and that's
incompatible with `rewrite-timestamp` (which the CLI sets for
reproducible-build hashing).

Adds a simple env-var opt-out so contributors can keep using their
default builder. The flag is only read by the local-build code path;
remote/cloud builds are unaffected.

```bash
TRIGGER_BUILD_SKIP_REWRITE_TIMESTAMP=1 \
  pnpm exec trigger deploy --profile default --local-build --push --builder orbstack
```

The trade-off: skipping `rewrite-timestamp` means layer timestamps
reflect actual build time, so two identical builds produce different
layer hashes. Fine for a local-dev registry; the only real consumer of
timestamp-stability is registry-layer cache hit rates.

## Test plan

- [x] Manual: ran `trigger deploy --profile default --local-build --push
--builder orbstack` against the localhost webapp + a local Docker
registry on port 5001 — first failed with the rewrite-timestamp/unpack
error, then succeeded after setting
`TRIGGER_BUILD_SKIP_REWRITE_TIMESTAMP=1`.
- [x] Full chat.agent smoke sweep (15 tests, including suspend/resume,
deepResearch subtask, AgentChat orchestrator) against the deployed image
— all pass.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
2026-05-17 09:28:57 +01:00
Eric Allam 627e059298 feat(trigger-sdk): add streamBaseURL to TriggerChatTransport (#3641)
`TriggerChatTransport` had a single `baseURL` option covering both the
`.in/append` POSTs and the long-lived `.out` SSE subscription. Customers
wanting to route the SSE through a proxy (e.g. a Cloudflare worker
capturing JA4 fingerprints for bot detection) had to send every append
through the proxy too, adding a hop to every user message.

New optional `streamBaseURL` overrides the SSE base URL only; appends
keep using `baseURL`. Falls back to `baseURL` when unset, so existing
transports are unchanged.

```ts
const transport = new TriggerChatTransport({
  task: "ai-chat",
  baseURL: "https://api.trigger.dev",
  streamBaseURL: "https://chat-proxy.example.com",
  accessToken,
  startSession,
});
```

Verified with a new test in `chat.test.ts` that asserts `.in/append`
routes through `baseURL` and `.out` SSE routes through `streamBaseURL`.
All existing tests still pass.
2026-05-17 07:55:12 +03:00
nicktrn a8280f125c ci: fix code path filter negation (#3637)
`dorny/paths-filter` defaults to OR semantics across the pattern array,
so the leading `**` matched every file and the `!...` excludes were
no-ops. The `code` filter has been returning `true` for every PR since
#3615.

Split into two filter steps: `code` moves into its own step with
`predicate-quantifier: every` so excludes actually subtract. The two
re-include workflow files become a separate `typecheck_self` filter that
the `typecheck` job ORs into its `if:`.

Side effect: workflow-file-only PRs that don't touch `pr_checks.yml` or
`typecheck.yml` no longer trigger typecheck. Previously they did because
the filter was broken-true.
2026-05-16 12:51:35 +03:00
Eric Allam 05d3ab1059 docs(clickhouse): require max+1 numbering and idempotent DDL (#3633)
## Summary

Codify two rules for ClickHouse migration authors that came out of the
029/030 ordering incident on the TRI-9367 test cloud deploy:

1. **Number files to `max(existing) + 1`, never slot in below the
latest.** Goose runs in strict mode in the cloud deploy pipeline and
refuses to apply a missing version below the current version — slotting
a file in below an already-applied number blocks the next deploy.
2. **DDL must be idempotent** (`ADD COLUMN IF NOT EXISTS`, `DROP COLUMN
IF EXISTS`, `CREATE TABLE IF NOT EXISTS`, etc.) so a retry or
out-of-order apply (`goose up --allow-missing` for local recovery,
manual fixups) is a no-op rather than an error.

## Where the rules live

- `internal-packages/clickhouse/CLAUDE.md` — full rules + example for
migration authors (and AI agents writing migrations).
- `.claude/REVIEW.md` — added a 🔴 finding under "What makes a 🔴
Important finding" so PR reviewers flag either fault as blocking.

The existing migration files are left untouched; the idempotency
requirement applies going forward.

## Test plan

- [ ] Next ClickHouse migration PR uses `IF NOT EXISTS` / `IF EXISTS`
forms
- [ ] No new migration files numbered below an already-applied version
on test/prod
2026-05-15 17:25:29 +00:00
Eric Allam 032b5a117a fix(clickhouse): renumber task_kind migration 029 → 031 (#3631)
## Summary

Renumber `029_add_task_kind_to_task_runs_v2.sql` →
`031_add_task_kind_to_task_runs_v2.sql` to fix a deploy-blocking
out-of-order migration, and make the DDL idempotent with `ADD COLUMN IF
NOT EXISTS` / `DROP COLUMN IF EXISTS`.

## Root cause

- Migration `030_create_sessions_v1.sql` landed on main on 2026-04-28
(PR #3417) and was applied to test cloud ClickHouse on a subsequent
deploy. Current goose version on test ClickHouse: **30**.
- Migration `029_add_task_kind_to_task_runs_v2.sql` was authored later
on 2026-05-10 as part of the Sessions primitive PR series (`be1a6cf8`).
- The next test cloud deploy failed because goose strict-mode refused to
apply a missing version *before* the current version:

```
goose run: error: found 1 missing migrations before current version 30:
  version 29: 029_add_task_kind_to_task_runs_v2.sql
```

## Fix

1. **Rename to `031_*`** (next available number after 030). Goose now
treats it as a new migration after 030 and applies it cleanly on
test/prod where the column does not yet exist.
2. **Make the DDL idempotent** (`ADD COLUMN IF NOT EXISTS`). The
original 029 may have been applied in environments that ran goose with
`--allow-missing` (e.g. some local dev databases) — those would have the
column already, and the rename causes goose to see 031 as new and
re-attempt the ADD. Idempotent DDL keeps that path safe. The `Down`
mirrors with `DROP COLUMN IF EXISTS`.

## Test plan

- [ ] Test cloud deploy (after this lands) successfully runs the
ClickHouse migration step
- [ ] `task_kind` column shows up on `trigger_dev.task_runs_v2`
post-migration
- [ ] Local environments that had previously applied 029 do not error on
the next `goose up`
2026-05-15 17:17:17 +00:00
Eric Allam 5788573b4f chore: enter prerelease mode (rc) to ship v4.5.0-rc.0 (#3630)
## Summary

Adds `.changeset/pre.json` to put the repo into changesets pre mode with
dist-tag `rc`. After this merges, the changesets bot regenerates the
existing release PR as `chore: release v4.5.0-rc.0`. Merging that PR
publishes the first release candidate of 4.5.0 to npm under `@rc`.

The pre-mode plumbing landed in #3628. The release content (chat.agent +
sessions + ai prompts + dashboard server-changes) landed in #3629.

## What ships when the bot PR merges

Under dist-tag `rc`:
-
`@trigger.dev/{sdk,core,build,react-hooks,redis-worker,plugins,python,rsc,schema-to-json}@4.5.0-rc.0`
- `trigger.dev@4.5.0-rc.0`

Plus:
- Docker image `ghcr.io/triggerdotdev/trigger.dev:v4.5.0-rc.0`
(immutable tag only — `:v4-beta` is not touched)
- Helm chart `oci://ghcr.io/triggerdotdev/charts/trigger.dev:4.5.0-rc.0`
- GitHub release `v4.5.0-rc.0` marked as Pre-release (no Latest badge)

What does NOT happen:
- npm `latest` stays at 4.4.6
- No marketing-site changelog PR (gated on `is_prerelease != 'true'`)
- Docker `:latest` not touched (we never push it anyway in this repo)

## Iteration

For subsequent rc.N: add a regular changeset to main, bot regenerates
the release PR as `v4.5.0-rc.N`. Merge to ship.

## Exiting pre mode

When ready to ship stable: `pnpm exec changeset pre exit`, push, merge
regenerated PR. That publishes `4.5.0` under `latest` and fires the
marketing-site dispatch.
2026-05-15 16:19:42 +00:00
Eric Allam eedde2793d chore: rewrite v4.5.0 release content around AI Agents (#3629)
## Summary

Refocuses the v4.5.0 changeset and server-changes content on the
public-facing AI features story, replacing the pre-release-internal diff
framing that had accumulated in `.changeset/` and `.server-changes/`.
Pairs with the RC support PR — the next bot regeneration will pick up
this content.

## What's in here

### Changeset rewrites

- **`chat-agent.md` rewritten as the headline AI Agents entry** —
written from the `docs/ai-chat/` surface (not from internal pre-release
diffs). Covers useChat integration, multi-turn durability via Sessions,
lifecycle hooks, stop generation, tool approvals (HITL), pending
messages + background injection, actions, typed state primitives,
`chat.toStreamTextOptions()`, multi-tab coordination, network
resilience, and the first-turn fast path (`chat.headStart`).
- **New `ai-prompts.md`** — announces the Prompts feature publicly for
the first time. Code-defined templates, deploy-versioning, dashboard
overrides, AI SDK telemetry integration, `chat.agent` integration via
`chat.prompt.set()` + `chat.toStreamTextOptions()`, full management SDK.
- **`sessions-primitive.md` expanded** — calls out
`tasks.triggerAndSubscribe()` and `sessions.list` as standalone
primitives (not just chat.agent infrastructure).
- **`chat-agent-on-boot-hook.md` trimmed** — drops "if you previously…"
pre-release migration framing.
- **Deletes 4 changesets** that described pre-release-internal
migrations or were circular ("groundwork for the upcoming chat.agent" —
chat.agent ships in the same release).

### Server-changes rewrites (`.server-changes/`)

Five new entries for the dashboard surface of the AI feature set:
- Agents list page
- Agent Playground
- Sessions dashboard
- Prompts dashboard (list with usage sparklines + detail with template /
Generations / Metrics / Versions tabs + override UI)
- Models registry (provider-grouped catalog with cross-tenant usage
metrics)
- AI generation span inspector on run traces
- Runs list Task source filter (Standard / Scheduled / Agent)
- Run-detail Agent view (segmented control)

Each entry is 1–2 sentences, no bullets, no implementation file paths —
fits as a single bullet in a future changelog.

Three older `.server-changes/` files were merged or split into the
cleaner taxonomy above and deleted.

## Out of scope

Non-AI-feature server-changes (admin-tabs, queue-length-cap fix,
worker-deployment race, streamdown upgrade, etc.) and changesets
(idempotency-key cap, sigsegv retry, locals-key fix, plugin auth, region
filters, etc.) are untouched.
2026-05-15 16:43:42 +01:00
Eric Allam dfa3ede209 feat(ci): support release candidates via changesets pre mode (#3628)
## Summary

Enables shipping `X.Y.Z-rc.N` prereleases of `@trigger.dev/*` via
changesets pre mode. RCs publish under the `rc` npm dist-tag, never
claim `latest`, and don't trigger marketing-site changelog PRs. The
plumbing is hyphen-in-version detection in `release.yml` — no separate
workflow, no opt-in flag at publish time.

Validated end-to-end against a sandbox repo (real npm publishes, Docker
builds, Helm chart pushes, GitHub releases) before porting back. Full RC
lifecycle tested: pre enter → rc.0 → iterate to rc.1 → pre exit →
stable. Plus interaction with the existing release-branch hotfix flow.

## What changes

### `release.yml`
- New `is_prerelease` output (hyphen-in-version)
- GitHub release adds `--prerelease` flag for RC publishes (Pre-release
badge, not Latest)
- `dispatch-changelog` job gated on `is_prerelease != 'true'` — no
marketing-site PR per RC

### Docker workflows
- Removes the `:v4-beta` floating tag entirely from `publish-webapp.yml`
and `publish-worker-v4.yml`. v4 is GA; the tag is a misnomer and is
already inconsistent with the npm side (npm `v4-beta` dist-tag was
frozen at 4.0.4 months ago while Docker `:v4-beta` kept bumping).
Self-hosters should pin to a versioned tag going forward — the last
value of `:v4-beta` stays frozen wherever it currently points.

### CLI version-check fix
(`packages/cli-v3/src/utilities/initialBanner.ts`)
Switches the "new version available" comparison from JavaScript
`localeCompare` to `semver.lt`. The old comparison handled `X.Y.Z-rc.N`
vs `X.Y.Z` incorrectly — a user on `4.5.0-rc.0` would never be prompted
to upgrade once `4.5.0` stable shipped (lex order put the prerelease
ahead of the bare version). Real semver gets this right.

Stable users were never affected: the check queries the `@latest`
dist-tag, which by convention never points at a prerelease.

## How an RC actually publishes after this

1. `pnpm exec changeset pre enter rc` on main, push the `pre.json`
2. Bot regenerates the release PR as `chore: release v<X.Y.Z>-rc.0`
3. Merge → `release.yml` runs `changeset publish` which reads
`pre.json.tag` and publishes under `--tag rc`. GitHub release marked
Pre-release. No marketing-site dispatch.
4. Iterate by adding changesets normally; bot bumps to `rc.1`, `rc.2`, …
5. When ready: `pnpm exec changeset pre exit`, push, merge regenerated
PR → stable ships under `latest` and the marketing-site dispatch fires.
2026-05-15 16:43:26 +01:00
Eric Allam 4c42f6cc0b feat(webapp,core,cli): filter runs by region in dashboard, API, and MCP (#3612)
## Summary

Adds a Region column and Region filter (under More filters) to the runs
list dashboard, the same filter on the public runs list API
(`filter[region]`), and a matching `region` input on the MCP `list_runs`
tool. Each run's executing region is also surfaced as a new optional
`region` field on the runs list and run retrieve responses, populated
from the worker instance group's `masterQueue` identifier.

Useful when you run tasks across multiple regions and want to slice the
runs list — or your existing run-querying scripts — by where the run
actually executed.

## Design

The filter value in the URL / API is the `masterQueue` identifier (the
same string already persisted on `TaskRun` and replicated to ClickHouse
as `worker_queue`), so the query just becomes `worker_queue IN (...)`
with no server-side translation. The Region dropdown options come from a
new resource loader backed by `RegionsPresenter`, which now also exposes
`masterQueue` alongside the existing region metadata.

```ts
// public API
const runs = await runs.list({ region: ["us-east-1", "eu-west-1"] });
// each item: { id, status, ..., region?: "us-east-1" }
```

```ts
// MCP
list_runs({ environment: "prod", region: "us-east-1" })
```
2026-05-15 15:32:13 +00:00
Eric Allam 454f0c949a perf(webapp): cache task metadata in Redis for the trigger hotpath (#3625)
## Summary

The trigger-task hotpath used to early-return without a DB query when a
caller passed both a queue override and a per-trigger TTL — the hottest
configuration on the trigger API. Adding `triggerSource` to the resolver
so the runs-list "Source" filter could distinguish STANDARD / SCHEDULED
/
AGENT runs removed those early-returns, costing +2 DB queries per
trigger
on non-locked calls and +1 on locked calls.

This change caches `BackgroundWorkerTask` metadata (`ttl`,
`triggerSource`,
`queueId`, `queueName`) in Redis so the resolver can satisfy every
caller
configuration with a single `HGET` on the warm path. PG fallback on miss
back-fills the cache.

Follow-up to #3542.

## Design

Two key spaces:

- `task-meta:env:{envId}` — the "current worker" view, refreshed at
every
  deploy promotion. 24h safety TTL.
- `task-meta:by-worker:{workerId}` — used for `lockToVersion` triggers.
  Immutable post-create. 30d sliding TTL so historical workers age out.

Cache writes use Lua scripts via `defineCommand` so `DEL` + `HSET` +
`EXPIRE` land atomically — concurrent readers never see the empty
intermediate state of a naive pipeline. Read-path back-fill uses
single-field upserts so concurrent back-fills don't wipe each other's
siblings.

The cache lives behind its own `TASK_META_CACHE_REDIS_*` env-var prefix
that falls back to the default `REDIS_*` set, so operators can route the
cache to a dedicated Redis instance if they want.

The service/instance file split (`taskMetadataCache.server.ts` for the
pure class, `taskMetadataCacheInstance.server.ts` for the env-wired
singleton) mirrors the existing `runsReplicationService` /
`runsReplicationInstance` pattern.

## Test plan

- [ ] `pnpm run typecheck --filter webapp`
- [ ] `pnpm run test ./test/engine/triggerTask.test.ts --run` — 8
      existing tests untouched + 5 new tests covering warm cache, cold
      miss with back-fill, queue + ttl path, by-worker vs env keyspace,
      and the promotion cache write
- [ ] End-to-end against a dev worker: registering writes both keyspaces
with the expected TTLs, and `redis-cli HGETALL
"tr:task-meta:env:<envId>"`
      returns the cached entries


## Benchmark

Measured `DefaultQueueManager.resolveQueueProperties` against a real
Postgres + Redis (vitest `containerTest`, single-host docker). 500
sequential calls and 2,000 parallel calls (concurrency=50) per scenario,
request shaped as `{ taskId, queue: "bench-queue", ttl: "5m" }` — the
hot path this PR restores.

```
sequential (one in flight at a time):
[noop cache (baseline)]  n=500   mean=1.423ms  p50=1.394ms  p95=1.735ms  p99=2.629ms  max=11.100ms
[redis cache, cold   ]  n=500   mean=1.346ms  p50=1.283ms  p95=1.688ms  p99=2.463ms  max=5.058ms
[redis cache, warm   ]  n=500   mean=0.084ms  p50=0.078ms  p95=0.105ms  p99=0.156ms  max=1.129ms
speedup (warm vs baseline, sequential): 16.95x

parallel (concurrency=50):
[noop cache (baseline)]  n=2000  mean=10.069ms  p50=8.850ms  p95=14.718ms  p99=31.887ms  total=405ms  ops/s=4,940
[redis cache, warm   ]  n=2000  mean=0.614ms   p50=0.568ms  p95=1.189ms   p99=1.432ms   total=25ms   ops/s=80,389
throughput speedup (warm vs baseline, parallel): 16.27x
```

Read:

- **Warm cache cuts resolver latency 17×** at p50 — from ~1.4 ms to ~78
µs per call.
- **Cold cache is on par with baseline** — the extra `HGET` miss adds
<50 µs against the two Postgres queries that follow, so the worst case
is not worse than today.
- **Under burst load (50 concurrent triggers)**, the baseline's p99
jumps to ~32 ms as Postgres connections queue up; warm stays at ~1.4 ms.
The cache moves the saturation point from ~5k ops/s (PG pool) to ~80k
ops/s (single-client Redis pipelining).

Caveats: single-host docker, local Postgres + Redis, resolver-only
measurement (excludes the rest of the trigger transaction). Prod adds
region-local Redis RTT (~0.3–0.8 ms) which shifts warm absolute numbers
up but keeps the ratio intact.
2026-05-15 11:52:53 +01:00
Daniel Sutton bff4b46b22 fix(webapp): log Google auth conflict as warn instead of error (#3627)
## Summary

A "Google auth conflict" Sentry alert fires whenever a user signs in via
Google whose Google account is linked to one user row but whose
Google-provided email is now on a *different* user row. The handler in
`apps/webapp/app/models/user.server.ts:236` already does the right thing
— it returns the existing auth-linked user and skips the update path so
neither row gets mutated — but it logs the situation with
`logger.error`, which routes to Sentry as an exception and pages the
on-call channel.

There's no exception to chase here: the branch is the intended outcome
for a known data shape (user changed their email on one account after
originally signing up via Google on another). Downgrading the call to
`logger.warn` keeps the diagnostic record in our logs (with all the same
context fields — email, both user IDs, authIdentifier) but stops it
firing the production error alert.

## Change

- `logger.error` → `logger.warn` for the conflict branch in
`findOrCreateGoogleUser`. Context payload is unchanged.

## Test plan

- [x] Typecheck only — there's no behavioural change to test, the log
level is the entire diff.
2026-05-15 09:42:17 +00:00
Eric Allam ac02c0f709 fix(core): drop unique-symbol brand on LocalsKey to fix dual-package builds (#3626)
## Summary

`LocalsKey<T>` (the type returned by `locals.create()`) was branded with
a
module-level `declare const __local: unique symbol`. Each such
declaration
is its own nominal type, and `tshy` emits separate `.d.ts` files for the
ESM and CJS outputs — each gets its own `__local` symbol. Under certain
pnpm hoisting layouts a single TypeScript compilation can resolve
`LocalsKey` from both the ESM source path and the CJS dist path within
the same call site, producing two structurally-incompatible variants of
the same type. TS surfaces this as the misleading error:

```
Argument of type 'LocalsKey<X>' is not assignable to parameter of type
'LocalsKey<X>'. Property '[__local]' is missing in type 'LocalsKey<X>'
but required in type 'BrandLocal<X>'.
```

The error has been hitting CI on PRs opened since the chat.agent stack
landed (e.g. #3625 typecheck job), but doesn't reproduce on developer
machines where the pnpm node_modules layout was built up incrementally.

## Fix

Replace the `unique symbol` brand with an optional phantom field that
carries `T` at the type level:

```ts
// before
declare const __local: unique symbol;
type BrandLocal<T> = { [__local]: T };
export type LocalsKey<T> = BrandLocal<T> & {
  readonly id: string;
  readonly __type: unique symbol;
};

// after
export type LocalsKey<T> = {
  readonly id: string;
  readonly __type: symbol;
  /** Phantom carrier for the value type — never read at runtime. */
  readonly __valueType?: T;
};
```

The ESM and CJS `.d.ts` outputs now produce structurally identical
types,
so cross-output resolution no longer produces a mismatch. `T` is still
carried at the type level via the optional phantom field. The runtime
shape is unchanged — `manager.ts` was already casting via `as unknown`,
which is no longer needed.

## Test plan

- [ ] `pnpm run typecheck --filter @trigger.dev/core --filter
@trigger.dev/sdk`
- [ ] `pnpm run build --filter @trigger.dev/core --filter
@trigger.dev/sdk`
      (clean rebuild) — confirms the ESM and CJS dist `.d.ts` outputs
      no longer carry distinct `unique symbol` declarations
- [ ] `pnpm --filter @trigger.dev/core test test/mockTaskContext.test.ts
--run`
- [ ] `pnpm --filter @trigger.dev/sdk test test/mockChatAgent.test.ts
--run`
2026-05-15 08:23:01 +01:00
nicktrn 0510fd6661 ci: skip typecheck for refs and non-code file PRs (#3624)
Follow-up to #3615. The `code` filter currently fires typecheck for any
change outside `docs/`, `.changeset/`, `hosting/`, or `.github/` - so a
docs-only PR like #3623 (touching `references/ai-chat/.env.example` +
`README.md`) triggered the typecheck job. None of the `references/*`
packages declare a `typecheck` script either, so even when a real code
change lands there, `turbo run typecheck` skips them. Running the job is
pure cost.

Tightens the filter to also exclude:

- `references/**` - playground projects, none of them contribute to
`turbo run typecheck` today
- `**/*.md` - markdown anywhere
- `**/.env.example` - example env files anywhere

Two known gaps left open:

- references/ have no real CI typecheck coverage. Separate question -
either add `typecheck` scripts to each (or top-level `tsc -p`), or
accept playground status.
- `changes` job still runs (it's a path-filter step) but the dependent
jobs all skip on irrelevant PRs.
2026-05-14 19:36:09 +01:00
Eric Allam e59291e35f docs(ai-chat): clarify local setup with .env.example (#3623) 2026-05-14 17:41:38 +00:00
Eric Allam 15b7cde8e7 feat: ai-chat reference project + MCP agent-chat tooling (4/4) (#3546)
## Summary

A complete Next.js reference project that exercises `chat.agent`
end-to-end, plus the CLI MCP tools that let Claude Code, Cursor, and
similar IDE agents drive a deployed `chat.agent` task from the editor.
Builds on #3545.

## Design

`references/ai-chat` is a full Next.js app: prisma-backed persistence,
multi-chat sidebar, per-chat model picker, debug panel, tool examples
(`getCurrentTime`, `searchHackerNews`, `createGithubIssue`, PR review
helpers, code sandbox), and smoke tests. It's intended both as a
copy-paste starting point and as a place to regression-test SDK changes.

The CLI gains MCP tools (`start_agent_chat`, `send_agent_message`,
`close_agent_chat`, `list_agents`) so an IDE agent can converse with a
deployed `chat.agent` task. The dev runtime adds one-shot OOM kill on
the run controller and skills bundling in the build pipeline.
2026-05-14 17:55:23 +01:00
Eric Allam 8673d42c80 feat: ai-chat reference project + MCP agent-chat tooling
Top of the chat.agent stack: a full Next.js reference project that
exercises chat.agent end-to-end, plus the CLI MCP tools that drive
agent runs from Claude Code / Cursor / etc.

references/ai-chat:
- Full Next.js app with prisma persistence, multi-chat sidebar,
  per-chat model picker, debug panel, tool examples, smoke tests
- Reference tools: getCurrentTime, searchHackerNews, createGithubIssue,
  PR review helpers, code sandbox
- chat-client-test orchestrator for concurrent-send stress
- references/hello-world chatAgent + triggerAndSubscribe examples

CLI MCP tooling for chat.agent:
- mcp/tools/agentChat.ts (start_agent_chat, send_agent_message,
  close_agent_chat)
- mcp/tools/agents.ts + tasks.ts (list agents, agent run details)
- dev-run-controller OOM kill + taskRunProcessPool tweaks
- dev/managed entry-point hooks for skills bundling
- buildWorker + bundleSkills (agent skills support)

Includes ai-tool-helpers + mcp-agent-chat-sessions changesets, plus
the streamdown@2 patch and pnpm-lock reconciliation.

(Will be renamed to feature/ai-chat-reference-and-cli before push.)

fix(cli): preserve lastEventId after sendMessage fallback to avoid stale turn-complete replay
2026-05-14 17:35:56 +01:00
Eric Allam 16538f6fdb feat(webapp): agent-view dashboard for chat.agent runs (3/4) (#3545)
## Summary

A chat-aware run inspector and a `/playground` UI for testing
`chat.agent` tasks interactively. Builds on #3543's runtime.

## Design

The run inspector grows a new tab that renders the conversation chain
for any `chat.agent`-kind run. It subscribes to the run's session
streams, threads chat parts through a per-message renderer, and uses a
shared markdown + Shiki component for code highlighting (also used by
the test-payload panel).

The playground is a standalone `/playground` route that lets you drive a
deployed chat agent from the dashboard — pick a task, send messages,
watch tool calls render, and see span detail on every turn. The matching
`/agents` list view shows all deployed agents in the project.
2026-05-14 16:58:15 +01:00
Eric Allam 23ff763359 feat(webapp): agent-view dashboard for chat.agent runs
Dashboard surfaces for inspecting and debugging chat.agent runs.
Depends on the Sessions primitive (L1) and chat.agent runtime (L2+L3).

Run inspector — chat-aware:
- AgentView + AgentMessageView (run inspector tab for chat.agent runs)
- AIChatMessages + AISpanDetails + types.ts (per-span chat message
  rendering, tool-call/tool-output handling)
- PromptSpanDetails (gen_ai.* span detail panel)
- StreamdownRenderer + shikiTheme (markdown renderer with shiki
  highlighting and v2 patch)
- useAutoScrollToBottom hook

Playground UI (interactive chat.agent debugger):
- /playground index + /playground/$agentParam routes
- /agents route + AgentListPresenter
- PlaygroundPresenter (per-org basin variants, clientData wiring)
- realtime session routes for playground + run inspector chat
- AI-generate-payload + AIPayloadTabContent for the test panel

Navigation + theming:
- SideMenu links for Agents and Playground
- BlankStatePanels copy updates
- tailwind config + tailwind.css storybook hooks
- streamdown@2 dep in apps/webapp/package.json

Includes agent-view-sessions, playground-trigger-config-fields,
run-agent-view, and streamdown-v2-upgrade .server-changes.
2026-05-14 16:45:53 +01:00
Daniel Sutton 5022769f70 fix(webapp): retry on version collision when initializing a deployment (#3610)
Concurrent `POST /api/v1/deployments` requests for the same environment
race on the `WorkerDeployment(environmentId, version)` unique
constraint. Both requests read the same latest deployment via
`findFirst`, compute the same next version via
`calculateNextBuildVersion`, and both attempt
`prisma.workerDeployment.create()` — one wins, the other crashes with
Prisma `P2002`. The bug is a classic TOCTOU between the version read and
the version write; it's been latent since the version-assignment logic
was first added but only fires when two deploys land within milliseconds
of each other (CI matrices, retried CLI calls, webhook-triggered
redeploys).

## Approach

Extracts the version assignment + create into a small helper
`createDeploymentWithNextVersion`
(`apps/webapp/app/v3/services/initializeDeployment/createDeploymentWithNextVersion.server.ts`).
The helper retries on `P2002 (environmentId, version)` up to 5 times
with randomised 5–50ms jitter so N concurrent racers don't loop in
lockstep. Each attempt re-reads the latest version, recomputes via
`calculateNextBuildVersion`, and re-runs the caller's `buildData`
callback so version-dependent fields (image ref tag, friendlyId) are
always consistent with the version actually persisted. A `logger.warn`
fires per collision so the retry rate is observable in production logs.

When retries are exhausted, the helper throws a dedicated
`DeploymentVersionCollisionError` carrying `environmentId`, `attempts`,
and `lastAttemptedVersion`, with the original
`PrismaClientKnownRequestError` attached as `cause`. Sentry walks the
`cause` chain natively, so contention exhaustion shows up as a
distinguishable wrapper exception linked to the underlying `P2002`
rather than a generic unique-constraint violation that looks identical
to every other duplicate-key bug.

The behavioural change is limited to "catch P2002 and retry instead of
crashing." The image ref computation stays inside the builder callback
(same call site as before the refactor), so ECR / non-ECR behaviour, S2
stream creation order, and all downstream side effects are unchanged.

## Non-goals

- No new database migrations, no schema changes, no isolation-level /
locking changes. A serialisable transaction or advisory lock would also
fix this; retry-on-conflict is the smaller change that keeps the
existing version-allocation logic intact.
- Does not touch the analogous `calculateNextBuildVersion` call in
`createBackgroundWorker.server.ts`, which likely has the same race shape
against `BackgroundWorker`'s unique constraint — flagged as a follow-up.

## Test plan

- [x] `pnpm run typecheck --filter webapp` passes (no new errors in the
modified files).
- [x] Three real-Postgres tests in
`apps/webapp/test/createDeploymentWithNextVersion.test.ts` via
`containerTest`:
- 5 concurrent calls all produce distinct, persistable versions
(`Set(versions).size === concurrency`). The naive read-then-create
version of the helper fails this test with the exact same `P2002` seen
in production; the retry version passes.
- Non-`P2002` errors raised from the `buildData` callback propagate
immediately without retry, builder invoked exactly once.
- With `maxRetries: 0`, concurrent racers surface the wrapped
`DeploymentVersionCollisionError` (not a raw `P2002`); `environmentId`,
`attempts`, `lastAttemptedVersion` are populated and `error.cause.code
=== "P2002"`.
- [x] Existing `apps/webapp/test/getDeploymentImageRef.test.ts` still
green (the file was untouched in the final diff).

## Follow-ups (not in this PR)

- `createBackgroundWorker.server.ts` likely has the same TOCTOU shape
against its background-worker version unique constraint — should use the
same helper.
- Sentry visibility check: confirm `error.cause` chain renders as a
linked exception in the Sentry UI when the wrapped error fires (requires
a sandboxed triggering of the exhaustion path).

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-14 16:23:32 +01:00
Eric Allam a97365d1d5 feat(sdk): chat.agent — runtime + browser transport (2/4) (#3543)
## Summary

Adds `chat.agent({...})`, a durable conversational task runtime, plus
the browser-side `TriggerChatTransport` + `AgentChat` that drive it from
a React or Next.js app. Conversations survive page refreshes, network
blips, idle suspend, and process restarts, with built-in tools, HITL
approvals, multi-turn state, and stop-mid-stream cancellation. Builds on
#3542.

## Design

Each `/in/append` request carries at most one new message. The agent
reconstructs prior history at run boot from an object-store snapshot
plus a `session.out` replay tail, so conversation context lives
server-side instead of bloating the wire. Awaited snapshot writes after
every `onTurnComplete` keep the chain durable across idle suspend.
Registering `hydrateMessages` short-circuits both paths for customers
who own their own conversation store.

Lifecycle hooks — `onChatStart`, `onTurnStart`, `onTurnComplete`,
`onAction`, `onValidateMessages`, `hydrateMessages` — cover validation,
persistence, and post-turn work. `chat.history` exposes read primitives
(`getPendingToolCalls`, `getResolvedToolCalls`, `extractNewToolResults`,
`findMessage`, `all`) for HITL flows. `chat.local` gives per-run typed
state with Proxy access and dirty tracking. `chat.headStart` bridges
first-turn TTFC via a customer HTTP handler. `oomMachine` opts a chat
into one-shot OOM-retry on a larger machine.

`TriggerChatTransport` is a `Transport` implementation for Vercel's
ai-sdk `useChat`: delta-only wire sends, SSE reconnection with
`lastEventId` resume, stop/abort cleanup, dynamic `accessToken` refresh,
`X-Peek-Settled` fast-close. `AgentChat` is the direct programmatic
equivalent. A cross-tab coordinator does leader election so multiple
open tabs share a single SSE.

```ts
import { chat } from "@trigger.dev/sdk/ai";
import { streamText } from "ai";

export const myChat = chat.agent({
  id: "my-chat",
  run: async ({ messages, signal }) =>
    streamText({ model: openai("gpt-4o"), messages, abortSignal: signal }),
});
```
2026-05-14 15:29:42 +01:00
Eric Allam 16720a5e62 feat(sdk): chat.agent — runtime + browser transport
Adds the chat.agent({...}) task definition (server runtime) and the
browser-side TriggerChatTransport + AgentChat that drives it from a
React or Next.js app. The runtime sits on top of the Sessions primitive
and handles the durable conversational task lifecycle.

Server runtime:
- chat.agent({...}) — session-aware task definition
- Lifecycle hooks: onChatStart, onTurnStart, onTurnComplete, onAction,
  onValidateMessages, hydrateMessages
- chat.history read primitives for HITL flows
- chat.local, chat.headStart, chat.handover, oomMachine
- Delta-only wire + S3 snapshot reconstruction at run boot
- Actions are no longer turns

Browser transport:
- TriggerChatTransport (ai-sdk Transport): delta-only wire sends,
  SSE reconnection with lastEventId resume, stop/abort cleanup,
  dynamic accessToken refresh
- AgentChat: direct programmatic API
- useTriggerChatTransport (React hook)
- chat-tab-coordinator: cross-tab leader election

Includes the chat-agent, chat-agent-delta-wire-snapshots,
chat-history-read-primitives, chat-head-start, chat-actions-no-turn,
chat-session-attributes, agent-skills, and mock-chat-agent-test-harness
changesets.
2026-05-14 15:09:25 +01:00
nicktrn b62c8a19c2 ci: add dependabot weekly summary workflow (#3616)
Adds a Mon 08:00 UTC workflow that posts a summary of open Dependabot
alerts and PRs to Slack. Uses env-scoped secrets so the alerts PAT and
Slack token are only available to this workflow.
2026-05-14 15:44:37 +02:00
nicktrn 9caf4ceaf1 ci: skip typecheck for workflow-only PRs (#3619)
The `code` paths filter currently matches `**` minus a tiny exclusion
list, so a PR that only touches `.github/workflows/*.yml` still flips
`code == true` and runs typecheck (~2 min on the runner).

Exclude `.github/**` from `code`, then re-include just `pr_checks.yml`
and `typecheck.yml` so a change to either of those still triggers the
full code check matrix.

Effect:
- workflow-only PRs (this one, future dependabot/codeql/etc.) skip
typecheck; `all-checks` treats the skipped job as non-failure so the
required status passes.
- modifying `pr_checks.yml` or `typecheck.yml` themselves still triggers
typecheck.
- the existing per-suite filters (`webapp`, `packages`, `internal`,
`cli`, `sdk`) already re-include the specific workflows that gate them,
so they're unaffected.
2026-05-14 14:37:55 +01:00
Eric Allam 979655c281 feat: Sessions dashboard, task_kind, and chat-ready hardening (1/4) (#3542)
## Summary

A `/sessions` dashboard for inspecting durable Sessions, an `AGENT` /
`SCHEDULED` task-kind filter for the runs list, and the server-side
hardening (rate-limit exemption for packets, retry-with-backoff on
stream appends, typed too-large-chunk error) that the `chat.agent`
runtime in #3543 needs. Builds on the Sessions primitive shipped in
#3417.

## Design

The Sessions list + detail routes mirror the run inspector pattern.
`TaskTriggerSource` gains `AGENT` and `SCHEDULED` values, persisted on
`BackgroundWorker.taskKind` and `TaskRun.taskKind` (plus a matching
Clickhouse column), so the runs list can filter by kind.

New `@trigger.dev/core` modules — `sessionStreams`, `inputStreams`, a
`sessionStreamInstance` for realtime streams, and the
`realtime-streams-api` / `session-streams-api` surfaces — expose the
typed shapes that chat.agent will use to drive `session.out`.
`ChatChunkTooLargeError` lets the runtime drop oversized chunks with a
typed surface instead of failing the run. `s2Append` retries transient
failures with exponential backoff. `/api/v[12]/packets/*` is exempt from
customer rate limits so chat snapshot reads and writes don't get
throttled under load.

## Stack

Part of a 4-PR stack. Merge bottom-up.

1. **This PR** (#3542) → `main`
2. #3543#3542 — `chat.agent` runtime + browser transport
3. #3545#3543 — agent-view dashboard
4. #3546#3545 — ai-chat reference + MCP tooling

Replaces #3173 (closed).

<!-- GitButler Footer Boundary Top -->
---
This is **part 5 of 5 in a stack** made with GitButler:
- <kbd>&nbsp;5&nbsp;</kbd> #3612
- <kbd>&nbsp;4&nbsp;</kbd> #3546
- <kbd>&nbsp;3&nbsp;</kbd> #3545
- <kbd>&nbsp;2&nbsp;</kbd> #3543
- <kbd>&nbsp;1&nbsp;</kbd> #3542 👈 
<!-- GitButler Footer Boundary Bottom -->
2026-05-14 13:41:29 +01:00
Daniel Sutton 09f5354a03 fix(core): cap idempotencyKey length at the API boundary (#3560)
`tasks.trigger`, `tasks.batchTrigger`, `batch.create`,
`wait.createToken`, `wait.forDuration`, and the input/session stream
waitpoint endpoints all accept a caller-supplied `idempotencyKey` and
store it verbatim against a composite-unique index on `TaskRun`,
`BatchTaskRun`, or `Waitpoint`. The schemas had no length cap, so a
sufficiently long high-entropy key produced an index row larger than the
underlying storage layer can hold. The insert failed at the database,
and the caller saw a generic 500 from
`RunEngineTriggerTaskService.call()` / `CreateBatchService` / waitpoint
creation, depending on the endpoint.

Keys produced by `idempotencyKeys.create()` are 64-character SHA-256
hashes and never trip this — it only manifests for direct REST callers
(or SDK callers passing a raw string they generated themselves).
Low-entropy keys also sail through, because the storage layer compresses
repeated bytes before they reach the index, which is why the failure
mode is intermittent and tied to caller-side key shape.

## Fix

Add `.max(2048, "<field> must be 2048 characters or less")` to the seven
schemas that feed an indexed `idempotencyKey` column:

- `TriggerTaskRequestBody.options.idempotencyKey`
- `BatchTriggerTaskItem.options.idempotencyKey`
- `CreateBatchRequestBody.idempotencyKey`
- `CreateWaitpointTokenRequestBody.idempotencyKey`
- `CreateInputStreamWaitpointRequestBody.idempotencyKey`
- `CreateSessionStreamWaitpointRequestBody.idempotencyKey`
- `WaitForDurationRequestBody.idempotencyKey`

Plus the `idempotency-key` HTTP header on the trigger route (and the
three batch routes that re-export `HeadersSchema`). The header schema is
lifted out of `api.v1.tasks.$taskId.trigger.ts` into
`apps/webapp/app/v3/triggerHeaders.server.ts` so it can be exercised in
tests without dragging the route's import-time side effects.

The 2048 character ceiling is chosen to sit safely under the per-row
index limit while staying generous against existing callers — keys that
fit before still fit. Oversized keys now return a structured Zod 400
instead of a generic 500.

Limit is documented under `Idempotency key` in `docs/limits.mdx` and as
a `<Note>` on `docs/idempotency.mdx`.

## Test plan

- [x] 15 schema unit tests added
(`packages/core/src/v3/schemas/idempotencyKey.test.ts`,
`apps/webapp/test/routes/triggerHeaders.test.ts`) —
rejection-with-message + boundary acceptance for each capped schema. The
webapp test exercises the extracted `TriggerHeadersSchema` directly with
no mocks.
- [x] `pnpm run build --filter @trigger.dev/core`
- [x] `pnpm run typecheck --filter webapp`
- [x] End-to-end verified locally: baseline (small key) → 200; 3000-char
high-entropy header → 400 with the expected Zod error; same key at the
2048 boundary → 200; same key with the cap reverted → the database
rejected the insert and the route returned 500 to the caller. Cap
restored.

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-14 13:24:50 +01:00
Eric Allam be1a6cf8de feat: Sessions primitive — durable run-aware streams + dashboard
Adds Sessions, a durable, run-aware stream primitive that scopes
session.in / session.out records to a session (not a single run).
Records survive run boundaries; reconnect-from-last-event-id is built in.

Server foundation:
- New /realtime/v1/sessions/:session/:io/append + /records routes
- sessionRunManager + sessionsRepository + clickhouseSessionsRepository
- mintRunToken for short-lived per-session tokens
- s2Append retry-with-backoff + undici cause diagnostics
- /api/v[12]/packets/* exempt from customer rate limits
- BackgroundWorker schema gains taskKind enum (TASK, AGENT, SCHEDULED)
- TaskRun.taskKind column + clickhouse 029_add_task_kind_to_task_runs_v2

Core types:
- new sessionStreams, inputStreams, realtimeStreams packages in @trigger.dev/core
- session-streams-api / realtime-streams-api surface

Sessions dashboard UI (the primitive's own viewer):
- /sessions index + detail routes
- SessionsTable, SessionFilters, SessionStatus, CloseSessionDialog
- AGENT/SCHEDULED filter in RunFilters + TaskTriggerSource

Includes the sessions-primitive changeset.
2026-05-14 13:12:36 +01:00
nicktrn 8ba067d8b0 feat(webapp): preserve admin tabs search query between Users and Organizations (#3609)
Switching between the Users and Organizations tabs in the admin
dashboard now keeps the current `?search=` value, so you can flip
between the two without re-typing your filter. Other admin tabs don't
take `search` and so don't carry it.
2026-05-13 22:43:44 +00:00
nicktrn d144220174 ci: path-based skip and gate job in pr_checks (#3615)
`pr_checks` runs the full matrix on every PR. #3609 touched only
`apps/webapp/app/routes/admin.tsx` and still ran the 4-job CLI e2e
matrix and 5-job sdk-compat suite.

Adds a `changes` job using `dorny/paths-filter` and gates each tier:

- webapp + e2e-webapp: `apps/webapp/**`, `packages/**`,
`internal-packages/**`
- packages: `packages/**`
- internal: `internal-packages/**` + `packages/**` (cross-deps)
- e2e (cli-v3): `packages/{cli-v3,build,core,schema-to-json}/**`
- sdk-compat: `packages/{trigger-sdk,core}/**`

`.configs/**`, `package.json`, `pnpm-lock.yaml`, `pnpm-workspace.yaml`,
`turbo.json` are also included in every filter since they affect the
whole workspace.

Inlines the `units` reusable-workflow children so each can be gated
independently (status check names also flatten from `units / webapp /
...` to `webapp / ...`). `unit-tests.yml` is unaffected - still used by
`publish.yml`.

Adds an `all-checks` gate that always runs and short-circuits to success
when every dependent is success-or-skipped. With this in place a single
required status check (`All PR Checks`) is enough; before this,
`paths-ignore` would have left required checks Pending on docs/changeset
PRs ([gh
docs](https://docs.github.com/en/actions/managing-workflow-runs/skipping-workflow-runs)).
2026-05-13 22:37:20 +00:00
Iss b23740bf7d docs(self-hosting): NodeLocal DNS and ClickHouse task events (#3568)
## Summary
- Recommend deploying NodeLocal DNS and lowering `ndots` to `1` in the
Kubernetes self-hosting guide.
- Recommend storing task events in ClickHouse
(`EVENT_REPOSITORY_DEFAULT_STORE=clickhouse_v2`) in both the Docker and
Kubernetes guides, plus a new row in the webapp env var reference.
2026-05-13 22:06:13 +00:00
James Ritchie 759214eabd fix(webapp): Evict legacy resizable-panel localStorage on client boot (#3564)
## Summary

- Users on production are hitting `QuotaExceededError: Failed to execute
'setItem' on 'Storage'` when navigating runs, because their localStorage
is full of orphaned `panel-group-react-aria<n>-:<rid>:` entries.
- Each entry is a session-unique key written by the resizable panel
library; they accumulated to thousands per user over the last two months
and now block legitimate `setItem` calls (the run-view inspector can no
longer persist its layout, and the page crashes mid-render).
- This PR evicts the legacy entries once on client boot. The leak itself
is already plugged by the v1.1.3 upgrade in #XXXX — this is the cleanup
that recovers the wasted quota on existing users' machines.

## Root cause (already fixed, for context)

In v0.4.1 of the underlying library, `PanelGroupImpl` defaulted
`autosaveStrategy` to `"localStorage"` unconditionally — so *every*
`PanelGroup` wrote to localStorage on every autosave trigger, including
the four in `QueryEditor`, the one in `ReplayRunDialog`, the storybook
routes, etc. Without an `autosaveId`, the key fell back to
`panel-group-${useId()}`, and React Aria's `useId()` produces a new
session-unique prefix each visit. Result: entries accumulated without
bound across sessions.

The condition was introduced when
[#3282](https://github.com/triggerdotdev/trigger.dev/pull/3282) removed
the wrapper's explicit `autosaveStrategy="cookie"` override (to fix HTTP
431 cookie-size errors). That worked, but the library default that took
over silently caused this leak.

The v1.1.3 upgrade in the resizable-panel PR changed the default to
`autosaveStrategy = autosaveId ? "localStorage" : undefined`, so no new
entries are being written. Existing residue still needs to be removed
from users' browsers.

## Changes

- New file
[`apps/webapp/app/clientBeforeFirstRender.ts`](apps/webapp/app/clientBeforeFirstRender.ts)
— exports a `clientBeforeFirstRender()` function that runs
synchronously, before React hydrates. Encapsulates a small cleanup
helper that scans `localStorage` and removes:
- Every key starting with `panel-group-react-aria` (the legacy
auto-generated keys).
- The orphan `panel-run-parent-v2` key from before the autosaveId v2→v3
bump.
- [`apps/webapp/app/entry.client.tsx`](apps/webapp/app/entry.client.tsx)
— imports and invokes `clientBeforeFirstRender()` once, before
`hydrateRoot()`. This guarantees the cleanup completes before any
`ResizablePanelGroup` mounts and tries to write.

The cleanup is wrapped in `try/catch` so private-browsing /
disabled-storage scenarios fail silently. Idempotent: subsequent loads
find no matching keys and exit immediately.

## Test plan

- [x] Locally seed ~50 fake `panel-group-react-aria…` entries plus a
`panel-run-parent-v2` entry via DevTools console, hard reload → legacy
entries gone, real entries (`panel-run-parent-v3`, `panel-run-tree`)
preserved.
- [x] Idempotency: reload a second time, no errors, no state changes.
- [x] Add a control entry (`panel-run-parent-v3-but-different-suffix`) —
confirmed not over-matched.
- [x] Simulate broken `Storage.setItem` throwing — page still renders,
cleanup swallows the error.
- [x] Typecheck clean.

## Notes

- Customer report: `QuotaExceededError: Failed to execute 'setItem' on
'Storage': Setting the value of 'panel-run-parent-v3' exceeded the
quota.`
- The cleanup runs once per page load. Once a user has loaded the app
after this deploys, their localStorage is clean and the function becomes
a no-op forever.
2026-05-13 09:26:01 +01:00
Eric Allam 6b0e78f1db chore: raise REVIEW.md drift-audit turn budget and steer selective sampling (#3567)
## Summary

Follow-up to #3561. The drift-audit workflow timed out on PR #3542 (92
files, +5962 lines) by hitting `--max-turns 15` before reaching a
verdict, leaving a red  on that PR with no sticky comment.

## Changes

- `--max-turns` bumped from 15 to 30.
- Prompt now opens with an explicit "Strategy" section: read REVIEW.md
once, scan the file-list only, open at most 5 files (3-5 on PRs >50
files), and bias toward finishing over exploring.
- Final rule: *"when in doubt between one more file read and finish now
— finish now."*

The audit is allowed to miss things. It is not allowed to time out and
leave a red X.

## Test plan

- [ ] Verify this PR's audit posts ` REVIEW.md looks current for this
PR.` (small diff)
- [ ] After merge, retry the audit on #3542 or a similarly large PR and
confirm it completes
2026-05-12 21:24:09 +01:00
Eric Allam 5c4e06479d chore(docker): disable ClickHouse system log tables in local dev (#3565)
## Summary

Local ClickHouse was burning ~325% CPU endlessly merging its own
telemetry tables (`metric_log`, `asynchronous_metric_log`, `part_log`,
`trace_log`) after the container had been running long enough to
accumulate hundreds of GB of system-log data. OrbStack Helper reflected
this on the host (~400% CPU).

These tables are not used by anything in the dev stack. They only exist
for ClickHouse to log itself, so disabling them eliminates the merge
churn entirely.

## Changes

- Adds `docker/config/clickhouse-disable-system-logs.xml`, mounted into
`/etc/clickhouse-server/config.d/`, that removes the noisy system log
tables via `<table remove="1"/>`.
- Mounts the override file in `docker/docker-compose.yml`.

After applying, idle CPU dropped from 325% to ~12% on my machine.

## Test plan

- [ ] `pnpm run docker` brings up the stack cleanly
- [ ] `docker stats clickhouse` shows low idle CPU
- [ ] App functionality unaffected (system log tables are not queried by
the webapp)
2026-05-12 19:01:38 +01:00
Eric Allam e8ef374fe0 fix(webapp,run-engine): honor per-queue length cap on concurrency-key queues (#3558)
## Summary

Queues that use concurrency keys can no longer bypass the per-queue
length cap, and the "Queued | Running" columns in the dashboard now show
the true total across all CK variants instead of 0.

The cap and the dashboard both relied on `ZCARD` of the base queue key,
but CK-keyed runs live under `<base>:ck:<variant>` keys. Any queue that
used concurrency keys read 0 — letting a single CK variant grow
unbounded past the user's configured cap.

## Fix

Two per-base-queue counters are maintained inside the CK Lua scripts:
`<base>:lengthCounter` and `<base>:runningCounter`. Non-CK
enqueue/dequeue paths are untouched.

Counters are lazy-initialized the first time a CK enqueue (or nack)
lands on a queue: the Lua script sums `ZCARD` across the variants
tracked by `ckIndex`, sets the counter, then `INCR`s. Pre-existing CK
backlog on already-populated queues is captured automatically — no batch
migration required.

`INCR`/`DECR` is gated on `ZADD`/`SADD` returning 1 (a new entry vs an
idempotent no-op), so duplicate enqueues or re-dequeues don't inflate
the counter.

The counter is `SET` with a 24-hour TTL on init. `INCR`/`DECR` do not
extend the TTL, so the counter expires daily and the next CK operation
re-seeds it from `ckIndex`. This bounds any drift that accumulates
during the rolling-deploy overlap window — where old (un-Tracked) and
new (Tracked) webapp instances briefly coexist — to ≤24 hours, with no
admin sweep or background reconciler needed.

Read paths pipeline `ZCARD`/`SCARD` on the base key + `GET` on the
counter and sum. A missing counter is treated as 0, so pure non-CK
queues see the same answer as before.

The counter-aware scripts ship alongside the originals with a `Tracked`
suffix for rolling-deploy safety; a follow-up PR will drop the originals
once this has rolled out.

## Test plan

- [ ] `pnpm run test --filter @internal/run-engine` — 116 tests pass,
including a new `ckCounters.test.ts` covering lazy init from
pre-existing backlog, churn, floor-at-zero, the non-CK regression case,
mixed CK + non-CK on the same base queue, idempotent re-enqueue
(ZADD-already-exists), 24h TTL on the counter, and nack re-seeding after
counter expiry.
- [ ] Verified end-to-end against a live local environment:
- Triggered 24 CK enqueues across 4 variants → `lengthCounter=16`,
`runningCounter=8`, dashboard showed Queued=16 / Running=8 for the CK
queue.
- Set the env queue cap to 16, triggered 12 more enqueues → 8 succeeded,
4 rejected with `QueueSizeLimitExceededError`.
- Deleted the counter on a queue with 31 messages already sitting in CK
variants, triggered one more enqueue → counter materialized to 31 from
the `ckIndex` sum, then INCR'd.
2026-05-12 18:37:19 +01:00
Matt Aitken e4981d1b11 feat(webapp): consolidate auth path + add comprehensive auth tests (#3499)
## Summary

Consolidates the webapp's authentication and authorization into a small
set of route helpers, replacing the ad-hoc `requireUser` /
`requireUserId` / `authenticatedEnvironmentForAuthentication` calls
scattered across routes. Same security model, but the per-request flow
(authenticate → authorize → load) now lives in one place per route
family.

Introduces a plugin seam (`@trigger.dev/plugins`) that lets the cloud
build install a richer RBAC implementation without touching webapp code.
The OSS fallback keeps the pre-RBAC permissive behaviour intact, so
self-hosted deployments work unchanged.

Adds a comprehensive end-to-end auth test suite that didn't exist before
— 193 `it()` blocks (vitest reports ~199 after `it.each` expansion)
covering API key, PAT and JWT auth across the public API surface, plus
dashboard session auth for admin pages.

## Changes

### Plugin contract — `@trigger.dev/plugins`

`RoleBaseAccessController` interface authoritative for both OSS
(fallback) and cloud (enterprise plugin):
- `authenticateBearer(request, { allowJWT? })` — API-key / public-JWT
auth, returns env + ability
- `authenticateSession(request, { userId, organizationId?, projectId?
})` — dashboard auth, caller resolves `userId` from the session cookie
and passes it in (no `helpers.getSessionUserId` callback — decouples the
plugin host from session-cookie code)
- `authenticatePat(request, { organizationId?, projectId? })` — PAT
auth, returns identity + `lastAccessedAt` so the host can throttle the
per-request update
- `authenticateAuthorize*` variants for the auth-and-check-in-one-call
cases
- `isUsingPlugin(): Promise<boolean>` — capability flag for UI /
branching where plugin-present-ness matters; replaces the
sentinel-string coupling that had `personalAccessToken.server` matching
`"RBAC plugin not installed"` literally

### Dashboard auth (started, partial rollout)

Admin and settings pages migrated to a unified `dashboardLoader` /
`dashboardAction` helper that authenticates the session, runs an
authorization check, and exposes the result to the route. Other
dashboard routes still on the old pattern; remaining migration tracked
in TRI-8730.

Migrated routes:
- `admin.*` (14 admin / back-office / feature-flags / LLM-models /
notifications / orgs / concurrency pages)
- `_app.orgs.$organizationSlug.settings.team`
- `_app.orgs.$organizationSlug.settings.roles`

### API / realtime / engine auth (complete for the migrated families)

71 routes migrated to a unified `apiBuilder` that centralizes Bearer /
PAT / Public-JWT authentication and applies the per-route authorization
check before the handler runs. Includes:
- `api.v1.*` and `api.v2.*` and `api.v3.*` — tasks, runs, batches,
queues, prompts, deployments, query, sessions, waitpoints, packets,
workers, idempotency keys
- `realtime.v1.*` — runs, batches, sessions, streams
- `engine.v1.*` — dev / worker-action protocols

29 routes still on the legacy `authenticateApiRequest*` helpers —
tracked as a post-deploy follow-up in TRI-9228.

Multi-resource auth direction is now explicit at the call site via
`anyResource(...)` (OR) and `everyResource(...)` (AND). Bare arrays no
longer typecheck — fixes a class of bug where a JWT scoped to one
resource could implicitly access others under OR semantics.

PAT auth path consolidated: was three DB queries per request (legacy
`authenticateApiRequestWithPersonalAccessToken` findFirst +
`rbac.authenticatePat` join + `lastAccessedAt` update). Now one query in
the steady state — plugin returns `lastAccessedAt`, host smart-skips the
update via JS-side throttle when fresh.

Side effect: action aliases preserved historic JWT scope semantics where
the new model is stricter (e.g. a `write:tasks` JWT now also satisfies
`trigger` / `batchTrigger` / `update` actions on the same resource —
matched at the auth boundary, not in the route handler).

### Backwards-compat fixes

The strict-match model regressed several real-world JWT shapes. Each
preserved via explicit `anyResource(...)` entries in the route's authz
block:

- **Batch retrieve routes** (`api.v1.batches.$batchId`, `api.v2.*`,
`realtime.v1.batches.*`) accept `read:runs` JWTs again (pre-RBAC
literal-match superScope behaviour)
- **Runs list routes** (`api.v1.runs`, `realtime.v1.runs`) accept
type-level `read:tasks` / `read:tags` on unfiltered queries (matched the
legacy `Object.keys` iteration semantic)
- **PAT/OAT auth shape** normalized through `toAuthenticated` so all
auth methods return the same slim `AuthenticatedEnvironment` (was:
API-key returned the slim shape but PAT/OAT returned raw Prisma
`Decimal` / no `orgMember`)
- **Scope `:` preservation** in resource ids — `read:tags:env:staging`
now correctly identifies the tag id as `env:staging`, not `env`

### Slim `AuthenticatedEnvironment`

Extracted to `@trigger.dev/core/v3/auth/environment` — a structural
shape independent of `@trigger.dev/database`. The plugin contract
returns this; webapp consumers import from there; the cloud plugin
(Drizzle) returns the same shape without Prisma's `Decimal` class
leaking into the public surface. Lets internal-packages (run-engine,
etc.) refer to `AuthenticatedEnvironment` without pulling Prisma in.

### Auth test suite (new — `*.e2e.full.test.ts`)

193 e2e tests run against a real spawned webapp + Postgres (no mocks).
Coverage matrix:

- **API key auth** — read / write / trigger / batchTrigger / deploy
actions across runs, batches, deployments, prompts, queues, query,
sessions, input-streams, waitpoints, tasks, idempotency keys; multi-key
resources (a run carries batch / tag / task identifiers — auth must
accept any matching scope)
- **Personal Access Token auth** — comprehensive matrix: scope match,
scope mismatch, missing scope, expired token, malformed token
- **Public JWT auth** — sub-vs-URL environment resolution, expired JWTs,
signature verification, scope checking, otu (one-time-use) token
semantics, branch-environment signing-key fallback
- **Dashboard session auth** — admin-only pages reject non-admins;
per-action gating
- **Cross-cutting edge cases** — revoked API key grace window, JWT
cross-environment isolation, MissingResource branch behaviour

### Hygiene cleanups

- Deleted dead `app/services/authorization.server.ts` (legacy
`checkAuthorization` + types — no live consumers post-migration) and its
orphaned test
- Dropped the never-populated `scopes` field from
`ApiAuthenticationResultSuccess`
- `scheduleEmail` moved out of `email.server.ts` into its own module —
breaks a `commonWorker → marqs/V1` import chain that was poisoning the
auth test graph
- OSS Roles page shows a deployment-aware empty state ("Roles aren't
available in this self-hosted deployment" vs the plan-upsell copy) via
`rbac.isUsingPlugin()`
- Team action handler: explicit per-intent ability gates
(`manage:billing` for purchase-seats, `manage:members` for set-role +
remove-member with self-leave carve-out)

### Cross-repo coordination

All public-package contract changes paired in `triggerdotdev/cloud#763`
(rbac-packages branch) — the enterprise plugin implements the same
`RoleBaseAccessController` interface against Drizzle.

## Test plan

- [x] `pnpm run typecheck --filter webapp` clean
- [x] `pnpm --filter webapp exec vitest run --config
vitest.e2e.full.config.ts` — 193/193 pass (requires Docker for
testcontainers)
- [x] Spot-check an authed API endpoint with a valid + invalid API key
against a local stack
- [x] Spot-check the migrated admin pages render and gate non-admins

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 17:16:20 +01:00
Eric Allam 3cbe9f2307 chore: add .claude/REVIEW.md with CI drift check (#3561)
## Summary

Adds `.claude/REVIEW.md` — a repo-specific source of truth for what AI /
agent code reviewers should treat as critical in this codebase
(rolling-deploy safety, hot-table indexes, recovery-path queries,
testcontainers usage, etc.). Pairs with a Claude-based PR audit that
flags drift between REVIEW.md and the code as it evolves.

## How the audit works

Mirrors the existing `.github/workflows/claude-md-audit.yml` pattern. On
non-draft, non-fork PRs that touch code, `anthropics/claude-code-action`
reads REVIEW.md, samples the PR diff, and posts a sticky comment with up
to 3 of:

- `[stale]` — rule cites a path / function / table that's been removed
or renamed
- `[contradiction]` — code in the PR violates a current rule
- `[missing]` — PR introduces a new pattern future reviewers should know
about
- `[obsolete]` — rule asserts a constraint the repo has moved past

If nothing's off, posts ` REVIEW.md looks current for this PR.`

## Test plan

- [ ] Convert this PR to ready-for-review, confirm the audit runs and
posts a sticky comment
- [ ] Verify the audit doesn't run on fork PRs (gated by
`head.repo.full_name == github.repository`)
- [ ] Verify suggestions are actionable on at least one follow-up PR
2026-05-12 16:21:30 +01:00
Matt Aitken 8e675a4e93 fix(core): retry TASK_PROCESS_SIGSEGV under the user's retry policy (#3552)
Closes
[TRI-9234](https://linear.app/triggerdotdev/issue/TRI-9234/retry-task-process-sigsegv-errors-respecting-user-retry-config)

## What this changes

SIGSEGV crashes (`TASK_PROCESS_SIGSEGV`) will now be **retried when an
attempt fails**, in line with the task's configured retry settings
(`retry.maxAttempts` etc.) — the same path SIGTERM and uncaught
exceptions already use. Previously SIGSEGV was hard-classified as
non-retriable and failed the run on the first segfault, ignoring the
user's retry policy.

Tasks without a retry policy still fail fast on the first SIGSEGV.
Behaviour is unchanged for OOM kills (separate machine-bump retry path)
and SIGKILL_TIMEOUT.

## Deploy

**Only the webapp needs to ship.** The retry decision lives entirely in
the webapp:
- V2 path: `internal-packages/run-engine` (bundled into the webapp)
- V1 path: `apps/webapp/app/v3/services/completeAttempt.server.ts`

No supervisor, CLI, SDK, or customer-task-image changes required.
Customers do not need to redeploy. The `@trigger.dev/core` changeset is
just keeping the public package in sync — the published npm version
isn't what makes the fix work.

## Why retry

SIGSEGV in Node tasks is frequently non-deterministic across processes:

- **Native addon races** (`sharp`, `canvas`, `better-sqlite3`,
`node-rdkafka`, `bcrypt`, …) — libuv thread-pool work stepping on V8
handles. Different heap layout / thread schedule on a fresh process →
retry often succeeds.
- **JIT / GC interaction** — V8 turbofan deopt or GC during a native
callback. Timing-dependent.
- **Near-OOM in native code** — when RSS approaches the cgroup limit,
native allocations fail and poorly-written addons dereference NULL →
SIGSEGV instead of clean OOM-kill.
- **Host / hardware issues** — bit flips, kernel quirks. Retry lands on
a different host.

The genuinely deterministic case (a user-code bug always tripping the
same addon) is real, but a subset — and `maxAttempts` bounds the damage.

## Pre-existing inconsistency this resolves

- `shouldRetryError` returned `false` for `TASK_PROCESS_SIGSEGV` →
`fail_run`.
- `shouldLookupRetrySettings` already listed `TASK_PROCESS_SIGSEGV` as
retry-config-aware — but that branch was unreachable because
`shouldRetryError` short-circuited first in `retrying.ts:86-90`.
- We already retry `TASK_RUN_UNCAUGHT_EXCEPTION` (clearly a user-code
bug) under the user's retry policy; refusing to retry SIGSEGV was the
odd one out.

## Test plan

- [x] `pnpm exec vitest run test/errors.test.ts` in `packages/core` —
26/26 pass (4 new)
- [x] `pnpm run build --filter @trigger.dev/core`
- [ ] CI green on PR

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 15:05:01 +01:00
github-actions[bot] 41a486ea7e chore: release v4.4.6 (#3501)
🚀 Publish Trigger.dev Docker / typecheck (push) Failing after 0s
🚀 Publish Trigger.dev Docker / publish-webapp (push) Has been skipped
🚀 Publish Trigger.dev Docker / publish-worker (push) Has been skipped
🚀 Publish Trigger.dev Docker / publish-worker-v4 (push) Has been skipped
🚀 Publish Trigger.dev Docker / units (push) Failing after 5s
🧭 Helm Chart Release / lint-and-test (push) Has been cancelled
🧭 Helm Chart Release / release (push) Has been cancelled
## Summary
1 improvement, 1 bug fix.

## Improvements
- Fail attempts on uncaught exceptions instead of hanging to
`MAX_DURATION_EXCEEDED`. A Node `EventEmitter` (e.g. `node-redis`)
emitting `"error"` with no `.on("error", ...)` listener escalates to
`uncaughtException`, which the worker previously reported but did not
act on — runs drifted to maxDuration with empty attempts. They now fail
fast with the original error and status `FAILED`, and respect the task's
normal retry policy. You should still attach `.on("error", ...)`
listeners to long-lived clients to handle errors gracefully.
([#3529](https://github.com/triggerdotdev/trigger.dev/pull/3529))

## Bug fixes
- Fix dev workers spinning at 100% CPU after the parent CLI disconnects.
Orphaned `trigger-dev-run-worker` (and indexer) processes were caught in
an `uncaughtException` feedback loop: a periodic IPC send via
`process.send` would throw `ERR_IPC_CHANNEL_CLOSED` once the parent
closed the channel, which re-entered the same handler that itself called
`process.send`, scheduled via `setImmediate` and amplified by
source-map-support's `prepareStackTrace`. Fixed by (1) silently dropping
packets in `ZodIpcConnection` when the channel is disconnected, (2)
adding a `process.on("disconnect", ...)` handler in dev workers so they
exit cleanly when the CLI closes the IPC channel, and (3) wrapping all
`uncaughtException`-path `process.send` calls in a `safeSend` guard that
checks `process.connected` and swallows synchronous throws.
([#3491](https://github.com/triggerdotdev/trigger.dev/pull/3491))

<details>
<summary>Raw changeset output</summary>

# Releases
## @trigger.dev/build@4.4.6

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.4.6`

## trigger.dev@4.4.6

### Patch Changes

- Fix dev workers spinning at 100% CPU after the parent CLI disconnects.
Orphaned `trigger-dev-run-worker` (and indexer) processes were caught in
an `uncaughtException` feedback loop: a periodic IPC send via
`process.send` would throw `ERR_IPC_CHANNEL_CLOSED` once the parent
closed the channel, which re-entered the same handler that itself called
`process.send`, scheduled via `setImmediate` and amplified by
source-map-support's `prepareStackTrace`. Fixed by (1) silently dropping
packets in `ZodIpcConnection` when the channel is disconnected, (2)
adding a `process.on("disconnect", ...)` handler in dev workers so they
exit cleanly when the CLI closes the IPC channel, and (3) wrapping all
`uncaughtException`-path `process.send` calls in a `safeSend` guard that
checks `process.connected` and swallows synchronous throws.
([#3491](https://github.com/triggerdotdev/trigger.dev/pull/3491))
- Fail attempts on uncaught exceptions instead of hanging to
`MAX_DURATION_EXCEEDED`. A Node `EventEmitter` (e.g. `node-redis`)
emitting `"error"` with no `.on("error", ...)` listener escalates to
`uncaughtException`, which the worker previously reported but did not
act on — runs drifted to maxDuration with empty attempts. They now fail
fast with the original error and status `FAILED`, and respect the task's
normal retry policy. You should still attach `.on("error", ...)`
listeners to long-lived clients to handle errors gracefully.
([#3529](https://github.com/triggerdotdev/trigger.dev/pull/3529))
-   Updated dependencies:
    -   `@trigger.dev/core@4.4.6`
    -   `@trigger.dev/build@4.4.6`
    -   `@trigger.dev/schema-to-json@4.4.6`

## @trigger.dev/core@4.4.6

### Patch Changes

- Fix dev workers spinning at 100% CPU after the parent CLI disconnects.
Orphaned `trigger-dev-run-worker` (and indexer) processes were caught in
an `uncaughtException` feedback loop: a periodic IPC send via
`process.send` would throw `ERR_IPC_CHANNEL_CLOSED` once the parent
closed the channel, which re-entered the same handler that itself called
`process.send`, scheduled via `setImmediate` and amplified by
source-map-support's `prepareStackTrace`. Fixed by (1) silently dropping
packets in `ZodIpcConnection` when the channel is disconnected, (2)
adding a `process.on("disconnect", ...)` handler in dev workers so they
exit cleanly when the CLI closes the IPC channel, and (3) wrapping all
`uncaughtException`-path `process.send` calls in a `safeSend` guard that
checks `process.connected` and swallows synchronous throws.
([#3491](https://github.com/triggerdotdev/trigger.dev/pull/3491))
- Fail attempts on uncaught exceptions instead of hanging to
`MAX_DURATION_EXCEEDED`. A Node `EventEmitter` (e.g. `node-redis`)
emitting `"error"` with no `.on("error", ...)` listener escalates to
`uncaughtException`, which the worker previously reported but did not
act on — runs drifted to maxDuration with empty attempts. They now fail
fast with the original error and status `FAILED`, and respect the task's
normal retry policy. You should still attach `.on("error", ...)`
listeners to long-lived clients to handle errors gracefully.
([#3529](https://github.com/triggerdotdev/trigger.dev/pull/3529))

## @trigger.dev/python@4.4.6

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.4.6`
    -   `@trigger.dev/build@4.4.6`
    -   `@trigger.dev/sdk@4.4.6`

## @trigger.dev/react-hooks@4.4.6

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.4.6`

## @trigger.dev/redis-worker@4.4.6

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.4.6`

## @trigger.dev/rsc@4.4.6

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.4.6`

## @trigger.dev/schema-to-json@4.4.6

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.4.6`

## @trigger.dev/sdk@4.4.6

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.4.6`

</details>

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-05-12 11:33:00 +01:00
James Ritchie 1e4b896c30 Fix(webapp): Notification style updates (#3553)
### Style updates to the notifications
- Tightened up the typography
- Brighter background to make it stand out a bit more
- A bit more padding to make it more readable
- Show the close button on hover instead
- Turned the notification into a separate component as it's shared on
the admin page modal
- Minor tweaks to the behavior of toggling the notification beween
open/closed side menu states

### Before
<img width="224" height="313" alt="before"
src="https://github.com/user-attachments/assets/c9a9377c-4a3b-4477-921a-3c86385d3f0b"
/>

### After (with image)
<img width="239" height="284" alt="CleanShot 2026-05-11 at 17 22 01"
src="https://github.com/user-attachments/assets/311b4dbc-4853-4e6c-9f83-8173b38bd466"
/>

### After (no image)
<img width="239" height="189" alt="after"
src="https://github.com/user-attachments/assets/884e062b-3608-4cb3-a462-d50597257753"
/>

---------

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
2026-05-12 11:23:36 +01:00
Eric Allam 2301ed608c refactor(run-engine): make taskIdentifier optional on run-queue messages (#3559)
## Summary

Make `taskIdentifier` optional on the run-queue message schema. No
behavior change in this PR; readers continue to accept payloads that
include the field. A separate change will stop writing it on the wire to
shrink the per-run payload that lives in Redis while runs wait to be
dequeued.

## Design

The field is written into every payload at enqueue time but no consumer
reads it back on the dequeue path. Both the run-engine and supervisor
derive `taskIdentifier` from the loaded `TaskRun` row instead. Relaxing
the schema first means readers tolerate payloads that omit it, so the
writer-side change can ship without producing schema-parse errors during
a rolling deploy.

`projectId` is left required: `WorkerQueueResolver.#getOverride` reads
it for project-scoped runtime worker-queue overrides.

## Test plan

- [x] `pnpm run typecheck --filter @internal/run-engine`
- [x] `pnpm run typecheck --filter webapp`
- [x] `pnpm run test ./src/run-queue/tests/enqueueMessage.test.ts
./src/run-queue/tests/workerQueueResolver.test.ts --run` (28/28 passing)
2026-05-12 11:03:34 +01:00
Eric Allam 5f1a3e1653 refactor(run-engine): route TTL expiration through the batch path only (#3554)
## Summary

TTL expiration on queued runs was being scheduled twice: once via a
per-run `expireRun` worker job (the original implementation) and once
via the batch TTL system (added more recently). Both paths attempt to
flip the same run to `EXPIRED`. The per-run job almost always won the
race, leaving the batch consumer to observe runs already expired by the
older path.

This collapses TTL expiration onto the batch path so every queued TTLed
run goes through a single Redis-backed sorted set + batch consumer
instead of also getting its own scheduled redis-worker job.

## Design

`engine.trigger` and `delayedRunSystem.enqueueDelayedRun` no longer call
`ttlSystem.scheduleExpireRun`. The remaining `enqueueSystem.enqueueRun({
includeTtl: true })` already adds the run to the TTL sorted set;
`TtlSystem.expireRunsBatch` flips it to `EXPIRED` when the TTL fires.

Delayed runs get the same coverage by passing `includeTtl: true` on
their post-delay enqueue, so the TTL is armed from the moment the run
enters the queue (matching how the old job behaved —
`parseNaturalLanguageDuration` is evaluated at enqueue time).

The new path explicitly does not re-expire runs once they have been
allocated a concurrency slot. That is intentional: TTL is for runs that
are queued and have never started. Once a run has a slot it is on its
way to executing.

## Test plan

- [x] `pnpm run test --filter @internal/run-engine
./src/engine/tests/ttl.test.ts` — 15 tests, including a new "Re-enqueued
runs are not expired by TTL once they have started" that locks in the
queued-and-never-started contract.
- [x] `pnpm run test --filter @internal/run-engine
./src/engine/tests/delays.test.ts` — 5 tests, including "Delayed run
with a ttl" which now also asserts the TTL is armed from queue-enter
time, not `createdAt`.
- [x] `pnpm run test --filter @internal/run-engine
./src/engine/tests/lazyWaitpoint.test.ts` — 12 tests.
- [x] `pnpm run typecheck --filter @internal/run-engine`.
2026-05-12 07:58:42 +01:00
Iss 2b845455b0 feat(webapp): admin back-office editors for org max projects and batch rate limit (#3475)
## Summary

- Adds admin-only editors on the back-office org page for
`Organization.maximumProjectCount` and
`Organization.batchRateLimitConfig`, alongside the existing API rate
limit editor.
- Splits the back-office org page into per-section components
(`ApiRateLimitSection`, `BatchRateLimitSection`, `MaxProjectsSection`)
so each tool is self-contained — adding new sections later doesn't bloat
the route.
- Generalizes the rate-limit form into a reusable `RateLimitSection`
component + `RateLimitDomain` server config so API and batch share the
same UI, validation, and action handler. Each domain only owns its env
defaults, DB column, and logger key.
- "Saved." banner and validation errors are scoped to the section that
submitted, not the page.

Heads-up: the API rate-limit log key was renamed
`admin.backOffice.rateLimit` → `admin.backOffice.apiRateLimit` for
symmetry with the new `admin.backOffice.batchRateLimit`.

## Test plan

- [ ] As an admin, visit `/admin/back-office/orgs/:orgId` and confirm
all three sections render with the org's current values (or system
defaults).
- [ ] Edit and save each section; confirm only that section shows the
"Saved." banner.
- [ ] Submit invalid input (e.g. `0` tokens, malformed interval);
confirm errors render in the offending form only and the other sections
stay closed.
- [ ] Confirm a non-admin user is redirected away from the route.
- [ ] After saving a rate-limit override, hit the org with traffic and
confirm the new limit is enforced (API rate limit + batch rate limit
code paths read the column at request time).
2026-05-11 10:19:41 -04:00
Eric Allam a5ba406530 feat(webapp,redis): handle UNBLOCKED during ElastiCache role change (#3549)
## Summary

When ElastiCache demotes a primary to replica — during a Multi-AZ
failover or a vertical node-type change — the demoting primary issues an
`UNBLOCKED` reply to any in-flight blocking commands (`BLPOP`, `BRPOP`,
`BLMOVE`, `XREADGROUP ... BLOCK`, etc.) to clear them before the role
flips. ioredis surfaces these as `ReplyError` to caller code.

The shared `defaultReconnectOnError` added in #3548 only matches
`READONLY` and `LOADING`. This extends it to `UNBLOCKED` so the
disconnect-reconnect-retry cycle handles BLPOP-shaped errors the same
way the existing two cases handle non-blocking-command errors.

## Fix

```ts
export function defaultReconnectOnError(err: Error): boolean | 1 | 2 {
  const msg = err.message ?? "";
  if (
    msg.startsWith("READONLY") ||
    msg.startsWith("LOADING") ||
    msg.startsWith("UNBLOCKED")
  ) {
    return 2;
  }
  return false;
}
```

Returning `2` tells ioredis to disconnect, reconnect, and re-issue the
command. For a BLPOP that means a fresh BLPOP against the new primary
instead of the `UNBLOCKED` error escaping to the caller.

## Test plan

- [ ] CI green
- [ ] Trigger a Multi-AZ failover or a vertical scale event on an
ElastiCache replication group whose clients are running blocking
commands and confirm no `UNBLOCKED` errors surface to caller code during
the cutover.
2026-05-11 11:02:40 +01:00
Eric Allam 567e2a2c32 feat(webapp,redis): handle READONLY / LOADING during ElastiCache failover (#3548)
## Summary

During an ElastiCache role swap (failover) or node-type change (vertical
scale), the ioredis TCP/TLS connection stays open but the server starts
answering with `READONLY` (the client is talking to a node that became a
replica) or `LOADING` (node still loading data from disk). Without an
explicit hook, those errors surface to caller code as `ReplyError`
instances — every write op on the affected connection fails until the
cluster fully cuts over.

This PR adds `reconnectOnError` to every prod ioredis client so the
disconnect + reconnect + retry cycle absorbs these errors and caller
code never sees them.

## Fix

```ts
export function defaultReconnectOnError(err: Error): boolean | 1 | 2 {
  const msg = err.message ?? "";
  if (msg.startsWith("READONLY") || msg.startsWith("LOADING")) return 2;
  return false;
}
```

Returning `2` tells ioredis to disconnect, reconnect, and re-issue the
failed command. After reconnect, DNS / SG state routes the new socket to
a writable node.

The helper lives in `@internal/redis` and is wired into both the shared
`createRedisClient` (which covers RunQueue, schedule-engine,
redis-worker, and every other internal-package consumer) and the direct
`new Redis(...)` call sites in the webapp.

V1-only marqs files are intentionally not migrated.

## Test plan

- [x] `pnpm run typecheck --filter webapp`
- [x] `pnpm run typecheck --filter @internal/run-engine`
- [x] Verified end-to-end against a live ElastiCache vertical-scale
event — caller-surfaced errors went from tens of thousands during the
cutover window down to a handful per ioredis client
- [ ] Confirm steady-state behavior unchanged after deploy
2026-05-11 07:17:07 +01:00
James Ritchie 6cdd8814a3 fix(webapp): Fix for resizable side panel getting stuck at its min-size (#3538)
## Summary

- Run-view inspector panel was glitching out on Firefox: visual flicker
on close, locking up at min size, and intermittent `panelHasSpace`
invariant errors. Root cause is the underlying `react-window-splitter`
library's collapse animation, which uses `@react-spring/rafz` and
interacts poorly with Firefox.
- Disabled the library's collapse animation on Firefox only, app-wide
(every consumer of `RESIZABLE_PANEL_ANIMATION`). Chromium and Safari
behaviour is unchanged.

## Changes

- **Firefox animation skip** in `RESIZABLE_PANEL_ANIMATION` —
UA-detected at module load, resolves to `undefined` for Firefox so the
library's animation actor completes in one frame instead of running its
rAF loop.
- **Inspector min raised 50px → 250px** so dragging can't shrink the
panel into a near-useless width.
- **`autosaveId` bumped `v2` → `v3`** to invalidate stale persisted
snapshots (the library has a `// TODO` branch that ignores prop changes
for already-registered panels, so existing users would otherwise still
see the old 50px min).
- **`react-window-splitter` pinned** to exact `0.4.1` to protect the
patch from drifting if line offsets change in a patch release.
- **Two hunks added to the existing `@window-splitter/state` patch:**
- Removed the library's auto-collapse-on-drag block entirely. Every
collapsible panel in the app is parent-controlled, and that block was
triggering state-machine deadlocks when handlers were no-ops.
Drag-to-collapse is now disabled across the app; collapse is only
triggered explicitly (close button, ESC, URL change, etc.).
- In `getDeltaForEvent`, fall back to the panel's `default` before its
`min` when expanding — so the first ever click on a span opens the
inspector at 500px, not 250px.

## Local testing confirmed

- [x] Firefox: open a run, click various spans → panel opens instantly
at 500px, drags freely between 250px and max, closes instantly to 0. No
console errors.
- [x] Chrome/Chromium: same flow, but with smooth open/close animation
as before.
- [x] Safari: same as Chrome.
- [x] Reload mid-session → panel restores cleanly to the dragged size.
- [x] Other resizable panels in the app (logs, deployments, schedules,
batches, bulk-actions, runs index) still animate on Chromium/Safari.

## Notes

- Linear: TRI-8584
- Branch contains intermediate commits exploring an unsuccessful
snapshot-validator approach; they're reverted by the final commit.
Cumulative diff is 6 files. Squash on merge if you'd prefer a clean
history.

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 21:31:01 +01:00
Eric Allam ead1e5a53d feat(webapp): reload LLM pricing registry on Redis pub/sub (#3534)
## Summary

Adds a Redis pub/sub reload path to the webapp's in-memory LLM pricing
registry. When enabled on a process, the registry reloads from the
database whenever a publish lands on the configured channel — instead of
waiting for the existing 5-minute interval. Lets pricing/model changes
propagate to cost enrichment within seconds.

Subscription is **off by default** and opt-in per process. Only
OTel-ingesting services need real-time freshness; dashboard and worker
services run fine on the periodic interval and shouldn't pile onto each
publish with a full-table reload.

## Design

When `LLM_PRICING_RELOAD_PUBSUB_ENABLED=true`, subscribes via
`createRedisClient` against `COMMON_WORKER_REDIS_*` and listens on
`LLM_PRICING_RELOAD_CHANNEL` (default `llm-registry:reload`). The
5-minute periodic reload stays as a backstop, and a SIGTERM/SIGINT
handler closes the subscription cleanly.

The publisher side lives outside this PR — any process running in the
same Redis namespace can trigger a reload by `PUBLISH
llm-registry:reload <anything>`. Includes a `.server-changes/` note for
the changelog.

### Debounced reload

Bursts of publishes are coalesced. The first publish schedules a reload
at T+`LLM_PRICING_RELOAD_DEBOUNCE_MS` (default 1s); subsequent publishes
during that window are no-ops because the trailing reload picks up
everything when it queries the DB. Bounds reload rate to at most 1 per
debounce window regardless of publisher chattiness, so a runaway
upstream publisher can't fan out into a flood of full-table-scan
reloads.

## Test plan

- [ ] With `LLM_PRICING_RELOAD_PUBSUB_ENABLED=false` (default):
`redis-cli PUBSUB NUMSUB llm-registry:reload` returns `0` while the
webapp is up
- [ ] With it set to `true`: returns `>= 1`
- [ ] `redis-cli PUBLISH llm-registry:reload test` returns `1` (one
subscriber received) on a subscribed process
- [ ] Mutate an `LlmModel` row externally, publish on the channel,
observe the registry's match() picks up the change without waiting for
the 5-min tick
- [ ] Publish 100x in rapid succession; confirm only one reload fires
within the debounce window
2026-05-09 08:59:40 +01:00
James Ritchie f7a2bc7c96 Feat(webapp) filters UX update (#3451)
## Lots of filter UX improvements across lots of routes

### General
- Promoted important filters out of the "More filters" so they're always
visible
- SearchInput primitive is now reusable and Esc now clears the field (AI
filter input also clears with Esc)
- Tooltips + keyboard shortcuts on every primary filter button
- Brighter text on selected filter items / queue items 
- Filter dropdowns reordered for better hierarchy
- Removed debounce on Tasks page search for faster filtering

### Tasks page search
- Esc now clears the field
- ENTER submits a search to improve performance when you have lots of
tasks


https://github.com/user-attachments/assets/4b30521e-dbc4-4468-b2af-8c85bdfb9002

### Runs filters
- Moves Status and Tasks out of the More filters menu
- "Root only" toggle is set to false when you filter for a Task. This
state isn't stored and flips back to the stored value if filters are
cleared
<img width="1690" height="986" alt="CleanShot 2026-04-26 at 19 24 08@2x"
src="https://github.com/user-attachments/assets/b07da73c-140e-451f-a7bf-c32129317f63"
/>

### Batches filters
- General consistency improvements
<img width="1429" height="948" alt="CleanShot 2026-05-08 at 09 50 35"
src="https://github.com/user-attachments/assets/e5ec267f-2aa3-43ef-991e-93bf01bdaea5"
/>

### Schedules
- General consistency improvements
<img width="1567" height="1141" alt="CleanShot 2026-05-08 at 09 51 11"
src="https://github.com/user-attachments/assets/34b7da88-87c6-4e4d-a70f-fe13ea9f87ec"
/>

### Queues
- General consistency improvements
<img width="824" height="416" alt="CleanShot 2026-05-08 at 09 52 02"
src="https://github.com/user-attachments/assets/b4adc102-8192-4a68-b199-a175c2645a6c"
/>

### Waitpoint tokens
- General consistency improvements
<img width="941" height="363" alt="CleanShot 2026-05-08 at 09 52 19"
src="https://github.com/user-attachments/assets/d43aeb3f-7f80-454d-b183-fd077a4e3ff7"
/>

### Models
- General consistency improvements
<img width="1570" height="509" alt="CleanShot 2026-05-08 at 09 53 17"
src="https://github.com/user-attachments/assets/066d7646-4672-4cae-8ec0-e30a82889914"
/>

### AI metrics
- General consistency improvements
<img width="1568" height="624" alt="CleanShot 2026-05-08 at 09 53 43"
src="https://github.com/user-attachments/assets/fdfc4806-26fa-458d-a5ed-5c226b3bbc9f"
/>

### Logs
- General consistency improvements
<img width="1267" height="752" alt="CleanShot 2026-05-08 at 09 54 30"
src="https://github.com/user-attachments/assets/3e9ba871-b9dd-490e-aded-5d87134fd2bb"
/>

### Errors
- General consistency improvements
<img width="1568" height="670" alt="CleanShot 2026-05-08 at 09 54 50"
src="https://github.com/user-attachments/assets/fdda027a-e24f-4804-b4bb-203a6c2db960"
/>

### Query
- General consistency improvements
- History, Scope, Triggered (date) filters all have shortcut tooltips
- Scope filter now reuses the metrics ScopeFilter component
<img width="1566" height="716" alt="CleanShot 2026-05-08 at 09 55 22"
src="https://github.com/user-attachments/assets/0130b4a2-9daf-4edc-bada-3380aff4022a"
/>

### Dashboards
- General consistency improvements
- Scope filter gets nicer icons and a shortcut
- Nice icons for the Scope menu items
<img width="1567" height="769" alt="CleanShot 2026-05-08 at 09 56 10"
src="https://github.com/user-attachments/assets/7bea25f7-6c33-4d4a-a36d-3a1cb56afe09"
/>

### Custom dashboard
- General consistency improvements
- Add chart, Add title, and the kebab menu now have tooltips + shortcuts
<img width="1566" height="782" alt="CleanShot 2026-05-08 at 09 58 11"
src="https://github.com/user-attachments/assets/9df4db25-b2c0-43a2-b92f-00256337d5a9"
/>

### Environment variables
- General consistency improvements
<img width="1569" height="930" alt="CleanShot 2026-05-08 at 09 58 55"
src="https://github.com/user-attachments/assets/26e614b4-88e7-400b-aa6d-a96bad488fb8"
/>

### Preview branches
- General consistency improvements
<img width="1570" height="986" alt="CleanShot 2026-05-08 at 09 59 17"
src="https://github.com/user-attachments/assets/57a2b939-3670-4252-ab2c-d6dc65bdda1b"
/>
2026-05-08 17:35:09 +01:00
Iss 3cb6b5e9c4 docs(bun): note WebSocket limitation with remote browser connections (#3537) 2026-05-08 17:25:28 +01:00
Iss f8ddb766fa feat: Plain customer cards (#2933) 2026-05-08 11:51:57 -04:00
Daniel Sutton 61ae67cc02 fix(webapp): stop leaking exception messages on 5xx API responses (#3536)
When a webapp API route's catch-all 500 branch handles a non-typed
exception, it returns the raw `error.message` to the caller. If the
exception originates from an internal subsystem (the ORM client, an
infra dependency, etc.) the server-side error string is surfaced
verbatim in the response body — exposing implementation details the API
surface shouldn't carry.

The leak shows up in three shapes across the routes:

- `return json({ error: error.message }, { status: 500 })`
- `return json({ error: error instanceof Error ? error.message :
"Internal Server Error" }, { status: 500 })`
- ``return json({ error: `Internal server error: ${error.message}` }, {
status: 500 })``

(plus a couple of analogous neverthrow-Result variants on admin routes.)

## Fix

Across 19 webapp routes, replace each leaking branch with a generic body
(`"Something went wrong"` / `"Internal Server Error"` to match the
file's existing fallback) and add `logger.error(...)` so full visibility
is preserved server-side. Catch blocks that branch on typed user-input
errors (`ServiceValidationError`, `EngineServiceValidationError`,
`OutOfEntitlementError`, `PrismaClientKnownRequestError`) are left
intact — those messages are constructed deliberately and intended to be
customer-facing.

## Test plan

- [x] `pnpm run typecheck --filter webapp`
- [x] Per-route manual probe: inject a synthetic `Error` at the top of
the catch'd `try` block (or fake the wrapped call's rejection / Result
error), curl the route with the dev API key, confirm the response body
changed from the synthetic message verbatim → generic body. 21/21 leak
sites verified end-to-end.
- [x] 4xx-typed-error paths spot-checked: throwing
`ServiceValidationError` from inside the catch'd try still surfaces its
message at 422 as intended.
2026-05-08 16:25:53 +01:00
Daniel Sutton 749dc467f1 feat(webapp): link Sentry events to OTel traces via trace_id (#3531)
## Summary

Stamps the active OpenTelemetry `trace_id` and `span_id` onto every
Sentry event captured from the webapp, so engineers can copy a
`trace_id` from a Sentry issue and search for the corresponding trace in
any OTel-aware backend. Also adds an `otel_sampled` tag to indicate
whether the trace was head-sampled — a cheap signal for whether the link
will resolve to span data or hit a missing trace.

## Why

Sentry and OTel were OTel-disconnected: `apps/webapp/sentry.server.ts`
initialised Sentry with `skipOpenTelemetrySetup: true`, and no
error-capture site (`logger.server.ts`, the Remix-wrapped `handleError`,
the root `ErrorBoundary`) attached OTel context to the event. With many
spans/sec across services, getting from a Sentry issue to its trace was
guesswork.

## Approach

Single global Sentry event processor, registered immediately after
`Sentry.init`. On each event it reads
`trace.getActiveSpan()?.spanContext()` via `@opentelemetry/api`, then
writes:

- `event.contexts.trace.trace_id` and `event.contexts.trace.span_id`
(Sentry's native trace context fields)
- `event.tags.otel_sampled` = `"true"` | `"false"` (derived from
`traceFlags`)

If no active span (module-load errors, scheduled timers without a
context, primary cluster process), the processor returns the event
unmodified — Sentry's default propagation context fills in.

Implementation is co-located in `apps/webapp/sentry.server.ts` (no
separate helper module — `sentry.server.ts` is built standalone by
esbuild and a separate import would have required a new bundling step).
Helper functions are exported so the unit tests can reach them without
re-running `Sentry.init`.

## Non-goals (deliberate)

- No sample rate change. ~95% of Sentry events will carry a `trace_id`
that returns no spans in the tracing backend (head-sampled out). The
`otel_sampled` tag makes that obvious at a glance. Raising find-rate is
a separate conversation with cost trade-offs.
- No user/org tags or `Sentry.setUser` (would need auth-helper +
per-request scope wiring across multiple worker entrypoints — separate
ticket).
- Webapp image only. No changes to supervisor or CLI workers.

## Test plan

- [x] Unit tests in `apps/webapp/test/sentryTraceContext.server.test.ts`
— 9 tests covering: helper returns \`undefined\` with no active span;
returns \`traceId\`/\`spanId\`/\`sampled=true\` for a recording span;
returns \`sampled=false\` for a non-recording span; processor leaves the
event unchanged with no active span; processor stamps
\`trace_id\`/\`span_id\` onto \`contexts.trace\`; preserves existing
\`contexts.trace\` fields; tags \`otel_sampled\` correctly for both
sampled and non-sampled cases; never throws if \`@opentelemetry/api\`
access throws.
- [x] \`pnpm run typecheck --filter webapp\` passes.
- [x] Manually verified end-to-end against a sandboxed Sentry project:
confirmed both sampled and non-sampled traces correctly populate
\`contexts.trace.trace_id\` matching the OTel ids logged from the
loader, and the \`otel_sampled\` tag appears with the expected value.

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-08 11:47:24 +01:00
Eric Allam 3e6458f9b5 ci(claude): switch Claude Code actions to ANTHROPIC_API_KEY (#3532)
## Summary

Both Claude Code workflows (`claude.yml` and `claude-md-audit.yml`)
authenticated via `CLAUDE_CODE_OAUTH_TOKEN`, which broke when the org
disabled Claude subscription access for Claude Code:

> Your organization has disabled Claude subscription access for Claude
Code · Use an Anthropic API key instead, or ask your admin to enable
access

This switches both workflows to `anthropic_api_key: ${{
secrets.ANTHROPIC_API_KEY }}` (secret already added to the repo).

## Test plan

- [ ] Confirm `📝 CLAUDE.md Audit` runs to completion on this PR
- [ ] Confirm `@claude` mention in a PR comment still triggers the
`Claude Code` workflow successfully
2026-05-07 11:52:58 +01:00
Matt Aitken 62e006617e fix(cli): fail attempt on uncaught exception instead of hanging to maxDuration (TRI-9117) (#3529)
⚒️ Publish Worker (v4) / build (supervisor) (push) Has been cancelled
When a Node EventEmitter (e.g. node-redis) emits an "error" event with
no
listener attached, Node escalates it to process.on("uncaughtException")
in
the task worker. The worker reported the error via the
UNCAUGHT_EXCEPTION
IPC event but did not exit, and the supervisor-side handler in
taskRunProcess only logged the message at debug level — leaving the
run()
promise orphaned until maxDuration fired and producing empty attempts
(durationMs=0, costInCents=0).

The supervisor now rejects the in-flight attempt with an
UncaughtExceptionError and gracefully terminates the worker (preserving
the OTEL flush window) on UNCAUGHT_EXCEPTION. The attempt fails fast
with
TASK_EXECUTION_FAILED, surfacing the original error name, message, and
stack trace, and falls under the normal retry policy. This mirrors the
existing indexing-side behavior in indexWorkerManifest. Apply the same
handling to unhandled promise rejections, which Node already routes
through uncaughtException by default.
2026-05-06 19:35:43 +01:00
Saadi Myftija 6e8b039a4e ci: GHCR commit-SHA tag, OCI labels, and build provenance (#3528)
- Tags webapp images by full commit SHA on `main` pushes
(`ghcr.io/triggerdotdev/trigger.dev:<sha>`) so any commit can be
resolved to a digest easily.
- Adds OCI labels (`source`, `revision`, `version`, `created`) so
`docker inspect`, vulnerability scanners, and
registry browsers see source/commit/version directly.
- Signs each pushed digest with SLSA build provenance via
`actions/attest-build-provenance@v4.1.0` (pinned by SHA), enabling `gh
attestation verify oci://...` against the source commit and workflow.
2026-05-06 09:40:15 +02:00
Eric Allam 31999afcaa perf(webapp): trim BackgroundWorker.metadata to the schedule slice on create (#3525)
Large deploys (projects with many tasks or source files) blocked the
webapp event loop for several seconds inside Prisma's client-side
serializer on `BackgroundWorker.create`, tail-latencying every other
in-flight request on the same Node process. The `metadata` JSON column
was being written with the full deploy manifest — every task's config,
every queue and prompt, and the full source of every file — all of which
already live on dedicated columns or in dedicated tables.

Fix: project the manifest to `{ packageVersion, contentHash, tasks: [{
id, filePath, schedule }] }` on insert. The only post-write read site is
`changeCurrentDeployment`, which feeds `tasks[].schedule` into
`syncDeclarativeSchedules` at deploy promotion. The retained top-level
keys and per-task `filePath` are kept solely so
`BackgroundWorkerMetadata.safeParse` still succeeds on read.

## Test plan

- [ ] Deploy a project with declarative schedules; verify schedules are
created on first deploy
- [ ] Modify / remove schedules across subsequent deploys; verify sync
- [ ] Roll back to a previous deploy; verify `changeCurrentDeployment`
re-syncs schedules
- [ ] Inspect `BackgroundWorker.metadata` on a fresh deploy — should be
a small object, not the full manifest
2026-05-05 14:52:36 +01:00
Daniel Sutton 14920ce2c4 fix(webapp): downgrade expected user-input error logs to warn (#3523)
`dac9c83bd` added `ignoreErrors: /^ServiceValidationError(?::|$)/` in
`apps/webapp/sentry.server.ts` to drop SVEs before they reach Sentry.
The
filter only matches when the captured event's *type* is
`ServiceValidationError`, but nine call sites in the webapp catch SVE
(and
analogous user-input error types — `OutOfEntitlementError`,
`CreateDeclarativeScheduleError`, `QueryError`) and call
`logger.error("wrapper message", { error: e })` *before* the type check.
The captured event is then titled with the wrapper message, with the
inner
error buried in `extra.error` — invisible to the SDK filter. Result: a
steady stream of expected user-input failures escalating as
`error`-level
events when they should be `warn`.

Each catch block now type-discriminates first, logs expected types at
`warn`,
and keeps unknown-error fall-throughs at `error`. For service sites that
wrap into SVE (`createBackgroundWorker`,
`createDeploymentBackgroundWorkerV4`),
the inner error is logged at `error` before wrapping — mirrors the
`waitpointCompletionPacket.server.ts` pattern from `dac9c83bd`.

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 11:31:39 +01:00
nicktrn a8966a40ab fix(helm): bump clickhouse subchart 9.3.7 -> 9.4.4 (clickhouse 25.7.5) (#3524)
Fixes #3520. The bundled bitnami clickhouse subchart was pinned at
`9.3.7` (clickhouse `25.6.1-debian-12-r0`), which hits a memory-tracker
accounting bug under sustained ingest - the global counter overflows to
~7 EiB and every query gets rejected by OvercommitTracker until the pod
is restarted. Self-hosters running 4.0.5 through 4.4.5 are exposed
regardless of chart version since the subchart pin hadn't moved.

Bumping to `9.4.4` (clickhouse `25.7.5-debian-12-r0`) pulls in the
25.7.x memory-tracker fixes. This is also the latest publicly packaged
release at `oci://registry-1.docker.io/bitnamicharts` - that registry
has been frozen since 2025-08-28 (Bitnami catalog changes), but the
chart source remains under Apache 2 on `bitnami/charts`. The image
continues to resolve via `bitnamilegacy/clickhouse` per the existing
`values.yaml` override, since `bitnami/clickhouse` itself moved to
paid-only.

Verified locally: `helm dependency update` + `helm lint` + `helm
template` + kubeconform across all 57 rendered manifests. Rendered
statefulset image is
`docker.io/bitnamilegacy/clickhouse:25.7.5-debian-12-r0`.
2026-05-05 10:49:54 +01:00
Eric Allam 386b4f65ff feat(webapp): per-org S2 basin migration (#3516)
## Summary

Move from a single shared S2 basin to **per-org basins** with retention
tied to the org's billing plan. Stops S2 from deleting streams out from
under live chat sessions when basin retention fires before the chat
ends, and unlocks per-org cost attribution.

OSS / s2-lite installs are unaffected: provisioning is gated by
`REALTIME_STREAMS_PER_ORG_BASINS_ENABLED` (default `false`), and the
read precedence falls back to the global basin env var when an entity
has no stamped basin.

```
basin = run.streamBasinName ?? session.streamBasinName ?? env.REALTIME_STREAMS_S2_BASIN
```

## Design

Three nullable `streamBasinName` columns (`Organization`, `TaskRun`,
`Session`) plus a provisioner that idempotently creates the basin and
reconfigures retention on plan changes. The trigger and session-create
paths stamp the org's basin onto new rows; the realtime read path picks
the basin from the entity context.

Admin routes back-fill existing orgs and force-reconfigure a single org.

## Test plan

- [x] `pnpm run typecheck --filter webapp --filter @internal/run-engine`
- [x] Backfill admin route end-to-end (provision + DB stamp + S2 basin
config).
- [x] Reconfigure on plan change (all retention tiers).
- [x] chat.agent multi-turn drives streams into the per-org basin.
- [x] Legacy fallback when entity has no stamped basin.
- [x] Provisioner is a no-op when the flag is off.
2026-05-05 10:06:58 +01:00
nicktrn 3d418a9482 ci: add zizmor workflow security scanner (#3506)
Adds zizmor alongside the actionlint job from #3503. Both now run as
parallel jobs in a single `.github/workflows/workflow-checks.yml`,
triggered on `.github/workflows/**` and `.github/actions/**` changes.

Zizmor is configured with `unpinned-uses: hash-pin` policy via
`.github/zizmor.yml`, so any future unpinned action will fail CI.
Findings upload SARIF to the Security tab alongside CodeQL.

Bulk of the diff is cleanup of the findings zizmor surfaced on first
run. `zizmor --fix=all` handled most of them mechanically; the rest were
judgment calls.
2026-05-05 09:20:08 +01:00
Oskar Otwinowski 5dab2ae714 docs(private links): refresh PrivateLink setup screenshots, add ElastiCache IP-finding tip and NLB inbound-rules step (#3517) 2026-05-04 16:33:56 +02:00
James Ritchie 45ec23cc73 feat(webapp): app auto session logout (#3473)
<img width="2284" height="2028" alt="CleanShot 2026-05-01 at 18 53
50@2x"
src="https://github.com/user-attachments/assets/4f58cbb1-0168-40fb-a523-017f2ba625a1"
/>


## Performance
- **Per-request DB hit**: `getUserId` runs `getEffectiveSessionDuration`
(User lookup + Org `aggregate`) on *every* authenticated request,
including each fetcher poll. Consider caching the effective duration in
the session cookie with a short TTL (e.g. 60s) and revalidating in the
background.
- **Double session commit in `root.tsx`**: `getUser` already runs the
expiry check; then `commitAuthenticatedSessionLazy` commits the cookie
again. Fine, but doubles `Set-Cookie` headers on every page load — worth
a quick perf check.

## Correctness / Edge cases
- **Lazy backfill assumes a root.tsx hit first**: users whose first
post-deploy request is a fetcher/API route (`/resources/*`) skip the
backfill until they navigate to a page. Not a security hole, but
`getUserId` could backfill itself for completeness.
- **No upper bound on `Organization.maxSessionDuration`**: admin API
accepts `1` second, which would instant-logout every member on next
request. Add a `min(60)` (or `min(300)` to match the lowest user option)
to the Zod schema.
- **No clock-skew tolerance**: `isSessionExpired` is exact-millisecond.
Multi-instance deploys with skewed clocks could log users out a few
seconds early/late. Probably fine for the 5-min minimum, but worth
noting.

## Security
- **Auto-logout audit log lacks IP/orgId**: HIPAA forensics typically
wants source IP and which org context. Currently logs only `userId` +
path. IP isn't PII for audit purposes; orgIds help correlate. Add both.
- **Cookie `Max-Age` is 1 year regardless of user's setting**:
intentional (server-side `issuedAt` is the source of truth), but
reviewers will ask. Add a one-line comment on the cookie config
explaining why.

## API surface
- **`maxSessionDuration` is admin-PAT only**: no in-app UI for org
owners to set/change their own cap. If this is "Trigger staff sets it
during HIPAA onboarding", say so in the PR description; otherwise add an
org-settings UI.
- **Auto-submit dropdown has no confirmation**: misclicking "5 minutes"
immediately shortens the user's session window with no undo. Consider a
save button or 3-sec undo toast.

## Schema / migration
- **`User.sessionDuration NOT NULL DEFAULT 31556952`**: instant on PG
11+ (metadata-only), but call out in the PR description so reviewers
don't worry about a table rewrite on the User table.
- **No DB-level constraint matching `SESSION_DURATION_OPTIONS`**: if the
option list changes, existing users keep orphaned values. The dropdown's
tag-along behaviour hides this — fine for now, but if you ever drop an
option you'll need a backfill.

## UX
- **Session expiry only fires on next request**: an idle authenticated
tab keeps showing UI past the cap (until SSE/polling catches it, ~60s).
Add a client-side timer based on the user's effective duration that
triggers a fetcher to `/account` or `/logout` at expiry.
- **No "you were signed out" message on logout**: users hitting their
cap are bounced to `/` with no explanation. Was intentionally reverted
in this PR — call that out so reviewers don't request it.

## Tests
- Unit coverage on `sessionDuration.server.ts` is solid (215 lines).
Missing: integration test for `getUserId` → expired session → redirect
to `/logout`, and one for the loader's clamping fix (the most recent
bug). Add at least the second one to lock in the regression.

---------

Co-authored-by: Matt Aitken <matt@mattaitken.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-04 15:02:26 +01:00
nicktrn 8c56d85f90 fix(helm): supervisor OTLP endpoint resolves cross-namespace (#3504)
Reported by external contributor. The supervisor template hardcoded a
short DNS name for `OTEL_EXPORTER_OTLP_ENDPOINT`, which the supervisor
then propagates verbatim into runner pods
(`apps/supervisor/src/workloadManager/kubernetes.ts:196`). When runners
are spawned in a different namespace via
`supervisor.config.kubernetes.namespace`, the short name doesn't resolve
and span/log export silently fails - runs complete fine but the
dashboard shows nothing.

Same FQDN pattern the chart already uses for
`TRIGGER_WORKLOAD_API_DOMAIN` (line 203). Verified with `helm template
trigger . --namespace my-ns` - renders
`http://trigger-webapp.my-ns.svc.cluster.local:3030/otel`.

Cheers Niels
2026-05-02 09:58:27 +01:00
Eric Allam dc98ae4e28 chore: clean up stranded .server-changes/ files from v4.4.5 (#3509)
## Summary

Delete 34 `.server-changes/*.md` files that should have been cleaned up
automatically when v4.4.5 (#3406) was merged but were stranded by a
workflow race.

## Why these are stale

The `update-lockfile` job in `.github/workflows/changesets-pr.yml` is
what cleans up consumed `.server-changes/*.md` files on the release
branch. When v4.4.5 was merged on 2026-05-01, the post-merge workflow
run on `main` failed at `pnpm install --frozen-lockfile` (stale lockfile
in the merge commit), and `cancel-in-progress: true` cancelled the
in-flight run from the previous push — so `update-lockfile` never
reached the cleanup step.

Result: the 34 files described changes that v4.4.5 already shipped, and
they were re-appearing in the v4.4.6 release PR (#3501) under "Server
changes" plus showing up as deletions in its diff.

## What this PR keeps

- `fix-rollback-schedule-sync.md` — genuinely new for v4.4.6 (#3468),
the only server change introduced after v4.4.5
- `README.md`, `.gitkeep` — directory infrastructure
- `dev-cli-disconnect-md` — leaving alone (typo'd filename from March,
no `.md` extension, not picked up by the cleanup glob anyway)

## After merge

The next run of `changesets-pr.yml` will refresh #3501 with a "Server
changes" section that only lists the v4.4.6 entry, and the only
`.server-changes/` deletion in its diff will be
`fix-rollback-schedule-sync.md`.

## Related

- #3505 is the proper underlying fix — collapses the three-job graph
into a single atomic commit by `changesets/action` so this race can't
strand the cleanup again. This PR is just the one-time catch-up for the
files that already got stranded.
2026-05-02 09:55:08 +01:00
nicktrn cad8791859 chore: make changeset:version atomic (#3505)
Follow-up to the v4.4.5 release incident where the release PR (#3406)
was merged with a stale lockfile and stale Chart.yaml, breaking npm +
helm releases. The two automation jobs (`update-lockfile`,
`bump-chart-version`) got cancelled mid-flight by `cancel-in-progress`
when the merge fired the workflow again on `main`.

This restructures `changeset:version` so all the post-version-bump
fixups happen in the same script and end up in a single atomic commit on
`changeset-release/main`, via `changesets/action`'s normal commit step.

Pattern borrowed from Cloudflare workers-sdk, Astro, shadcn/ui.

## Before

```
push: main
└── release-pr (changeset version → bumps package.jsons, opens PR)
    └── update-lockfile (separate job, separate commit)
        └── bump-chart-version (separate job, separate commit)
```

Three jobs, three commits to the release branch.

## After

```
push: main
└── release-pr
    └── changesets/action runs:
          changeset version
          pnpm install --lockfile-only
          node scripts/bump-helm-chart.mjs
          node scripts/cleanup-server-changes.mjs
        ...all staged and committed as ONE commit by the action
```

One job, one commit.
2026-05-02 09:45:44 +01:00
nicktrn b19cf6df25 ci: add actionlint workflow (#3503)
Adds an `actionlint` job that runs on changes to `.github/workflows/**`
and `.github/actions/**`. Catches workflow bugs at PR time — expression
typos, deprecated runner labels, broken matrices, and shellcheck issues
in `run:` blocks.

Run from the official `docker://rhysd/actionlint` image, digest-pinned
alongside everything else.

Existing workflows had 6 shellcheck findings, all fixed.
2026-05-01 18:22:50 +01:00
nicktrn 57cca979c6 docs: refresh compute private beta page with may 1 updates (#3502)
Updates the compute private beta page with the May 1 release entry, plus
a deploy-time warning when `us-east-1-next` is the project default.

The new What's new entry, verbatim:

### May 1, 2026

- **Cold starts are faster across all machine sizes.** Every preset
starts faster, including `micro` and `small-1x` - there's no longer a
cold-start penalty for picking a smaller machine.
- **First runs after a deploy are faster on every preset.** Boot
snapshot creation is significantly quicker across the board, so the cold
path is consistently snappier.
- **`large-1x` and `large-2x` no longer hard-fail.** They're still not
recommended - cold-start performance trails the smaller presets and
we're ironing out reliability issues.

Follow-up to #3472 and #3479.
2026-05-01 17:32:13 +01:00
Eric Allam b65a04eb37 fix(cli,core): stop dev workers spinning at 100% CPU after parent CLI disconnect (#3491)
Orphaned `trigger-dev-run-worker` processes were pinning CPU at 100%
after the dev CLI exited — stuck in an uncaughtException feedback loop
where a closed IPC channel kept throwing `ERR_IPC_CHANNEL_CLOSED` back
into a handler that itself called `process.send`.

Fix:
- `ZodIpcConnection` no-ops sends when the channel is disconnected.
- Dev workers exit on `process.disconnect` instead of being re-parented
to init.
- All worker `uncaughtException` handlers route through a `safeSend`
guard so the handler can never re-enter itself.

Verified end-to-end: `kill -9` of the dev CLI now cleans up all child
workers within ~2s.
2026-05-01 17:11:06 +01:00
nicktrn 706a0b88c9 chore: upgrade pnpm to 10.33.2 with security hardening (#3489)
## Summary

- Upgrade pnpm from 10.23.0 → 10.33.2 (latest minor)
- Enable `blockExoticSubdeps: true` for supply-chain defense
- Update all version references across the repo

## Security improvements in 10.28.2+

- Path traversal protection in `directories.bin`
- Symlink-escape protection for `file:/git:` dependencies (prevents
reading `/etc/passwd`, `~/.ssh/...`)
- https://pnpm.io/settings#blockexoticsubdeps

## Files updated

- `package.json` — `packageManager` field
- `docker/Dockerfile` — 5 `corepack prepare` calls
- `apps/supervisor/Containerfile` — 1 `corepack prepare` call
- `pnpm-workspace.yaml` — added `blockExoticSubdeps: true`
- `CLAUDE.md`, `AGENTS.md`, `CONTRIBUTING.md`, `ai/references/repo.md` —
version references

## Verification

- `pnpm install --frozen-lockfile` succeeds (no lockfile regen needed)
- `pnpm install` (plain) produces zero lockfile diff
- All CI checks pass

Slack thread:
https://triggerdotdev.slack.com/archives/C061L2MHW93/p1777625600974279?thread_ts=1777622248.762639&cid=C061L2MHW93

https://claude.ai/code/session_01G759MUqmjsPh9k1qDxbdjG

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-05-01 16:24:26 +01:00
nicktrn 159408074a ci: swap buildjet/setup-node for actions/setup-node v6.4.0 (#3497)
Last action still firing the Node 20 deprecation warning after #3494.
`buildjet/setup-node@v4.0.4` (the latest tag) declares `runs: using:
'node20'` and the repo hasn't shipped a node24 update.

Workflows here run on `ubuntu-latest` (not buildjet runners), so the
buildjet fork wasn't giving us anything we don't get from
`actions/setup-node` directly. Swapping to `actions/setup-node@v6.4.0`
(node24 runtime) silences the warning.
2026-05-01 16:19:03 +01:00
Eric Allam 04bdf4b90b perf(webapp): throttle PAT + OAT lastAccessedAt writes to once per 5 min (#3493)
## Summary

Each successful PAT (`PersonalAccessToken`) or OAT
(`OrganizationAccessToken`) authentication issues a `prisma.X.update({
lastAccessedAt: new Date() })` to bump the timestamp. For tokens used at
high frequency (CLI clients, integrations) this generates a per-request
DB write that is mostly redundant — the `lastAccessedAt` field is only
surfaced on the settings page so users can decide which tokens to
revoke, and "within the last 5 minutes" is plenty of granularity for
that.

## Design

Replace each unconditional `update` with a conditional `updateMany`
whose `WHERE` requires the existing `lastAccessedAt` to be `NULL` or
strictly older than 5 minutes:

```ts
await prisma.personalAccessToken.updateMany({
  where: {
    id: personalAccessToken.id,
    OR: [
      { lastAccessedAt: null },
      { lastAccessedAt: { lt: new Date(Date.now() - PAT_LAST_ACCESSED_THROTTLE_MS) } },
    ],
  },
  data: { lastAccessedAt: new Date() },
});
```

The conditional runs inside the SQL `UPDATE`, so concurrent auths can't
race into a double-write.

No schema change. No migration. No new infrastructure. Throttle is a
hardcoded constant (`5 * 60 * 1000`) — easy to revisit.

## Test plan

- [x] `pnpm run typecheck --filter webapp`
- [x] `pnpm vitest run ./test/services/personalAccessToken.test.ts
./test/services/organizationAccessToken.test.ts` — 6/6 pass, verifying
the throttle `WHERE` clause is constructed correctly and the `update` is
skipped on token-not-found / wrong-prefix paths
2026-05-01 16:15:51 +01:00
nicktrn 1acdc506ea chore: bump helm chart version to 4.4.5 (#3500)
🧭 Helm Chart Release / lint-and-test (push) Has been cancelled
🧭 Helm Chart Release / release (push) Has been cancelled
Follow-up to v4.4.5 release. The `bump-chart-version` job on the release
PR was cancelled before it could run, so Chart.yaml was merged still
pointing at 4.4.4. The helm release job ([failed
run](https://github.com/triggerdotdev/trigger.dev/actions/runs/25218553990/job/73947054128))
caught it via its version-match guard.

Once this merges I'll re-run the helm release workflow manually.
2026-05-01 16:13:32 +01:00
devin-ai-integration[bot] 30bd567d48 fix: sync declarative schedules on deployment rollback (#3468)
##  Checklist

- [x] I have followed every step in the [contributing
guide](https://github.com/triggerdotdev/trigger.dev/blob/main/CONTRIBUTING.md)
- [x] The PR title follows the convention.
- [x] I ran and tested the code works

---

## Testing

- Reviewed the code flow for deployment rollback
(`ChangeCurrentDeploymentService`) and confirmed it was missing schedule
sync
- Verified all 4 callers of `ChangeCurrentDeploymentService` (UI
rollback, UI promote, API promote, finalize deployment) are now covered
- Ran `pnpm run typecheck --filter webapp` — passes cleanly

---

## Changelog

When rolling back (or manually promoting) a deployment, declarative
schedules were not being synced to match the target deployment's worker
metadata. Schedules remained as configured by the most recent deployment
rather than reflecting the target version's schedule configuration.

This fix adds a call to `syncDeclarativeSchedules` in
`ChangeCurrentDeploymentService` after the deployment promotion is
updated. It parses the target deployment's stored
`BackgroundWorkerMetadata` to restore the correct schedule state. This
covers both rollback and promote paths (UI and API). Errors are handled
gracefully so they don't block the deployment change itself.

---

## Screenshots

N/A — backend-only change.

💯

Link to Devin session:
https://app.devin.ai/sessions/0debf012b58c4132be778f8ea88cd2b6

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: nick <55853254+nicktrn@users.noreply.github.com>
2026-05-01 15:39:19 +01:00
Eric Allam 139cccf27e fix: update pnpm-lock.yaml for v4.4.5 release (#3498)
🚀 Publish Trigger.dev Docker / typecheck (push) Failing after 0s
🚀 Publish Trigger.dev Docker / publish-webapp (push) Has been skipped
🚀 Publish Trigger.dev Docker / publish-worker (push) Has been skipped
🚀 Publish Trigger.dev Docker / publish-worker-v4 (push) Has been skipped
🚀 Publish Trigger.dev Docker / units (push) Failing after 0s
## Summary

The v4.4.5 release PR (#3406) was merged before the automated
lockfile-update job in
[\`changesets-pr.yml\`](.github/workflows/changesets-pr.yml) could push
its commit. As a result main now has \`package.json\` bumped to
\`4.4.5\` but \`pnpm-lock.yaml\` still pinned to \`4.4.4\`.

This blocks every subsequent \`pnpm install --frozen-lockfile\` run,
including:
- \`release.yml\` for v4.4.5 publish ([run
#25217579660](https://github.com/triggerdotdev/trigger.dev/actions/runs/25217579660))
— never published packages to npm
- \`changesets-pr.yml\` on the next push to main ([run
#25217579645](https://github.com/triggerdotdev/trigger.dev/actions/runs/25217579645))

## Root cause (from CI logs)

\`\`\`
ERR_PNPM_OUTDATED_LOCKFILE Cannot install with "frozen-lockfile" because
pnpm-lock.yaml is not up to date with <ROOT>/packages/build/package.json
- @trigger.dev/core (lockfile: workspace:4.4.4, manifest:
workspace:4.4.5)
\`\`\`

Regenerated via \`pnpm install --lockfile-only\` against current main.
The diff is exactly what the canceled \`update-lockfile\` job would have
produced:

- 12 \`workspace:4.4.4\` → \`workspace:4.4.5\` specifier bumps
- pnpm metadata refresh (deprecation annotations on transitive deps, one
optional \`bufferutil\` peer resolution on \`react-email\`)

No new direct dependencies, no version drops.

## Follow-ups (separate PRs)

1. **Re-run release.yml** via \`workflow_dispatch\` (\`type: release\`,
\`ref\` = merge commit on main once this lands) to actually publish
4.4.5 to npm.
2. **Workflow fix** to prevent recurrence: fold the lockfile update into
\`changeset:version\` so the \`release-pr\` job creates a single commit
with version bumps + lockfile in sync. Removes the race window where the
release PR is mergeable before \`update-lockfile\` runs.
2026-05-01 15:37:59 +01:00
nicktrn cb94382ffb ci: vouch dependabot[bot] (#3496)
Dependabot's first auto-bump PR (#3495) was auto-closed because
`dependabot[bot]` isn't in the vouch list and isn't exempt from the
require-draft check.

Two changes:

- Add `dependabot[bot]` to `.github/VOUCHED.td` so the vouch check
passes.
- Add `dependabot[bot]` to the require-draft exception in
`vouch-check-pr.yml` (alongside `devin-ai-integration[bot]`) so its PRs
aren't closed for being non-draft.

Without both, dependabot bumps will keep getting closed and we lose the
weekly action update flow that #3494 set up.
2026-05-01 15:30:11 +01:00
github-actions[bot] d825427cbc chore: release v4.4.5 (#3406)
## Summary
8 new features, 18 improvements, 11 bug fixes.

## Breaking changes
- Add server-side deprecation gate for deploys from v3 CLI versions
(gated by `DEPRECATE_V3_CLI_DEPLOYS_ENABLED`). v4 CLI deploys are
unaffected.
([#3415](https://github.com/triggerdotdev/trigger.dev/pull/3415))

## Improvements
- Add `--no-browser` flag to `init` and `login` to skip auto-opening the
browser during authentication. Also error loudly when `init` is run
without `--yes` under non-TTY stdin (previously default-and-exited
silently, leaving the project half-initialized). Both commands now show
an `Examples` section in `--help`.
([#3483](https://github.com/triggerdotdev/trigger.dev/pull/3483))
- Add `isReplay` boolean to the run context (`ctx.run.isReplay`),
derived from the existing `replayedFromTaskRunFriendlyId` database
field. Defaults to `false` for backwards compatibility.
([#3454](https://github.com/triggerdotdev/trigger.dev/pull/3454))
- Redact the `resolveWaitpoint` runtime log so it only emits `id` and
`type` instead of the full completed waitpoint. Previously the log
printed the entire waitpoint (including `output`) to stdout in
production runs, which could leak sensitive payloads. The value returned
by `wait.forToken()` is unchanged.
([#3490](https://github.com/triggerdotdev/trigger.dev/pull/3490))
- Add `SessionId` friendly ID generator and schemas for the new durable
Session primitive. Exported from `@trigger.dev/core/v3/isomorphic`
alongside `RunId`, `BatchId`, etc. Ships the
`CreateSessionStreamWaitpoint` request/response schemas alongside the
main Session CRUD.
([#3417](https://github.com/triggerdotdev/trigger.dev/pull/3417))
- Truncate large error stacks and messages to prevent OOM crashes. Stack
traces are capped at 50 frames (keeping top 5 + bottom 45 with an
omission notice), individual stack lines at 1024 chars, and error
messages at 1000 chars. Applied in parseError, sanitizeError, and OTel
span recording.
([#3405](https://github.com/triggerdotdev/trigger.dev/pull/3405))

## Server changes

These changes affect the self-hosted Docker image and Trigger.dev Cloud:

- Add a "Back office" tab to `/admin` and a per-organization detail page
at `/admin/back-office/orgs/:orgId`. The first action available on that
page is editing the org's API rate limit: admins can save a
`tokenBucket` override (refill rate, interval, max tokens) and see a
plain-English preview of the resulting sustained rate and burst
allowance. Writes are audit-logged via the server logger.
([#3434](https://github.com/triggerdotdev/trigger.dev/pull/3434))
- Optional `DEPLOY_REGISTRY_ECR_DEFAULT_REPOSITORY_POLICY` env var to
apply a default repository policy when the webapp creates new ECR repos
([#3467](https://github.com/triggerdotdev/trigger.dev/pull/3467))
- Ship the Errors page to all users, with a polish + bug-fix pass:
pinned "No channel" item in the Slack alert channel picker,
viewer-timezone alert timestamps via Slack's `<!date^>` token, Activity
sparkline peak tooltip, centered loading spinner and bug-icon empty
state on the error detail page, ellipsis on the Configure alerts
trigger.
([#3477](https://github.com/triggerdotdev/trigger.dev/pull/3477))
- Configure the set of machine presets to build boot snapshots for at
deploy time via `COMPUTE_TEMPLATE_MACHINE_PRESETS` (CSV of preset names,
default `small-1x`). Use `COMPUTE_TEMPLATE_MACHINE_PRESETS_REQUIRED`
(CSV, default = full PRESETS list) to scope which preset failures fail a
required-mode deploy. Optional preset failures are logged and don't
block the deploy.
([#3492](https://github.com/triggerdotdev/trigger.dev/pull/3492))
- Regenerating a RuntimeEnvironment API key no longer invalidates the
previous key immediately. The old key is recorded in a new
`RevokedApiKey` table with a 24 hour grace window, and
`findEnvironmentByApiKey` falls back to it when the submitted key
doesn't match any live environment. The grace window can be ended early
(or extended) by updating `expiresAt` on the row.
([#3420](https://github.com/triggerdotdev/trigger.dev/pull/3420))
- Add the `Session` primitive — a durable, task-bound, bidirectional I/O
channel that outlives a single run and acts as the run manager for
`chat.agent`. Ships the Postgres `Session` + `SessionRun` tables,
ClickHouse `sessions_v1` + replication service, the `sessions` JWT
scope, and the public CRUD + realtime routes (`/api/v1/sessions`,
`/realtime/v1/sessions/:session/:io`) including `end-and-continue` for
server-orchestrated run handoffs and session-stream waitpoints.
([#3417](https://github.com/triggerdotdev/trigger.dev/pull/3417))
- Add `KUBERNETES_POD_DNS_NDOTS_OVERRIDE_ENABLED` flag (off by default)
that overrides the cluster default and sets `dnsConfig.options.ndots` on
runner pods (defaulting to 2, configurable via
`KUBERNETES_POD_DNS_NDOTS`). Kubernetes defaults pods to `ndots: 5`, so
any name with fewer than 5 dots — including typical external domains
like `api.example.com` — is first walked through every entry in the
cluster search list (`<ns>.svc.cluster.local`, `svc.cluster.local`,
`cluster.local`) before being tried as-is, turning one resolution into
4+ CoreDNS queries (×2 with A+AAAA). Using a lower `ndots` value reduces
DNS query amplification in the `cluster.local` zone.
  
Note: before enabling, make sure no code path relies on search-list
expansion for names with dots ≥ the configured value — those names will
hit their as-is form first and could resolve externally before falling
back to the cluster search path.
([#3441](https://github.com/triggerdotdev/trigger.dev/pull/3441))
- Vercel integration option to disable auto promotions
([#3376](https://github.com/triggerdotdev/trigger.dev/pull/3376))
- Make it clear in the admin that feature flags are global and should
rarely be changed.
([#3408](https://github.com/triggerdotdev/trigger.dev/pull/3408))
- Admin worker groups API: add GET loader and expose more fields on
POST. ([#3390](https://github.com/triggerdotdev/trigger.dev/pull/3390))
- Add 60s fresh / 60s stale SWR cache to `getEntitlement` in
`platform.v3.server.ts`. Eliminates a synchronous billing-service HTTP
round trip on every trigger. Reuses the existing `platformCache` (LRU
memory + Redis) pattern already used for `limits` and `usage`. Cache key
is `${orgId}`. Errors return a permissive `{ hasAccess: true }` fallback
(existing behavior) and are also cached to prevent thundering-herd on
billing outages.
([#3388](https://github.com/triggerdotdev/trigger.dev/pull/3388))
- Show a `MicroVM` badge next to the region name on the regions page.
([#3407](https://github.com/triggerdotdev/trigger.dev/pull/3407))
- Increase default maximum project count per organization from 10 to 25
([#3409](https://github.com/triggerdotdev/trigger.dev/pull/3409))
- Merge execution snapshot creation into the dequeue taskRun.update
transaction, reducing 2 DB commits to 1 per dequeue operation
([#3395](https://github.com/triggerdotdev/trigger.dev/pull/3395))
- Add per-worker Node.js heap metrics to the OTel meter —
`nodejs.memory.heap.used`, `nodejs.memory.heap.total`,
`nodejs.memory.heap.limit`, `nodejs.memory.external`,
`nodejs.memory.array_buffers`, `nodejs.memory.rss`. Host-metrics only
publishes RSS, which overstates V8 heap by the external + native
footprint; these give direct heap visibility per cluster worker so
`NODE_MAX_OLD_SPACE_SIZE` can be sized against observed heap peaks
rather than RSS.
([#3437](https://github.com/triggerdotdev/trigger.dev/pull/3437))
- Tag Prisma spans with `db.datasource: "writer" | "replica"` so
monitors and trace queries can distinguish the writer pool from the
replica pool. Applies to all `prisma:engine:*` spans (including
`prisma:engine:connection` used by the connection-pool monitors) and the
outer `prisma:client:operation` span.
([#3422](https://github.com/triggerdotdev/trigger.dev/pull/3422))
- Clarify the cross-region intent in the Terraform and AI-prompt helpers
on the Add Private Connection page. Both already default
`supported_regions` to `["us-east-1", "eu-central-1"]`; added an inline
comment / parenthetical so the user understands why both regions are
listed (Trigger.dev runs in both, so the service must be consumable from
either).
([#3465](https://github.com/triggerdotdev/trigger.dev/pull/3465))
- Add `RUN_ENGINE_READ_REPLICA_SNAPSHOTS_SINCE_ENABLED` flag (default
off) to route the Prisma reads inside `RunEngine.getSnapshotsSince`
through the read-only replica client. Offloads the snapshot polling
queries (fired by every running task runner) from the primary. When
disabled, behavior is unchanged.
([#3423](https://github.com/triggerdotdev/trigger.dev/pull/3423))
- Stop creating TaskRunTag records and _TaskRunToTaskRunTag join table
entries during task triggering. The denormalized runTags string array on
TaskRun already stores tag names, making the M2M relation redundant
write overhead.
([#3369](https://github.com/triggerdotdev/trigger.dev/pull/3369))
- Stop writing per-tick state (`lastScheduledTimestamp`,
`nextScheduledTimestamp`, `lastRunTriggeredAt`) on `TaskSchedule` and
`TaskScheduleInstance`. The schedule engine now carries the previous
fire time forward via the worker queue payload, eliminating ~270K
dead-tuple-driven autovacuums per year on these hot tables and the
associated `IO:XactSync` mini-spikes on the writer. Customer-facing
`payload.lastTimestamp` semantics are unchanged.
([#3476](https://github.com/triggerdotdev/trigger.dev/pull/3476))
- Replace the expensive DISTINCT query for task filter dropdowns with a
dedicated TaskIdentifier registry table backed by Redis. Environments
migrate automatically on their next deploy, with a transparent fallback
to the legacy query for unmigrated environments. Also fixes duplicate
dropdown entries when a task changes trigger source, and adds
active/archived grouping for removed tasks. Moves BackgroundWorkerTask
reads in the trigger hot path to the read replica.
([#3368](https://github.com/triggerdotdev/trigger.dev/pull/3368))
- Public Access Tokens (PATs) minted before an API key rotation now keep
working during the 24h grace window. `validatePublicJwtKey` falls back
to any non-expired `RevokedApiKey` rows for the signing environment when
the primary signature check against the env's current `apiKey` fails.
The fallback query only runs on the failure path, so the hot success
path is unchanged.
([#3464](https://github.com/triggerdotdev/trigger.dev/pull/3464))
- Batch items that hit the environment queue size limit now fast-fail
without
retries and without creating pre-failed TaskRuns.
([#3352](https://github.com/triggerdotdev/trigger.dev/pull/3352))
- Show the cancel button in the runs list for runs in `DEQUEUED` status.
`DEQUEUED` was missing from `NON_FINAL_RUN_STATUSES` so the list hid the
button even though the single run page allowed it.
([#3421](https://github.com/triggerdotdev/trigger.dev/pull/3421))
- Reduce 5xx feedback loops on hot debounce keys by quantizing
`delayUntil`,
  adding an unlocked fast-path skip, and gracefully handling redlock
contention in `handleDebounce` so the SDK no longer retries into a herd.
([#3453](https://github.com/triggerdotdev/trigger.dev/pull/3453))
- Fix RSS memory leak in the realtime proxy routes. `/realtime/v1/runs`,
`/realtime/v1/runs/:id`, and `/realtime/v1/batches/:id` called `fetch()`
into Electric with no abort signal, so when a client disconnected mid
long-poll, undici kept the upstream socket open and buffered response
chunks that would never be consumed — retained only in RSS, invisible to
V8 heap tooling. Thread `getRequestAbortSignal()` through
`RealtimeClient.streamRun/streamRuns/streamBatch` to `longPollingFetch`
and cancel the upstream body in the error path. Isolated reproducer
showed ~44 KB retained per leaked request; signal propagation releases
it cleanly.
([#3442](https://github.com/triggerdotdev/trigger.dev/pull/3442))
- Fix memory leak where every aborted SSE connection pinned the full
request/response graph on Node 20, caused by `AbortSignal.any()` in
`sse.ts` retaining its source signals indefinitely (see
nodejs/node#54614, nodejs/node#55351). Also clear the
`setTimeout(abort)` timer in `entry.server.tsx` so successful HTML
renders don't pin the React tree for 30s per request.
([#3430](https://github.com/triggerdotdev/trigger.dev/pull/3430))
- Preserve filters on the queues page when submitting modal actions.
([#3471](https://github.com/triggerdotdev/trigger.dev/pull/3471))
- Fix Redis connection leak in realtime streams and broken abort signal
propagation.
  
**Redis connections**: Non-blocking methods (ingestData, appendPart,
getLastChunkIndex) now share a single Redis connection instead of
creating one per request. streamResponse still uses dedicated
connections (required for XREAD BLOCK) but now tears them down
immediately via disconnect() instead of graceful quit(), with a 15s
inactivity fallback.
  
**Abort signal**: request.signal is broken in Remix/Express due to a
Node.js undici GC bug (nodejs/node#55428) that severs the signal chain
when Remix clones the Request internally. Added getRequestAbortSignal()
wired to Express res.on("close") via httpAsyncStorage, which fires
reliably on client disconnect. All SSE/streaming routes updated to use
it. ([#3399](https://github.com/triggerdotdev/trigger.dev/pull/3399))
- Prevent dashboard crash (React error #31) when span accessory item
text is not a string. Filters out malformed accessory items in
SpanCodePathAccessory instead of passing objects to React as children.
([#3400](https://github.com/triggerdotdev/trigger.dev/pull/3400))
- Upgrade Remix packages from 2.1.0 to 2.17.4 to address security
vulnerabilities in React Router
([#3372](https://github.com/triggerdotdev/trigger.dev/pull/3372))
- Fix Vercel integration settings page (remove redundant section
toggles) and improve the Vercel onboarding flow so the modal closes
after connecting a GitHub repo and the marketplace `next` URL is
preserved across the GitHub app install redirect.
([#3424](https://github.com/triggerdotdev/trigger.dev/pull/3424))

<details>
<summary>Raw changeset output</summary>

# Releases
## @trigger.dev/build@4.4.5

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.4.5`

## trigger.dev@4.4.5

### Patch Changes

- Add `--no-browser` flag to `init` and `login` to skip auto-opening the
browser during authentication. Also error loudly when `init` is run
without `--yes` under non-TTY stdin (previously default-and-exited
silently, leaving the project half-initialized). Both commands now show
an `Examples` section in `--help`.
([#3483](https://github.com/triggerdotdev/trigger.dev/pull/3483))
-   Updated dependencies:
    -   `@trigger.dev/core@4.4.5`
    -   `@trigger.dev/build@4.4.5`
    -   `@trigger.dev/schema-to-json@4.4.5`

## @trigger.dev/core@4.4.5

### Patch Changes

- Add `isReplay` boolean to the run context (`ctx.run.isReplay`),
derived from the existing `replayedFromTaskRunFriendlyId` database
field. Defaults to `false` for backwards compatibility.
([#3454](https://github.com/triggerdotdev/trigger.dev/pull/3454))
- Redact the `resolveWaitpoint` runtime log so it only emits `id` and
`type` instead of the full completed waitpoint. Previously the log
printed the entire waitpoint (including `output`) to stdout in
production runs, which could leak sensitive payloads. The value returned
by `wait.forToken()` is unchanged.
([#3490](https://github.com/triggerdotdev/trigger.dev/pull/3490))
- Add `SessionId` friendly ID generator and schemas for the new durable
Session primitive. Exported from `@trigger.dev/core/v3/isomorphic`
alongside `RunId`, `BatchId`, etc. Ships the
`CreateSessionStreamWaitpoint` request/response schemas alongside the
main Session CRUD.
([#3417](https://github.com/triggerdotdev/trigger.dev/pull/3417))
- Truncate large error stacks and messages to prevent OOM crashes. Stack
traces are capped at 50 frames (keeping top 5 + bottom 45 with an
omission notice), individual stack lines at 1024 chars, and error
messages at 1000 chars. Applied in parseError, sanitizeError, and OTel
span recording.
([#3405](https://github.com/triggerdotdev/trigger.dev/pull/3405))

## @trigger.dev/python@4.4.5

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.4.5`
    -   `@trigger.dev/build@4.4.5`
    -   `@trigger.dev/sdk@4.4.5`

## @trigger.dev/react-hooks@4.4.5

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.4.5`

## @trigger.dev/redis-worker@4.4.5

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.4.5`

## @trigger.dev/rsc@4.4.5

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.4.5`

## @trigger.dev/schema-to-json@4.4.5

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.4.5`

## @trigger.dev/sdk@4.4.5

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.4.5`

</details>

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-05-01 15:13:11 +01:00
nicktrn ee3887a321 feat(webapp): configurable deploy template machine presets (#3492)
The webapp's compute template creation hardcoded a single machine preset
(`small-1x`) at deploy time, regardless of which presets a project
actually uses. Tasks running on any other preset paid full cold-snapshot
creation cost on first run.

Two new env vars:

- `COMPUTE_TEMPLATE_MACHINE_PRESETS` - CSV of preset names to build boot
snapshots for during deploy. Defaults to `small-1x` so existing deploys
don't change behavior.
- `COMPUTE_TEMPLATE_MACHINE_PRESETS_REQUIRED` - CSV of presets whose
failure fails a required-mode deploy. Defaults to the full `PRESETS`
list. Optional preset failures are logged but don't block the deploy.

The compute client now sends the multi-config request shape; the service
evaluates per-preset outcomes against the required set and surfaces a
combined failure message when a required preset fails.

Both env vars are validated at boot via the env schema - unknown preset
names or `_REQUIRED` entries that aren't a subset of `_PRESETS` fail
loudly at startup rather than silently per-deploy.
2026-05-01 15:10:26 +01:00
nicktrn 39baea8094 ci: pin actions to SHAs and add dependabot config (#3494)
Most actions in this repo were several major versions behind, which is
why every CI run has been emitting Node 20 deprecation warnings.

Pinning every action to a commit SHA (with the version as a trailing
comment) means each CI run uses the exact code that was reviewed when
the bump landed, instead of whatever a maintainer last pointed the major
tag at. Dependabot is configured to group all action bumps into one
weekly PR with a 7-day cooldown.

Worth flagging:

- The Claude Code action ships ~daily but the model is set separately
via `--model` in `claude_args`, so SHA-pinning the action gives
reproducibility without locking the model.
- The kubeconform container is digest-pinned
(`docker://image:tag@sha256:...`). Dependabot's github-actions ecosystem
doesn't track `docker://` references ([explicit TODO in
dependabot-core](https://github.com/dependabot/dependabot-core/blob/main/github_actions/lib/dependabot/github_actions/file_parser.rb)),
so it needs manual bumps either way - but the digest pin protects
against tag repointing for free.
2026-05-01 14:08:43 +00:00
Matt Aitken 7c7d785552 Don't log waitpoint output when resolving (#3490)
Redact the `resolveWaitpoint` runtime log so it only emits `id` and
`type` instead of the full completed waitpoint. Previously the log
printed the entire waitpoint (including `output`) to stdout in
production runs, which could leak sensitive payloads. The value returned
by `wait.forToken()` is unchanged.
2026-05-01 12:37:57 +01:00
DKP 1dfd595986 fix(webapp): invalid HTML nesting in errors Activity tooltip (#3488)
The Activity peak count tooltip in the errors list rendered a `<button>`
(from `SimpleTooltip`'s default `TooltipTrigger`) inside the row's `<a>`
link (`TableCell to={errorPath}`). Interactive content nested inside
other interactive content is invalid HTML and triggers accessibility
warnings. Adding `asChild` to `SimpleTooltip` makes the existing
`<span>` the trigger directly, removing the nested `<button>`.
2026-05-01 10:31:25 +01:00
nicktrn e2b9e0f9f5 feat(cli-v3): add --no-browser flag and examples to init/login --help (#3483)
Closes the most common friction point hit while setting up a fresh
project from an agent harness: the CLI auto-opens the user's default
browser during auth and there is no supported way to skip it (the
existing `isLinuxServer()` path only triggers when `xdg-open` is missing
entirely).

`--no-browser` on `login` and `init` prints the URL and waits to be
visited from any browser. The flag threads through the embedded
`login()` call inside `init`.

While here:
- `init` now errors loudly when stdin is non-TTY without `--yes` instead
of default-and-exiting silently at the first prompt (which left the
project half-initialized: deps installed, no config or example file).
- Both commands gain an `Examples` block in `--help` rendered between
the description and the arguments/options list, so `--help | head`
surfaces the common invocations.

Other commands also call `login()` embedded and would benefit from
`--no-browser` too, but kept this PR scoped to the cases the friction
log called out.
2026-05-01 09:39:22 +01:00
Eric Allam ac7177d61f feat(schedule-engine): stop persisting per-tick schedule state (#3476)
## Summary

Each scheduled-task tick previously issued **3 Prisma `UPDATE`s**
against
`TaskSchedule.lastRunTriggeredAt`,
`TaskScheduleInstance.lastScheduledTimestamp`,
and `TaskScheduleInstance.nextScheduledTimestamp`. All three were pure
denormalization — every value can be derived without persisting.

After this PR `TaskSchedule` and `TaskScheduleInstance` become **near
read-only**:
writes happen only on schedule create / update / delete (rare admin
actions),
so the per-tick autovacuum churn on these hot tables disappears.

## Design

The previous fire time travels forward through the **schedule worker
payload**,
not through the database. Concretely:

- The `schedule.triggerScheduledTask` worker payload gains an optional
  `lastScheduleTime: z.coerce.date().optional()` field.
- When the engine fires a schedule, it re-enqueues the next tick with
  `lastScheduleTime = scheduleTimestamp` (the just-fired time).
- When the next tick dequeues, `payload.lastTimestamp` is sourced from
`params.lastScheduleTime` directly. No DB round-trip, no cron-derivation
  drift across DST boundaries, no caveats around recently-edited cron
  expressions.

`payload.lastTimestamp` keeps its `Date | undefined` SDK shape.
First-ever
fires still report `undefined`, so customer `if
(!payload.lastTimestamp)`
first-run patterns keep working.

For Redis jobs that were enqueued **before** this change (which lack
`lastScheduleTime` in their payload), the engine falls back to
`instance.lastScheduledTimestamp` once. Once those drain, the column is
never read again. Revert is code-only; the columns stay in place and can
be dropped in a follow-up once the rollout is stable.

## Files

- `internal-packages/schedule-engine/*` — engine refactor,
`workerCatalog`
schema field, `TriggerScheduleParams` extension, tests updated to assert
  on the worker-payload flow rather than DB readbacks.
- `internal-packages/database/prisma/schema.prisma` — `/// @deprecated`
  triple-slash docstrings on the three columns. No migration.
- `apps/webapp/app/presenters/v3/ScheduleListPresenter.server.ts` —
drops
the `lastRunTriggeredAt` Prisma select; "Last run" cell is approximated
from the cron expression's previous slot, gated on `schedule.createdAt`
so brand-new schedules show "–". UI is best-effort; the runs page is the
  source of truth.
- `apps/webapp/app/v3/utils/calculateNextSchedule.server.ts` — adds a
  `previousScheduledTimestamp` helper for the UI cell above. Public API
responses (`api.v1.schedules.*`) already compute `nextRun` from cron and
  don't expose `lastTimestamp` — no public API change.
- `references/scheduled-tasks/` — new reference project with declarative
  schedules at multiple cadences and three throw-on-fail validators
(`first-fire-detector`, `interval-validator`, `upcoming-validator`) for
  E2E-verifying the worker-payload flow.

Refs TRI-8891

## Test plan

- [x] `pnpm run typecheck --filter @internal/schedule-engine --filter
webapp`
- [x] `pnpm run build --filter @trigger.dev/core`
- [x] `pnpm run test --filter @internal/schedule-engine` — integration
test
asserts first-fire `lastTimestamp === undefined`, second fire carries
      the previous fire's timestamp exactly.
- [x] E2E against local webapp via `references/scheduled-tasks`:
- Fresh schedules attached → all three deprecated columns stay `NULL`
after
    multiple fires.
  - Redis payload at second fire contains
    `"lastScheduleTime":"<previous fire timestamp>"`.
- `TaskRun.payload` and the every-minute task's returned output both
confirm
`lastTimestamp = null` on first fire and `lastTimestamp = <prev fire>`
on
    second fire, exactly 60s apart.
  - All three throw-on-FAIL validators completed successfully on every
    non-first fire.
- [x] Schedules REST API end-to-end (`POST` / `GET` / `PUT` / `activate`
/
`deactivate` / `DELETE`) — `nextRun` recomputed live from cron + tz on
      every response, no reads of deprecated columns.
2026-05-01 08:22:39 +01:00
DKP 19c16759f6 feat(webapp): errors page polish and GA rollout (#3477)
## What this does

Polish + bug-fix pass on the Errors page so it can ship to everyone.
Touches the Slack alert config UX, errors list, error detail page, and
unhides the SideMenu entry for non-admins.
## Decisions

**"No channel" item over standalone Remove button**
Chose pinning a `<XMarkIcon /> No channel` `SelectItem` above the
channel list. Rejected the standalone "Remove channel" link in a
`<Hint>` — color/hover behaviour clashed with the sibling `<TextLink>`,
and "channel selection" is the right context for clearing. Server action
already deletes the channel when `slackChannel=""` is submitted.

**Slack `<!date^>` token over per-user TZ field for alerts**
Chose Slack's native `<!date^TS^…>` token so each viewer sees timestamps
in their own timezone (UTC fallback). Rejected per-user/per-org TZ
schema work — works for multi-region channels for free. Email/dashboard
TZ source-of-truth filed as TRI-8885 / TRI-8886.

**Make errors GA**
2026-04-30 17:56:51 +01:00
nicktrn 24de77c4ab docs: call out compute private beta limitations (#3479)
Updates the private beta page with current caveats so beta orgs aren't
surprised.

Refs TRI-8900.
2026-04-30 17:40:10 +01:00
Eric Allam 04b4d85f50 fix(webapp): allow JWT auth on POST /api/v1/sessions (#3474)
## Summary

`POST /api/v1/sessions` was secret-key-only because the customer browser
flow runs through `chat.createStartSessionAction` (server-side, holds
the secret key). But the `cli-v3` MCP `start_agent_chat` tool is itself
a server-side surface — developer's CLI/IDE acting as their own server —
and only holds a JWT minted from the user's PAT. Without JWT support on
this route the entire MCP agent toolkit (`start_agent_chat`,
`send_agent_message`, `close_agent_chat`) is blocked at session
creation.

Add `allowJWT: true` plus an `authorization` block requiring the
`write:sessions` (or `admin`) super-scope.

## Why a wildcard `sessions` resource

Resource scoping by `taskIdentifier` isn't possible at auth-resolve time
— action routes don't pass `body` to the `resource` callback, and the
task name only lives in the body. So the resource is `sessions: "*"` and
the super-scope does the actual gating. The JWT-issuer (cli-v3 MCP,
customer servers wrapping their own auth helpers, etc.) decides which
scopes to mint, which is where per-task narrowing lives.

## Test plan

- [x] Verified end-to-end against local:
`mcp__trigger__start_agent_chat` → `send_agent_message("pong")` →
`send_agent_message("echo")` → `close_agent_chat` all succeed. Two
assistant turns reuse the same runId (continuation in the idle window).
- [ ] Browser-mediated `chat.createStartSessionAction` flow continues to
work unchanged (still uses secret-key path under the hood).
- [ ] Loader (GET) and other session routes — unchanged, no scope drift.

## Notes

This unblocks T17 in the [ai-chat e2e smoke
catalog](https://github.com/triggerdotdev/trigger.dev/blob/feature/tri-7532-ai-sdk-chat-transport-and-chat-task-system/.claude/skills/ai-chat-e2e/SMOKE-TESTS.md)
(which lives in the feature branch's skill catalog, not this repo).
Pairs with the cli-v3 MCP fix on the feature branch (`feat: AI SDK
custom useChat transport & chat.task harness`, PR #3173) — that PR's
`agentChat.ts` change makes the call shape correct (`taskIdentifier` +
`triggerConfig`); this PR opens the door for the JWT to actually pass.
2026-04-30 09:56:45 +01:00
ThullyoCunha f1736595cd feat(webapp): apply default repository policy on ECR repo creation (#3467)
🚀 Publish Trigger.dev Docker / units (push) Failing after 13m3s
🚀 Publish Trigger.dev Docker / typecheck (push) Failing after 13m3s
🚀 Publish Trigger.dev Docker / publish-webapp (push) Has been skipped
🚀 Publish Trigger.dev Docker / publish-worker (push) Has been skipped
🚀 Publish Trigger.dev Docker / publish-worker-v4 (push) Has been skipped
## Summary

Self-hosters that operate the webapp's ECR account separately from the
account running the EKS workers (e.g., a shared platform account that
hosts the registry plus per-team accounts that host clusters) currently
hit a 403 Forbidden the first time **any** project is deployed:

```
Failed to pull image "<acct-A>.dkr.ecr.<region>.amazonaws.com/<namespace>/proj_…:…":
unexpected status from HEAD request to .../v2/.../manifests/sha256:…: 403 Forbidden
```

`ensureEcrRepositoryExists` in
`apps/webapp/app/v3/getDeploymentImageRef.server.ts` calls
`CreateRepository` and `PutLifecyclePolicy`, but never
`SetRepositoryPolicy` — so the new repo inherits the AWS default (only
the registry-owner account can read/pull). Workers in the cluster
account get 403 every single deploy. The only workarounds today are
running a one-off post-create script or pre-creating every repo by hand.

## Proposed change

Add an optional env var:

```
DEPLOY_REGISTRY_ECR_DEFAULT_REPOSITORY_POLICY  (V4 mirror: V4_DEPLOY_REGISTRY_ECR_DEFAULT_REPOSITORY_POLICY)
```

Raw IAM policy JSON. When set, the webapp calls `SetRepositoryPolicy`
immediately after `CreateRepository` so every new repo carries that
policy from creation. Operators control the principal/actions; we don't
bake in any opinions about cross-account boundaries.

Example value (for the typical self-host case — grant pull to the
cluster account):

```json
{
  "Version": "2012-10-17",
  "Statement": [{
    "Sid": "AllowClusterAccountPull",
    "Effect": "Allow",
    "Principal": {"AWS": "arn:aws:iam::<cluster-account-id>:root"},
    "Action": [
      "ecr:GetDownloadUrlForLayer",
      "ecr:BatchGetImage",
      "ecr:BatchCheckLayerAvailability"
    ]
  }]
}
```

## Why env var (not a chart-level field)

- Mirrors the shape of the sibling vars (`DEPLOY_REGISTRY_ECR_TAGS`,
`DEPLOY_REGISTRY_ECR_ASSUME_ROLE_ARN`, etc.) which are already
operator-supplied via `webapp.extraEnvVars` in self-host setups.
- Cloud is unaffected — the env var is optional, unset by default;
existing behavior unchanged.
- Existing repos are unaffected — only newly-created repos get the
policy.
- `RepositoryCreationTemplate` from the AWS provider isn't an
alternative here: it only applies to repos created via
pull-through-cache or replication, not to `ecr:CreateRepository` API
calls.

## Implementation

- `apps/webapp/app/env.server.ts` — declare
`DEPLOY_REGISTRY_ECR_DEFAULT_REPOSITORY_POLICY` and the V4 fallback.
- `apps/webapp/app/v3/registryConfig.server.ts` — propagate
`ecrDefaultRepositoryPolicy` to `RegistryConfig`.
- `apps/webapp/app/v3/getDeploymentImageRef.server.ts` —
`createEcrRepository` accepts the policy; if set, calls
`SetRepositoryPolicy` after `PutLifecyclePolicy`.
- `docs/self-hosting/env/webapp.mdx` — documentation row added under
**Deploy & Registry**.

## Verification

Verified end-to-end against a self-hosted Trigger.dev on EKS where the
ECR account is separate from the cluster account:

- **Without the env var** (current `main`): the new project's first run
pod stays in `ImagePullBackOff` with `403 Forbidden`.
- **With the env var set** to a JSON granting
`ecr:BatchGetImage`/`GetDownloadUrlForLayer`/`BatchCheckLayerAvailability`
to the cluster account: a fresh `trigger.dev deploy --env prod` followed
by a `hello-world` run completes in ~5s end-to-end on the first try.

Manually also confirmed that existing repos are untouched (the call only
fires inside `createEcrRepository`, which only runs when
`DescribeRepositories` returned `RepositoryNotFoundException`).

## Out of scope

- Chart values surface for this — operators already pass the existing
ECR vars via `webapp.extraEnvVars`, so this follows the same pattern.
Happy to add a first-class chart field in a follow-up if that's the
preferred direction.
- IAM-policy validation in the webapp — we forward the JSON verbatim to
AWS and surface AWS's error messages on misuse, matching how
`DEPLOY_REGISTRY_ECR_TAGS` is handled today.

This is a draft pending CI / CodeRabbit pass — happy to iterate on
direction (e.g., split into per-action env vars, or extend the chart
values schema) if any of the above choices feels off.

---------

Co-authored-by: nicktrn <55853254+nicktrn@users.noreply.github.com>
2026-04-29 15:17:23 +01:00
nicktrn 8e368cc3d7 docs: add compute private beta page (#3472) 2026-04-29 13:42:29 +01:00
nicktrn 226b93edf9 fix(webapp): preserve filters on queues page action redirects (#3471)
Queues page action handler was rebuilding the redirect URL with only
`?page=`, so any pause/resume/override modal confirmation wiped the
user's search query. With hundreds of queues filtered down to a handful,
every confirmation dropped you back to the unfiltered list - and
pagination still pointed at the previous numeric page, so you'd land on
a different slice than you came from.

Swap the manual rebuild for `url.search` so the full querystring
(including any future filter params) flows through. Drops the now-unused
`SearchParamsSchema.parse` call inside `action`; the loader still
validates on the way back.
2026-04-29 13:06:14 +01:00
devin-ai-integration[bot] b0131352f6 fix(webapp): constrain usage chart height to 320px (#3469)
##  Checklist

- [x] I have followed every step in the [contributing
guide](https://github.com/triggerdotdev/trigger.dev/blob/main/CONTRIBUTING.md)
- [x] The PR title follows the convention.
- [x] I ran and tested the code works

---

## Testing

Ran the webapp locally with `CLOUD_ENV=development` and verified the
usage page chart height at different viewport sizes. The chart now
renders at a fixed 320px height instead of expanding to fill the
viewport.

---

## Changelog

Fix the "Usage by day" chart on the usage settings page taking up 100%
of the viewport height.

The regression was introduced in PR #2905 when the `UsageChart` was
migrated from using `ChartContainer` directly (with `max-h-96 min-h-40
w-full`) to the new `Chart.Root` compound component. The
`ChartContainer` base class includes `aspect-video` (16:9 ratio), and
the `max-h-96` constraint was lost during migration, causing the chart
to scale its height based on viewport width.

Fix: wrap `Chart.Root` in a fixed-height container (`h-80` = 320px) and
use the `fillContainer` prop, which applies `!aspect-auto` to override
the `aspect-video` ratio.

---

## Screenshots

Before (chart fills entire viewport):

![before](https://app.devin.ai/api/presigned_proxy?token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJvcmdfaWQiOiJvcmctMWYzMWQ4ZTQ4NjI3NGRjYTg5MDM5MGQ4ZWY2YWVjNzQiLCJ1c2VyX2lkIjpudWxsLCJidWNrZXRfbmFtZSI6ImRldmluYXR0YWNobWVudHMiLCJidWNrZXRfa2V5IjoiYXR0YWNobWVudHNfcHJpdmF0ZS9vcmctMWYzMWQ4ZTQ4NjI3NGRjYTg5MDM5MGQ4ZWY2YWVjNzQvOGRhMTdiMDEtZThiOC00MGI4LWIxOGUtNmJmMzc0ZjYzYjMxIiwiaWF0IjoxNzc3NDU1Mjg2LCJleHAiOjE3NzgwNjAwODYsImZpbGVuYW1lIjoiY3VycmVudC1icm9rZW4ucG5nIn0.LU6idghnalKHL2ZAJGVxWTmPnuUgoL-6QNRdk3bDJqg)

After (chart constrained to 320px):

![after](https://app.devin.ai/api/presigned_proxy?token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJvcmdfaWQiOiJvcmctMWYzMWQ4ZTQ4NjI3NGRjYTg5MDM5MGQ4ZWY2YWVjNzQiLCJ1c2VyX2lkIjpudWxsLCJidWNrZXRfbmFtZSI6ImRldmluYXR0YWNobWVudHMiLCJidWNrZXRfa2V5IjoiYXR0YWNobWVudHNfcHJpdmF0ZS9vcmctMWYzMWQ4ZTQ4NjI3NGRjYTg5MDM5MGQ4ZWY2YWVjNzQvMDliN2E2YzktZjIzYi00YTJiLWE0N2EtMTk3NzBhZGU1MmU3IiwiaWF0IjoxNzc3NDU1Mjg2LCJleHAiOjE3NzgwNjAwODYsImZpbGVuYW1lIjoib3B0aW9uLWItaDgwLWZpbGxDb250YWluZXIucG5nIn0.uKi3Yw4Z6GvoNuEVO8B4hwSvvHtyYzXUhtSEHcuSvTY)

💯

Link to Devin session:
https://app.devin.ai/sessions/6e5ed40516d3448db85950feb1115ab3

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: nick <55853254+nicktrn@users.noreply.github.com>
2026-04-29 10:56:35 +01:00
Eric Allam 99dfee3a57 fix(webapp): honor RevokedApiKey grace window for public access tokens (#3464)
## Summary

Follow-up to #3420. PATs (public access tokens) minted before an API key
rotation 401'd immediately on the realtime stream endpoints, even though
the rotation flow advertises a 24h overlap. This fixes the gap.

## Root cause

PATs are JWTs signed with the env's `apiKey` at mint time. When that
secret is rotated, `validatePublicJwtKey`
(`apps/webapp/app/services/realtime/jwtAuth.server.ts`) only verifies
the signature against `environment.parentEnvironment?.apiKey ??
environment.apiKey` — i.e. the env's *current* canonical key. Any PAT in
the wild signed with the previous key fails signature verification →
401, even within the grace window.

#3420 wired up the grace-window fallback in two places —
`findEnvironmentByApiKey` (raw secret-key auth) and `api.v1.auth.jwt.ts`
(signs new JWTs with the canonical key when minting from an old one) —
but the *verify* path for already-issued PATs was never updated.

In a typical app, `POST /api/v1/tasks/.../trigger` (Bearer secret) keeps
working through rotation because that path has the fallback, but `GET
/realtime/v1/streams/run_*/...` and `POST
/realtime/v1/streams/run_*/input/...` 401 for runs that were already in
flight when the rotation happened.

## Fix

After the primary `validateJWT` against the env's current `apiKey`, fall
back to non-expired `RevokedApiKey` rows for the signing env (parent env
when the request is against a child) — but **only on the failure path**,
so the hot success path is unchanged. Uses `$replica` to match the rest
of the auth path.

Symmetrical to the `findEnvironmentByApiKey` two-step from #3420.

## Changes

- `apps/webapp/app/services/realtime/jwtAuth.server.ts` —
`validateAgainstRevokedApiKeys` helper invoked only on `!result.ok`
- `apps/webapp/app/models/runtimeEnvironment.server.ts` —
`findEnvironmentById` also selects `parentEnvironment.id` so we can
scope the revoked-keys lookup to the correct env

## Test plan

E2E verified locally via curl against `GET /realtime/v1/runs/{runId}`
(PAT-authenticated):

- [x] Pre-rotation, PAT signed with K1 → **200** with run body
- [x] Simulate rotation (insert `RevokedApiKey` row + flip env `apiKey`
to K2 in a single transaction, mirroring `regenerateApiKey`)
- [x] Same PAT (K1) within grace window → **200** with run body —
fallback hits
- [x] Fresh PAT signed with K2 → **200** — current key still works
- [x] Set `RevokedApiKey.expiresAt` to past → **401** — fallback finds
no live row
- [x] Bogus signature (no rotation) → **401**
- [x] Cleanup verified: env `apiKey` restored, `RevokedApiKey` row
deleted
- [x] `pnpm run typecheck --filter webapp` passes
2026-04-29 10:00:33 +01:00
Eric Allam dac9c83bdc chore(webapp,run-engine): downgrade boundary log noise to warn (#3462)
## Summary

Several boundary catches and customer-input validation paths were
logging at `error` level for failures the system already handles
gracefully — disconnect on auth failure, return undefined, skip retries,
etc. This batch routes them to `warn` (which stays in stdout) or counts
them as OTel metrics, so visibility is preserved without surfacing them
as alerts.

## Changes

**New helper / pattern:**
- `apiBuilder.server.ts` — `logBoundaryError(message, error, url)`
inspects the inner error type at loader/action boundary catches;
downgrades to `warn` for `AbortError`, `ServiceValidationError`, and
`EngineServiceValidationError`.
- `platform.v3.server.ts` — `platform_client.failures_total` OTel
counter with `{function, kind}` labels; helper
`recordPlatformFailure(fn, kind)` replaces the previous error-level
logging across all `BillingClient` wrappers.

**Log-level downgrades:**
- `handleSocketIo.server.ts` — `Worker authentication failed` → warn
(system disconnects on failure; refs TRI-8863)
- `waitpointSystem.ts` — when `runStatus === "CANCELED"` in the
suspended-without-checkpoint branch, skip the throw and warn instead
(benign cancel-vs-resume race, nothing to resume)
- `runAttemptSystem.ts` — `flushedMetadata` parse/validate failures →
warn (customer-side data shape, system returns gracefully)
- `batch-queue/index.ts` — final-attempt failures with
`result.skipRetries` → warn (callbacks already opted out of retry, e.g.
queue size limit hit)
- `queryPerformanceMonitor.server.ts` — slow queries → warn
(observability signal, not an application error)
- `timeoutDeployment.server.ts` — deployment-state mismatch in the
timeout job → warn (timeout-vs-completion race)

**Inner error preservation:**
- `waitpointCompletionPacket.server.ts` — `logger.error(uploadError)`
before throwing the `ServiceValidationError` wrapper, so the underlying
upload error stays visible.

## Why

The pattern across all of these is the same: a boundary log treated any
thrown/returned error as `error` regardless of cause, even when the
cause was an expected, system-handled condition (client disconnect,
customer quota, race condition, schema validation of customer data).
That made the logs noisy and made it harder to spot real bugs.

Where the underlying signal is still useful operationally (slow queries,
billing call failures), we route it to OTel metrics with low-cardinality
labels so dashboards and alerts can be tuned independently of error
logs.

## Test plan

- [ ] `pnpm run typecheck --filter webapp`
- [ ] `pnpm run build --filter @internal/run-engine`
- [ ] Trigger a run on hello-world and verify task lifecycle is
unaffected
- [ ] Cancel a suspended run and verify the cancel-while-suspended
branch in `waitpointSystem.ts` returns `{status: "skipped"}` instead of
throwing
- [ ] Confirm `platform_client.failures_total` counter shows up in
metrics with `{function, kind}` labels when the billing client errors
2026-04-29 10:00:22 +01:00
Oskar Otwinowski 1a7943ce1b feat(docs): Private Links official documentation (#3466)
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-04-28 21:20:52 +02:00
Oskar Otwinowski 5fe72eefa5 feat(webapp): Private Links setup wizard UI tweaks (#3465) 2026-04-28 21:20:11 +02:00
nicktrn fefe61f006 ci(helm): roll prereleases on main pushes + manual trigger (#3461)
Today the helm prerelease workflow only fires on PRs that touch
`hosting/k8s/helm/**`. Two consequences we ran into:

1. The `changeset-release/main` PR's prerelease comment goes stale once
the release branch gets force-pushed without a helm-touching commit (the
bot's `Chart.yaml` bump alone doesn't seem to refire the trigger
reliably).
2. The release PR's chart references an `appVersion` (e.g. `v4.4.5`)
whose Docker images don't exist until *after* merge + tag. So that
prerelease chart can't actually be installed end-to-end.

Renames the workflow to `helm-prerelease.yml` and adds two new triggers:

- **`push: main`** with `paths: hosting/k8s/helm/**` -> rolling
prereleases versioned `<base>-main.<sha>`. `appVersion` stays at
whatever `Chart.yaml` has (i.e. last released), so installs pull real
images. Tests that chart structure is deployable, even if the app code
is one release behind.
- **`workflow_dispatch`** with optional `app_version` input -> manually
trigger a prerelease and optionally override `appVersion` (e.g. pin to
`main` or a specific tag). Useful for testing chart + app-version
combinations on demand.

PR behavior unchanged: same `<base>-pr<N>.<sha>` versioning, same
posted/updated comment.

Why not also bypass paths for `changeset-release/main`? The release PR's
chart references not-yet-built `v4.4.5` images, so those prereleases
aren't actually installable. The rolling main prerelease covers the
testable case better.

Why not SHA-pin `appVersion` to a built image like `main-<sha>`? Bigger
change - the docker publish workflows currently only push `:main` (no
SHA-suffixed tag). Worth doing later if we want first-class "install one
chart, get exactly that commit's app code" testing, but out of scope
here.

Diff is mostly a rename. Substantive changes:
- new `push` and `workflow_dispatch` triggers
- `prerelease` job `if:` extended for the new event types
- version logic branches per event
- new "Override appVersion" step (workflow_dispatch only)
- new "Write run summary" step so non-PR runs surface the install
instructions
- PR comment steps gated on `github.event_name == 'pull_request'`
- concurrency group falls back to `github.ref` for non-PR runs
2026-04-28 15:42:12 +01:00
Eric Allam c69e939c34 feat: Sessions - bidirectional durable agent streams (#3417)
> ⚠️ **Not released yet.** This PR is the server-side foundation only.
The SDK changes that customers will actually use (`chat.agent`
migration, `chat.createStartSessionAction`, `useTriggerChatTransport`
updates) live on a separate branch and ship together in an upcoming
`@trigger.dev/sdk` prerelease. Until that prerelease is published, this
surface is reachable only via direct HTTP.

## What this gives Trigger.dev users

A new first-class primitive, **Session**, for durable, task-bound,
bidirectional I/O that outlives any single run. Sessions are the run
manager for `chat.agent` going forward, and they unblock anything else
that needs "one identifier, many runs over time" with a stable channel
pair the client can write to and subscribe to.

### Use cases unblocked

- **Chat agents that persist across many runs.** One session per chat
(keyed on your own `chatId` via `externalId`), turns 1..N attach to the
same Session, the UI subscribes once and keeps receiving output as new
runs take over.
- **Approval loops and long-running tasks with user feedback.** The task
waits on `.in`, the client writes to `.in`, the server enforces
no-writes-after-close.
- **Workflow progress streams that live past the run.** Subscribe to
`.out` after the task finishes to replay history.
- **Resume-next-day flows.** A session is a durable row, not a transient
stream. Send a message a day later and the server triggers a fresh run
on the same session.

### How it works (Session-as-run-manager)

A Session row is task-bound (`taskIdentifier` + `triggerConfig` are
required) and owns its current run via `currentRunId` +
`currentRunVersion` for optimistic claim. Three trigger paths:

1. **Session create** — `POST /api/v1/sessions` creates the row and
triggers the first run synchronously.
2. **Append-time probe** — `POST
/realtime/v1/sessions/:session/in/append` checks if the current run is
alive; if it has terminated (idle exit, crash, etc.), the server
triggers a new run before processing the append.
3. **End-and-continue handoff** — `POST
/api/v1/sessions/:session/end-and-continue`, called by the running
agent, triggers a fresh run and atomically swaps `currentRunId`. Used by
`chat.requestUpgrade()` for version handoffs.

Every triggered run is recorded in the `SessionRun` audit table with a
reason (`initial`, `continuation`, `upgrade`, `manual`).

## Public API surface

### Control plane

- `POST /api/v1/sessions` — create. Idempotent on `(env, externalId)`.
Triggers the first run, returns the session and a session-scoped public
access token. Returns 409 if the upserted row is already closed.
- `GET /api/v1/sessions/:session` — retrieve by friendlyId
(`session_abc...`) or by your own externalId (server disambiguates by
prefix).
- `GET /api/v1/sessions` — list with filters (`type`, `tag`,
`taskIdentifier`, `externalId`, derived `status` ACTIVE/CLOSED/EXPIRED,
created-at range) and cursor pagination. Backed by ClickHouse.
- `PATCH /api/v1/sessions/:session` — update tags / metadata /
externalId.
- `POST /api/v1/sessions/:session/close` — terminate. Idempotent,
hard-blocks new server-brokered writes.
- `POST /api/v1/sessions/:session/end-and-continue` — agent-only handoff
to a fresh run.

### Realtime

- `PUT /realtime/v1/sessions/:session/:io` — initialize a channel.
Returns S2 credentials in headers so high-throughput clients can write
direct to S2.
- `GET /realtime/v1/sessions/:session/:io` — SSE subscribe. Supports
Last-Event-ID resume and an opt-in `X-Peek-Settled: 1` header that
fast-closes the stream when the upstream is already settled
(`trigger:turn-complete`), eliminating long-poll wait on
reconnect-on-reload paths.
- `POST /realtime/v1/sessions/:session/:io/append` — server-side
appends.
- `POST /api/v1/runs/:runFriendlyId/session-streams/wait` — runs wait on
a session stream as a waitpoint, with a race-check to avoid suspending
if data already landed.

### Auth scopes

`sessions` is a new resource type. `read:sessions:{id}`,
`write:sessions:{id}`, `admin:sessions:{id}` flow through the existing
JWT validator. Session-scoped public access tokens minted by the server
replace browser-held trigger-task tokens for chat-style flows — the
browser never sees a run identifier or a run-scoped token in steady
state.

## What's coming after this PR

- **SDK + chat.agent migration**: separate branch, separate PR, ships in
the next `@trigger.dev/sdk` prerelease alongside this server deploy.
Customers using the prerelease `chat.agent` will follow the [upgrade
guide](https://github.com/triggerdotdev/trigger.dev/blob/docs/tri-7532-ai-sdk-chat-transport-and-chat-task-system/docs/ai-chat/upgrade-guide.mdx).
- **Dashboard surfaces**: dedicated agent list, agent playground, agent
view on the run dashboard. Tracking separately.

## Implementation notes

- **Postgres `Session` table**: scalar scoping columns (`projectId`,
`runtimeEnvironmentId`, `environmentType`, `organizationId`) without
FKs, matching the January TaskRun FK-removal decision. Point-lookup
indexes only — list queries go to ClickHouse. Terminal markers
(`closedAt`, `expiresAt`) are write-once.
- **ClickHouse `sessions_v1`**: ReplacingMergeTree, partitioned by
month, ordered by `(org_id, project_id, environment_id, created_at,
session_id)`. Tags indexed via `tokenbf_v1` skip index.
- **`SessionsReplicationService`**: mirrors `RunsReplicationService`
exactly — leader-locked logical replication consumer,
`ConcurrentFlushScheduler`, retry with exponential backoff + jitter,
identical metric shape. Dedicated slot + publication so the two consume
independently.
- **S2 keys**: `sessions/{addressingKey}/{out|in}`. The existing
`runs/{runId}/{streamId}` key format for run-scoped streams is
untouched.
- **Optimistic claim**: `ensureRunForSession` triggers a run upfront
(cheap to cancel if it loses the race), then attempts an `updateMany`
keyed on `currentRunVersion`. Loser cancels its triggered run and reuses
the winner's. No DB lock held across the trigger.

### What did NOT change

Run-scoped `streams.pipe` / `streams.input` and the existing
`/realtime/v1/streams/{runId}/...` routes are unchanged. Sessions are
net-new — not a reshaping of the current streams API.

## Deploy notes

- Set `SESSION_REPLICATION_CLICKHOUSE_URL` and
`SESSION_REPLICATION_ENABLED=1` to enable the replication consumer.
- The `Session` table needs `REPLICA IDENTITY FULL` set on the prod
source DB before the publication is created (same one-time DDL we did
for `TaskRun`). Required for delete events to carry full column values.
- Cross-form authorization on the `GET /api/v1/sessions/:session` loader
(a JWT minted for either form authorizes both URL forms). Action routes
are URL-form-specific, matching how the SDK mints PATs.

## Verification

- Webapp typecheck clean (10/10).
- `apps/webapp/test/sessionsReplicationService.test.ts` — round-trip
tests for insert/update/delete through Postgres logical replication into
ClickHouse via testcontainers.
- Live end-to-end against local dev: create + retrieve (both forms) +
update + close, `.out.initialize` + `.out.append` x2 + `.in.send` +
`.out.subscribe` over SSE, list with all filter combinations +
pagination, `end-and-continue` swap, `X-Peek-Settled` fast-close
(verified in browser via reconnect-on-reload and via curl). Replicated
row lands in ClickHouse within ~1s.
- Multi-round Devin + CodeRabbit review feedback addressed
(read-after-write paths use `prisma` writer, info-leak on auth-routes
masked as 403, peek-settled discriminator parsing fix, etc.).

## Test plan

- [ ] `pnpm run typecheck --filter webapp`
- [ ] `pnpm run test --filter webapp
./test/sessionsReplicationService.test.ts --run`
- [ ] Start the webapp with `SESSION_REPLICATION_CLICKHOUSE_URL` and
`SESSION_REPLICATION_ENABLED=1`. Confirm the slot and publication
auto-create on boot.
- [ ] `POST /api/v1/sessions` and verify the row replicates to
`trigger_dev.sessions_v1` within a couple of seconds.
- [ ] `POST /api/v1/sessions/:id/close`, then confirm `POST
/realtime/v1/sessions/:id/out/append` returns 400.
- [ ] Reuse a closed session's `externalId` on `POST /api/v1/sessions`
and confirm 409.
- [ ] `GET /realtime/v1/sessions/:id/out` with `X-Peek-Settled: 1` after
a turn completes and confirm `X-Session-Settled: true` response header +
immediate close.
2026-04-28 12:35:55 +01:00
Eric Allam e134da7306 fix(run-engine): debounce hot-key lock contention and 5xx feedback loop (#3453)
## Changes

Three changes in
`internal-packages/run-engine/src/engine/systems/debounceSystem.ts`, in
order of impact:

1. **Fast-path skip before the lock.** In `handleExistingRun`, do an
unlocked read of `delayUntil` (and `createdAt` for the max-duration
check) from the run row before entering `runLock.lock("handleDebounce",
...)`. If `newDelayUntil <= currentDelayUntil` and the run is still
within its max-duration window, return the existing run immediately
without taking the lock. Safe because debounce is monotonic-forward only
— a stale read either matches reality or undershoots, both of which
decay correctly (re-checked properly inside the lock by whichever caller
is actually pushing forward). Trailing-mode triggers carrying
`updateData` still take the lock so the data update is applied.

2. **Quantize `newDelayUntil`.** Round the computed `newDelayUntil` to
1-second buckets (configurable via `quantizeNewDelayUntilMs`, set to 0
to disable). Without quantization, every call has a slightly larger
`newDelayUntil` than the last and they all pass the fast-path check.
With it, concurrent callers on the same key share a target time and ~95%
short-circuit. User-visible effect: a debounced run might fire up to 1s
earlier than the strict spec — non-issue for typical debounce use cases
(chat summarization, batched notifications, etc.).

3. **Graceful lock-contention fallback.** Wrap the `runLock.lock(...)`
call so `LockAcquisitionTimeoutError` and Redlock `ExecutionError` /
`ResourceLockedError` return the existing run id with success instead of
propagating a 5xx. Debounce is best-effort: if we can't take the lock,
the herd is already updating it for us; fall in line. This kills the 5xx
→ SDK-retry feedback loop. With (1)+(2) this rarely fires; without them
it's the difference between 5xx and 200.

Defaults preserve current behaviour aside from quantization (1s) and
fast-path (on). Both are configurable via `RunEngineOptions.debounce`.

##  Checklist

- [x] I have followed every step in the [contributing
guide](https://github.com/triggerdotdev/trigger.dev/blob/main/CONTRIBUTING.md)
- [x] The PR title follows the convention.
- [x] I ran and tested the code works


---

## Changelog

Reduce 5xx feedback loops on hot debounce keys by quantizing
`delayUntil`, adding an unlocked fast-path skip before the redlock, and
gracefully handling redlock contention in `handleDebounce` so the SDK no
longer retries into a herd.

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-04-28 11:22:00 +01:00
devin-ai-integration[bot] 4b28080ed4 feat: add isReplay to run context (#3454)
## Summary

Adds `isReplay` boolean to the run context (`ctx.run.isReplay`),
following the same pattern as the existing `isTest`. The value is
derived from the existing `replayedFromTaskRunFriendlyId` database
field, so no schema migration is needed.

##  Checklist

- [x] I have followed every step in the [contributing
guide](https://github.com/triggerdotdev/trigger.dev/blob/main/CONTRIBUTING.md)
- [x] The PR title follows the convention.
- [x] I ran and tested the code works

---

## Testing

- Verified `@trigger.dev/core` builds successfully
- Verified `webapp` typechecks successfully
- All new fields use `default(false)` for backwards compatibility

---

## Changelog

- Added `isReplay` to `TaskRun` and `V3TaskRun` schemas in `common.ts`
- Added `RUN_IS_REPLAY` semantic attribute and wired it in `taskContext`
- Propagated `isReplay` through the dequeue system, run attempt system,
and all execution context construction paths (V1 + V2)
- Added `isReplay` to `DequeuedMessage` and
`TaskRunExecutionLazyAttemptPayload` schemas
- Added patch changeset for `@trigger.dev/core`
- Updated docs: added `isReplay` to context reference, added "Detecting
replays" section to replaying page

---

💯

Link to Devin session:
https://app.devin.ai/sessions/1d6f1b3cc39a4623b72d05bf00f2d70c

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: nick <55853254+nicktrn@users.noreply.github.com>
2026-04-28 11:57:44 +02:00
nicktrn 91fd8a8a03 chore(security): close dependabot alerts q2 (#3456)
Closes ~80 dependabot alerts (3 critical, ~25 high, ~31 medium) by
bumping direct deps where possible and narrowly overriding the rest.
Cloud uses `resend` email transport and Node 20 - all bumps are safe for
both cloud and self-hosters.

## Direct upgrades

| Package | Where | From | To | Why |
|---|---|---|---|---|
| `vite` | root devDeps | ^5.4.21 | *(removed)* | dead pin; vitest pulls
vite transitively |
| `dompurify` | apps/webapp | ^3.2.6 | ^3.4.1 | XSS CVEs |
| `effect` | apps/webapp | ^3.11.7 | ^3.21.2 | AsyncLocalStorage CVE in
Effect fibers |
| `nodemailer` | internal-packages/emails | ^7.0.11 | ^8.0.6 | SMTP CRLF
injection (only affects self-hosters w/ smtp/aws-ses transport) |
| `uuid` | apps/webapp | ^9.0.0 | ^14.0.0 | buffer bounds check;
ESM-only but bundled by Remix |
| `uuid` + `@types/uuid` | packages/trigger-sdk | ^9.0.0 | *(removed)* |
dead deps, no usage |
| `@types/uuid` | apps/webapp | ^9.0.0 | *(removed)* | uuid 14 ships its
own types |
| `tar` | packages/cli-v3 | ^7.5.4 | ^7.5.13 | path traversal CVEs |
| `testcontainers` + `@testcontainers/postgresql` +
`@testcontainers/redis` | internal-packages/testcontainers | ^10.28.0 |
^11.14.0 | dev/test cleanup; one-line API fix for
`RedisContainer(image)` |
| `rimraf` | webapp + 6 packages | ^3.0.2 / ^5.0.7 | ^6.0.1 | dev/build
tool consolidation |

## Scoped overrides

All bound by both `>=` and `<` to avoid major-version yanks.

| Override | Closes |
|---|---|
| `tar@>=7 <7.5.11` → `^7.5.11` | supervisor's `@kubernetes/client-node
1.0.0` chain |
| `axios@>=1.0.0 <1.15.0` → `^1.15.0` | replaces older 1.9.0 pin |
| `systeminformation@>=5.0.0 <5.31.0` → `^5.31.0` | bumps existing
5.27.14 pin |
| `lodash@>=4.0.0 <4.18.0` → `^4.18.0` | bumps existing 4.17.23 pin |
| `lodash-es@>=4.0.0 <4.18.0` → `^4.18.0` | new (mirrors lodash) |
| `dompurify@>=3 <3.4.0` → `^3.4.1` | catches transitive dompurify via
mermaid |
| `vite@>=5.0.0 <6.4.2` → `^6.4.2` | path traversal; vite 5 has no patch
|
| `rollup@>=4 <4.59.0` → `^4.59.0` | path traversal in vite/vitest chain
|
| `flatted@>=3 <3.4.2` → `^3.4.2` | prototype pollution in eslint
flat-cache |
| `picomatch@>=2 <2.3.2` → `^2.3.2` | ReDoS in 2.x branch (transitive) |
| `picomatch@>=4 <4.0.4` → `^4.0.4` | ReDoS in 4.x branch
(vitest/tinyglobby) |
| `minimatch@>=3 <3.1.3` → `^3.1.3` | ReDoS in eslint 8 chain |
| `protobufjs@>=7 <7.5.5` → `^7.5.5` | **critical** RCE via
@opentelemetry/otlp-transformer |
| `fast-xml-parser@>=4 <4.5.5` → `^4.5.5` | DOCTYPE bypass + others (4.x
branch via aws-sdk in supervisor) |
| `fast-xml-parser@>=5 <5.7.0` → `^5.7.0` | **critical** + others (5.x
branch via aws-sdk in webapp) |
| `path-to-regexp@>=0.1 <0.1.13` → `^0.1.13` | ReDoS in express 4 /
@remix-run/express |
| `ajv@>=8 <8.18.0` → `^8.18.0` | DoS |
| `socket.io-parser@>=4 <4.2.6` → `^4.2.6` | DoS in @trigger.dev/core's
socket.io |
| `postcss@>=8 <8.5.10` → `^8.5.10` | XSS via stringify |
| `yaml@>=2 <2.8.3` → `^2.8.3` | DoS |
| `semver@>=5 <5.7.2` → `^5.7.2` | ReDoS in 5.x |
| `defu@>=6 <6.1.5` → `^6.1.5` | prototype pollution via __proto__ in
@prisma/config c12 chain |

## Dismissed (~47)

| Reason | Cluster | Count |
|---|---|---|
| `not_used` | langsmith + next 15.x in references/* | 10 |
| `not_used` | minimatch 8.x via prisma-generator-ts-enums
(references/prisma-6) | 3 |
| `not_used` | basic-ftp via puppeteer in references/hello-world +
references/seed | 2 |
| `not_used` | hono / @hono/node-server / express-rate-limit /
path-to-regexp 8.x / @modelcontextprotocol/sdk - all via mcp-sdk chain
(dormant in webapp; dev-only localhost in cli-v3) | 22 |
| `not_used` | fastify / @fastify/static / file-type via evalite devDep
| 5 |
| `tolerable_risk` | rollup 3 + minimatch 5/8/9/10 dev/build tooling |
13 |

## Notes

- **mcp-sdk chain**: `@vercel/sdk` in webapp imports `Vercel` API client
only; `mcp-server/*` subpath isn't loaded at runtime. cli-v3's MCP
server runs only via `trigger mcp` on developer machines. Bumping
`@modelcontextprotocol/sdk` to latest (1.29.0) wouldn't close these
alerts anyway - it ships hono ^4.11.4 which is still vulnerable - so
dismissal is the cleaner call.
- **References ignore list**: confirmed with current dependabot ignore
config; added `references/seed/package.json` (only gap).
- **undici** alerts (CVE-2026-1527, 4 alerts) will auto-close: lockfile
already at 6.25.0 > patched 6.24.0; just needs Dependabot rescan.
- **Effect 3.20 fix** is a runtime-only scheduler fix, no public API
changes - verified with research agent against our four `effect/*`
imports.
- **uuid 14** is ESM-only; we only call `validate`/`version` (no crypto
needed) so Node 20 requirement isn't load-bearing for us.
## Public packages (`packages/*`)

Minimal surface, deliberately. None of these change published runtime
behaviour - all changesets-worthy public package changes are deferred to
a regular release pass.

| Package | Change | Runtime impact |
|---|---|---|
| `packages/trigger-sdk` | Removed dead `uuid` dep (no source imports) |
None - dep was unused |
| `packages/cli-v3` | `tar` ^7.5.4 → ^7.5.13 | Patch bump within
already-allowed 7.x range; nothing CLI consumers see |
| `packages/core` / `packages/build` / `packages/python` /
`packages/rsc` / `packages/react-hooks` / `packages/schema-to-json` |
`rimraf` ^3.0.2 → ^6.0.1 in devDeps | Build-time only, no runtime change
|

No changeset added because nothing in these packages affects what
published consumers run.

## Validation

- Webapp typecheck (forced, no cache) passes after every commit
- Smoke-tested testcontainers v11 changes via real `postgresTest` +
`redisTest` (sync.test.ts, releaseConcurrency.test.ts) - both pass
- Webapp built + verified `require("uuid")` no longer in CJS server
output (now bundled inline)
- Test env webapp deployed at `dependabot-q2.rc0` (cloud#740) - no
issues observed
- Test suite run with package prerelease passed
2026-04-28 10:22:44 +01:00
nicktrn 9e99c81f64 ci: skip privileged PR jobs on fork PRs (#3458)
Fork PRs can't access org secrets or push to GHCR, so these two
`pull_request` jobs hard-fail with no path to passing:

- `claude-md-audit` - needs `CLAUDE_CODE_OAUTH_TOKEN`
- `helm-pr-prerelease` `prerelease` job - needs `packages: write` to
push the chart

Hit this on #3449. Approving the run didn't help; the jobs ran and
failed at the privileged step. The chart-validation `lint-and-test` job
is fork-safe and stays untouched - that remains the merge gate for Helm
changes.

Gate both jobs on same-repo head:

```yaml
if: github.event.pull_request.head.repo.full_name == github.repository
```

Other PR workflows already handle forks fine: `pr_checks`
(typecheck/units/e2e/sdk-compat) falls back to anonymous DockerHub pulls
when secrets are missing.
2026-04-28 09:54:16 +01:00
ThullyoCunha e8f1a7a0a1 fix(helm): expand CLICKHOUSE_PASSWORD in webapp CLICKHOUSE_URL via kubelet (#3449)
## Summary

When the official Helm chart is deployed with an external ClickHouse and
`clickhouse.external.existingSecret` set — the documented path for not
committing secrets to `values.yaml` — the webapp pod crash-loops on
startup:

```
goose run: parse "http://default:${CLICKHOUSE_PASSWORD}@<host>:8123?secure=false": net/url: invalid userinfo
```

Context in vouch request #3443. Re-opening in draft status per bot
policy (previous attempt was #3445, closed by automation because it
wasn't draft; no changes to the patch).

## Root cause

Two pieces interact:

1. `hosting/k8s/helm/templates/_helpers.tpl` renders `CLICKHOUSE_URL`
(and `RUN_REPLICATION_CLICKHOUSE_URL`) with a shell-style literal
`${CLICKHOUSE_PASSWORD}` expecting bash expansion at container start.
2. `docker/scripts/entrypoint.sh` does `export
GOOSE_DBSTRING="$CLICKHOUSE_URL"` — single-pass POSIX sh substitution,
so the inner `${...}` survives as literal text and goose rejects it.

Reproduces against the latest published chart
(`oci://ghcr.io/triggerdotdev/charts/trigger:4.0.5`) and `main`.

## Fix

Switch the two helpers (external + `existingSecret` branch) from
shell-style `${CLICKHOUSE_PASSWORD}` to Kubernetes'
`$(CLICKHOUSE_PASSWORD)`. Kubelet substitutes `$(VAR)` at pod-creation
time from earlier env entries, and the chart already declares
`CLICKHOUSE_PASSWORD` from the Secret immediately before
`CLICKHOUSE_URL`, so the URL reaches the entrypoint with the real
password already inlined. No entrypoint change, no image change. The
plain-password branch (no `existingSecret`) is unchanged.

Operator caveat added as template comments: `CLICKHOUSE_PASSWORD` must
be URL-userinfo-safe since kubelet substitutes verbatim without
percent-encoding. Hex-encoded passwords (e.g. `openssl rand -hex 32`)
are safe by construction.

## Verification

- `helm template` against `external.existingSecret` now renders `value:
"http://default:$(CLICKHOUSE_PASSWORD)@<host>:8123?secure=false"` (was
`${CLICKHOUSE_PASSWORD}`).
- `helm template` against the plain-password branch is byte-identical to
before.
- Deployed end-to-end on a staging EKS cluster (Meistrari platform):
webapp container reaches `goose: successfully migrated database to
version: 6`, Node.js ClickHouse client connects at runtime.

## Alternatives considered

- **Change `entrypoint.sh`** to `eval` / `envsubst` the URL — larger
surface, touches every deployment mode (Docker Compose + k8s) and every
container image.
- **Mirror the Postgres pattern** (chart reads the full URL via
`valueFrom.secretKeyRef`, as in `trigger-v4.postgres.useSecretUrl`) —
cleaner long-term but requires a new `values.yaml` field and a migration
path for existing users. Happy to follow up with that as a separate PR
if the minimal fix here isn't the preferred direction.

## Changeset

None added — the Helm chart isn't versioned through `@changesets/cli`
(docs/chart-only PRs historically merge without a changeset, e.g.
#2671). Happy to add one if the policy changed.

Closes #3443.
2026-04-27 22:15:27 +01:00
Eric Allam 4dced14ad5 chore: fix CONTRIBUTING.md setup steps and scope db:seed to webapp (#3450)
## Summary

Two fixes that together get a fresh-machine setup working from
`CONTRIBUTING.md` end-to-end with no manual workarounds:

### `CONTRIBUTING.md`

- Fix wrong path in the migration walkthrough: `cd packages/database` →
`cd internal-packages/database`. The current path doesn't exist; this
breaks step 2 for every contributor adding a migration.
- Renumber duplicate `4.` steps in **Adding migrations** and the skipped
`5.` in the hello-world **Running** section.
- Combine three sequential `pnpm run build --filter ...` calls into one
(Turbo parallelizes filters): `pnpm run build --filter webapp --filter
trigger.dev --filter @trigger.dev/sdk`.
- Add a `pnpm run db:seed` step after migrate. The seed creates the
local user, `References` org, and reference projects (including
`hello-world` with the stable `proj_rrkpdguyagvsoktglnod`). Removes the
manual instruction to edit the `externalRef` column in Postgres.
- Mention ClickHouse and the ClickHouse migrator alongside
Postgres/Redis in the Docker step (they're already part of `pnpm run
docker`, just invisible in the docs).
- Remove the V1-era **Add sample jobs** section.
`references/job-catalog` no longer exists; the hello-world flow above
replaces it.

### `turbo.json`

Scope `db:seed` to `webapp#db:seed → webapp#build`. The previous
root-level entry queued `build` for every workspace package — including
`references-*`, `docs`, `kubernetes-provider`, `coordinator`, etc. Only
`webapp` actually has a `db:seed` script, so the rest of those builds
were dead weight. Worse: a single broken reference (today,
`references-realtime-hooks-test` failing under Turbopack with
`node:fs/promises`) kills the whole seed pipeline.

After the change, `turbo run db:seed --dry-run` plan drops from 27 tasks
to 20 — only `webapp` and its real transitive workspace deps. Reference
projects no longer block seeding.

## Test plan

- [x] Fresh-machine setup followed end-to-end on a wiped Postgres +
ClickHouse: migrate → seed → build → webapp → CLI login → `trigger dev`
→ triggered `hello-world`, run completed with `{"message":"Hello,
world!"}`.
- [x] `turbo run db:seed --dry-run=json` confirms 20 tasks, all webapp
deps, no reference packages.
- [ ] CI green on the renamed turbo task name.
2026-04-27 18:13:29 +01:00
Eric Allam 5693b62cfb fix(webapp): propagate abort signal through realtime proxy fetch (#3442)
## Summary

Fixes an RSS-only memory leak in the three realtime proxy routes
(`/realtime/v1/runs`, `/realtime/v1/runs/:id`,
`/realtime/v1/batches/:id`). Client disconnects during an in-flight
long-poll would leave the upstream fetch to Electric running with no way
to abort it, so undici kept the socket open and buffered response chunks
that would never be consumed.

## Root cause

All three routes flow through
`RealtimeClient.streamRun/streamRuns/streamBatch` → `#streamRunsWhere` →
`#performElectricRequest` → `longPollingFetch(url, { signal })`. The
chain was already signal-aware, but `#streamRunsWhere` hardcoded
`signal=undefined` when calling `#performElectricRequest`, so no signal
ever reached `longPollingFetch`.

When a downstream client aborts a long-poll mid-flight:
1. Express tears down the downstream response socket.
2. The `longPollingFetch` promise has already resolved (it returns as
soon as upstream headers arrive) and handed back `new
Response(upstream.body, {...})`.
3. `undici` keeps the upstream socket open and continues buffering
chunks into the `ReadableStream` that nothing will ever read from.
4. The upstream connection is eventually closed by Electric's own poll
timeout (~20s). During that window the per-request buffers stay in
native memory.

These buffers live below V8's accounting — no `heapUsed` or `external`
growth, no sign in heap snapshots, only RSS. An isolated standalone
reproducer (`fetch` against a slow-streaming upstream, discard the
`Response` before consuming its body) measures **~44 KB retained per
leaked request** after GC. That's consistent with the undici socket +
receive buffer + HTTP parser state for a long-lived chunked response.
The pattern is the shape documented in
[nodejs/undici#1108](https://github.com/nodejs/undici/issues/1108) and
[#2143](https://github.com/nodejs/undici/issues/2143).

## What changed

- **`realtimeClient.server.ts`** — add optional `signal` parameter to
`streamRun`, `streamRuns`, `streamBatch`, and the shared
`#streamRunsWhere`; thread it through to `#performElectricRequest`
instead of hardcoding `undefined`.
- **`realtime.v1.runs.$runId.ts`, `realtime.v1.runs.ts`,
`realtime.v1.batches.$batchId.ts`** — pass `getRequestAbortSignal()`
(from `httpAsyncStorage.server.ts`) at the call site. This is the signal
wired to `res.on('close')` and fires reliably on downstream disconnect.
- **`longPollingFetch.ts`** — belt-and-suspenders: cancel the upstream
body explicitly in the error path, and treat `AbortError` as a clean
`499` instead of a `500`. This both releases undici's buffers
deterministically on error and avoids spurious 500s in request logs when
a client legitimately walks away.

## Verification

Standalone reproducer: slow upstream server streams 32 KB chunks every
100 ms for 5 seconds per request. The proxy does `fetch(url)` with
varying signal/cancel strategies, creates `new Response(upstream.body,
...)`, and discards it without consuming the body (simulating the leak
path).

Results from 1 000 parallel fetches per variant, measured post-GC:

| variant | Δ heap | Δ external | Δ RSS |
| --- | --- | --- | --- |
| A. no signal, body never consumed (the bug) | +0.3 MB | 0 MB | **+59.4
MB** |
| B. signal propagated, aborted after headers (this fix) | −0.1 MB | 0
MB | +15.4 MB |
| C. no signal, explicit `res.body.cancel()` | 0 MB | 0 MB | −25.4 MB |

10-round sustained test of variant B to distinguish accumulating
retention from one-time allocator overhead:
```
round  1/10  Δ=+3.2 MB     round  6/10  Δ=-12.5 MB
round  2/10  Δ=-7.6 MB     round  7/10  Δ=-11.9 MB
round  3/10  Δ=-11.7 MB    round  8/10  Δ=-2.6 MB
round  4/10  Δ=+3.2 MB     round  9/10  Δ=-8.0 MB
round  5/10  Δ=-1.2 MB     round 10/10  Δ=-12.6 MB
```
RSS oscillates in a 49-65 MB band with no upward trend — signal
propagation fully releases the buffers.

## Risk

- Behavior change only on aborted long-polls: the upstream fetch now
cancels promptly instead of running to its natural timeout. This saves
both memory and outbound traffic to Electric.
- `AbortError` now surfaces as `499` rather than `500`. Any dashboard or
alert that counts 500s in request logs will see slightly fewer of them;
this is the intended behavior.
- Signal-aware parameter is optional on
`RealtimeClient.streamRun/streamRuns/streamBatch`, so callers that don't
opt in get the previous behavior.

## Test plan

- [ ] Existing realtime integration tests pass
- [ ] Dashboard realtime views (runs list, batch details) continue
working normally across tab open/close cycles
- [ ] Under a burst of aborted long-polls, server RSS returns to
baseline rather than climbing
2026-04-24 16:00:02 +01:00
Eric Allam 2ce981d454 chore: add GautamBytes to vouch list (#3447)
Adds @GautamBytes to the vouch list so they can contribute to the
repository.

Closes #3307

Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Eric Allam <ericallam@users.noreply.github.com>
2026-04-24 15:58:32 +01:00
nicktrn d2059556c3 chore: vouch for ThullyoCunha (#3444)
closes #3443
2026-04-24 15:29:30 +01:00
Iss 8dd1fc12c6 docs: document auth.withAuth scoped authentication helper (#3436) 2026-04-24 13:57:15 +01:00
Matt Aitken 8aa1e55588 test: e2e auth baseline tests + webapp testcontainer infrastructure (#3438)
Adds a minimal end-to-end test harness that spawns the compiled webapp
as a child
process against a throwaway Postgres container, plus a baseline of 8
auth-behaviour
tests. These tests will be used as a regression check before and after
the upcoming
apiBuilder RBAC migration to confirm auth behaviour is unchanged.

## What's included

**`internal-packages/testcontainers/src/webapp.ts`** (new)
Spawns `build/server.js` with a dynamically allocated port, polls
`/healthcheck`,
and exposes `WebappInstance` and `startTestServer()` (postgres container
+ webapp +
PrismaClient in one call). Key details:
- Uses `process.execPath` so the correct Node binary is found in forked
test processes
- Sets `NODE_PATH` to `node_modules/.pnpm/node_modules` so pnpm-hoisted
transitive
deps (e.g. `eventsource-parser`) resolve correctly inside the subprocess
- Overrides both `PORT` and `REMIX_APP_PORT` so Vite's automatic `.env`
loading
  doesn't override the dynamically allocated port

**`internal-packages/testcontainers/package.json`**
Adds `./webapp` sub-path export so tests can `import from
"@internal/testcontainers/webapp"`.

**`internal-packages/testcontainers/src/index.ts`**
Exports `createPostgresContainer` (used internally by `webapp.ts`).

**`apps/webapp/test/helpers/seedTestEnvironment.ts`** (new)
Creates a minimal org → project → environment row set with random
suffixes.

**`apps/webapp/test/api-auth.e2e.test.ts`** (new)
8 tests across two suites:
- API-key bearer: valid key (auth passes, 404), missing header (401),
invalid key (401), error body shape
- JWT bearer: valid JWT on JWT-enabled route (passes), valid JWT on
non-JWT route (401), empty-scope JWT (403), wrong signing key (401)

## How to run

```bash
# Build required first (one-time)
pnpm run build --filter webapp

cd apps/webapp && pnpm exec vitest run test/api-auth.e2e.test.ts
```

## Test plan
- [x] All 8 tests pass against the current webapp build
- [x] Webapp healthcheck returns 200 on startup
- [ ] CI passes

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-04-24 12:06:35 +01:00
Saadi Myftija 496ac78484 feat(supervisor): optional ndots override for runner pods (#3441)
⚒️ Publish Worker (v4) / build (supervisor) (push) Has been cancelled
Adds `KUBERNETES_POD_DNS_NDOTS_OVERRIDE_ENABLED` flag (off by default)
that overrides the cluster default and sets `dnsConfig.options.ndots` on
runner pods (defaulting to 2, configurable via
`KUBERNETES_POD_DNS_NDOTS`).

Kubernetes defaults pods to `ndots: 5`, so any name with fewer than 5
dots, including typical external domains like `api.example.com`, is
first walked through every entry in the cluster search list
(`<ns>.svc.cluster.local`, `svc.cluster.local`, `cluster.local`) before
being tried as-is, turning one resolution into 4+ CoreDNS queries (×2
with A+AAAA).

Using a lower `ndots` value reduces DNS query amplification in the
`cluster.local` zone.
2026-04-24 13:05:55 +02:00
Matt Aitken f7aefb705a fix: disable RunQueue Worker in priority tests to prevent partial-batch race (#3440)
## Summary

- The `processMasterQueueForEnvironment` call in the priority test was
racing against background `processQueueForWorkerQueue` jobs scheduled
50ms after each trigger
- With a 50ms debounce (`processWorkerQueueDebounceMs: 50`) and runs
triggered sequentially, the RunQueue Worker could process those jobs
mid-sequence, pushing partial batches to the worker queue in the wrong
overall priority order
- `masterQueueConsumersDisabled: true` only blocks the shard-level
polling loops — it does not prevent the RunQueue's own Worker from
processing these debounced jobs
- Fix: add `worker.disabled: true` to the test 1 engine config, which
propagates to `workerOptions.disabled` in the RunQueue constructor and
prevents the Worker from starting

## Test plan

- [x] Both priority tests pass: `pnpm run test
./src/engine/tests/priority.test.ts --run`
- [x] Test 1 log confirms no ` Starting run engine worker` or worker
loop messages — workers fully disabled
- [x] Test 2 unaffected (uses master queue consumers for automatic
promotion, no `disabled` flag added)

🤖 Generated with [Claude Code](https://claude.com/claude-code)
2026-04-24 12:03:47 +01:00
nicktrn 3446be7b32 ci: automate helm chart release alongside package release (#3439)
Wires up automatic Helm chart releases to ride along with the existing
changeset-driven package release flow.

Today `Chart.yaml` is bumped by hand and `release-helm.yml` fires only
when a human pushes a `helm-v*` tag. With this, the changeset release PR
also carries a `Chart.yaml` bump so main always matches the published
version, and `release.yml` invokes `release-helm.yml` via
`workflow_call` after Docker images are published.

`helm-v${VERSION}` tag is pushed as a marker (same GITHUB_TOKEN trick as
`v.docker.*`). Manual `helm-v*` tag flow still works. Chart.yaml
consistency check in `release-helm.yml` is the safety net if the bump
job ever drifts.

First rollout: the open `changeset-release/main` PR has stale
Chart.yaml. Bump it manually on that branch before merging, otherwise
the first automated helm release fails at the consistency check.
2026-04-24 07:05:55 +01:00
Eric Allam ca399565ba feat(webapp): add per-worker Node.js heap metrics (#3437)
## Summary

Adds direct V8 heap and process-memory gauges to the webapp's
OpenTelemetry meter. The webapp already exports per-cluster-worker
Node.js runtime metrics (event-loop lag / utilization, active handles,
active requests, libuv threadpool size) via a custom meter under the
`trigger.dev` scope. Heap and memory were missing; this PR adds them
alongside, in the same observable-batch pattern.

## New gauges

| Metric | Source | Unit |
| --- | --- | --- |
| `nodejs.memory.heap.used` | `process.memoryUsage().heapUsed` | bytes |
| `nodejs.memory.heap.total` | `process.memoryUsage().heapTotal` | bytes
|
| `nodejs.memory.heap.limit` | `v8.getHeapStatistics().heap_size_limit`
| bytes |
| `nodejs.memory.external` | `process.memoryUsage().external` | bytes |
| `nodejs.memory.array_buffers` | `process.memoryUsage().arrayBuffers` |
bytes |
| `nodejs.memory.rss` | `process.memoryUsage().rss` | bytes |

Gated by the existing `INTERNAL_OTEL_NODEJS_METRICS_ENABLED` flag, same
as the adjacent event-loop / handle gauges. Zero overhead when disabled.

## Why

`@opentelemetry/host-metrics` publishes `process.memory.usage`, which is
RSS only. RSS is the sum of V8 heap, external memory (Buffers, etc.),
native code, and thread stacks. Without a direct heap metric it is not
possible to size the V8 heap cap (`--max-old-space-size`) from metrics
alone, because RSS overstates heap by the external + native footprint. A
worker can have a 4 GB RSS with a 2.5 GB heap and 1.5 GB of buffers; the
former constrains `--max-old-space-size`, the latter does not.

`nodejs.memory.heap.limit` also surfaces the configured
`--max-old-space-size` (read from
`v8.getHeapStatistics().heap_size_limit`), so operators can see the
current limit in the same dashboard as actual usage rather than
cross-referencing container environment variables.

## Risk

Minimal. Observable gauges are sampled at the configured metric-export
interval. `v8.getHeapStatistics()` and `process.memoryUsage()` are each
microsecond-level calls, and six gauges are added to the same batch
callback that already reads ~20 other Node.js runtime values per sample.
Same registration pattern as the existing event-loop metrics in the
file.

## Test plan

- [ ] Deploy and confirm the six new gauges appear at the configured
exporter
- [ ] In cluster mode, confirm per-worker granularity (one series per
cluster worker, tagged by `process.executable.name` /
`service.instance.id`)
- [ ] Confirm `nodejs.memory.heap.limit` reports the configured
`--max-old-space-size` value in bytes
2026-04-23 22:41:24 +01:00
Iss 41434b536b feat(webapp): admin Back Office tab with org API rate limit editor (#3434)
## Summary
- New **Back office** tab at `/admin`, per-org detail page at
`/admin/back-office/orgs/:orgId` designed to host future per-org admin
actions (project count, delete account, YC deals).
- First action: edit an organization's API rate limit — tokenBucket
override (refill rate, interval, max tokens), with a live plain-English
preview (e.g. *"1,500 requests per minute · 750 request burst
allowance"*). Writes are audit-logged via the server logger.
- Cleanup: removed unused `v2?` / `v3?` columns from the admin orgs list
(display only — Prisma select untouched).

## Test plan
- [ ] Back office tab visible in admin nav and highlighted when on a
sub-route
- [ ] `/admin/orgs` shows a Back office "Open" link per row; no v2/v3
columns
- [ ] Empty state at `/admin/back-office` links back to `/admin/orgs`
- [ ] Detail page renders the effective rate limit in view mode; Edit
reveals the form
- [ ] Save writes `Organization.apiRateLimiterConfig`, returns to view
mode, shows "Rate limit saved." banner
- [ ] Invalid values surface inline field errors and keep edit mode
- [ ] Non-admins hitting any new route are redirected to `/`
- [ ] Server logs show `admin.backOffice.rateLimit` info line per
mutation
2026-04-23 12:51:49 -04:00
devin-ai-integration[bot] cbb1f35ef0 chore(helm): bump appVersion to v4.4.4 (#3432)
🧭 Helm Chart Release / lint-and-test (push) Has been cancelled
🧭 Helm Chart Release / release (push) Has been cancelled
2026-04-23 18:47:44 +02:00
Eric Allam 486f49791d fix(webapp): eliminate SSE abort-signal memory leak (#3430)
## Summary

Fixes a server-side memory leak in the webapp's SSE helper. Every
aborted SSE connection (client tab close, navigation, timeout) was
pinning its full request/response graph indefinitely on Node 20, so any
long-running webapp process accumulated retained memory proportional to
streaming-request churn.

## Root cause

`apps/webapp/app/utils/sse.ts` combined four abort signals via
`AbortSignal.any([requestAbortSignal, timeoutSignal,
internalController.signal])`. The composite signal tracks its source
signals in an internal `Set<WeakRef>` registered against a
`FinalizationRegistry`; under sustained traffic those entries accumulate
faster than they're cleaned up, pinning every source signal (and its
listeners, and anything those listeners close over) until the parent
signal itself is GC'd or aborts.

This is a long-standing Node issue with multiple open reports:

- [nodejs/node#54614](https://github.com/nodejs/node/issues/54614) —
original report, still open. A [follow-up from
ChainSafe](https://github.com/nodejs/node/issues/54614#issuecomment-4055656572)
describes the exact same shape in a Lodestar production workload (req +
timeout signals composed per request accumulating in long-running
worker) and the same mitigation: drop `AbortSignal.any`, compose
manually.
- [nodejs/node#55351](https://github.com/nodejs/node/issues/55351) —
mechanism confirmed by Node member @jasnell: *"the set of dependent
signals known to the AbortSignal are kept in an internal Set using
WeakRefs. The AbortSignals are being properly gc'd but the Set is never
cleaned out of the WeakRefs making those leak."* Partially fixed by [PR
#55354](https://github.com/nodejs/node/pull/55354), shipped in Node
22.12.0 — but only covers the tight-loop case, not long-lived parent
signals.
- [nodejs/node#57584](https://github.com/nodejs/node/issues/57584) —
circular-dependency variant, still open.
- [nodejs/node#62363](https://github.com/nodejs/node/issues/62363) —
regression in Node 24/25 from an unrelated V8 change ("Don't pretenure
WeakCells"). Different root cause, same symptom.

A separate issue in `apps/webapp/app/entry.server.tsx` —
`setTimeout(abort, ABORT_DELAY)` with no `clearTimeout` on success paths
— kept the React render tree + `remixContext` alive for 30s per
successful HTML request. Same pattern fixed upstream in React Router
templates
([react-router#14200](https://github.com/remix-run/react-router/pull/14200)),
never backported to Remix v2.

## What changed

- **`apps/webapp/app/utils/sse.ts`** — single-signal abort chain.
`AbortSignal.any` removed; `AbortSignal.timeout` replaced by a plain
`setTimeout` cleared when the controller aborts; named sentinel
constants used as stackless abort reasons; request-abort handler
explicitly removed on cleanup.
- **`apps/webapp/app/entry.server.tsx`** — clears the `setTimeout(abort,
ABORT_DELAY)` timer in `onShellReady` / `onAllReady` / `onShellError`.
- **`apps/webapp/app/v3/tracer.server.ts` + `env.server.ts`** — gates
OpenTelemetry `HttpInstrumentation` and `ExpressInstrumentation` behind
`DISABLE_HTTP_INSTRUMENTATION=true` as an escape hatch for future
OTel-listener retention patterns. Defaults to enabled.
- **`apps/webapp/app/presenters/v3/RunStreamPresenter.server.ts`** —
uses the shared `ABORT_REASON_SEND_ERROR` sentinel.

## Verification

### Full-app reproduction (memlab)

Isolated local harness, 500 abrupt SSE disconnects against a
dev-presence route, GC between passes, heap snapshot diff with
[memlab](https://facebook.github.io/memlab/):

| Run | Heap delta after 500 conns + GC | memlab retained leaks |
| --- | --- | --- |
| Before | +16.0 MB (linear with request count) | 158 clusters; 250
`ServerResponse`, 1000 `AbortController`, 250 `SpanImpl` retained |
| After | **+3.3 MB (noise)** | **0 app-code leaks** |

### Standalone mechanism isolation

To confirm *which* axis of the change is load-bearing, a separate
standalone Node script (`/tmp/abort-leak-test.mjs`) ran 2000 requests ×
200 KB payload per variant:

| Variant | Heap delta after GC |
| --- | --- |
| baseline (no signal machinery) | 0 MB |
| V1: `AbortSignal.any` + string abort reason | **+9.1 MB** |
| V2: `AbortSignal.any` only (no reason) | **+10.8 MB** |
| V3: string reason only (no `AbortSignal.any`) | 0 MB |
| V4: neither (the fix) | 0 MB |
| V5: `AbortSignal.any` with no listener on the composite | **+10.2 MB**
|

This proves `AbortSignal.any` is the sole mechanism. The reason type
(`.abort()` vs `.abort("string")`) is irrelevant for retention — V3 is
clean, V5 leaks even without a listener on the composite.

## Risk

- `sse.ts` is used by the dev-presence routes. Behaviour is equivalent —
timeouts and client disconnects still abort the stream. `signal.reason`
is now a named string sentinel (`"timeout"`, `"request_aborted"`, etc.)
instead of the previous string arg or default `AbortError`. No in-tree
reader of `signal.reason` exists.
- `entry.server.tsx` change is a standard cleanup of an abort timer,
matches upstream React Router guidance.
- `tracer.server.ts` change is env-gated and defaults to current
behaviour.
- Three other webapp `AbortSignal.timeout()` callsites (alert delivery,
remote-build status) are fire-and-forget passed directly to `fetch` —
not composed with anything long-lived, no retention risk, untouched.

## Test plan

- [ ] Existing SSE integration tests pass
- [ ] Dev-presence SSE behaves normally across tab open/close cycles
- [ ] No heap growth under sustained aborted-connection traffic (heap
snapshot diff)

## Follow-up

The same `AbortSignal.any([userSignal, internalSignal])` pattern exists
in several SDK/core callsites that ship to customers
(`packages/core/src/v3/realtimeStreams/manager.ts`,
`packages/trigger-sdk/src/v3/{ai,chat,chat-client,sessions}.ts`,
`packages/core/src/v3/workers/warmStartClient.ts`). Whether those leak
in practice depends on the user passing a long-lived signal. Tracked
separately.
2026-04-23 15:54:05 +02:00
nicktrn 87b6716535 fix(helm): support webapp serviceAccount annotations for IRSA (#3429)
Mirrors the existing `supervisor.serviceAccount` pattern onto webapp so
operators can annotate the SA (IRSA `eks.amazonaws.com/role-arn`,
Workload Identity, etc.) or bring their own SA. Without this,
`webapp.serviceAccount.annotations` isn't exposed and operators have to
patch the SA out-of-band.

```yaml
webapp:
  serviceAccount:
    create: true
    name: ""
    annotations:
      eks.amazonaws.com/role-arn: arn:aws:iam::123456789012:role/trigger-webapp
```

Three pieces, same as supervisor:
- `webapp.serviceAccount.create` toggle on the SA block
- `webapp.serviceAccount.annotations` + `name` values
- `trigger-v4.webappServiceAccountName` helper, used by the SA, the
token-syncer RoleBinding subject, and the Deployment's
`serviceAccountName`

Role + RoleBinding are left unguarded (matching supervisor's shape where
`rbac.create` is a separate toggle from `serviceAccount.create`) -
BYO-SA users take on the responsibility of ensuring the SA they supply
has the permissions the RoleBinding grants.

Verified with `helm template` against default values, an IRSA annotation
override, and `create: false` with a custom name.
2026-04-23 15:08:54 +02:00
Matt Aitken fc71e7dd75 fix: handle fast-completion race in batch streaming seal check (#3427)
## Problem

When `batchTrigger()` is called with large payloads, each item's payload
is uploaded to R2 server-side during the streaming loop before being
enqueued. This makes the loop slow — around 3 seconds per item. Workers
pick up and execute each item as it's enqueued, running concurrently
with the ongoing stream.

For the last item in the batch, a race exists between the streaming loop
finishing and the batch completion cleanup:

1. The loop enqueues the last item and returns from `enqueueBatchItem()`
2. A waiting worker picks up the item almost instantly and executes it
3. `recordSuccess()` fires, `processedCount` hits the expected total,
`finalizeBatch()` runs
4. `cleanup()` deletes all Redis keys for the batch, including
`enqueuedItemsKey`
5. The streaming loop exits and calls `getBatchEnqueuedCount()` — reads
the now-deleted key — returns 0

The count check finds `enqueuedCount (0) !== batch.runCount`, falls
through to a Postgres fallback, but the fallback only checked `sealed`.
The BatchQueue completion path sets `status = COMPLETED` in Postgres
without setting `sealed = true` (that's the streaming endpoint's job),
so the fallback misses it too.

This causes the endpoint to return `sealed: false`. The SDK treats this
as retryable and retries up to 5 times with exponential backoff. Each
retry calls `enqueueBatchItem()`, which reads the batch meta key from
Redis — also deleted by `cleanup()` — and throws "Batch not found or not
initialized" (500). The final retry gets a 422 because the batch is
already COMPLETED, which the SDK does not retry, causing an `ApiError`
to be thrown from `await batchTrigger()` in the parent run — even though
all child runs completed successfully.

## Fix

In the Postgres fallback inside `StreamBatchItemsService`, also check
`status === "COMPLETED"` alongside `sealed`. This covers the
fast-completion path where the BatchQueue finishes all runs before the
streaming endpoint gets to seal the batch normally.

Also switches `findUnique` to `findFirst` per webapp convention.

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-04-23 13:45:41 +01:00
Oskar Otwinowski 8eb596f3fe fix(vercel): Fix vercel settings page (#3424) 2026-04-22 19:01:34 +02:00
Eric Allam 2d3b2e82e6 feat(run-engine): flag to route getSnapshotsSince through read replica (#3423)
## Summary

Adds `RUN_ENGINE_READ_REPLICA_SNAPSHOTS_SINCE_ENABLED` (default `"0"`).
When enabled, the Prisma reads inside `RunEngine.getSnapshotsSince` run
against the read-only replica client instead of the primary. Offloads
the snapshot-polling queries fired by every running task runner off the
writer.

## Why

`getSnapshotsSince` is called from the managed runner's
fetch-and-process loop (once per poll interval, plus on every
snapshot-change notification). It runs four sequential reads per call —
one `findFirst` by snapshot id, one `findMany` on snapshots with
`createdAt > X`, one raw SQL against `_completedWaitpoints`, and chunked
`findMany` on `waitpoint`. Per concurrent run, every few seconds. It's
read-only, tolerates a small amount of staleness, and is an obvious
candidate for the replica.

## Replica-lag considerations

- **Step 1 "since snapshot not found"**: if the runner just received a
snapshot id from the primary and asks the replica before it replicates,
the function throws and the caller treats the response as an error
(runner falls back to a metadata refresh). Self-correcting, not silent.
- **Step 2 missing newly-created snapshots**: the next poll's `createdAt
> sinceSnapshot.createdAt` filter still picks them up once the replica
catches up.
- **Waitpoint junction race**: the riskiest path — if a latest snapshot
is replicated but its `_completedWaitpoints` join rows aren't yet, the
runner could advance past that snapshot with `completedWaitpoints: []`.
WAL/storage-level replication replays commits in order, so in practice
both should appear atomically on the reader, but the race window is why
the flag ships disabled.

Aurora reader shrinks all three windows to single-digit ms in typical
conditions, and its storage-level replication gives atomic visibility of
committed transactions on the reader.

## Test plan

- [ ] Flip the flag on in a non-prod environment, confirm snapshot
polling behaves normally and `getSnapshotsSince` errors in Sentry stay
flat.
- [ ] Verify writer query volume drops and reader query volume rises on
the snapshot-polling queries.
- [ ] Keep an eye on `AuroraReplicaLag` (or equivalent) during rollout.
2026-04-22 11:48:04 +01:00
Eric Allam 7c95ee498e feat(webapp): tag Prisma spans with db.datasource attribute (#3422)
## Summary

Stamp every Prisma span with `db.datasource: "writer" | "replica"` so
traces can distinguish which client the query went through.

Both `PrismaClient` instances share the same global
`@prisma/instrumentation`, so their spans come out with identical names
and attributes today. This makes them trivially filterable.

## How

Two pieces in `apps/webapp/app/`:

1. **`v3/tracer.server.ts`** — a `DatasourceAttributeSpanProcessor`
reads an OTel context key in `onStart` and calls
`span.setAttribute("db.datasource", value)`. Registered as the first
span processor.
2. **`db.server.ts`** — `tagDatasource(datasource, client)` wraps each
`PrismaClient` with `$extends({ query: { $allOperations } })`. The
middleware sets the context key around the query and directly tags the
active span (to catch `prisma:client:operation`, which Prisma creates
before the middleware fires).

### Context-propagation gotcha

`PrismaPromise` is lazy — `query(args)` returns a thenable that only
starts when someone `.then()`s it. The naive `context.with(ctx, () =>
query(args))` restores ALS synchronously, so when Prisma's internal code
awaits the thenable later, the engine spans fire with the original ALS.
Wrapping as `async () => await query(args)` forces the `.then()` inside
the `context.with` callback, so ALS stays on our context for the engine
spans.

### Coverage

- **Tagged**: all `prisma:engine:*` (`connection`, `db_query`,
`serialize`, `query`, etc.), `prisma:client:operation`,
`prisma:client:serialize`, `prisma:client:connect`
- **Not tagged**: `prisma:client:load_engine` — one-time startup, fires
before any query

Concurrent `Promise.all([writer.x, replica.y])` correctly tags each pool
separately (ALS isolates per-Promise chain).

### Performance

One `context.with` (~200ns) and one `setAttribute` per span (effectively
free per OTel JS benchmarks) per Prisma op. Negligible against a query
path measured in milliseconds.

## Test plan

- [ ] Verify `db.datasource` appears on `prisma:engine:connection` spans
after the webapp is restarted
- [ ] Spot-check a handful of real traces carry the attribute
2026-04-21 16:56:17 +01:00
nicktrn b570586899 fix(webapp): allow cancelling runs in DEQUEUED status from the runs list (#3421)
The cancel button was missing from the runs list for runs in `DEQUEUED`
status. The runs list gates the button on `run.isCancellable`, which
goes through `isCancellableRunStatus` -> `CANCELLABLE_RUN_STATUSES` =
`NON_FINAL_RUN_STATUSES`. `DEQUEUED` was never added to that list when
it was introduced in the run engine.

The single run page uses a separate check (`!run.isFinished`, i.e. the
inverse of `FINAL_RUN_STATUSES`), so cancellation already worked there -
only the list was affected.

Adding `DEQUEUED` to `NON_FINAL_RUN_STATUSES` also flips
`isCrashableRunStatus` and `isFailableRunStatus`, but:

- The crash path is the right behaviour - a `DEQUEUED` run (worker has
claimed but not yet executing) can legitimately crash before
`EXECUTING`, same as `PENDING`/`DELAYED` already do.
- The fail path (`failedTaskRun.server.ts`) is only reached from V1 code
paths (marqs consumers, v1 heartbeat handler). `DEQUEUED` is a
V2-engine-only status, so V1 consumers never see it.

When cancelling a `DEQUEUED` run the execution snapshot goes to
`PENDING_CANCEL` (worker must ack) but `TaskRun.status` flips to
`CANCELED` immediately - the UI reflects cancellation without waiting
for the worker. Added an integration test in
`run-engine/src/engine/tests/cancelling.test.ts` covering the full
trigger -> dequeue -> cancel -> worker-ack flow.

## Stall safety

The stall recovery path (PENDING_EXECUTING heartbeat miss ->
nack-and-requeue -> back to QUEUED) lives entirely inside
`@internal/run-engine` and never touches the webapp's `taskStatus.ts`
helpers - the engine has zero imports from `~/v3/taskStatus` and doesn't
know `CrashTaskRunService` / `FailedTaskRunService` exist. A stalled
DEQUEUED run still goes back to the queue for retry; this change cannot
cause stalls to crash or fail.

The only realistic impact is the intended UI fix - the theoretical V1
crash/fail branches for DEQUEUED are unreachable in practice because V1
runs never have DEQUEUED status.
2026-04-21 11:33:17 +01:00
Eric Allam 03e4d5fe31 feat(webapp,database): API key rotation grace period (#3420)
## Summary

Regenerating a RuntimeEnvironment API key no longer immediately
invalidates the previous one. Rotation is now overlap-based: the old key
keeps working for 24 hours so customers can roll it out in their env
vars without downtime, then stops working.

## Design

- **New `RevokedApiKey` table** (one row per revocation). Holds the
archived `apiKey`, a FK to the env, an `expiresAt`, and a `createdAt`.
Indexed on `apiKey` (high-cardinality equality — single-row hits) and on
`runtimeEnvironmentId`.
- **`regenerateApiKey` wraps both writes in a single `$transaction`:**
insert a `RevokedApiKey` with `expiresAt = now + 24h`, update the env
with the new `apiKey`/`pkApiKey`.
- **`findEnvironmentByApiKey` does a two-step lookup:** primary
unique-index hit on `RuntimeEnvironment.apiKey` first; on miss,
`RevokedApiKey.findFirst({ apiKey, expiresAt: { gt: now } })` with an
`include: { runtimeEnvironment }`. Two-step (not `OR`-join) keeps the
hot path identical to today and puts the fallback cost only on invalid
keys. Both lookups use `$replica`.
- **Admin endpoint** `POST /admin/api/v1/revoked-api-keys/:id` accepts
`{ expiresAt }` and updates the row. Setting to `now` ends the grace
window immediately; setting to the future extends it.
- **Modal copy** on the regenerate dialog updated — previously warned of
downtime, now explains the 24h overlap.

## Why a separate table instead of columns on `RuntimeEnvironment`

- Keeps the hot auth path's primary lookup unchanged — no
OR/nullable-apiKey semantics to reason about.
- Naturally supports multiple in-flight grace windows (regenerate twice
in a day → two old keys valid until their independent expiries).
- FK + cascade cleans up correctly when an env is deleted; nothing to
backfill.

## Test plan

Verified locally against hello-world with dev and prod env keys:

- [x] baseline — current key authenticates (`GET /api/v1/runs`) → `200`
- [x] regenerate via UI — DB shows old key in `RevokedApiKey` with
`expiresAt ≈ now+24h`, env has new key
- [x] grace window — both old and new keys → `200`; bogus key → `401`
- [x] admin endpoint: `expiresAt = now` → old key `401`
- [x] admin endpoint: `expiresAt = +1h` (after early-expire) → old key
`200` again
- [x] admin endpoint: `expiresAt = past` → old key `401`
- [x] admin 400 (invalid body), 404 (unknown id), 401 (missing/non-admin
PAT)
- [x] same flow exercised end-to-end on a PROD-typed env — behavior
identical
- [x] `pnpm run typecheck --filter webapp` passes
2026-04-20 18:28:16 +01:00
Iss de3b9a158b docs: document secret env vars and Vercel sync behavior (#3419) 2026-04-20 13:19:33 -04:00
DKP be6b490790 docs: skills page update (#3418) 2026-04-20 17:06:01 +01:00
Eric Allam 881288c615 feat(webapp): deprecate v3 CLI deploys server-side (#3415)
##  Checklist

- [x] I have followed every step in the [contributing
guide](https://github.com/triggerdotdev/trigger.dev/blob/main/CONTRIBUTING.md)
- [x] The PR title follows the convention.
- [x] I ran and tested the code works

---

## Summary

Adds a server-side gate that detects deploy attempts from v3 CLI
versions (i.e. `trigger.dev@3.x`) at the `POST /api/v1/deployments`
entry point and, when enabled, rejects them with a clear upgrade
message. v4 CLI deploys are completely unaffected.

The last 3.x CLI release was `3.3.7`, which we can't update. This
approach short-circuits the deploy before any DB writes, image-ref
generation, S2 stream creation, or queue enqueue — no side effects in
either mode.

## How v3 vs v4 are distinguished

I pulled the published CLI tarballs for `trigger.dev@3.3.7`, `4.0.0`,
`4.0.1`, `4.0.5`, `4.1.0`, `4.2.0`, and the current `4.4.4` in the repo.
The cleanest, most reliable signal is the request body to `POST
/api/v1/deployments`:

| Field on initialize | v3.3.7 CLI | v4.x CLI |
|---|---|---|
| `type` | **never sent** | always sent — `"MANAGED"` (run_engine_v2) or
`"V1"` |
| `isNativeBuild` / `gitMeta` / `triggeredVia` / `runtime` | not sent |
sent |
| `registryHost` / `namespace` | sent (v3-only; stripped by current Zod
schema) | not sent |

Every v4 call site I inspected sets `type: features.run_engine_v2 ?
"MANAGED" : "V1"` unconditionally. `payload.type` is `undefined` if and
only if the client is a 3.x CLI.

## Behavior

- Detection always runs and emits `logger.warn("Detected deploy from
deprecated v3 CLI", { environmentId, projectId, organizationId, enforced
})`, which lets us watch how many v3 deploys are still happening before
enforcement is flipped.
- Enforcement is gated behind `DEPRECATE_V3_CLI_DEPLOYS_ENABLED`
(default `"0"`, off). When `"1"`, the server returns `400` with:

> The trigger.dev CLI v3 is no longer supported for deployments. Please
upgrade your project to v4: https://trigger.dev/docs/migrating-from-v3

The v3 CLI surfaces this verbatim as `Failed to start deployment:
<message>` because `zodfetch` throws `ApiError` for non-retryable 4xx
(400/422) and `deploy.js` in 3.3.7 prints `error.message`.

## Out of scope (intentionally)

- `api.v1.deployments.$deploymentId.finalize.ts` /
`FinalizeDeploymentService` /
`createDeploymentBackgroundWorkerV3.server.ts` are V1-engine paths, not
the v3 CLI gate. Leaving them alone per review.
- Container-side `createDeploymentBackgroundWorker` call in
`managed-index-controller.ts` is still used by v4's in-image indexer.
Not touched.
- v3 `trigger dev` flow (different code path) — separate deprecation
if/when needed.

## Testing

- Ran `pnpm run typecheck --filter webapp` locally — passes.
- Verified v4 tarballs (4.0.0, 4.0.1, 4.0.5, 4.1.0, 4.2.0, 4.4.4) all
include `type:` in the `initializeDeployment` call site, so none will be
accidentally blocked.
- Verified v3.3.7 tarball's `initializeDeployment` payload has no `type`
field.

Rollout plan after merge:
1. Deploy with `DEPRECATE_V3_CLI_DEPLOYS_ENABLED` unset → watch
`Detected deploy from deprecated v3 CLI` log volume.
2. When comfortable, set `DEPRECATE_V3_CLI_DEPLOYS_ENABLED=1` to
enforce.

---

## Changelog

Detect v3 CLI deploys on `/api/v1/deployments` and, when
`DEPRECATE_V3_CLI_DEPLOYS_ENABLED=1`, reject them with an upgrade
message pointing at https://trigger.dev/docs/migrating-from-v3. v4 CLI
deploys are unaffected.


Link to Devin session:
https://app.devin.ai/sessions/b242c11bd86e4099aeec8b59bab62143
Requested by: @ericallam
2026-04-20 15:26:57 +01:00
Matt Aitken 6e6deb41e1 Admin endpoint to set concurrency burst factor (#3412)
Example cURL call using an admin user PAT (replace with a real one):

```sh
curl -X PUT https://cloud.trigger.dev/admin/api/v1/environments/<environmentId>/burst-factor \
    -H "Authorization: Bearer tr_pat_1234" \
    -H "Content-Type: application/json" \
    -d '{"burstFactor": 1.5}'
```
2026-04-19 19:35:04 +01:00
Iss 7d7ebdde52 feat: Increase default project limit per org from 10 to 25 (#3409) 2026-04-17 11:05:57 -04:00
nicktrn 9a988ab885 chore(webapp): clarify admin feature flags are global (#3408)
global flags are global.
2026-04-17 13:49:30 +01:00
nicktrn 581db83f64 feat(webapp): highlight microVM regions on the regions page (#3407)
Adds a `MicroVM` badge next to the region name on the regions page. Uses
the existing `small` badge variant for visual consistency with the
`Default` badge already on this page.
2026-04-17 13:40:05 +01:00
Eric Allam 69acdc2b32 fix(core): truncate large error stacks and messages to prevent OOM (#3405)
## Summary

Large error stacks and messages can OOM the worker process when
serialized into OTel spans or `TaskRunError` objects. This was reported
when throwing an error with a massive `.stack` property from a chat
agent hook.

This adds frame-based stack truncation (similar to Sentry's approach)
plus message length limits, applied consistently across all error
serialization paths.

### What changed

**`packages/core/src/v3/errors.ts`**
- `truncateStack()` — parses `error.stack` into message lines + frame
lines, caps at 50 frames (keep top 5 closest to throw + bottom 45 entry
points, with "... N frames omitted ..." in between). Individual lines
capped at 1024 chars.
- `truncateMessage()` — caps error messages at 1000 chars
- Applied in `parseError()` and `sanitizeError()`

**`packages/core/src/v3/otel/utils.ts`**
- `sanitizeSpanError()` now uses `truncateStack` and `truncateMessage`
from `errors.ts` instead of duplicating truncation logic
- Non-Error values (strings, JSON) capped at 5000 chars

**`packages/core/src/v3/tracer.ts`**
- `startActiveSpan` catch block now delegates to `recordSpanException()`
instead of calling `span.recordException()` directly

### Limits

| What | Limit | Rationale |
|------|-------|-----------|
| Stack frames | 50 | Matches Sentry's `STACKTRACE_FRAME_LIMIT` |
| Top frames kept | 5 | Closest to throw site |
| Bottom frames kept | 45 | Entry points / framework frames |
| Per-line length | 1024 | Matches Sentry, prevents regex DoS |
| Message length | 1000 | Bounded but generous |
| Generic string (non-Error) | 5000 | Fallback for JSON/string errors in
spans |

## Test plan

- [x] 17 unit tests in `packages/core/test/errors.test.ts`
- [x] E2E: threw a 300-frame / 5000-char-message error in the ai-chat
reference app, verified truncated stack and message in span via
`get_span_details`
- [x] Verified the run survived the error (no OOM, continued waiting for
next message)
2026-04-17 13:25:01 +01:00
Matt Aitken 45ba398c80 Error page graph: for a time bucket don't fill zeros for a version with no errors (#3402)
This caused performance issues with large numbers of versions, and bad
UX when hovering the graph (showing irrelevant versions)
2026-04-17 10:12:54 +01:00
Eric Allam 9636e43567 fix(webapp): reduce error-level log noise for handled/benign cases (#3403)
Two changes to cut error volume from logs that represent handled
conditions, not real errors (combined ~1600/hr in prod):

1. api.v1.waitpoints.tokens.$waitpointFriendlyId.complete.ts

The route throws `json(..., { status: 404 })` when a waitpoint
isn't found, but the generic catch block caught that Response,
logged it as an error (with an empty {} body because Error fields
are non-enumerable), and rethrew as a 500 — so clients saw a 500
instead of the intended 404, and every stale-waitpoint request
produced a Sentry event.

Fix: re-throw Response objects unchanged so the correct status
propagates and we don't log user 404s as errors. Also serialize
remaining Error instances explicitly (name/message/stack) so the
logs are actionable when we do hit a real error.

2. v3/marqs/sharedQueueConsumer.server.ts:603

"Task run has invalid status for execution. Going to ack" — the
message itself says we're handling it gracefully. Benign race
between dequeue and completion/cancellation. Demote to warn.
2026-04-17 09:34:13 +01:00
devin-ai-integration[bot] ff290dfe2f perf(run-engine): merge dequeue snapshot creation into taskRun.update transaction [TRI-8450] (#3395)
## Summary

Nests the `TaskRunExecutionSnapshot` creation inside the
`taskRun.update()` Prisma call in the dequeue flow, reducing **2 DB
commits → 1** per dequeue operation. This is the highest-volume of the
five unmerged flows identified in TRI-8450 (~9,200 commits/sec on the
engine service).

**Pattern**: Follows the same nested-write approach already used in the
completion path (`runAttemptSystem.ts:735`) and trigger path
(`engine/index.ts:674`).

**Changes**:
- `dequeueSystem.ts`: Moved snapshot creation into `executionSnapshots:
{ create: {...} }` within the existing `taskRun.update()`. Pre-generates
the snapshot ID via `generateInternalId()` (plain cuid, matching what
Prisma's `@default(cuid())` produces) so the event emission, heartbeat
enqueue, and return value can all be constructed from data already in
scope — **no extra DB read needed** after the merged write.
`SnapshotId.toFriendlyId()` is used only for the return value's
`friendlyId` field, matching the original `createExecutionSnapshot`
behavior.
- `executionSnapshotSystem.ts`: Added public
`enqueueHeartbeatIfNeeded()` method that exposes the heartbeat
scheduling logic (previously only available internally via
`createExecutionSnapshot`). This is needed because `PENDING_EXECUTING`
requires a heartbeat, unlike the `FINISHED` status in the completion
reference pattern. This method is reusable by future merge targets
(retry-immediate, checkpoint, cancel, requeue).

**Net DB change per dequeue**: eliminates 1 write transaction (the
separate `TaskRunExecutionSnapshot.create`). No extra reads added — the
snapshot ID is pre-generated and the `executionSnapshotCreated` event
payload is constructed inline from values already available in the
closure.

## Review & Testing Checklist for Human

- [ ] **Verify manually-constructed event payload matches DB state**:
The `executionSnapshotCreated` event is now built inline (not read back
from DB). Confirm the field values (`runStatus: "PENDING"`,
`attemptNumber`, `checkpointId`, `workerId`, `runnerId`,
`completedWaitpointIds`) match what Prisma actually writes. A mismatch
here would be silent — event consumers would get stale/wrong data.
- [ ] **Verify `attemptNumber` source is equivalent**: Old code used
`lockedTaskRun.attemptNumber` (post-update result). New code uses
`result.run.attemptNumber` (pre-update). The `taskRun.update()` data
payload does NOT include `attemptNumber`, so they should be identical —
but confirm this assumption holds for all dequeue scenarios (e.g.
retried runs).
- [ ] **Verify `isValid` defaults to `true` in schema**: The old
`createExecutionSnapshot` explicitly set `isValid: error ? false :
true`. The nested create omits `isValid` (no error in the dequeue happy
path). Confirm the Prisma schema default for
`TaskRunExecutionSnapshot.isValid` is `true`.
- [ ] **Verify `runStatus: "PENDING"` hardcoding matches the mapping**:
The old code passed `lockedTaskRun.status` ("DEQUEUED") to
`createExecutionSnapshot`, which mapped it to "PENDING" via `run.status
=== "DEQUEUED" ? "PENDING" : run.status`. The new code hardcodes
`"PENDING"` directly. This is correct but brittle if `status` ever
changes from "DEQUEUED" to something else upstream.
- [ ] **Spot-check `completedWaitpoints` connect + order logic**: The
nested create replicates the connect/order logic from
`createExecutionSnapshot` (lines 387-393). Verify the
`snapshot.completedWaitpoints` type provides `id` and `index` fields
compatible with this usage.
- [ ] **Verify `checkpoint` in return value**: The return now uses
`snapshot.checkpoint` (from the *previous* snapshot) instead of reading
the newly-created snapshot's checkpoint relation. Since `checkpointId`
is passed through unchanged, they should be identical — but worth a
sanity check.

**Recommended test plan**: deploy to staging, run the
`sample_pg_activity.py` sampler for a 5-minute window, and verify the
COMMIT count drop on the engine service + proportional `IO:XactSync`
reduction.

### Notes
- This only covers the **dequeue** flow (flow #1 from TRI-8450). The
remaining four flows (retry-immediate, checkpoint, requeue, cancel) are
separate follow-ups.
- The new `enqueueHeartbeatIfNeeded` method is deliberately designed for
reuse by those follow-up PRs.
- CI note: the `priority.test.ts` failure in shard 7 is a flaky ordering
assertion unrelated to this change (it compares `friendlyId` values in
dequeue order). The `audit` check is also pre-existing/unrelated.

Link to Devin session:
https://app.devin.ai/sessions/034fe0e7224f49278a2de260203e1377
Requested by: @ericallam

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Eric Allam <eallam@icloud.com>
2026-04-16 15:43:16 +01:00
Eric Allam 67d2025f33 feat(webapp): add 60s/60s SWR cache to getEntitlement (#3388)
Wraps getEntitlement in platform.v3.server.ts with the existing
platformCache (LRU memory + Redis) under a new `entitlement` namespace.
Eliminates a synchronous billing-service HTTP round trip on every
trigger.

Cache config: 60s fresh / 60s stale SWR. Cache key is the
organization id. Errors are caught inside the loader and return the
existing permissive { hasAccess: true } fallback, which is also
cached to prevent thundering-herd on billing outages.

Trade-off: plan upgrade/downgrade is now visible after up to ~120s
worst-case (60s fresh + 60s stale revalidation). Acceptable since
the existing limits and usage namespaces use 5min/10min, and the
defensive hasAccess: true fallback already exists.
2026-04-16 15:37:27 +01:00
Eric Allam 79b6053e13 feat(server): add TaskIdentifier registry to replace expensive distinct query (#3368)
Replace the expensive DISTINCT query for task filter dropdowns with a
dedicated TaskIdentifier registry table backed by Redis. Environments
migrate automatically on their next deploy, with a transparent fallback
to the legacy query for unmigrated environments. Also fixes duplicate
dropdown entries when a task changes trigger source, and adds
active/archived grouping for removed tasks. Moves BackgroundWorkerTask
reads in the trigger hot path to the read replica.
2026-04-16 15:22:19 +01:00
Eric Allam 94abe97132 fix(webapp): prevent dashboard crash when span accessory text is not a string (#3400) 2026-04-16 15:15:23 +01:00
Eric Allam 02d2334c8a fix(webapp): fix Redis connection leak in realtime streams and broken abort signal propagation (#3399)
Pool Redis connections for non-blocking ops (ingestData, appendPart,
getLastChunkIndex)
using a shared singleton instead of new Redis() per request. Use
redis.disconnect()
for immediate teardown in streamResponse cleanup. Add 15s inactivity
timeout fallback.

Fix broken request.signal in Remix/Express by wiring Express
res.on('close') to an
AbortController via httpAsyncStorage. All SSE/streaming routes now use
getRequestAbortSignal() which fires reliably on client disconnect,
bypassing the
Node.js undici GC bug (nodejs/node#55428) that severs the signal chain.
2026-04-16 15:15:10 +01:00
Eric Allam f5b4d34c4b chore: allow Devin bot in claude-code-action workflows (#3401) 2026-04-16 15:33:20 +02:00
nicktrn 93f2ca6bf4 feat(webapp): extend admin workers endpoint and unify admin api auth (#3390)
Extends the admin worker groups endpoint with a GET loader and more
fields on POST (type, hidden, workloadType, cloudProvider, location,
staticIPs, enableFastPath), and pulls the PAT + admin check that was
inlined or locally duplicated across every admin.api route into a shared
helper in personalAccessToken.server.ts. The generic
authenticateAdminRequest returns a discriminated result;
requireAdminApiRequest is the thin Remix loader/action wrapper that
throws. The neverthrow-style route (platform-notifications.ts) now
composes the generic helper instead of duplicating the check. Verified
locally against GET (listing) and POST (new fields, invalid enum,
minimal backwards-compat).
2026-04-16 13:46:53 +01:00
Eric Allam 0c33de88f0 chore: add Devin bot to vouch list and skip draft requirement (#3396) 2026-04-16 11:38:45 +01:00
Eric Allam 7c95207486 fix(run-engine): Stop querying for associated run tags during dequeue (#3379) 2026-04-15 16:38:32 +01:00
devin-ai-integration[bot] 7d82041809 fix(security): upgrade Remix packages 2.1.0 → 2.17.4 (#3372)
## Summary

Upgrades all `@remix-run/*` packages in `apps/webapp` from **2.1.0 →
2.17.4** to address security vulnerabilities. Recreation of #2951 on a
fresh checkout of `main`.

**Updated packages (`apps/webapp/package.json`):**
- `@remix-run/express`, `@remix-run/node`, `@remix-run/react`,
`@remix-run/serve`, `@remix-run/server-runtime`: 2.1.0 → 2.17.4
- `@remix-run/router`: ^1.15.3 → ^1.23.2
- `@remix-run/dev`, `@remix-run/eslint-config`, `@remix-run/testing`:
2.1.0 → 2.17.4

**Root `package.json` overrides:**
- `@remix-run/dev@2.17.4>tar-fs`: 2.1.3 → 2.1.4
- `testcontainers@10.28.0>tar-fs`: 3.0.9 → 3.1.1

**Documentation:** Updated Remix version references in `CLAUDE.md`,
`apps/webapp/CLAUDE.md`, and `.cursor/rules/webapp.mdc`.

**Server changes:** Added `.server-changes/upgrade-remix-security.md`
for release tracking per `CONTRIBUTING.md`.

No application code changes — only `package.json` files, documentation,
a server-changes entry, and the regenerated `pnpm-lock.yaml`.

### Updates since last revision

Addressed all 3 Devin Review findings:
1. **Missing `.server-changes/` file** — added
`.server-changes/upgrade-remix-security.md` (commit ce22a0bd4)
2. **Sentry Remix patch (`@sentry/remix@9.46.0`)** — verified the patch
at `patches/@sentry__remix@9.46.0.patch` applies cleanly against 2.17.4.
The patch modifies Sentry's own `RemixInstrumentation` wrapper (removing
`request.clone()` and form data attributes), not Remix internals. The
underlying Remix APIs it hooks into (`callRouteAction`,
`callRouteLoader`) are stable across 2.1→2.17.
3. **`remix-typedjson@0.3.1` compatibility** — peer deps declare
`@remix-run/react: ^1.16.0 || ^2.0`, covering 2.17.4. Confirmed working
at runtime across all 22 tested pages that use it (root.tsx, hooks,
route loaders).

### Verification performed during this session

- **Runtime:** Express+Remix integration, magic link login, client-side
routing, MetaFunction rendering
- **Operational:** hello-world task triggered via API, runs list, run
detail, tasks page
- **Comprehensive UI:** 22 pages, 11 filter types, environment/project
switchers, interactive elements
- **Docker:** Production Dockerfile (`docker/webapp/Dockerfile`) builds
successfully
- **Changelog audit:** All 16 minor versions reviewed — every breaking
change is behind opt-in future flags the webapp doesn't enable

## Review & Testing Checklist for Human

- [ ] **Verify auth flows in staging** — `remix-auth`,
`remix-auth-email-link`, and `remix-auth-github` declare peer deps on
`@remix-run/server-runtime@^1.x`, which is now 2.17.4. Login (magic link
+ OAuth) should be tested in a staging environment since local dev
testing may not exercise all auth code paths.
- [ ] **Verify tar-fs override versions** resolve the targeted security
advisories (2.1.4 and 3.1.1)
- [ ] **Review new transitive dependencies** added by the upgrade:
`turbo-stream@2.4.1`, `undici@6.25.0`, `valibot@1.3.1`, `ws@7.5.10`

Recommended test plan: deploy to staging and exercise core webapp flows
— login (email magic link + GitHub OAuth), dashboard navigation, task
triggering/viewing, and API endpoints — to catch runtime regressions not
covered by local testing.

### Notes
- Peer dependency warnings for `remix-auth-*` packages (expecting
`@remix-run/server-runtime@^1.x`) were present in the original PR #2951
as well and appear to be pre-existing
- The lockfile diff is large (~1200 lines) but mechanical — driven by
the Remix version bump cascading through transitive dependencies
- CI failures (`audit`, `units/internal/1-of-8`) are unrelated: `audit`
is a `claude-code-action` bot permissions issue; the internal test
failure is a ClickHouse testcontainers `Failed to connect to Reaper`
flake

Link to Devin session:
https://app.devin.ai/sessions/d9fa9953b9bf40e5a8d12b8f5ba5b86b
Requested by: @ericallam

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Eric Allam <eallam@icloud.com>
2026-04-15 13:50:22 +01:00
Eric Allam 73ea5865d8 fix(run-engine): distinguish oneTimeUseToken P2002 from idempotency key collision (#3374)
Prevent retrying when retrying won’t actually do any good
2026-04-15 10:42:58 +01:00
Oskar Otwinowski 097fab0d8c feat(webapp): Vercel integration - disable auto promotions (#3376)
<img width="1044" height="709" alt="image"
src="https://github.com/user-attachments/assets/7f2cf25c-3b74-46a9-8794-41be077e04bf"
/>
2026-04-14 19:29:31 +02:00
Iss 7fdb2c4d1e docs: troubleshooting and additional packages version pinning (#3092)
- Connection error troubleshooting
- Additional packages version pinning
- Realtime stream error troubleshooting
2026-04-14 15:11:06 +01:00
Iss f0f4527655 docs: added startup_timeout_sec note (#3124) 2026-04-14 15:10:49 +01:00
Iss b8ce6939b6 docs: adds deduplication key clarification (#3151) 2026-04-14 15:10:04 +01:00
Iss 54d22e9ee1 docs: add per-task middleware section to tasks overview (#3197) 2026-04-14 15:09:48 +01:00
Iss 8b4ac45aed docs: add Bun runtime setup for Sentry error tracking (#3233) 2026-04-14 15:09:32 +01:00
Iss 57a634ea50 docs: note retry.onThrow as a parallel wait (#3248) 2026-04-14 15:09:16 +01:00
Iss c37bb9abc8 docs: add Nango OAuth integration guide (#3262)
Adds a guide showing how to use Nango to make authenticated API calls
inside a Trigger.dev task, using GitHub + Claude as a concrete example.
2026-04-14 15:08:53 +01:00
Iss f2b1b76a07 docs: add list deployments endpoint, fix defaultMachine, and clarify wait token browser completion (#3350)
Adds missing list deployments API page, fixes defaultMachine → machine
in config docs, and clarifies browser CORS usage for wait token
completion with corrected warning placement
2026-04-14 15:08:33 +01:00
Eric Allam f739a5c545 fix(db): add index to ProjectAlertStorage to prevent sequence scans (#3349) 2026-04-14 15:04:16 +01:00
Eric Allam ed0c3e4f38 fix: stop creating TaskRunTag records and join table entries during triggering (#3369)
The TaskRun.runTags string array already stores tag names, making the
TaskRunTag M2M relation redundant write overhead. Remove createTags
calls, connect: tags, and join table writes from both V1 and V2 trigger
paths. Simplify the add-tags API to just push to runTags directly.
2026-04-14 05:40:51 +01:00
Eric Allam c09983ef19 docs(cli): Expand and improve the MCP server and dev CLI command (#3225)
Depends on #3224
2026-04-13 14:27:45 +01:00
Eric Allam 417ab876e3 fix(batch-queue): Batch items that hit the environment queue size limit now fast-fail (#3352) 2026-04-13 14:24:38 +01:00
nicktrn 4d7fbf0b1b docs: add task-level and config-level TTL documentation (#3200)
Documents TTL support at task-level and config-level. Companion to #3196
- merge after new packages are released.
2026-04-13 14:23:24 +01:00
github-actions[bot] 5ea36e08f2 chore: release v4.4.4 (#3228)
🚀 Publish Trigger.dev Docker / typecheck (push) Failing after 0s
🚀 Publish Trigger.dev Docker / publish-webapp (push) Has been skipped
🚀 Publish Trigger.dev Docker / publish-worker (push) Has been skipped
🚀 Publish Trigger.dev Docker / publish-worker-v4 (push) Has been skipped
🚀 Publish Trigger.dev Docker / units (push) Failing after 0s
## Summary
12 new features, 59 improvements, 17 bug fixes.

## Highlights

- Add support for setting TTL (time-to-live) defaults at the task level
and globally in trigger.config.ts, with per-trigger overrides still
taking precedence
([#3196](https://github.com/triggerdotdev/trigger.dev/pull/3196))
- Large run outputs can use the new API which allows switching object
storage providers.
([#3275](https://github.com/triggerdotdev/trigger.dev/pull/3275))

## Improvements
- Add platform notifications support to the CLI. The `trigger dev` and
`trigger login` commands now fetch and display platform notifications
(info, warn, error, success) from the server. Includes discovery-based
filtering to conditionally show notifications based on project file
patterns, color markup rendering for styled terminal output, and a
non-blocking display flow with a spinner fallback for slow fetches. Use
`--skip-platform-notifications` flag with `trigger dev` to disable the
notification check.
([#3254](https://github.com/triggerdotdev/trigger.dev/pull/3254))
- Add `get_span_details` MCP tool for inspecting individual spans within
a run trace.
([#3255](https://github.com/triggerdotdev/trigger.dev/pull/3255))
- New `get_span_details` tool returns full span attributes, timing,
events, and AI enrichment (model, tokens, cost, speed)
- Span IDs now shown in `get_run_details` trace output for easy
discovery
- New API endpoint `GET /api/v1/runs/:runId/spans/:spanId`
- New `retrieveSpan()` method on the API client
- `get_query_schema` — discover available TRQL tables and columns
- `query` — execute TRQL queries against your data
- `list_dashboards` — list built-in dashboards and their widgets
- `run_dashboard_query` — execute a single dashboard widget query
- `whoami` — show current profile, user, and API URL
- `list_profiles` — list all configured CLI profiles
- `switch_profile` — switch active profile for the MCP session
- `start_dev_server` — start `trigger dev` in the background and stream
output
- `stop_dev_server` — stop the running dev server
- `dev_server_status` — check dev server status and view recent logs
- `GET /api/v1/query/schema` — query table schema discovery
- `GET /api/v1/query/dashboards` — list built-in dashboards
- `--readonly` flag hides write tools (`deploy`, `trigger_task`,
`cancel_run`) so the AI cannot make changes
- `read:query` JWT scope for query endpoint authorization
- `get_run_details` trace output is now paginated with cursor support
- MCP tool annotations (`readOnlyHint`, `destructiveHint`) for all tools
- `get_query_schema` now requires a table name and returns only one
table's schema (was returning all tables)
- `get_current_worker` no longer inlines payload schemas; use new
`get_task_schema` tool instead
- Query results formatted as text tables instead of JSON (~50% fewer
tokens)
- `cancel_run`, `list_deploys`, `list_preview_branches` formatted as
text instead of raw JSON
- Schema and dashboard API responses cached to avoid redundant fetches
- Adapted the CLI API client to propagate the trigger source via http
headers.
([#3241](https://github.com/triggerdotdev/trigger.dev/pull/3241))
- Propagate run tags to span attributes so they can be extracted
server-side for LLM cost attribution metadata.
([#3213](https://github.com/triggerdotdev/trigger.dev/pull/3213))
- New `get_span_details` tool returns full span attributes, timing,
events, and AI enrichment (model, tokens, cost, speed)
- Span IDs now shown in `get_run_details` trace output for easy
discovery
- New API endpoint `GET /api/v1/runs/:runId/spans/:spanId`
- New `retrieveSpan()` method on the API client
- `get_query_schema` — discover available TRQL tables and columns
- `query` — execute TRQL queries against your data
- `list_dashboards` — list built-in dashboards and their widgets
- `run_dashboard_query` — execute a single dashboard widget query
- `whoami` — show current profile, user, and API URL
- `list_profiles` — list all configured CLI profiles
- `switch_profile` — switch active profile for the MCP session
- `start_dev_server` — start `trigger dev` in the background and stream
output
- `stop_dev_server` — stop the running dev server
- `dev_server_status` — check dev server status and view recent logs
- `GET /api/v1/query/schema` — query table schema discovery
- `GET /api/v1/query/dashboards` — list built-in dashboards
- `--readonly` flag hides write tools (`deploy`, `trigger_task`,
`cancel_run`) so the AI cannot make changes
- `read:query` JWT scope for query endpoint authorization
- `get_run_details` trace output is now paginated with cursor support
- MCP tool annotations (`readOnlyHint`, `destructiveHint`) for all tools
- `get_query_schema` now requires a table name and returns only one
table's schema (was returning all tables)
- `get_current_worker` no longer inlines payload schemas; use new
`get_task_schema` tool instead
- Query results formatted as text tables instead of JSON (~50% fewer
tokens)
- `cancel_run`, `list_deploys`, `list_preview_branches` formatted as
text instead of raw JSON
- Schema and dashboard API responses cached to avoid redundant fetches
- Add optional `hasPrivateLink` field to the dequeue message
organization object for private networking support
([#3264](https://github.com/triggerdotdev/trigger.dev/pull/3264))
- Define and manage AI prompts with `prompts.define()`. Create typesafe
prompt templates with variables, resolve them at runtime, and manage
versions and overrides from the dashboard without redeploying.
([#3244](https://github.com/triggerdotdev/trigger.dev/pull/3244))

## Bug fixes
- Fix dev CLI leaking build directories on rebuild, causing disk space
accumulation. Deprecated workers are now pruned (capped at 2 retained)
when no active runs reference them. The watchdog process also cleans up
`.trigger/tmp/` when the dev CLI is killed ungracefully (e.g. SIGKILL
from pnpm).
([#3224](https://github.com/triggerdotdev/trigger.dev/pull/3224))
- Fix `--load` flag being silently ignored on local/self-hosted builds.
([#3114](https://github.com/triggerdotdev/trigger.dev/pull/3114))
- Fixed `search_docs` tool failing due to renamed upstream Mintlify tool
(`SearchTriggerDev` → `search_trigger_dev`)
- Fixed `list_deploys` failing when deployments have null
`runtime`/`runtimeVersion` fields (#3139)
- Fixed `list_preview_branches` crashing due to incorrect response shape
access
- Fixed `metrics` table column documented as `value` instead of
`metric_value` in query docs
- Fixed dev CLI leaking build directories on rebuild — deprecated
workers now clean up their build dirs when their last run completes
- Fixed `search_docs` tool failing due to renamed upstream Mintlify tool
(`SearchTriggerDev` → `search_trigger_dev`)
- Fixed `list_deploys` failing when deployments have null
`runtime`/`runtimeVersion` fields (#3139)
- Fixed `list_preview_branches` crashing due to incorrect response shape
access
- Fixed `metrics` table column documented as `value` instead of
`metric_value` in query docs
- Fixed dev CLI leaking build directories on rebuild — deprecated
workers now clean up their build dirs when their last run completes

## Server changes

These changes affect the self-hosted Docker image and Trigger.dev Cloud:

- Add admin UI for viewing and editing feature flags (org-level
overrides and global defaults).
([#3291](https://github.com/triggerdotdev/trigger.dev/pull/3291))
- AI prompt management dashboard and enhanced span inspectors.
  
  **Prompt management:**
- Prompts list page with version status, model, override indicators, and
24h usage sparklines
- Prompt detail page with template viewer, variable preview, version
history timeline, and override editor
- Create, edit, and remove overrides to change prompt content or model
without redeploying
  - Promote any code-deployed version to current
- Generations tab with infinite scroll, live polling, and inline span
inspector
- Per-prompt metrics: total generations, avg tokens, avg cost, latency,
with version-level breakdowns
  
  **AI span inspectors:**
- Custom inspectors for `ai.generateText`, `ai.streamText`,
`ai.generateObject`, `ai.streamObject` parent spans
- `ai.toolCall` inspector showing tool name, call ID, and input
arguments
  - `ai.embed` inspector showing model, provider, and input text
- Prompt tab on AI spans linking to prompt version with template and
input variables
  - Compact timestamp and duration header on all AI span inspectors
  
  **AI metrics dashboard:**
- Operations, Providers, and Prompts filters on the AI Metrics dashboard
  - Cost by prompt widget
  - "AI" section in the sidebar with Prompts and AI Metrics links
  
  **Other improvements:**
  - Resizable panel sizes now persist across page refreshes
- Fixed `<div>` inside `<p>` DOM nesting warnings in span titles and
chat messages
([#3244](https://github.com/triggerdotdev/trigger.dev/pull/3244))
- Add allowRollbacks query param to the promote deployment API to enable
version downgrades
([#3214](https://github.com/triggerdotdev/trigger.dev/pull/3214))
- Pre-warm compute templates on deploy for orgs with compute access.
Required for projects using a compute region, background-only for
others.
([#3114](https://github.com/triggerdotdev/trigger.dev/pull/3114))
- Add automatic LLM cost calculation for spans with GenAI semantic
conventions. When a span arrives with `gen_ai.response.model` and token
usage data, costs are calculated from an in-memory pricing registry
backed by Postgres and dual-written to both span attributes
(`trigger.llm.*`) and a new `llm_metrics_v1` ClickHouse table that
captures usage, cost, performance (TTFC, tokens/sec), and behavioral
(finish reason, operation type) metrics.
([#3213](https://github.com/triggerdotdev/trigger.dev/pull/3213))
- Add API endpoint `GET /api/v1/runs/:runId/spans/:spanId` that returns
detailed span information including properties, events, AI enrichment
(model, tokens, cost), and triggered child runs.
([#3255](https://github.com/triggerdotdev/trigger.dev/pull/3255))
- Multi-provider object storage with protocol-based routing for
zero-downtime migration
([#3275](https://github.com/triggerdotdev/trigger.dev/pull/3275))
- Add IAM role-based auth support for object stores (no access keys
required).
([#3275](https://github.com/triggerdotdev/trigger.dev/pull/3275))
- Add platform notifications to inform users about new features,
changelogs, and platform events directly in the dashboard.
([#3254](https://github.com/triggerdotdev/trigger.dev/pull/3254))
- Add private networking support via AWS PrivateLink. Includes
BillingClient methods for managing private connections, org settings UI
pages for connection management, and supervisor changes to apply
`privatelink` pod labels for CiliumNetworkPolicy matching.
([#3264](https://github.com/triggerdotdev/trigger.dev/pull/3264))
- Reduce run start latency by skipping the intermediate queue when
concurrency is available. This optimization is rolled out per-region and
enabled automatically for development environments.
([#3299](https://github.com/triggerdotdev/trigger.dev/pull/3299))
- Extended the search filter on the environment variables page to match
on environment type (production, staging, development, preview) and
branch name, not just variable name and value.
([#3302](https://github.com/triggerdotdev/trigger.dev/pull/3302))
- Set `application_name` on Prisma connections from SERVICE_NAME so DB
load can be attributed by service
([#3348](https://github.com/triggerdotdev/trigger.dev/pull/3348))
- Fix transient R2/object store upload failures during batchTrigger()
item streaming.
  
- Added p-retry (3 attempts, 500ms–2s exponential backoff) around
`uploadPacketToObjectStore` in `BatchPayloadProcessor.process()` so
transient network errors self-heal server-side rather than aborting the
entire batch stream.
- Removed `x-should-retry: false` from the 500 response on the batch
items route so the SDK's existing 5xx retry path can recover if
server-side retries are exhausted. Item deduplication by index makes
full-stream retries safe.
([#3331](https://github.com/triggerdotdev/trigger.dev/pull/3331))
- Concurrency-keyed queues now use a single master queue entry per base
queue instead of one entry per key. Prevents high-CK-count tenants from
consuming the entire parentQueueLimit window and starving other tenants
on the same shard.
([#3219](https://github.com/triggerdotdev/trigger.dev/pull/3219))
- Reduce lock contention when processing large `batchTriggerAndWait`
batches. Previously, each batch item acquired a Redis lock on the parent
run to insert a `TaskRunWaitpoint` row, causing
`LockAcquisitionTimeoutError` with high concurrency (880 errors/24h in
prod). Since `blockRunWithCreatedBatch` already transitions the parent
to `EXECUTING_WITH_WAITPOINTS` before items are processed, the per-item
lock is unnecessary. The new `blockRunWithWaitpointLockless` method
performs only the idempotent CTE insert without acquiring the lock.
([#3232](https://github.com/triggerdotdev/trigger.dev/pull/3232))
- Strip `secure` query parameter from QUERY_CLICKHOUSE_URL before
passing to ClickHouse client. This was already done for the main and
logs ClickHouse clients but was missing for the query client, causing a
startup crash with `Error: Unknown URL parameters: secure`.
([#3204](https://github.com/triggerdotdev/trigger.dev/pull/3204))
- Fix `OrganizationsPresenter.#getEnvironment` matching the wrong
development environment on teams with multiple members. All dev
environments share the slug `"dev"`, so the previous `find` by slug
alone could return another member's environment. Now filters DEVELOPMENT
environments by `orgMember.userId` to ensure the logged-in user's dev
environment is selected.
([#3273](https://github.com/triggerdotdev/trigger.dev/pull/3273))

<details>
<summary>Raw changeset output</summary>

# Releases
## @trigger.dev/build@4.4.4

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.4.4`

## trigger.dev@4.4.4

### Patch Changes

- Add platform notifications support to the CLI. The `trigger dev` and
`trigger login` commands now fetch and display platform notifications
(info, warn, error, success) from the server. Includes discovery-based
filtering to conditionally show notifications based on project file
patterns, color markup rendering for styled terminal output, and a
non-blocking display flow with a spinner fallback for slow fetches. Use
`--skip-platform-notifications` flag with `trigger dev` to disable the
notification check.
([#3254](https://github.com/triggerdotdev/trigger.dev/pull/3254))

- Fix dev CLI leaking build directories on rebuild, causing disk space
accumulation. Deprecated workers are now pruned (capped at 2 retained)
when no active runs reference them. The watchdog process also cleans up
`.trigger/tmp/` when the dev CLI is killed ungracefully (e.g. SIGKILL
from pnpm).
([#3224](https://github.com/triggerdotdev/trigger.dev/pull/3224))

- Fix `--load` flag being silently ignored on local/self-hosted builds.
([#3114](https://github.com/triggerdotdev/trigger.dev/pull/3114))

- Add `get_span_details` MCP tool for inspecting individual spans within
a run trace.
([#3255](https://github.com/triggerdotdev/trigger.dev/pull/3255))

- New `get_span_details` tool returns full span attributes, timing,
events, and AI enrichment (model, tokens, cost, speed)
- Span IDs now shown in `get_run_details` trace output for easy
discovery
    -   New API endpoint `GET /api/v1/runs/:runId/spans/:spanId`
    -   New `retrieveSpan()` method on the API client

- MCP server improvements: new tools, bug fixes, and new flags.
([#3224](https://github.com/triggerdotdev/trigger.dev/pull/3224))

    **New tools:**

    -   `get_query_schema` — discover available TRQL tables and columns
    -   `query` — execute TRQL queries against your data
    -   `list_dashboards` — list built-in dashboards and their widgets
    -   `run_dashboard_query` — execute a single dashboard widget query
    -   `whoami` — show current profile, user, and API URL
    -   `list_profiles` — list all configured CLI profiles
    -   `switch_profile` — switch active profile for the MCP session
- `start_dev_server` — start `trigger dev` in the background and stream
output
    -   `stop_dev_server` — stop the running dev server
- `dev_server_status` — check dev server status and view recent logs

    **New API endpoints:**

    -   `GET /api/v1/query/schema` — query table schema discovery
    -   `GET /api/v1/query/dashboards` — list built-in dashboards

    **New features:**

- `--readonly` flag hides write tools (`deploy`, `trigger_task`,
`cancel_run`) so the AI cannot make changes
    -   `read:query` JWT scope for query endpoint authorization
- `get_run_details` trace output is now paginated with cursor support
- MCP tool annotations (`readOnlyHint`, `destructiveHint`) for all tools

    **Bug fixes:**

- Fixed `search_docs` tool failing due to renamed upstream Mintlify tool
(`SearchTriggerDev` → `search_trigger_dev`)
- Fixed `list_deploys` failing when deployments have null
`runtime`/`runtimeVersion` fields (#3139)
- Fixed `list_preview_branches` crashing due to incorrect response shape
access
- Fixed `metrics` table column documented as `value` instead of
`metric_value` in query docs
- Fixed dev CLI leaking build directories on rebuild — deprecated
workers now clean up their build dirs when their last run completes

    **Context optimizations:**

- `get_query_schema` now requires a table name and returns only one
table's schema (was returning all tables)
- `get_current_worker` no longer inlines payload schemas; use new
`get_task_schema` tool instead
- Query results formatted as text tables instead of JSON (~50% fewer
tokens)
- `cancel_run`, `list_deploys`, `list_preview_branches` formatted as
text instead of raw JSON
- Schema and dashboard API responses cached to avoid redundant fetches

- Add support for setting TTL (time-to-live) defaults at the task level
and globally in trigger.config.ts, with per-trigger overrides still
taking precedence
([#3196](https://github.com/triggerdotdev/trigger.dev/pull/3196))

- Adapted the CLI API client to propagate the trigger source via http
headers.
([#3241](https://github.com/triggerdotdev/trigger.dev/pull/3241))

-   Updated dependencies:
    -   `@trigger.dev/core@4.4.4`
    -   `@trigger.dev/build@4.4.4`
    -   `@trigger.dev/schema-to-json@4.4.4`

## @trigger.dev/core@4.4.4

### Patch Changes

- Fix `list_deploys` MCP tool failing when deployments have null
`runtime` or `runtimeVersion` fields.
([#3224](https://github.com/triggerdotdev/trigger.dev/pull/3224))

- Propagate run tags to span attributes so they can be extracted
server-side for LLM cost attribution metadata.
([#3213](https://github.com/triggerdotdev/trigger.dev/pull/3213))

- Add `get_span_details` MCP tool for inspecting individual spans within
a run trace.
([#3255](https://github.com/triggerdotdev/trigger.dev/pull/3255))

- New `get_span_details` tool returns full span attributes, timing,
events, and AI enrichment (model, tokens, cost, speed)
- Span IDs now shown in `get_run_details` trace output for easy
discovery
    -   New API endpoint `GET /api/v1/runs/:runId/spans/:spanId`
    -   New `retrieveSpan()` method on the API client

- MCP server improvements: new tools, bug fixes, and new flags.
([#3224](https://github.com/triggerdotdev/trigger.dev/pull/3224))

    **New tools:**

    -   `get_query_schema` — discover available TRQL tables and columns
    -   `query` — execute TRQL queries against your data
    -   `list_dashboards` — list built-in dashboards and their widgets
    -   `run_dashboard_query` — execute a single dashboard widget query
    -   `whoami` — show current profile, user, and API URL
    -   `list_profiles` — list all configured CLI profiles
    -   `switch_profile` — switch active profile for the MCP session
- `start_dev_server` — start `trigger dev` in the background and stream
output
    -   `stop_dev_server` — stop the running dev server
- `dev_server_status` — check dev server status and view recent logs

    **New API endpoints:**

    -   `GET /api/v1/query/schema` — query table schema discovery
    -   `GET /api/v1/query/dashboards` — list built-in dashboards

    **New features:**

- `--readonly` flag hides write tools (`deploy`, `trigger_task`,
`cancel_run`) so the AI cannot make changes
    -   `read:query` JWT scope for query endpoint authorization
- `get_run_details` trace output is now paginated with cursor support
- MCP tool annotations (`readOnlyHint`, `destructiveHint`) for all tools

    **Bug fixes:**

- Fixed `search_docs` tool failing due to renamed upstream Mintlify tool
(`SearchTriggerDev` → `search_trigger_dev`)
- Fixed `list_deploys` failing when deployments have null
`runtime`/`runtimeVersion` fields (#3139)
- Fixed `list_preview_branches` crashing due to incorrect response shape
access
- Fixed `metrics` table column documented as `value` instead of
`metric_value` in query docs
- Fixed dev CLI leaking build directories on rebuild — deprecated
workers now clean up their build dirs when their last run completes

    **Context optimizations:**

- `get_query_schema` now requires a table name and returns only one
table's schema (was returning all tables)
- `get_current_worker` no longer inlines payload schemas; use new
`get_task_schema` tool instead
- Query results formatted as text tables instead of JSON (~50% fewer
tokens)
- `cancel_run`, `list_deploys`, `list_preview_branches` formatted as
text instead of raw JSON
- Schema and dashboard API responses cached to avoid redundant fetches

- Large run outputs can use the new API which allows switching object
storage providers.
([#3275](https://github.com/triggerdotdev/trigger.dev/pull/3275))

- Add optional `hasPrivateLink` field to the dequeue message
organization object for private networking support
([#3264](https://github.com/triggerdotdev/trigger.dev/pull/3264))

- Add support for setting TTL (time-to-live) defaults at the task level
and globally in trigger.config.ts, with per-trigger overrides still
taking precedence
([#3196](https://github.com/triggerdotdev/trigger.dev/pull/3196))

- Adapted the CLI API client to propagate the trigger source via http
headers.
([#3241](https://github.com/triggerdotdev/trigger.dev/pull/3241))

## @trigger.dev/python@4.4.4

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/sdk@4.4.4`
    -   `@trigger.dev/core@4.4.4`
    -   `@trigger.dev/build@4.4.4`

## @trigger.dev/react-hooks@4.4.4

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.4.4`

## @trigger.dev/redis-worker@4.4.4

### Patch Changes

- Adapted the CLI API client to propagate the trigger source via http
headers.
([#3241](https://github.com/triggerdotdev/trigger.dev/pull/3241))
-   Updated dependencies:
    -   `@trigger.dev/core@4.4.4`

## @trigger.dev/rsc@4.4.4

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.4.4`

## @trigger.dev/schema-to-json@4.4.4

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.4.4`

## @trigger.dev/sdk@4.4.4

### Patch Changes

- Define and manage AI prompts with `prompts.define()`. Create typesafe
prompt templates with variables, resolve them at runtime, and manage
versions and overrides from the dashboard without redeploying.
([#3244](https://github.com/triggerdotdev/trigger.dev/pull/3244))
- Add support for setting TTL (time-to-live) defaults at the task level
and globally in trigger.config.ts, with per-trigger overrides still
taking precedence
([#3196](https://github.com/triggerdotdev/trigger.dev/pull/3196))
- Adapted the CLI API client to propagate the trigger source via http
headers.
([#3241](https://github.com/triggerdotdev/trigger.dev/pull/3241))
-   Updated dependencies:
    -   `@trigger.dev/core@4.4.4`

</details>

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2026-04-13 13:24:38 +01:00
Oskar Otwinowski 3c9647cb8c feat(webapp): Platform notifications admin imporovements (#3324)
- bugfix to show the changelog to the target audience
- more functionality for admins, to edit, delete and archive
notifications
2026-04-13 12:26:15 +02:00
nicktrn e59614a31c feat(webapp): gate microvm regions behind compute access feature flag (#3366)
Adds region-level gating so MICROVM regions are only visible and usable
by orgs with the `hasComputeAccess` feature flag. Admins and explicit
allowlist behavior unchanged.

- New shared helper (`regionAccess.server.ts`) with
`resolveComputeAccess`, `defaultVisibilityFilter`, and
`isComputeRegionAccessible`
- `RegionsPresenter` filters out MICROVM regions for non-compute orgs
- `SetDefaultRegionService` blocks setting a MICROVM region as default
without compute access
- `WorkerGroupService` blocks triggering runs in MICROVM regions without
compute access
- `computeTemplateCreation` refactored to use shared
`resolveComputeAccess`
- Updated snapshot callback schema
2026-04-13 11:24:00 +01:00
nicktrn bd41bb2cbd feat(webapp): set application_name on prisma connections (#3348)
Sets `application_name` on the Prisma writer and replica connection
strings using the existing `SERVICE_NAME` env var, so DB load can be
attributed by service.
2026-04-08 22:32:46 +01:00
Matt Aitken def21b26b6 fix(batch): retry R2 upload on transient failure in BatchPayloadProcessor (#3331)
⚒️ Publish Worker (v4) / build (supervisor) (push) Has been cancelled
A single "fetch failed" from the object store was aborting the entire
batch stream with no retry. Added p-retry (3 attempts, 500ms-2s backoff)
around ploadPacketToObjectStore so transient network errors self-heal
server-side instead of propagating to the SDK.
2026-04-07 15:29:10 +01:00
James Ritchie 4f2ff3d9de fix(wabapp): Fix for wrapping text on run inspector (#3328)
### Text wrapping fix

- Fixes message text not wrapping on the run inspector if there were no
spaces in the text
- Fixes inspector title truncation
- Adds a copy text button for the Message property

<img width="468" height="740" alt="CleanShot 2026-04-04 at 10 19 02@2x"
src="https://github.com/user-attachments/assets/71e42bf3-d103-44a2-b3b4-937c0b60a4bc"
/>
2026-04-04 11:02:12 +01:00
James Ritchie cf0fdde3c4 Feat(webapp): animated resizable panel (#3319)
This is a small improvement mainly with the UI Skills file:

- Animate open and close the Resizable panels
- Uses the built in animation hooks from react-window-splitter
- Includes a global variable for the animation easing and timing for
consistency


https://github.com/user-attachments/assets/50ed0019-ed12-4e08-b95c-7c6d1fe5bac0
2026-04-03 14:03:37 +01:00
James Ritchie e31b03eac2 fix(webapp): Responsive improvements for the onboarding screens (#3318)
Simple responsive breakpoint changes to the 3 onboarding screens + the
login screen to make it mobile friendlier

<img width="523" height="872" alt="CleanShot 2026-04-02 at 17 18 55"
src="https://github.com/user-attachments/assets/815d19ca-df9b-4b3e-8f6d-e00c81628679"
/>
2026-04-02 22:38:02 +01:00
Eric Allam f1f1d02f1d feat(dashboard): a few tweaks to the AI models page (#3315) 2026-04-02 16:48:36 +01:00
Oskar Otwinowski ed64042168 feat(private-links): Show assigned IPs for provisioned private links (#3317) 2026-04-02 16:47:55 +02:00
Matt Aitken 0e14b6d750 TaskRun optimizations: dropping FKs and some indexes (#3309)
## Summary

- Drop all 8 foreign key constraints on TaskRun. The run listing path is
now fully ClickHouse-backed so we no longer need Postgres to enforce
referential integrity on this table. The FK constraints add write
overhead on every insert/update with no remaining benefit. Prisma
queries are unaffected.
- Remove PostgresRunsRepository and its associated feature flag
(runsListRepository), which was the last remaining code path querying
TaskRun directly for list/count operations.
- Drop three indexes that were only useful for the Postgres run list
path and have no remaining query consumers:
- TaskRun_runtimeEnvironmentId_id_idx — was the cursor pagination index
for PostgresRunsRepository; superseded by the (runtimeEnvironmentId,
createdAt DESC) composite index
- TaskRun_scheduleId_idx — redundant with the (scheduleId, createdAt
DESC) composite index; no direct Postgres queries filter by scheduleId
alone
- TaskRun_rootTaskRunId_idx — no queries filter TaskRun by rootTaskRunId
as a WHERE clause anywhere in the codebase

All index drops use CONCURRENTLY IF EXISTS to avoid table locks in
production.

## Test plan

  - pnpm run db:migrate:deploy applies all migrations cleanly
  - pnpm run typecheck --filter webapp passes
  - Run list pages load correctly in the dashboard (ClickHouse path)
  - Scheduled task runs still trigger and appear correctly
2026-04-01 15:40:17 +01:00
James Ritchie bf736a79fc Feat(webapp): Models page UI improvements (#3308)
Lots of layout and UI improvements to the new Models page

<img width="2282" height="1356" alt="CleanShot 2026-04-01 at 14 14 35"
src="https://github.com/user-attachments/assets/4a13a291-5d5b-415f-962f-b9cfb25b887d"
/>
2026-04-01 14:48:05 +01:00
DKP f75d4d62af docs: Remove old idempotencyKey warning (#3306) 2026-04-01 12:04:28 +01:00
Matt Aitken 68e88d0d71 Object Storage seamless migration (#3275)
This allows seamless migration to different object storage.

Existing runs that have offloaded payloads/outputs will continue to use
the default object store (configured using `OBJECT_STORE_*` env vars).

You can add additional stores by setting new env vars:
- `OBJECT_STORE_DEFAULT_PROTOCOL` this determines where new run large
payloads will get stored.
- If you set that you need to set new env vars for that protocol.
  
Example:

```
OBJECT_STORE_DEFAULT_PROTOCOL=“s3"
OBJECT_STORE_S3_BASE_URL=https://s3.us-east-1.amazonaws.com
OBJECT_STORE_S3_ACCESS_KEY_ID=<val>
OBJECT_STORE_S3_SECRET_ACCESS_KEY=<val>
OBJECT_STORE_S3_REGION=us-east-1
OBJECT_STORE_S3_SERVICE=s3
```

---------

Co-authored-by: nicktrn <55853254+nicktrn@users.noreply.github.com>
2026-04-01 10:06:12 +01:00
Iss 2637e47195 docs: add migration guide from n8n to Trigger.dev (#3283)
Adds a migration reference for users moving from n8n to Trigger.dev.
Includes a concept map, four common patterns covering the
migration-specific gaps, and a full customer onboarding example. The
onboarding workflow highlights the 3-day wait pattern, an area where
n8n's execution model has known reliability issues at production scale
that Trigger.dev handles natively
2026-03-31 17:38:39 -04:00
Matt Aitken 0977c56efe Errors (versions) (#3187)
- Added versions filtering on the Errors list and page
- Added errors stacked bars to the graph on the individual error page

---------

Co-authored-by: James Ritchie <james@trigger.dev>
2026-03-31 19:06:54 +01:00
nicktrn 2ba77d89dd fix: add build step to @internal/compute package (#3303)
The @internal/compute package had its main/types pointing to
./src/index.ts with no build step. This works in dev (tsc resolves .ts
at compile time) but fails at runtime in Docker because Node.js can't
load .ts files directly.

Added tsconfig.build.json and build/clean/dev scripts matching the
pattern used by schedule-engine and other internal packages. Exports now
point to dist/.
2026-03-31 13:23:46 +01:00
Saadi Myftija 1307d97f7a fix(webapp): match environment when searching env variables (#3302)
Temporary workaround that enables filtering by environment in the
envvars page, without changing any UI.

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-03-31 12:53:05 +02:00
Eric Allam a3407287c9 feat(engine): enqueue fast path; skip the queue under certain conditions (#3299)
## Summary

Currently, every triggered run follows a two-step path through Redis:

1. **Enqueue** — A Lua script atomically adds the message to a queue
sorted set (ordered by priority-adjusted timestamp)
2. **Dequeue** — A debounced `processQueueForWorkerQueue` job fires
~500ms later, checks concurrency limits, removes the message from the
sorted set, and pushes it to a worker queue (Redis list) where workers
pick it up via `BLPOP`

This means every run pays at least ~500ms of latency between being
triggered and being available for a worker to execute, even when the
queue is empty and concurrency is wide open.

### What changed

The enqueue Lua scripts now atomically decide whether to **skip the
queue sorted set entirely** and push directly to the worker queue. This
happens inside the same Lua script that handles normal enqueue, so the
decision is atomic with respect to concurrency bookkeeping.

A run takes the **fast path** when all of these are true:
- **Fast path is enabled** for this worker queue (gated per
`WorkerInstanceGroup`)
- **No available messages** in the queue (`ZRANGEBYSCORE` finds nothing
with score ≤ now) — this respects priority ordering and allows fast path
even when the queue has future-scored messages (e.g. nacked retries with
delay)
- **Environment concurrency** has capacity
- **Queue concurrency** has capacity (including per-concurrency-key
limits for CK queues)

When the fast path is taken:
- The message is stored and pushed directly to the worker queue
(`RPUSH`)
- Concurrency slots are claimed (`SADD` to the same sets used by the
normal dequeue path)
- The `processQueueForWorkerQueue` job is **not scheduled** (no work to
do)
- TTL sorted set is skipped (the `expireRun` worker job handles TTL
independently)

When any condition fails, the existing slow path runs unchanged.

### Rollout gating

- **Development environments**: Fast path is always enabled
- **Production environments**: Gated by a new `enableFastPath` boolean
on `WorkerInstanceGroup` (defaults to `false`), allowing
region-by-region rollout

### Rolling deploy safety

Each process registers its own Lua scripts via `defineCommand`
(identified by SHA hash). Old and new processes never share scripts. The
Redis data structures are fully compatible in both directions — ack,
nack, and release operations work identically regardless of which path a
message took.

## Test plan

- [x] Fast path taken when queue is empty and concurrency available
- [x] Slow path when `enableFastPath` is false
- [x] Slow path when queue has available messages (respects priority
ordering)
- [x] Fast path when queue only has future-scored messages
- [x] Slow path when env concurrency is full
- [x] Fast-path message can be acknowledged correctly
- [x] Fast-path message can be nacked and re-enqueued to the queue
sorted set
- [x] Run all existing run-queue tests (ack, nack, CK, concurrency
sweeper, dequeue) to verify no regressions
- [x] Typecheck passes for run-engine and webapp
2026-03-31 10:16:49 +01:00
nicktrn 0e63f8317e feat: add ttl support at task and config levels (#3196)
Add TTL (time-to-live) defaults at task-level and config-level, with
precedence: per-trigger > task > config > dev default (10m).

Docs PR: #3200 (merge after packages are released)
2026-03-30 23:25:07 +01:00
Eric Allam b07567888d fix(llm-catalog): refresh default model pricing on sync (#3281)
- Rebuild llm_pricing_tiers and llm_prices in syncLlmCatalog for
source=default
- Add vitest config, sync regression tests, and pin vitest 3.1.4
- Update pnpm-lock.yaml for the new devDependency
2026-03-30 21:59:43 +01:00
nicktrn 5447cf4843 feat(webapp): admin UI for feature flag overrides (#3291)
Adds a dialog to the admin orgs page for viewing and editing per-org
feature flag overrides. Flags are introspected from the catalog so the
UI stays in sync with available flags automatically. Also adds a new tab
for global flags.

Refactors featureFlags.server.ts to split catalog definition (shared)
from server-only runtime (flags(), makeSetMultipleFlags). The shared
module exports flag metadata and validation so both the UI and API
routes can use it without pulling in server dependencies.
2026-03-30 16:55:23 +01:00
Saadi Myftija 7210bdee9b feat(supervisor): custom tolerations for scheduled runs (#3297)
Adds support for taint tolerations for scheduled runs. Useful for
selectively tolerating taints on dedicated node pools.

The new `KUBERNETES_SCHEDULED_RUN_TOLERATIONS` env variable accepts a
comma-separated list in the format key=value:effect (or key:effect for
the Exists operator).

Drive-by: renames all `KUBERNETES_SCHEDULE_*` affinity env vars to
KUBERNETES_SCHEDULED_RUN_* for clarity — this feature isn't used in
production yet or published in a tagged image; the name change is fine.
2026-03-30 13:04:13 +02:00
Oskar Otwinowski 66b70101e9 feat(webapp): Org level feature flags for Private Links (#3287) 2026-03-30 11:46:01 +02:00
nicktrn 9cb3dcb07c feat(supervisor): compute workload manager (#3114)
Adds the `ComputeWorkloadManager` for routing task execution through the
compute gateway, including full checkpoint/restore support, OTel trace
integration, and template pre-warming.

## Changes

**Compute workload manager**
(`apps/supervisor/src/workloadManager/compute.ts`)
- Routes instance create, snapshot, delete, and restore through the
compute gateway API
- Wide event logging on create with full timing and context
- Configurable gateway timeout, auth token, image digest stripping

**Compute snapshot service**
(`apps/supervisor/src/services/computeSnapshotService.ts`)
- Timer wheel for delayed snapshot dispatch (avoids wasted work on
short-lived waitpoints)
- Configurable dispatch concurrency limit
(`COMPUTE_SNAPSHOT_DISPATCH_LIMIT`)
- Snapshot-complete callback handler with suspend completion reporting
- Trace context management and OTel span emission for snapshot
operations

**OTel trace service**
(`apps/supervisor/src/services/otlpTraceService.ts`)
- Fire-and-forget OTLP span emission for compute operations (provision,
restore, snapshot)
- BigInt nanosecond conversion preserving sub-ms precision for span
ordering

**Template creation**
(`apps/webapp/app/v3/services/computeTemplateCreation.server.ts`)
- Three-mode rollout: required (MICROVM projects), shadow (feature flag
/ percentage), skip
- Integrated into deploy finalize flow

**Shared compute package** (`internal-packages/compute/`)
- Gateway client with namespace-based API (instances, templates,
snapshots)
- Zod schemas for all gateway request/response types

**Database**
- `COMPUTE` variant added to `TaskRunCheckpointType` enum
- `WorkloadType` enum and column on `WorkerInstanceGroup`
- `hasComputeAccess` feature flag

**Env / config**
- Compute gateway URL, auth token, timeout
- Snapshot enable flag, delay, dispatch limit
- Dedicated OTLP endpoint for compute spans
(`COMPUTE_TRACE_OTLP_ENDPOINT`)
2026-03-29 22:03:59 +01:00
Oskar Otwinowski 2366b217bb fix(private-links): Private Links plans adjustments (#3285) 2026-03-27 20:03:41 +01:00
Oskar Otwinowski efe24f9c2a feat(private-link): Add private links UI (#3264)
⚒️ Publish Worker (v4) / build (supervisor) (push) Has been cancelled
2026-03-27 15:34:39 +01:00
Eric Allam 21cf8f9965 fix(webapp): disable resizable panel cookie saves to prevent large cookies causing HTTP 431 errors (#3282) 2026-03-27 13:04:04 +00:00
Oskar Otwinowski 8244ac6f84 feat(Notifications): Platform notifications for CLI and Dashboard (#3254)
For human reviewer:

- Check if Redis connection + code makes sense
- Check CLI methods (it's on a hotpath)
- Check DB Migrations and new tables


##  Checklist

- [x] I have followed every step in the [contributing
guide](https://github.com/triggerdotdev/trigger.dev/blob/main/CONTRIBUTING.md)
- [x] The PR title follows the convention.
- [x] I ran and tested the code works

---

## Testing

Spawning new CLI / Dashboard notifications, check MVP, check if failures
not produce any problems with CLI/Dashboard

---

## Changelog

Added notifications mechanism for Dashboard and CLI

---

## Screenshots



💯
2026-03-26 14:19:17 +01:00
James Ritchie cd7d7593aa fix(webapp): Only show bright title text on the right hand inspector titles (#3276)
Only makes the span title text brighter when it's shown in the right
hand side inspector (in the spans list it stays dimmed)
2026-03-26 13:09:36 +00:00
Oskar Otwinowski 922a852a9b fix(Vercel): Initial deployment fix (#3263) 2026-03-26 11:21:13 +01:00
Saadi Myftija 38559480c9 feat: replicate trigger_source, root_trigger_source, and is_warm_start to ClickHouse (#3274)
Adds three new top-level columns to the ClickHouse task_runs_v2 table
primarily for analytics:

- `trigger_source` / `root_trigger_source` - extracted from the existing
TaskRun.annotations JSON during WAL
replication
- `is_warm_start` - new nullable boolean on TaskRun in Postgres, set in
the existing taskRun.update() at attempt
start (no additional write). null until the first attempt starts.

Run region is already available via the existing `worker_queue` column
in ClickHouse.
2026-03-26 10:27:31 +01:00
Eric Allam efcafdf388 fix(webapp): filter dev environments by userId in OrganizationsPresenter (#3273)
fix: filter dev environments by userId in OrganizationsPresenter
2026-03-26 07:59:39 +00:00
Saadi Myftija 97d2f72063 feat(supervisor): schedule-tree node affinity (#3271)
Scheduled runs create predictable hourly spikes that compete with
on-demand runs for node capacity. Runs triggered "on-demand" via the
SDK, API, or dashboard, are more sensitive to cold start latency since
users are typically
waiting on the result. When a burst of scheduled runs lands at the top
of the hour, it can saturate the shared pool resources causing
contention, affecting cold starts across the board.

The idea in this change is to absorb these periodic spikes in a
dedicated pool without affecting the cold starts of on-demand runs.
Scheduled runs are inherently less sensitive to cold starts.

### Changes in this PR

Follows up on run annotations (#3241), which made trigger origin
available on every run in the tree. This PR exposes
annotations at dequeue time to the supervisor. This enables scheduling
decisions based on trigger source.

The affinities are soft preferences at schedule time, so runs fall back
gracefully if the target pool is out out of capacity.
2026-03-25 18:24:19 +01:00
Eric Allam 1a6481a579 feat: add Model Registry feature with catalog pipeline, dashboard pages, and TSQL schema (#3270)
- Add llm-model-catalog package (renamed from llm-pricing) with Claude
CLI research pipeline
- Add Prisma schema: catalog columns + baseModelName on LlmModel
- Add ClickHouse: llm_model_aggregates MV + base_response_model column
- Add TSQL llm_models schema for query page integration
- Add ModelRegistryPresenter with catalog, metrics, and comparison
queries
- Add 3 dashboard pages: catalog (cards+table+filters), detail
(overview+metrics+cost estimator), compare
- Add sidebar navigation under AI section with hasAiAccess feature flag
- Add admin dashboard sync/seed for catalog metadata
- Add model variant grouping (dated snapshots under base models)
- Add shared formatters and design system component usage

refs TRI-7941
2026-03-25 16:30:08 +00:00
Eric Allam 947f33d55b fix: downgrade queue size limit errors to warnings (#3243)
Queue limit ServiceValidationErrors were being logged at error level.
These are
expected validation rejections, not bugs.

- Add logLevel property to ServiceValidationError (webapp + run-engine)
- Set logLevel: warn on all queue limit throws
- Schedule engine: detect queue limit failures and log as warn
- Redis-worker: respect logLevel on thrown errors
2026-03-25 16:08:30 +00:00
James Ritchie 2037254a9b Feat(webapp) AI screen UI improvements (#3269)
### Lots of UI improvements to the Prompts pages: 

#### New side menu icons
<img width="234" height="110" alt="CleanShot 2026-03-25 at 14 26 33"
src="https://github.com/user-attachments/assets/8039ee8f-92a0-477d-ac91-458dfa43020b"
/>

#### Compact horizontal start finish times so scanning generations list
is consistent
<img width="618" height="174" alt="CleanShot 2026-03-25 at 14 24 03"
src="https://github.com/user-attachments/assets/7eb69475-d539-4e34-b0b5-5cd5119c1aea"
/>

#### Tidied up the metrics view
<img width="1607" height="925" alt="CleanShot 2026-03-25 at 14 23 52"
src="https://github.com/user-attachments/assets/d4bf2761-8e09-4d91-bfb8-e10d1508042f"
/>

#### Copiable metadata
<img width="274" height="160" alt="CleanShot 2026-03-25 at 14 23 44"
src="https://github.com/user-attachments/assets/487c804d-c000-4cd0-9a19-30c2681712df"
/>

#### Cleaner versions list
<img width="463" height="264" alt="CleanShot 2026-03-25 at 14 23 28"
src="https://github.com/user-attachments/assets/a2dfede7-0e7d-4f9c-8b01-00ba13070f70"
/>

#### Overall consistency improvements, shortcut keys and UI behaviours
improvements
<img width="2279" height="1349" alt="CleanShot 2026-03-25 at 14 23 02"
src="https://github.com/user-attachments/assets/0c29257b-15a0-443c-b872-a9f4a7f6af13"
/>

---------

Co-authored-by: Eric Allam <eallam@icloud.com>
2026-03-25 16:02:41 +00:00
DKP 7f9b0463cf docs: Fixes and realtime improvements (#3265) 2026-03-25 15:28:41 +00:00
Eric Allam c00dae006b feat(mcp): add get_span_details tool (#3255)
Returns the fully detailed span with attributes and AI enrichment data
2026-03-24 13:18:49 +00:00
Eric Allam 774007e9b9 fix(mcp): get_run_details only caches completed runs (#3253) 2026-03-24 11:28:52 +00:00
Oskar Otwinowski f86e492c31 feat(vercel): Flow to support Vercel's template deployment (#3229) 2026-03-24 11:04:57 +01:00
Saadi Myftija d4772b5f60 feat: run annotations (#3241)
Adds an `annotations` JSONB column to task runs that captures where and
how each run was triggered.
This enables filtering and analyzing trigger origins without querying up
the run tree. Also enables making scheduling decisions based on the
trigger source, e.g., use separate affinities for scheduled runs.

Each run records:
- **triggerSource**: who initiated it (sdk, api, dashboard, cli, mcp,
schedule)
- **triggerAction**: what kind of action (trigger, replay, test)
- **rootTriggerSource**: the trigger source of the root ancestor,
propagated through the entire run
 tree
- **rootScheduleId**: schedule id, in case the run tree was triggered
from a schedule

Currently the main motivation for annotations it to determine whether a
run is part of a schedule-originated tree without traversing ancestors.

### A couple of design considerations
- **Decoupled source from method**: triggerSource and triggerAction are
separate fields to avoid
combinatorial explosion (every new source × every new action)
- **Server-side first**: all annotation values are primarily determined
on the server, only a minor SDK change needed
- **Forward-compatible**: annotation fields use
`z.enum([...]).or(anyString)` so new values can be
added without breaking validation; we currently don't need an explicit
version field for annotations.

Note: `metadata` would have been a more fitting name for the db column,
as it is consistent with other tables where we store this type of
information. It is already in use to store user metadata though, so we
go with `annotations` instead.
2026-03-23 16:07:30 +01:00
Matt Aitken 88f755082f Vouched: jrossi (#3247)
Fixes #3246
2026-03-23 12:18:27 +00:00
Eric Allam 54d95ee4b9 feat: AI prompt management dashboard and enhanced span inspectors (#3244)
- Full prompt management UI: list, detail, override, and version
management for AI prompts defined with `prompts.define()`
- Rich AI span inspectors for all AI SDK operations with token usage,
messages, and prompt context
- Real-time generation tracking with live polling and filtering

## Prompt management

Define prompts in your code with `prompts.define()`, then manage
versions and overrides from the dashboard without redeploying:

```typescript
import { task, prompts } from "@trigger.dev/sdk";
import { generateText } from "ai";
import { openai } from "@ai-sdk/openai";
import { z } from "zod";

const supportPrompt = prompts.define({
  id: "customer-support",
  model: "gpt-4o",
  variables: z.object({
    customerName: z.string(),
    plan: z.string(),
    issue: z.string(),
  }),
  content: `You are a support agent for Acme SaaS.
Customer: {{customerName}} ({{plan}} plan)
Issue: {{issue}}
Respond with empathy and precision.`,
});

export const supportTask = task({
  id: "handle-support",
  run: async (payload) => {
    const resolved = await supportPrompt.resolve({
      customerName: payload.name,
      plan: payload.plan,
      issue: payload.issue,
    });

    const result = await generateText({
      model: openai(resolved.model ?? "gpt-4o"),
      system: resolved.text,
      prompt: payload.issue,
      ...resolved.toAISDKTelemetry(),
    });

    return { response: result.text };
  },
});
```

The prompts list page shows each prompt with its current version, model,
override status, and a usage sparkline over the last 24 hours.

From the prompt detail page you can:

- **Create overrides** to change the prompt template or model without
redeploying. Overrides take priority over the deployed version when
`prompt.resolve()` is called.
- **Promote** any code-deployed version to be the current version
- **Browse generations** across all versions with infinite scroll and
live polling for new results
- **Filter** by version, model, operation type, and provider
- **View metrics** (total generations, avg tokens, avg cost, latency)
broken down by version

## AI span inspectors

Every AI SDK operation now gets a custom inspector in the run trace
view:

- **`ai.generateText` / `ai.streamText`** — Shows model, token usage,
cost, the full message thread (system prompt, user message, assistant
response), and linked prompt details
- **`ai.generateObject` / `ai.streamObject`** — Same as above plus the
JSON schema and structured output
- **`ai.toolCall`** — Shows tool name, call ID, and input arguments
- **`ai.embed`** — Shows model and the text being embedded

For generation spans linked to a prompt, a "Prompt" tab shows the prompt
metadata, the input variables passed to `resolve()`, and the template
content from the prompt version.

All AI span inspectors include a compact timestamp and duration header.

## Other improvements

- Resizable panel sizes now persist across page refreshes (patched
`@window-splitter/state` to fix snapshot restoration)
- Run page panels also persist their sizes
- Fixed `<div>` inside `<p>` DOM nesting warnings in span titles and
chat messages
- Added Operations and Providers filters to the AI metrics dashboard

## Screenshots

<img width="3680" height="2392" alt="CleanShot 2026-03-21 at 10 14
17@2x"
src="https://github.com/user-attachments/assets/f3e59989-a2fa-4990-a9d0-3cacda431868"
/>

<img width="3680" height="2392" alt="CleanShot 2026-03-21 at 10 15
37@2x"
src="https://github.com/user-attachments/assets/2f2d02df-2d2b-44fb-ac6f-9153f6a6c387"
/>

<img width="3680" height="2392" alt="CleanShot 2026-03-21 at 10 15
54@2x"
src="https://github.com/user-attachments/assets/baa161e0-ef91-4fa4-a55f-986b71cccdf0"
/>
2026-03-23 06:23:19 +00:00
Matt Aitken 35298ac357 Impersonating run and clearing fix (#3144)
- Automatically impersonate a run when visiting /runs/<run_id> if an
admin is logged in
- Clear existing impersonation when switching
2026-03-18 14:36:27 +00:00
Eric Allam 1cfc296c6b feat(ai): LLM metrics tracking and AI span inspector (#3213)
- Automatic LLM cost enrichment for AI SDK spans (streamText,
generateText, generateObject) or any other spans that use semantic
gen_ai attributes with support for 145+ models
- New AI span inspector sidebar showing model, tokens, cost, messages,
tool calls, and response text
- LLM metrics dual-write to ClickHouse `llm_metrics_v1` table for
analytics
- LLM metrics built-in dashboard (unlinked at the moment)
- Provider cost fallback — uses gateway/OpenRouter reported costs from
`providerMetadata` when registry pricing is unavailable
- Prefix-stripping for gateway/OpenRouter model names (e.g.
`mistral/mistral-large-3` matches `mistral-large-3` pricing)
- Admin dashboard for managing LLM model pricing (list, create, edit,
delete, search, test pattern matching)
- Missing models detection page — queries ClickHouse for unpriced models
with sample spans and Claude Code-ready prompts for adding pricing
- AI span seed script (`pnpm run db:seed:ai-spans`) with 51 spans across
12 provider systems for local dev testing
- UI fixes: `completionTokens`/`promptTokens` aliases,
`ai.response.object` display for generateObject, cache read/write token
breakdown

## Screenshots:

<img width="1030" height="104" alt="CleanShot 2026-03-17 at 16 48 54@2x"
src="https://github.com/user-attachments/assets/bc8fccda-e48b-4d0c-bfb1-e620064e5979"
/>

<img width="1094" height="1512" alt="CleanShot 2026-03-17 at 16 49
23@2x"
src="https://github.com/user-attachments/assets/c2424569-d07e-4d67-a436-e8250043a1ee"
/>

<img width="1074" height="1412" alt="CleanShot 2026-03-17 at 16 49
18@2x"
src="https://github.com/user-attachments/assets/22342ac4-4769-45d1-a328-a24fb9a82a50"
/>

<img width="1012" height="2292" alt="CleanShot 2026-03-17 at 16 39
01@2x"
src="https://github.com/user-attachments/assets/59e327d1-6652-4293-8be0-bb8326e5fbc5"
/>

<img width="3680" height="2392" alt="CleanShot 2026-03-15 at 08 29
38@2x"
src="https://github.com/user-attachments/assets/1f77beb8-de67-495b-b890-bcdb8d7f1fe8"
/>

---------

Co-authored-by: James Ritchie <james@trigger.dev>
2026-03-17 18:26:43 +00:00
Eric Allam 411803e49e fix(engine): lockless waitpoint insert for batch items to eliminate lock contention (#3232)
When processing batchTriggerAndWait items, each batch item was acquiring
a
Redis lock on the parent run to insert a TaskRunWaitpoint row. With high
concurrency (processingConcurrency=50), this caused
LockAcquisitionTimeoutError
(880 errors/24h in prod), orphaned runs, and stuck parent runs.

Since blockRunWithCreatedBatch already transitions the parent to
EXECUTING_WITH_WAITPOINTS before items are processed, the per-item lock
is
unnecessary. The new blockRunWithWaitpointLockless method performs only
the
idempotent CTE insert and timeout scheduling without acquiring the lock.
2026-03-17 16:42:01 +00:00
Eric Allam dbbe9f77f9 feat(cli): Expand and improve the MCP server and dev CLI command (#3224)
## Summary

Major expansion of the MCP server (14 → 25 tools), context efficiency
optimizations, new API endpoints, and a fix for the dev CLI leaking
build directories on disk.

### New MCP tools

- **Query & analytics**: `get_query_schema`, `query`, `list_dashboards`,
`run_dashboard_query` — query your data using TRQL directly from AI
assistants
- **Profile management**: `whoami`, `list_profiles`, `switch_profile` —
see and switch CLI profiles per-project (persisted to
`.trigger/mcp.json`)
- **Dev server control**: `start_dev_server`, `stop_dev_server`,
`dev_server_status` — start/stop `trigger dev` and stream build output
- **Task introspection**: `get_task_schema` — get payload schema for a
specific task (split out from `get_current_worker` to reduce context)

### New API endpoints

- `GET /api/v1/query/schema` — discover TRQL tables and columns
(server-driven, multi-table)
- `GET /api/v1/query/dashboards` — list built-in dashboard widgets and
their queries

### New features

- **`--readonly` flag** — hides write tools (`deploy`, `trigger_task`,
`cancel_run`) so agents can't make changes
- **`read:query` JWT scope** — new authorization scope for query
endpoints, with per-table granularity (`read:query:runs`,
`read:query:llm_metrics`, etc.)
- **Paginated trace output** — `get_run_details` now paginates trace
events via cursor, caching the full trace in a temp file so subsequent
pages don't re-fetch
- **MCP tool annotations** — all tools now have
`readOnlyHint`/`destructiveHint` annotations for clients that support
them
- **Project-scoped profile persistence** — `switch_profile` saves to
`.trigger/mcp.json` (gitignored), automatically loaded on next MCP
server start

### Context optimizations

- `get_query_schema` requires a table name — returns one table's schema
instead of all tables (60-80% fewer tokens)
- `get_current_worker` no longer inlines payload schemas — use
`get_task_schema` for specific tasks
- Query results formatted as text tables instead of JSON (~50% fewer
tokens for flat data)
- `cancel_run`, `list_deploys`, `list_preview_branches` formatted as
text instead of raw `JSON.stringify()`
- Schema and dashboard API responses cached (1hr and 5min respectively)

### Bug fixes

- Fixed `search_docs` failing due to renamed upstream Mintlify tool
(`SearchTriggerDev` → `search_trigger_dev`)
- Fixed `list_deploys` failing when deployments have null
`runtime`/`runtimeVersion` fields (fixes #3139)
- Fixed `list_preview_branches` crashing due to incorrect response shape
access
- Fixed `metrics` table column documented as `value` instead of
`metric_value` in query docs
- Fixed `/api/v1/query` not accepting JWT auth (added `allowJWT: true`)

### Dev CLI build directory fix

The dev CLI was leaking `build-*` directories in `.trigger/tmp/` on
every rebuild, accumulating hundreds of MB over time (842MB observed).
Three layers of protection added:

1. **During session**: deprecated workers are pruned (capped at 2
retained) when no active runs reference them, preventing unbounded
accumulation
2. **On SIGKILL/crash**: the watchdog process now cleans up
`.trigger/tmp/` when it detects the parent CLI was killed
3. **On next startup**: existing `clearTmpDirs()` wipes any remaining
orphans

## Test plan

- [ ] `pnpm run mcp:smoke` — 17 automated smoke tests for all read-only
MCP tools
- [ ] `pnpm run mcp:test list` — verify 25 tools registered (21 in
`--readonly` mode)
- [ ] `pnpm run mcp:test --readonly list` — verify write tools hidden
- [ ] Manual: start dev server, trigger task, rebuild multiple times,
verify build dirs stay capped at 4
- [ ] Manual: SIGKILL the dev CLI, verify watchdog cleans up
`.trigger/tmp/`
- [ ] Verify new API endpoints return correct data: `GET
/api/v1/query/schema`, `GET /api/v1/query/dashboards`

🤖 Generated with [Claude Code](https://claude.com/claude-code)
2026-03-17 11:38:23 +00:00
Jack Cheng f98e274b06 feat(webapp): allow version downgrades via promote API (#3214) 2026-03-17 07:38:20 +00:00
Matt Aitken cf0e1ff91a Vouched: bhekanik (#3227) 2026-03-16 22:24:49 +00:00
Matt Aitken 2406e850eb RunEngine readme updates (#3223) 2026-03-16 18:52:21 +00:00
Eric Allam 7672e8d998 fix(run-queue): prevent concurrency keys from bloating master queue shards (#3219)
Queues with concurrency keys now appear as a single entry in the master
queue instead of one entry per key. This prevents high-CK-count tenants
from consuming the entire `parentQueueLimit` window and starving other
tenants on the same shard.

A new per-queue **CK index** (sorted set) tracks active concurrency key
sub-queues. The master queue gets one `:ck:*` wildcard entry per base
queue. Dequeuing from that entry round-robins across sub-queues,
maintaining per-CK concurrency tracking and fairness.

All existing operations (enqueue, dequeue, ack, nack, DLQ, TTL expiry)
are CK-index-aware and keep the index consistent. Old-format entries
drain naturally during rollout — no migration step needed, single
deploy.
2026-03-14 13:37:54 +00:00
Eric Allam 2c688ca3d4 chore(repo): Add new contributing guidelines and auto-close outside PRs open in "ready to review" status (#3218) 2026-03-14 10:23:34 +00:00
Eric Allam 59ff68b00c chore(vouch): add bharathkumar39293 (#3217) 2026-03-14 09:12:51 +00:00
Eric Allam 440c16c4aa chore(repo): add agentcrumbs.dev support (#3206) 2026-03-13 13:37:09 +00:00
Eric Allam d4d8d9fabc fix(engine): add additional error logging around triggering runs (#3211) 2026-03-13 07:18:43 +00:00
Eric Allam 8108683073 vouch chengzp (#3212) 2026-03-12 23:33:25 +00:00
James Ritchie fea8ae4e53 feat(webapp): self serve preview branches and team members (#3201)
## Adds 2 self serve features

### 1. self serve preview branches

- Copies the patterns of the self serve concurrency
- Self serve only available on Pro plan (otherwise you are linked to the
billing plans page)
- Global self serve branches limit: 180 (+20 for the Pro plan). It can
be overridden per Org
- You need to archive branches before reducing the number of extra
branches you're paying for
- Branches are removed immediately but remain billed until the end of
the billing cycle like extra concurrency

### 2. self serve team members

- Copies the patterns of the self serve concurrency
- Self serve only available on Pro plan (otherwise you are linked to the
billing plans page)
- Global self serve members is unlimited but can be limited with the
same env var quota and overridden per org if needed
- You need to remove team members before reducing the number of members
you pay for
- Team members are removed immediately but remain billed until the end
of the billing cycle like extra concurrency
2026-03-12 14:56:10 +00:00
Iss 21fdb528f5 docs: deprecate syncVercelEnvVars extension and add conflict warning (#3208)
Deprecates the syncVercelEnvVars build extension and adds warnings in
both the Vercel integration docs and the extension's own page to prevent
conflicts with the native env var sync
2026-03-11 17:56:37 -04:00
Dinko Osrecki b8766e7a27 fix(webapp): strip secure param from query ClickHouse URL (#3204) 2026-03-11 06:42:49 +00:00
Eric Allam fef6d18e8f chore: add @edosrecki to vouched contributors (#3203) 2026-03-11 06:33:58 +00:00
github-actions[bot] c0b6309c72 chore: release v4.4.3 (#3182)
🚀 Publish Trigger.dev Docker / units (push) Failing after 4s
🚀 Publish Trigger.dev Docker / typecheck (push) Failing after 20s
🚀 Publish Trigger.dev Docker / publish-webapp (push) Has been skipped
🚀 Publish Trigger.dev Docker / publish-worker (push) Has been skipped
🚀 Publish Trigger.dev Docker / publish-worker-v4 (push) Has been skipped
## Summary
2 new features, 2 improvements.

## Improvements
- Add syncSupabaseEnvVars to pull database connection strings and save
them as trigger.dev environment variables
([#3152](https://github.com/triggerdotdev/trigger.dev/pull/3152))
- Auto-cancel in-flight dev runs when the CLI exits, using a detached
watchdog process that survives pnpm SIGKILL
([#3191](https://github.com/triggerdotdev/trigger.dev/pull/3191))

## Server changes

These changes affect the self-hosted Docker image and Trigger.dev Cloud:

- A new Errors page for viewing and tracking errors that cause runs to
fail
  
  - Errors are grouped using error fingerprinting
- View top errors for a time period, filter by task, or search the text
  - View occurrences over time
- View all the runs for an error and bulk replay them
([#3172](https://github.com/triggerdotdev/trigger.dev/pull/3172))
- Add sidebar tabs (Options, AI, Schema) to the Test page for schemaTask
payload generation and schema viewing.
([#3188](https://github.com/triggerdotdev/trigger.dev/pull/3188))

<details>
<summary>Raw changeset output</summary>

# Releases
## @trigger.dev/build@4.4.3

### Patch Changes

- Add syncSupabaseEnvVars to pull database connection strings and save
them as trigger.dev environment variables
([#3152](https://github.com/triggerdotdev/trigger.dev/pull/3152))
-   Updated dependencies:
    -   `@trigger.dev/core@4.4.3`

## trigger.dev@4.4.3

### Patch Changes

- Auto-cancel in-flight dev runs when the CLI exits, using a detached
watchdog process that survives pnpm SIGKILL
([#3191](https://github.com/triggerdotdev/trigger.dev/pull/3191))
-   Updated dependencies:
    -   `@trigger.dev/core@4.4.3`
    -   `@trigger.dev/build@4.4.3`
    -   `@trigger.dev/schema-to-json@4.4.3`

## @trigger.dev/core@4.4.3

### Patch Changes

- Auto-cancel in-flight dev runs when the CLI exits, using a detached
watchdog process that survives pnpm SIGKILL
([#3191](https://github.com/triggerdotdev/trigger.dev/pull/3191))

## @trigger.dev/python@4.4.3

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.4.3`
    -   `@trigger.dev/build@4.4.3`
    -   `@trigger.dev/sdk@4.4.3`

## @trigger.dev/react-hooks@4.4.3

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.4.3`

## @trigger.dev/redis-worker@4.4.3

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.4.3`

## @trigger.dev/rsc@4.4.3

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.4.3`

## @trigger.dev/schema-to-json@4.4.3

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.4.3`

## @trigger.dev/sdk@4.4.3

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.4.3`

</details>

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2026-03-10 10:17:24 +00:00
Eric Allam 436f20efc6 feat(cli): auto-cancel dev runs on CLI exit via detached watchdog (#3191)
When the dev CLI exits (e.g. ctrl+c via pnpm), runs that were
mid-execution
previously stayed stuck in EXECUTING status for up to 5 minutes until
the
heartbeat timeout fired. Now they are cancelled within seconds.

The dev CLI spawns a lightweight detached watchdog process at startup.
The
watchdog monitors the CLI process ID and, when it detects the CLI has
exited,
calls a new POST /engine/v1/dev/disconnect endpoint to cancel all
in-flight
runs immediately (skipping PENDING_CANCEL since the worker is known to
be dead).

Watchdog design:
- Fully detached (detached: true, stdio: ignore, unref()) so it survives
  even when pnpm sends SIGKILL to the process tree
- Active run IDs maintained via atomic file write
(.trigger/active-runs.json)
- Single-instance guarantee via PID file (.trigger/watchdog.pid)
- Safety timeout: exits after 24 hours to prevent zombie processes
- On clean shutdown, the watchdog is killed (no disconnect needed)

Disconnect endpoint:
- Rate-limited: 5 calls/min per environment
- Capped at 500 runs per call
- Small counts (<= 25): cancelled inline with pMap concurrency 10
- Large counts: delegated to the bulk action system
- Uses finalizeRun: true to skip PENDING_CANCEL and go straight to
FINISHED

Run engine change:
- cancelRun() now respects finalizeRun when the run is in EXECUTING
status,
skipping the PENDING_CANCEL waiting state and going directly to FINISHED
2026-03-09 12:14:43 +00:00
James Ritchie d1ea8d8f74 Fix(webapp) onboarding fixes (#3189)
### Fixes and improvements to the onboarding questions: 

**This change is worth double checking @matt-aitken** 
- Update to the Button.tsx file: it now takes `isLoading` that shows a
spinner in the middle of the button (replacing the button text and any
icons) and sets it to `disabled`. It does this nicely by keeping the
button width the same so there's no layout shift.

**Other fixes**
- Fixes an issue where if you type a custom option in the "What
technologies do you use" question, it doesn't check the list to see if
it matches. Now it checks the box if you've typed an option from that
list.
- When we randomize the list of onboarding question options, we now
store the position they appeared in the list
2026-03-09 10:09:21 +00:00
Eric Allam e64b101138 feat(webapp): Add test payload AI generation to the test page based on payload schemas (#3188)
<img width="2191" height="1023" alt="CleanShot 2026-03-06 at 13 36 53"
src="https://github.com/user-attachments/assets/4eba0d1a-1528-49a3-be5b-6bde89030193"
/>
<img width="411" height="1069" alt="CleanShot 2026-03-06 at 13 37 28"
src="https://github.com/user-attachments/assets/e5f7bb9c-c894-41cc-9ca6-96b43fcf6005"
/>

Add a tabbed sidebar to the Test page for standard tasks, reusing the
ClientTabs pattern from the Query page.

- Options tab: existing sidebar content (machine, version, queue, etc.)
- AI tab: AI-powered payload generation with streaming, supports JSON
  Schema, inferred schema from recent runs, and task source code lookup
  via tool calling for tasks without schemas
- Schema tab: displays payload JSON Schema (from schemaTask), inferred
  schema (from recent runs via @jsonhero/schema-infer), or empty state
  with schemaTask docs and example code

Data layer changes:
- Surface payloadSchema and inferredPayloadSchema from TestTaskPresenter
- Add payloadSchema and fileId to WorkerDeploymentWithWorkerTasks type
- Decompress zlib-deflated source files for AI context
2026-03-06 15:31:16 +00:00
Eric Allam 6f6523ff78 chore(repo): remove unnecessary "trigger.dev v4.4.2" header from the release PR description (#3183) 2026-03-06 14:07:32 +00:00
Matt Aitken d30ed3421e Logs query use_query_condition_cache=1 (#3186)
In theory this will make Log queries faster
2026-03-06 10:24:43 +00:00
Matt Aitken 5f359be286 feature: Errors page (#3172)
A top-level Errors page that aggregates errors from failed runs with
occurrences metrics.


https://github.com/user-attachments/assets/8f0ef55e-90dd-4faa-9051-59f4665181e4

Errors are “fingerprinted” so similar errors are grouped together (e.g.
has an ID in the error message).

You can view an individual error to view a timeline of when it fired,
the runs, and bulk replay them.
2026-03-05 11:05:45 +00:00
Oskar Otwinowski e49ccc1226 feat(buildExtensions): syncSupabaseEnvVars build extension (#3152)
with docs
2026-03-05 11:04:26 +01:00
Eric Allam c01332297b docs: update batch trigger concurrency limits (#3171) 2026-03-04 11:50:59 +00:00
Eric Allam c5ce2976be fix: Add repo flag when for updating the docker link during release (#3170) 2026-03-04 11:13:47 +00:00
Eric Allam e954a2c97a docs: realtime input streams (#3153)
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
2026-03-04 09:55:54 +00:00
github-actions[bot] e36b75785e chore: release v4.4.2 (#3127)
🚀 Publish Trigger.dev Docker / units (push) Failing after 4s
🚀 Publish Trigger.dev Docker / typecheck (push) Failing after 20s
🚀 Publish Trigger.dev Docker / publish-webapp (push) Has been skipped
🚀 Publish Trigger.dev Docker / publish-worker (push) Has been skipped
🚀 Publish Trigger.dev Docker / publish-worker-v4 (push) Has been skipped
# trigger.dev v4.4.2

## Summary
2 new features, 2 improvements, 8 bug fixes.

## Improvements
- Add input streams for bidirectional communication with running tasks.
Define typed input streams with `streams.input<T>({ id })`, then consume
inside tasks via `.wait()` (suspends the process), `.once()` (waits for
next message), or `.on()` (subscribes to a continuous stream). Send data
from backends with `.send(runId, data)` or from frontends with the new
`useInputStreamSend` React hook.
([#3146](https://github.com/triggerdotdev/trigger.dev/pull/3146))
- Add PAYLOAD_TOO_LARGE error to handle graceful recovery of sending
batch trigger items with payloads that exceed the maximum payload size
([#3137](https://github.com/triggerdotdev/trigger.dev/pull/3137))

## Bug fixes
- Fix slow batch queue processing by removing spurious cooloff on
concurrency blocks and fixing a race condition where retry attempt
counts were not atomically updated during message re-queue.
([#3079](https://github.com/triggerdotdev/trigger.dev/pull/3079))
- fix(sdk): batch triggerAndWait variants now return correct
run.taskIdentifier instead of unknown
([#3080](https://github.com/triggerdotdev/trigger.dev/pull/3080))

## Server changes

These changes affect the self-hosted Docker image and Trigger.dev Cloud:

- Two-level tenant dispatch architecture for batch queue processing.
Replaces the
single master queue with a two-level index: a dispatch index (tenant →
shard)
and per-tenant queue indexes (tenant → queues). This enables O(1) tenant
selection and fair scheduling across tenants regardless of queue count.
Improves batch queue processing performance.
([#3133](https://github.com/triggerdotdev/trigger.dev/pull/3133))
- Add input streams with API routes for sending data to running tasks,
SSE reading, and waitpoint creation. Includes Redis cache for fast
`.send()` to `.wait()` bridging, dashboard span support for input stream
operations, and s2-lite support with configurable S2 endpoint, access
token skipping, and S2-Basin headers for self-hosted deployments. Adds
s2-lite to Docker Compose for local development.
([#3146](https://github.com/triggerdotdev/trigger.dev/pull/3146))
- Speed up batch queue processing by disabling cooloff and increasing
the batch queue processing concurrency limits on the cloud:
  
  - Pro plan: increase to 50 from 10.
  - Hobby plan: increase to 10 from 5.
- Free plan: increase to 5 from 1.
([#3079](https://github.com/triggerdotdev/trigger.dev/pull/3079))
- Move batch queue global rate limiter from FairQueue claim phase to
BatchQueue worker queue consumer for accurate per-item rate limiting.
Add worker queue depth cap to prevent unbounded growth that could cause
visibility timeouts.
([#3166](https://github.com/triggerdotdev/trigger.dev/pull/3166))
- Fix a race condition in the waitpoint system where a run could be
blocked by a completed waitpoint but never be resumed because of a
PostgreSQL MVCC issue. This was most likely to occur when creating a
waitpoint via `wait.forToken()` at the same moment as completing the
token with `wait.completeToken()`. Other types of waitpoints (timed,
child runs) were not affected.
([#3075](https://github.com/triggerdotdev/trigger.dev/pull/3075))
- Fix metrics dashboard chart series colors going out of sync and
widgets not reloading stale data when scrolled back into view
([#3126](https://github.com/triggerdotdev/trigger.dev/pull/3126))
- Gracefully handle oversized batch items instead of aborting the
stream.
  
When an NDJSON batch item exceeds the maximum size, the parser now emits
an error marker instead of throwing, allowing the batch to seal
normally. The oversized item becomes a pre-failed run with
`PAYLOAD_TOO_LARGE` error code, while other items in the batch process
successfully. This prevents `batchTriggerAndWait` from seeing connection
errors and retrying with exponential backoff.
  
Also fixes the NDJSON parser not consuming the remainder of an oversized
line split across multiple chunks, which caused "Invalid JSON" errors on
subsequent lines.
([#3137](https://github.com/triggerdotdev/trigger.dev/pull/3137))
- Require the user is an admin during an impersonation session.
Previously only the impersonation cookie was checked; now the real
user's admin flag is verified on every request. If admin has been
revoked, the session falls back to the real user's ID.
([#3078](https://github.com/triggerdotdev/trigger.dev/pull/3078))

<details>
<summary>Raw changeset output</summary>

# Releases
## @trigger.dev/build@4.4.2

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.4.2`

## trigger.dev@4.4.2

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/build@4.4.2`
    -   `@trigger.dev/core@4.4.2`
    -   `@trigger.dev/schema-to-json@4.4.2`

## @trigger.dev/python@4.4.2

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/sdk@4.4.2`
    -   `@trigger.dev/build@4.4.2`
    -   `@trigger.dev/core@4.4.2`

## @trigger.dev/react-hooks@4.4.2

### Patch Changes

- Add input streams for bidirectional communication with running tasks.
Define typed input streams with `streams.input<T>({ id })`, then consume
inside tasks via `.wait()` (suspends the process), `.once()` (waits for
next message), or `.on()` (subscribes to a continuous stream). Send data
from backends with `.send(runId, data)` or from frontends with the new
`useInputStreamSend` React hook.
([#3146](https://github.com/triggerdotdev/trigger.dev/pull/3146))

Upgrade S2 SDK from 0.17 to 0.22 with support for custom endpoints
(s2-lite) via the new `endpoints` configuration, `AppendRecord.string()`
API, and `maxInflightBytes` session option.

-   Updated dependencies:
    -   `@trigger.dev/core@4.4.2`

## @trigger.dev/redis-worker@4.4.2

### Patch Changes

- Fix slow batch queue processing by removing spurious cooloff on
concurrency blocks and fixing a race condition where retry attempt
counts were not atomically updated during message re-queue.
([#3079](https://github.com/triggerdotdev/trigger.dev/pull/3079))
-   Updated dependencies:
    -   `@trigger.dev/core@4.4.2`

## @trigger.dev/rsc@4.4.2

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.4.2`

## @trigger.dev/schema-to-json@4.4.2

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.4.2`

## @trigger.dev/sdk@4.4.2

### Patch Changes

- Add input streams for bidirectional communication with running tasks.
Define typed input streams with `streams.input<T>({ id })`, then consume
inside tasks via `.wait()` (suspends the process), `.once()` (waits for
next message), or `.on()` (subscribes to a continuous stream). Send data
from backends with `.send(runId, data)` or from frontends with the new
`useInputStreamSend` React hook.
([#3146](https://github.com/triggerdotdev/trigger.dev/pull/3146))

Upgrade S2 SDK from 0.17 to 0.22 with support for custom endpoints
(s2-lite) via the new `endpoints` configuration, `AppendRecord.string()`
API, and `maxInflightBytes` session option.

- fix(sdk): batch triggerAndWait variants now return correct
run.taskIdentifier instead of unknown
([#3080](https://github.com/triggerdotdev/trigger.dev/pull/3080))

- Add PAYLOAD_TOO_LARGE error to handle graceful recovery of sending
batch trigger items with payloads that exceed the maximum payload size
([#3137](https://github.com/triggerdotdev/trigger.dev/pull/3137))

-   Updated dependencies:
    -   `@trigger.dev/core@4.4.2`

## @trigger.dev/core@4.4.2

</details>

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2026-03-04 09:47:24 +00:00
Eric Allam dee6f1d09e fix(batch): move batch queue global rate limiter to worker consumer level (#3166)
The global rate limiter was being applied at the FairQueue claim phase,
consuming 1 token per queue-claim-attempt rather than per item
processed.
  With many small queues (each batch is its own queue), consumers burned
  through tokens on empty or single-item queues, causing aggressive
  throttling well below the intended items/sec limit.

  Changes:
- Move rate limiter from FairQueue claim phase to BatchQueue worker
queue
    consumer loop (before blockingPop), so each token = 1 item processed
  - Replace the FairQueue rate limiter with a worker queue depth cap to
    prevent unbounded growth that could cause visibility timeouts
- Add BATCH_QUEUE_WORKER_QUEUE_MAX_DEPTH env var (optional, disabled by
default)
2026-03-03 14:44:27 +00:00
James Ritchie b192b71b93 Feat(webapp): add create custom dashboard button to metrics page (#3095)
Adds a "Create custom dashboard" button to the top right of the metrics
dashboard

 
<img width="3546" height="1934" alt="CleanShot 2026-02-19 at 11 25
12@2x"
src="https://github.com/user-attachments/assets/0bb46ade-47c9-4396-b62a-f4801d7d90b4"
/>
2026-03-02 18:23:02 +00:00
Eric Allam a9163dfd9d chore(docker): Pin goose in Dockerfile to v3.26.0 (#3163)
The latest goose requires go version 1.25:
https://github.com/pressly/goose/releases/tag/v3.27.0
2026-03-02 17:09:46 +00:00
Eric Allam 540e1c86a4 feat: Input Streams - Bidirectional task communication (#3146)
Input streams enable sending typed data to executing tasks from external
callers — backends, frontends, or other tasks. This unlocks interactive
use cases like approval UIs, cancel buttons, chat interfaces, and
human-in-the-loop AI workflows where the task needs to receive data
while running.

Three consumption patterns inside a task:

* `.wait()` — Suspend the task until data arrives (process freed, most
efficient)
* `.once()` — Wait for the next message (process stays alive)
* `.on()` — Subscribe to a continuous stream of messages

One send pattern from outside:

* `.send(runId, data)` — Send typed data to a specific run's input
stream

## User-facing API

### Define a typed input stream

```ts
import { streams, task } from "@trigger.dev/sdk";

const approval = streams.input<{ approved: boolean; reviewer: string }>({ id: "approval" });
```

### Consume inside a task

```ts
export const myTask = task({
  id: "my-task",
  run: async () => {
    // Pattern 1: Suspend until data arrives (most efficient — frees the process)
    const result = await approval.wait({ timeout: "5m" });

    // Pattern 2: Wait for next message (process stays alive)
    const data = await approval.once().unwrap();

    // Pattern 3: Subscribe to multiple messages
    approval.on((data) => { /* handle each message */ });
  },
});
```

### Send from outside

```ts
// From a backend (using secret API key)
await approval.send(runId, { approved: true, reviewer: "alice" });

// From a frontend (using public JWT token from trigger response)
const { send } = useInputStreamSend("approval", runId, { accessToken });
send({ approved: true, reviewer: "alice" });
```

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-03-02 16:49:54 +00:00
Eric Allam 2af5c860de chore(repo): Dispatch new-release to www.trigger.dev repo to generate version changelog draft PR (#3162) 2026-03-02 16:47:34 +00:00
James Ritchie a09038b066 Feature(webapp): new User and Project onboarding questions (#3109)
- New User onboarding questions added and stored in a new
`onboardingData` col
- Keeps the same Org creation screen and stores the data in the same
format in same DB column
- New Org onboarding questions addded and stored in a new
`onboardingData` col


https://github.com/user-attachments/assets/244e4bae-f74d-4ed4-a545-92c9b927e98b

---------

Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-03-02 14:45:14 +00:00
Eric Allam 2135dc56d6 chore(claude): Improve claude code instructions (#3161)
Also includes a claude.md audit workflow for PRs
2026-03-02 12:42:05 +00:00
Saadi Myftija 4a0fb2fc06 ci: pin vouch action version (#3160)
Pins vouch actions to `c6d80ead49839655b61b422700b7a3bc9d0804a9`
(v1.4.2) in favor of security practices. We were previously using the
`@main` tag.

Also removes the checkout steps as they're not needed in these
workflows.
2026-03-02 10:35:42 +01:00
James Ritchie e1f8134f44 fix(webapp): Removes the collapsible option from the query inspector to avoid firefox bug (#3155)
There's a bug in react-window-splitter on Firefox. When trying to expand
the inspector panel in the query editor, it checks if the main panel has
space but gets an object instead of a number for the auto-sized
query-main panel. This causes the expand calculation to fail and it
snaps it back to collapsed.

I've removed this behavior for now as it's not an important feature.
2026-02-28 20:09:13 +00:00
Oskar Otwinowski 10d6f01843 feat(vercel): Vercel SDK fixes and correct env vars behavior for staging envs (#3149) 2026-02-28 07:41:19 +00:00
Iss 24b92d3b68 docs: added runtime error note for supabase edge function (#3140) 2026-02-27 10:59:04 -05:00
Eric Allam 8003923598 feat(server): Gracefully handle oversized batch items instead of aborting the stream (#3137)
Gracefully handle oversized batch items instead of aborting the stream.

When an NDJSON batch item exceeds the maximum size, the parser now emits
an error marker instead of throwing, allowing the batch to seal
normally. The oversized item becomes a pre-failed run with
`PAYLOAD_TOO_LARGE` error code, while other items in the batch process
successfully. This prevents `batchTriggerAndWait` from seeing connection
errors and retrying with exponential backoff.

Also fixes the NDJSON parser not consuming the remainder of an oversized
line split across multiple chunks, which caused "Invalid JSON" errors on
subsequent lines.
2026-02-27 10:11:42 +00:00
Eric Allam cff45664fc fix: legacy master queue drain should never re-add entries (#3142) 2026-02-27 10:11:31 +00:00
Iss 51b6c3a580 docs: added note about Prisma 7.x for TASK_RUN_STALLED_EXECUTING error (#3138) 2026-02-26 16:01:31 -05:00
James Ritchie d5a27f08ed Fix(webapp): change "metrics" to "dashboard" (#3136)
<img width="249" height="245" alt="CleanShot 2026-02-26 at 16 44 46"
src="https://github.com/user-attachments/assets/2e38b60c-0fe4-4b88-b9b9-71df82943ace"
/>
2026-02-26 16:56:07 +00:00
Matt Aitken 719a44da01 Better explanation of batch processing concurrency (#3135) 2026-02-26 14:27:21 +00:00
Iss 92dfeb37b3 docs: Add workaround for Homebrew Bun ENOENT error to Bun guide (#3125) 2026-02-26 08:25:17 -05:00
Eric Allam b1e78a6590 feat(batch-queue): two-level tenant dispatch for fair queue (#3133)
Replace flat master queue index with two-level tenant dispatch to fix
noisy neighbor problem. When a tenant has many queues at capacity, the
scheduler now iterates tenants (Level 1) not queues, then fetches
per-tenant queues (Level 2) only for eligible tenants.

Single-deploy migration: new enqueues write to dispatch indexes only,
consumer drains old master queue alongside new dispatch path until
empty.
2026-02-26 13:07:27 +00:00
Eric Allam 5612383684 chore(repo): Improve formatting of server entries in release notes (#3134) 2026-02-26 11:59:47 +00:00
Iss 4451fcb84c docs: Query page output dot notation and metadata availability (#3132)
Clarifies in the Query docs that run metadata is not available on the
Query page and that the output column is JSON, so dot notation (e.g.
output.externalId) should be used for selecting and filtering. Adds an
example that filters by an output field in WHERE
2026-02-25 17:22:08 -05:00
Iss 863dbe8d60 docs: document waitpoint token API endpoints (#3130)
Adds REST API documentation for the 5 waitpoint token endpoints
(`/api/v1/waitpoints/tokens`), including create, list, retrieve,
complete, and HTTP callback. Also adds the `publicAccessToken` security
scheme used by the complete endpoint.

<!-- mintlify-editor-comments:start -->
Mintlify
---
0 threads from 0 users in Mintlify

- No unresolved comments
<!-- mintlify-editor-comments:end -->

<!-- mintlify-comment-->

<a
href="https://dashboard.mintlify.com/trigger/trigger/editor/docs%2Fdocument-waitpoint-endpoints?source=pr_comment"
target="_blank" rel="noopener noreferrer"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://d3gk2c5xim1je2.cloudfront.net/assets/open-mintlify-editor-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://d3gk2c5xim1je2.cloudfront.net/assets/open-mintlify-editor-light.svg"><img
src="https://d3gk2c5xim1je2.cloudfront.net/assets/open-mintlify-editor-light.svg"
alt="Open in Mintlify Editor"></picture></a>

<!-- /mintlify-comment -->
2026-02-25 12:52:14 -05:00
Eric Allam 39dd91b098 fix(engine) prevent MVCC race in blockRunWithWaitpoint pending check (#3075)
Split the CTE in blockRunWithWaitpoint so the pending waitpoint check
is a separate SQL statement. In READ COMMITTED isolation, each statement
gets its own snapshot, so a separate SELECT sees the latest committed
state from concurrent completeWaitpoint calls.

Previously, the CTE did INSERT + pending check in one statement (one
snapshot). If completeWaitpoint committed between the CTE start and
the SELECT, the SELECT would still see PENDING due to the stale
snapshot. Neither side would enqueue continueRunIfUnblocked, leaving
the run stuck forever.
2026-02-25 17:41:50 +00:00
Matt Aitken cf6b6e7063 Fix realtime connections pricing tier from 100 to 1000 (#3131)
Closes #

##  Checklist

- [ ] I have followed every step in the [contributing
guide](https://github.com/triggerdotdev/trigger.dev/blob/main/CONTRIBUTING.md)
- [ ] The PR title follows the convention.
- [ ] I ran and tested the code works

---

## Testing

Verified the pricing definition displays the correct tier amount on the
plan selection page.

---

## Changelog

Fixed incorrect pricing tier for additional realtime connections from
$10/month per 100 to $10/month per 1000.

---

## Screenshots

N/A

💯

https://claude.ai/code/session_015QrZZJHPWta3QCBnhX2Pff

Co-authored-by: Claude <noreply@anthropic.com>
2026-02-25 17:34:02 +00:00
Eric Allam bed3789c31 fix(batch-queue): speed up batch queue processing by disabling cooloff and fixing retry race (#3079)
Fix slow fair queue processing by removing spurious cooloff on
concurrency blocks and fixing a race condition where retry attempt
counts were not atomically updated during message re-queue.

Removed cooloff entirely from the batch queue
2026-02-25 17:33:01 +00:00
Eric Allam a482153365 feat(webapp): require the user is an admin during an impersonation session (#3078) 2026-02-25 16:07:10 +00:00
Eric Allam fe193418d0 chore(repo) auto-link server change entries to their PRs via GitHub API (#3129) 2026-02-25 16:02:32 +00:00
Matt Aitken e9fb8e3b52 Query fixes: stale widget fix, multiple series colors mismatch (#3126)
- Fix for series color assignment being out of sync with the graph
(ensures added series colors match their graph representation)
- Reload widgets when returning to screen if props changed (prevents
stale widgets after filtering and scrolling)
2026-02-25 15:50:17 +00:00
Eric Allam c05b30adfe chore(repo): fix enhanced release pr description to filter out dependency only updates (#3128) 2026-02-25 15:50:06 +00:00
Eric Allam f37bdaac84 fix(sdk): batch triggerAndWait variants now return correct run.taskIdentifier instead of unknown (#3080)
Fixes #2942
2026-02-25 15:37:04 +00:00
Eric Allam 3c0644a3b8 feat: unified GitHub release, server change tracking, and enhanced release PR (#3085)
- Add .server-changes/ convention for tracking server-only changes
- Create scripts/enhance-release-pr.mjs to deduplicate and categorize
changeset PR body
- Create scripts/generate-github-release.mjs to format unified GitHub
release body
- Change release.yml to create one unified GitHub release instead of
per-package releases
- Add update-release job to patch Docker image link after images are
pushed to GHCR
- Update changesets-pr.yml to trigger on .server-changes, enhance PR
body, and clean up consumed files
- Document server changes in CLAUDE.md, CONTRIBUTING.md, CHANGESETS.md,
and RELEASE.md
2026-02-25 13:54:19 +00:00
Eric Allam 19733c8338 docs(queues): Cover new queue limits and TTL system (#3030)
<!-- devin-review-badge-begin -->

---

<a
href="https://app.devin.ai/review/triggerdotdev/trigger.dev/pull/3030"
target="_blank">
  <picture>
<source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-open-in-devin-review-dark.svg?v=1">
<img
src="https://static.devin.ai/assets/gh-open-in-devin-review-light.svg?v=1"
alt="Open with Devin">
  </picture>
</a>
<!-- devin-review-badge-end -->
2026-02-25 11:18:44 +00:00
Matt Aitken 9ba608d2cf TRQL function tests and fixes (#3076)
What changed
- Fixed some functions like dateAdd, toString, ifNotFinite
- Removed all functions that accept lambdas as they're not supported
(yet)
- Added tests for all TRQL functions that use ClickHouse
2026-02-24 19:44:07 +00:00
Iss 89c73ed8ba docs: document run result and batch API endpoints (#3121)
Adds OpenAPI specs and sidebar pages for four previously undocumented
public endpoints: retrieve
run result, per-task batch trigger, retrieve batch, and retrieve batch
results.
2026-02-24 13:30:16 -05:00
Iss 97bf89873e docs: document undocumented run API endpoints (#3120)
Adds API reference pages for three previously undocumented run
endpoints: retrieve run events, retrieve run trace, and add tags to a
run.
2026-02-24 10:49:39 -05:00
Iss b60788df82 docs: note that onCancel only runs during active execution (#3119)
Adds a warning to the onCancel docs clarifying that the hook only fires
when a run is actively executing
2026-02-23 21:24:38 +00:00
Eric Allam 6409fea6ac fix(engine): allow disabling the ttl system consumers independently from the whole system (#3115) 2026-02-23 16:39:30 +00:00
Eric Allam ae46e3f7c8 feat(server): New TTL system, enforce max queue length limits, lazy waitpoint creation (#2980)
This PR implements a new run TTL system and queue size limits to prevent
unbounded queue growth which should help prevent situations where queues
enter a "death spiral" where the queue will never be able to catch up.

The main/correct way to battle this situation is to enforce a maximum
TTL on all runs (e.g. up to 14 days) where runs that have been queued
for that maximum TTL will get auto-expired, making room for newer runs
to execute. This required creating a new TTL system that can handle
higher workloads and is now deeply integrated into the RunQueue. When
runs are enqueued with a TTL, they are added to their normal queue as
well as to the TTL queue. When runs are dequeued, they are removed from
both their normal queue and the TTL queue. If runs are dequeued by the
TTL system, they are removed from their normal queue. Both these
dequeues happen automatically so there is no race condition.

The TTL expiration system is also made reliable by expiring runs via a
Redis worker, which is enqueued to atomically inside the TTL dequeue lua
script.

### Optional associated waitpoints

Additionally, this PR implements an optimization where runs that aren't
triggered with a dependent parent run will no longer create an
associated waitpoint. Associated waitpoints are then lazily created if a
dependent run wants to wait for the child run post-facto (via debounce
or idempotency), which is a rare situation but is possible. This means
fewer waitpoint creations but also fewer waitpoint completions for runs
with no dependencies.

### Environment Queue Limits

Prevents any single queue growing too large by enforcing queue size
limits at trigger time.

- Queue size checks happen at trigger time - runs are rejected if queue
would exceed limit
- Dashboard UI shows queue limits on both the Queues page and a new
Limits page
- In-memory caching for queue size checks to reduce Redis load

### Batch trigger fixes

Currently when a batch item cannot be created for whatever reason (e.g.
queue limits) the run will never get created, which means a stalled run
if using `batchTriggerAndWait`. We've updated the system to handle this
differently: now when a batch item cannot be triggered and converted
into a run, we will eventually (after retrying 8 times up to 30s) we
will create a "pre-failed" run with the error details, correctly
resolving the batchTriggerAndWait.
2026-02-23 15:57:32 +00:00
Oskar Otwinowski 69dc7bcde8 feat(webapp): Vercel / Slack integrations improvements (#3108)
##  Checklist

- [x] I have followed every step in the [contributing
guide](https://github.com/triggerdotdev/trigger.dev/blob/main/CONTRIBUTING.md)
- [x] The PR title follows the convention.
- [x] I ran and tested the code works

---

## Testing

Slack + GitHub + Vercel + Builds + Deployments

---

## Changelog

Settings changes:
- Split general from integrations
- Add new Slack section to org level integrations
Vercel improvements:
- bugfix for TRIGGER_SECRET_KEY collision
- onboarding improvements for connecting to projects
- new loops event
Slack improvements:
- nicer alerts
Webhook/Email alerts:
- rich events with Github & Vercel integration data

---

## Screenshots


<img width="2550" height="652" alt="Screenshot 2026-02-20 at 21 53 34"
src="https://github.com/user-attachments/assets/8d7c9f1d-5fe9-4516-8fb3-885460b4207f"
/>
<img width="843" height="710" alt="Screenshot 2026-02-23 at 10 55 54"
src="https://github.com/user-attachments/assets/8ea72c1f-431b-493c-b9a9-8076cce12262"
/>
<img width="765" height="466" alt="Screenshot 2026-02-20 at 21 52 46"
src="https://github.com/user-attachments/assets/157fafb8-b7bf-499d-8953-c2aed5e44ce0"
/>
<img width="691" height="261" alt="Screenshot 2026-02-20 at 22 04 24"
src="https://github.com/user-attachments/assets/3aea7369-2008-4af8-a9c0-5fbfa2cc381d"
/>
<img width="2032" height="1114" alt="Screenshot 2026-02-19 at 14 48 49"
src="https://github.com/user-attachments/assets/dc10c14e-cd15-445a-b5be-d694d29d20e5"
/>
<img width="2032" height="1114" alt="Screenshot 2026-02-19 at 14 49 04"
src="https://github.com/user-attachments/assets/1ef591fd-fd00-430a-9649-8b18cff9586d"
/>
<img width="1583" height="1115" alt="Screenshot 2026-02-19 at 17 32 56"
src="https://github.com/user-attachments/assets/c5c8f318-d193-4dd4-86f7-1cc4bbcc4e0c"
/>
<img width="422" height="187" alt="Screenshot 2026-02-20 at 21 57 41"
src="https://github.com/user-attachments/assets/37865cb6-4c0d-40ef-9c60-7b057d546c61"
/>
<img width="1583" height="1115" alt="Screenshot 2026-02-19 at 17 33 06"
src="https://github.com/user-attachments/assets/e9180e8e-e611-4734-9232-80c62ff863ad"
/>

💯
2026-02-23 13:48:09 +00:00
Matt Aitken 3a7054628f Fix: show the deprecation panel if it's an old project and v3 (#3113)
Without doing an expensive query we can’t tell if it’s definitely a v3
projects – like getting run counts.
So let’s just assume if the project hasn’t been upgraded to v4 (by
running dev/deploy CLI with v4) AND the project is older than the v4
release then it’s v3.
2026-02-23 11:10:37 +00:00
Eric Allam 676525279a docs: otel metrics (#3096) 2026-02-20 16:51:48 +00:00
James Ritchie 79f8cdef72 Fix(webapp): logs button + logs table row link fix (#3107)
Small fixes and improvements to the logs page:

- Clicking the Run ID didn't open inspector
- Swapped the "open link in tab" icon with Runs icon
- Prevent tooltip hovering on Level info

<img width="350" height="206" alt="CleanShot 2026-02-20 at 10 00 37@2x"
src="https://github.com/user-attachments/assets/3e82f24a-c0a1-4c01-a8e9-9e06a8af982a"
/>
2026-02-20 16:44:55 +00:00
github-actions[bot] 98bf706437 chore: release v4.4.1 (#3100)
🚀 Publish Trigger.dev Docker / units (push) Failing after 0s
🚀 Publish Trigger.dev Docker / typecheck (push) Failing after 6s
🚀 Publish Trigger.dev Docker / publish-webapp (push) Has been skipped
🚀 Publish Trigger.dev Docker / publish-worker (push) Has been skipped
🚀 Publish Trigger.dev Docker / publish-worker-v4 (push) Has been skipped
This PR was opened by the [Changesets
release](https://github.com/changesets/action) GitHub action. When
you're ready to do a release, you can merge this and publish to npm
yourself or [setup this action to publish
automatically](https://github.com/changesets/action#with-publishing). If
you're not ready to do a release yet, that's fine, whenever you add more
changesets to main, this PR will be updated.


# Releases
## @trigger.dev/build@4.4.1

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.4.1`

## trigger.dev@4.4.1

### Patch Changes

- Add OTEL metrics pipeline for task workers. Workers collect process
CPU/memory, Node.js runtime metrics (event loop utilization, event loop
delay, heap usage), and user-defined custom metrics via
`otel.metrics.getMeter()`. Metrics are exported to ClickHouse with
10-second aggregation buckets and 1m/5m rollups, and are queryable
through the dashboard query engine with typed attribute columns,
`prettyFormat()` for human-readable values, and AI query support.
([#3061](https://github.com/triggerdotdev/trigger.dev/pull/3061))
-   Updated dependencies:
    -   `@trigger.dev/build@4.4.1`
    -   `@trigger.dev/core@4.4.1`
    -   `@trigger.dev/schema-to-json@4.4.1`

## @trigger.dev/python@4.4.1

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/sdk@4.4.1`
    -   `@trigger.dev/build@4.4.1`
    -   `@trigger.dev/core@4.4.1`

## @trigger.dev/react-hooks@4.4.1

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.4.1`

## @trigger.dev/redis-worker@4.4.1

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.4.1`

## @trigger.dev/rsc@4.4.1

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.4.1`

## @trigger.dev/schema-to-json@4.4.1

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.4.1`

## @trigger.dev/sdk@4.4.1

### Patch Changes

- Add OTEL metrics pipeline for task workers. Workers collect process
CPU/memory, Node.js runtime metrics (event loop utilization, event loop
delay, heap usage), and user-defined custom metrics via
`otel.metrics.getMeter()`. Metrics are exported to ClickHouse with
10-second aggregation buckets and 1m/5m rollups, and are queryable
through the dashboard query engine with typed attribute columns,
`prettyFormat()` for human-readable values, and AI query support.
([#3061](https://github.com/triggerdotdev/trigger.dev/pull/3061))
-   Updated dependencies:
    -   `@trigger.dev/core@4.4.1`

## @trigger.dev/core@4.4.1

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2026-02-20 16:40:06 +00:00
Iss 354363e408 docs: Vercel integration – marketplace link, Build options, Root Directory warning (#3097)
Adds a direct Vercel Marketplace link, documents configuring build
options via the project config page, and adds a warning and workaround
for projects using a Vercel Root Directory
2026-02-20 10:56:44 -05:00
Iss 525145def0 docs: v3 deprecation notice and Migrate using AI updates on migrating-from-v3 (#3098)
Adds a deprecation warning at the top of the migrating-from-v3 page and
updates the “Migrate using AI” prompt and intro
2026-02-20 10:55:48 -05:00
Eric Allam 23c327ea16 fix(metrics): disable collecting system and filesystem metrics in deployed workers (#3106) 2026-02-20 15:45:35 +00:00
Eric Allam d794101c67 fix(webapp): fix broken MFA by only committing one auth session set-cookie call (#3104)
Co-authored-by: Oskar Otwinowski <oskar.otwinowski@gmail.com>
2026-02-20 15:09:53 +00:00
Eric Allam f325638892 fix(tests): fix flaky getSnapshotsSince test (#3103) 2026-02-20 14:52:59 +00:00
Eric Allam 68e3f8c9db chore(clickhouse): more clickhouse migration conflict fixes (#3102) 2026-02-20 13:28:58 +00:00
Eric Allam 2071090042 chore(clickhouse): fix clickhouse migration version conflict (#3101) 2026-02-20 14:24:56 +01:00
Eric Allam 469b039090 feat: OTEL metrics pipeline for task workers (#3061)
- Adds an end-to-end OTEL metrics pipeline: task workers collect and
export metrics via OpenTelemetry, the webapp ingests them into
ClickHouse, and they're queryable through the existing dashboard query
engine
- Workers emit process CPU/memory metrics (via
`@opentelemetry/host-metrics`) and Node.js runtime metrics (event loop
utilization, event loop delay, heap usage)
- Users can create custom metrics in their tasks via
`otel.metrics.getMeter()` from `@trigger.dev/sdk`
- Metrics are automatically tagged with run context (run ID, task slug,
machine, worker version) so they can be sliced per-run, per-task, or
per-machine
- The TSQL query engine gains metrics table support with typed attribute
columns, `prettyFormat()` for human-readable values, and per-schema time
bucket thresholds
- Includes reference tasks
(`references/hello-world/src/trigger/metrics.ts`) demonstrating
CPU-intensive, memory-ramp, bursty workload, and custom metrics patterns

## What changed

### Metrics collection (packages/core, packages/cli-v3)
- **Metrics export pipeline** — `TracingSDK` now sets up a
`MeterProvider` with a `PeriodicExportingMetricReader` that chains
through `TaskContextMetricExporter` (adds run context attributes) and
`BufferingMetricExporter` (batches exports to reduce overhead)
- **Host metrics** — Enabled `@opentelemetry/host-metrics` for process
CPU, memory, and system-level metrics
- **Node.js runtime metrics** — New `nodejsRuntimeMetrics.ts` module
using `performance.eventLoopUtilization()`, `monitorEventLoopDelay()`,
and `process.memoryUsage()` to emit 6 observable gauges
- File system and diskio metrics
- **Custom metrics** — Exposed `otel.metrics` from `@trigger.dev/sdk` so
users can create counters, histograms, and gauges in their tasks
- **Machine ID** — Stable per-worker machine identifier for grouping
metrics
- **Dev worker** — Drops `system.*` metrics to reduce noise, keeps
sending metrics between runs in warm workers

### Metrics ingestion (apps/webapp)
- **OTEL endpoint** — `otel.v1.metrics.ts` accepts OTEL metric export
requests (JSON and protobuf), converts to ClickHouse rows
- **ClickHouse schema** — `017_create_metrics_v1.sql` with 10-second
aggregation buckets, JSON attributes column, 60-day TTLs

### Query engine (internal-packages/tsql, apps/webapp)
- **Metrics query schema** — Typed columns for metric attributes
(`task_identifier`, `run_id`, `machine_name`, `worker_version`, etc.)
extracted from the JSON attributes column
- **`prettyFormat()`** — TSQL function that annotates columns with
format hints (`bytes`, `percent`, `durationSeconds`) for frontend
rendering without changing the underlying data
- **Per-schema time buckets** — Different tables can define their own
time bucket thresholds (metrics uses tighter intervals than runs)
- **AI query integration** — The AI query service knows about the
metrics table and can generate metric queries
- **Chart improvements** — Better formatting for byte values,
percentages, and durations in charts and tables

### Reference project
- **`references/hello-world/src/trigger/metrics.ts`** — 6 example tasks:
`cpu-intensive`, `memory-ramp`, `bursty-workload`, `sustained-workload`,
`concurrent-load`, `custom-metrics`

## Test plan

- [ ] Build all packages and webapp
- [ ] Start dev worker with hello-world reference project
- [ ] Run `cpu-intensive`, `memory-ramp`, and `custom-metrics` tasks
- [ ] Verify metrics in ClickHouse: `SELECT DISTINCT metric_name FROM
metrics_v1`
- [ ] Query via dashboard AI: "show me CPU utilization over time"
- [ ] Verify `prettyFormat` renders correctly in chart tooltips and
table cells
- [ ] Confirm dev worker drops `system.*` metrics but keeps `process.*`
and `nodejs.*`
2026-02-20 13:16:34 +00:00
Eric Allam 38981f5e70 chore(docs): cover maxDelay debounce option (#2985)
<!-- devin-review-badge-begin -->

---

<a
href="https://app.devin.ai/review/triggerdotdev/trigger.dev/pull/2985">
  <picture>
<source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-open-in-devin-review-dark.svg?v=1">
<img
src="https://static.devin.ai/assets/gh-open-in-devin-review-light.svg?v=1"
alt="Open with Devin">
  </picture>
</a>
<!-- devin-review-badge-end -->
2026-02-19 13:50:33 +00:00
Matt Aitken 22505e9803 Query and Metrics docs (#3074)
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
2026-02-19 13:50:13 +00:00
github-actions[bot] 84a00b6eb6 chore: release v4.4.0 (#2941)
🚀 Publish Trigger.dev Docker / typecheck (push) Failing after 0s
🚀 Publish Trigger.dev Docker / publish-webapp (push) Has been skipped
🚀 Publish Trigger.dev Docker / publish-worker (push) Has been skipped
🚀 Publish Trigger.dev Docker / publish-worker-v4 (push) Has been skipped
🚀 Publish Trigger.dev Docker / units (push) Failing after 3s
This PR was opened by the [Changesets
release](https://github.com/changesets/action) GitHub action. When
you're ready to do a release, you can merge this and publish to npm
yourself or [setup this action to publish
automatically](https://github.com/changesets/action#with-publishing). If
you're not ready to do a release yet, that's fine, whenever you add more
changesets to main, this PR will be updated.


# Releases
## @trigger.dev/sdk@4.4.0

### Minor Changes

- Added `query.execute()` which lets you query your Trigger.dev data
using TRQL (Trigger Query Language) and returns results as typed JSON
rows or CSV. It supports configurable scope (environment, project, or
organization), time filtering via `period` or `from`/`to` ranges, and a
`format` option for JSON or CSV output.
([#3060](https://github.com/triggerdotdev/trigger.dev/pull/3060))

    ```typescript
    import { query } from "@trigger.dev/sdk";
    import type { QueryTable } from "@trigger.dev/sdk";

    // Basic untyped query
const result = await query.execute("SELECT run_id, status FROM runs
LIMIT 10");

    // Type-safe query using QueryTable to pick specific columns
const typedResult = await query.execute<QueryTable<"runs", "run_id" |
"status" | "triggered_at">>(
      "SELECT run_id, status, triggered_at FROM runs LIMIT 10"
    );
    typedResult.results.forEach((row) => {
      console.log(row.run_id, row.status); // Fully typed
    });

    // Aggregation query with inline types
const stats = await query.execute<{ status: string; count: number }>(
      "SELECT status, COUNT(*) as count FROM runs GROUP BY status",
      { scope: "project", period: "30d" }
    );

    // CSV export
    const csv = await query.execute("SELECT run_id, status FROM runs", {
      format: "csv",
      period: "7d",
    });
    console.log(csv.results); // Raw CSV string
    ```

### Patch Changes

- Add `maxDelay` option to debounce feature. This allows setting a
maximum time limit for how long a debounced run can be delayed, ensuring
execution happens within a specified window even with continuous
triggers.
([#2984](https://github.com/triggerdotdev/trigger.dev/pull/2984))

    ```typescript
    await myTask.trigger(payload, {
      debounce: {
        key: "my-key",
        delay: "5s",
maxDelay: "30m", // Execute within 30 minutes regardless of continuous
triggers
      },
    });
    ```

- Aligned the SDK's `getRunIdForOptions` logic with the Core package to
handle semantic targets (`root`, `parent`) in root tasks.
([#2874](https://github.com/triggerdotdev/trigger.dev/pull/2874))

- Export `AnyOnStartAttemptHookFunction` type to allow defining
`onStartAttempt` hooks for individual tasks.
([#2966](https://github.com/triggerdotdev/trigger.dev/pull/2966))

- Fixed a minor issue in the deployment command on distinguishing
between local builds for the cloud vs local builds for self-hosting
setups.
([#3070](https://github.com/triggerdotdev/trigger.dev/pull/3070))

-   Updated dependencies:
    -   `@trigger.dev/core@4.4.0`

## @trigger.dev/build@4.4.0

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.4.0`

## trigger.dev@4.4.0

### Patch Changes

- Fix runner getting stuck indefinitely when `execute()` is called on a
dead child process.
([#2978](https://github.com/triggerdotdev/trigger.dev/pull/2978))
- Add optional `timeoutInSeconds` parameter to the
`wait_for_run_to_complete` MCP tool. Defaults to 60 seconds. If the run
doesn't complete within the timeout, the current state of the run is
returned instead of waiting indefinitely.
([#3035](https://github.com/triggerdotdev/trigger.dev/pull/3035))
- Fixed a minor issue in the deployment command on distinguishing
between local builds for the cloud vs local builds for self-hosting
setups.
([#3070](https://github.com/triggerdotdev/trigger.dev/pull/3070))
-   Updated dependencies:
    -   `@trigger.dev/core@4.4.0`
    -   `@trigger.dev/build@4.4.0`
    -   `@trigger.dev/schema-to-json@4.4.0`

## @trigger.dev/core@4.4.0

### Patch Changes

- Add `maxDelay` option to debounce feature. This allows setting a
maximum time limit for how long a debounced run can be delayed, ensuring
execution happens within a specified window even with continuous
triggers.
([#2984](https://github.com/triggerdotdev/trigger.dev/pull/2984))

    ```typescript
    await myTask.trigger(payload, {
      debounce: {
        key: "my-key",
        delay: "5s",
maxDelay: "30m", // Execute within 30 minutes regardless of continuous
triggers
      },
    });
    ```

- Fixed a minor issue in the deployment command on distinguishing
between local builds for the cloud vs local builds for self-hosting
setups.
([#3070](https://github.com/triggerdotdev/trigger.dev/pull/3070))

- fix: vendor superjson to fix ESM/CJS compatibility
([#2949](https://github.com/triggerdotdev/trigger.dev/pull/2949))

Bundle superjson during build to avoid `ERR_REQUIRE_ESM` errors on
Node.js versions that don't support `require(ESM)` by default (&lt;
22.12.0) and AWS Lambda which intentionally disables it.

- Add Vercel integration support to API schemas: `commitSHA` and
`integrationDeployments` on deployment responses, and `source` field for
environment variable imports.
([#2994](https://github.com/triggerdotdev/trigger.dev/pull/2994))

## @trigger.dev/python@4.4.0

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.4.0`
    -   `@trigger.dev/sdk@4.4.0`
    -   `@trigger.dev/build@4.4.0`

## @trigger.dev/react-hooks@4.4.0

### Patch Changes

- Fix `onComplete` callback firing prematurely when the realtime stream
disconnects before the run finishes.
([#2929](https://github.com/triggerdotdev/trigger.dev/pull/2929))
-   Updated dependencies:
    -   `@trigger.dev/core@4.4.0`

## @trigger.dev/redis-worker@4.4.0

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.4.0`

## @trigger.dev/rsc@4.4.0

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.4.0`

## @trigger.dev/schema-to-json@4.4.0

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.4.0`

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2026-02-19 13:38:10 +00:00
James Ritchie 506b161751 feature(webapp): Show a v3 deprecation notice in the side bar (#3090)
- If your project is v3, show a v3 deprecation panel in the side menu
- If there is an active incident, show the incident panel instead
- Links to the Migration guide in the docs
- Displays when the side menu is collapsed



https://github.com/user-attachments/assets/f8492713-c58b-4f83-bcce-0e85f4a967ef

<img width="972" height="694" alt="CleanShot 2026-02-19 at 08 13 59@2x"
src="https://github.com/user-attachments/assets/0599dd20-d598-48c6-b83c-208648cee071"
/>
2026-02-19 11:02:10 +00:00
Matt Aitken 1ec7722690 Add Queue Management API endpoints and SDK documentation (#3087)
Closes #<issue>

##  Checklist

- [ ] I have followed every step in the [contributing
guide](https://github.com/triggerdotdev/trigger.dev/blob/main/CONTRIBUTING.md)
- [ ] The PR title follows the convention.
- [ ] I ran and tested the code works

---

## Testing

N/A - Documentation and OpenAPI schema updates only.

---

## Changelog

Added comprehensive Queue Management API support:

**OpenAPI Endpoints:**
- `GET /api/v1/queues` - List all queues with pagination support
- `GET /api/v1/queues/{queueParam}` - Retrieve a specific queue by ID,
task ID, or custom queue name
- `POST /api/v1/queues/{queueParam}/pause` - Pause or resume a queue
- `POST /api/v1/queues/{queueParam}/concurrency/override` - Override
queue concurrency limits
- `POST /api/v1/queues/{queueParam}/concurrency/reset` - Reset
concurrency limits to base values

**Schema Definitions:**
- `QueueObject` - Complete queue representation with concurrency details
- `ListQueuesResult` - Paginated queue listing response

**Documentation:**
- Updated `queue-concurrency.mdx` with SDK usage examples for queue
management
- Added 5 new management API documentation pages for each endpoint
- Updated `docs.json` navigation structure with new "Queues API" section

All endpoints support flexible queue identification (by ID, task ID, or
custom queue name) and include TypeScript code samples.

---

## Screenshots

N/A

💯

https://claude.ai/code/session_01LyrXwxHCbejvi34fykifPP

Co-authored-by: Claude <noreply@anthropic.com>
2026-02-18 22:35:39 +00:00
Iss 2825a200e6 docs: added v3 to v4 migreation note about IP change (#3086) 2026-02-18 18:08:00 +00:00
Matt Aitken eb0f963393 feat(metrics): Dashboard charts performance improvements (#3083)
Summary
- Only render the top 50 series
- Improved rendering performance on bar charts
2026-02-18 16:12:00 +00:00
nicktrn 59b6eb9a3e feat(webapp): add region selector to test and replay task (#3082)
Adds a region selector to the Test task page and Replay run dialog, so
users can override the region from the dashboard. Disabled with a
placeholder for dev environments.

Closes #3016
2026-02-18 10:39:38 +00:00
Mihai Popescu b793f33e59 Logs: new materialized view and some UI improvements (#3069)
This will prevent internal logs to be added to the
task_events_search_table

Closes #<issue>

##  Checklist

- [x] I have followed every step in the [contributing
guide](https://github.com/triggerdotdev/trigger.dev/blob/main/CONTRIBUTING.md)
- [x] The PR title follows the convention.
- [x] I ran and tested the code works

---

## Testing

Ran the migration, deleted the old invalid rows and ran new tasks.
The undesired logs are not added to the table.

---

## Changelog

Updated the MATERIALIZED VIEW to also filter for `trace_id != ''`

---------

Co-authored-by: Matt Aitken <matt@mattaitken.com>
2026-02-18 09:48:52 +00:00
Oskar Otwinowski 6e0b54b081 docs(vercel): add comprehensive Vercel integration docs (#3081)
Expand documentation for the Vercel integration with detailed usage,
installation, environment variable sync, atomic deployments, and
environment mapping. Replace the previous "coming soon" placeholder with
complete instructions and UI flow for connecting via the Trigger.dev
dashboard or the Vercel Marketplace. Explain required GitHub
integration,
how env vars sync in both directions, which vars are excluded, and how
to control sync behavior. Describe atomic deployments (default for
production), how they gate Vercel deployments to ensure task/app
consistency, and note related configuration changes. Add tips and notes
to guide setup and troubleshooting.

This provides users with actionable guidance to connect Vercel, map
environments, and keep app and tasks in sync without custom CI scripts.
2026-02-17 18:19:00 +01:00
Eric Allam 4c986ad1eb fix(docs): Fix loop to iterate over results.runs (#3077)
To get access to the runs from `batch.triggerAndWait` use `results.runs`
2026-02-17 13:52:58 +01:00
Saadi Myftija 7af789bc3e fix(deployments): external build token generation issue (#3070)
Fixes an issue introduced in #3024.

The behavior for local builds in older CLI versions relies on
`externalBuildData` to be defined to distinguish from the self-hosting
local build path, even though it doesn't actually use the token.
2026-02-16 19:05:40 +01:00
Oskar Otwinowski 72ce6a8300 fix(vercel): Keep search params for OAuth redirects (#3071) 2026-02-16 18:51:33 +01:00
Pramod Dhungana 921285ca32 add friendly messages for common http error codes (#3001)
Closes #<issue>

##  Checklist

- [ x] I have followed every step in the [contributing
guide](https://github.com/triggerdotdev/trigger.dev/blob/main/CONTRIBUTING.md)
- [x ] The PR title follows the convention.
- [ x] I ran and tested the code works

---

## Testing

Verified the error message mapping logic locally and ensured existing
behavior remains unchanged.


---

## Changelog

Added friendly messages for HTTP 401, 403, and 429 errors.





<!-- devin-review-badge-begin -->

---

<a
href="https://app.devin.ai/review/triggerdotdev/trigger.dev/pull/3001">
  <picture>
<source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-open-in-devin-review-dark.svg?v=1">
<img
src="https://static.devin.ai/assets/gh-open-in-devin-review-light.svg?v=1"
alt="Open with Devin">
  </picture>
</a>
<!-- devin-review-badge-end -->
2026-02-16 11:42:26 +00:00
Iss 667a93ca3c docs: add ClickHouse migration troubleshooting tip (#3011)
Adds troubleshooting tip for when ClickHouse migrations report "no
migrations to run" but the schema is missing. This happens when the
goose migration tracker is out of sync with the actual schema state.

<!-- devin-review-badge-begin -->

---

<a
href="https://app.devin.ai/review/triggerdotdev/trigger.dev/pull/3011"
target="_blank">
  <picture>
<source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-open-in-devin-review-dark.svg?v=1">
<img
src="https://static.devin.ai/assets/gh-open-in-devin-review-light.svg?v=1"
alt="Open with Devin">
  </picture>
</a>
<!-- devin-review-badge-end -->
2026-02-16 11:41:31 +00:00
DKP 4dfa65809a Dropped bracket highlight and border by 50% (#3068) 2026-02-16 11:31:02 +00:00
Matt Aitken d4cd34094e Query API and SDK (#3060)
Summary
- Add API endpoint to run TRQL queries
- Implement SDK function for executing queries

## SDK
Added `query.execute()` which lets you query your Trigger.dev data using
TRQL (Trigger Query Language) and returns results as typed JSON rows or
CSV. It supports configurable scope (environment, project, or
organization), time filtering via `period` or `from`/`to` ranges, and a
`format` option for JSON or CSV output.

```typescript
import { query } from "@trigger.dev/sdk";
import type { QueryTable } from "@trigger.dev/sdk";

// Basic untyped query
const result = await query.execute("SELECT run_id, status FROM runs LIMIT 10");

// Type-safe query using QueryTable to pick specific columns
const typedResult = await query.execute<QueryTable<"runs", "run_id" | "status" | "triggered_at">>(
  "SELECT run_id, status, triggered_at FROM runs LIMIT 10"
);
typedResult.results.forEach(row => {
  console.log(row.run_id, row.status); // Fully typed
});

// Aggregation query with inline types
const stats = await query.execute<{ status: string; count: number }>(
  "SELECT status, COUNT(*) as count FROM runs GROUP BY status",
  { scope: "project", period: "30d" }
);

// CSV export
const csv = await query.execute(
  "SELECT run_id, status FROM runs",
  { format: "csv", period: "7d" }
);
console.log(csv.results); // Raw CSV string
```
2026-02-16 10:53:41 +00:00
James Ritchie 796ad29386 Fix(webapp): consistent query button width (#3067)
Prevents layout shift by keeping the button width the same when
isLoading

Normal state
<img width="423" height="186" alt="CleanShot 2026-02-16 at 10 31 23"
src="https://github.com/user-attachments/assets/232bc14f-dc2c-4092-9b46-f6d652568633"
/>

isLoading
<img width="403" height="137" alt="CleanShot 2026-02-16 at 10 31 43"
src="https://github.com/user-attachments/assets/e9215b48-c2cc-4ede-91ff-08bbdb2382a0"
/>
2026-02-16 10:49:37 +00:00
James Ritchie b4e08bddec Fix(webapp): metrics UI improvements (#3063)
- Lots of small UI improvements
- Toggle full screen charts with "V" shortcut

<img width="3504" height="2286" alt="CleanShot 2026-02-14 at 20 06
30@2x"
src="https://github.com/user-attachments/assets/32403661-06b3-4f6c-a074-243b3f43197d"
/>

<img width="362" height="222" alt="CleanShot 2026-02-14 at 20 06 58@2x"
src="https://github.com/user-attachments/assets/ee4f5859-4b71-482e-9636-5abd78d9f623"
/>

<img width="434" height="360" alt="CleanShot 2026-02-14 at 20 06 53@2x"
src="https://github.com/user-attachments/assets/44339348-f3f3-425b-bbf3-bb41ed6a8e59"
/>

<img width="460" height="310" alt="CleanShot 2026-02-14 at 20 06 48@2x"
src="https://github.com/user-attachments/assets/eb3b6af6-b88a-4677-8a48-acfa6b768770"
/>

<img width="502" height="332" alt="CleanShot 2026-02-14 at 20 06 45@2x"
src="https://github.com/user-attachments/assets/b788d78b-8f4a-4b18-94c8-487673f3ec7b"
/>

<img width="370" height="257" alt="CleanShot 2026-02-14 at 20 10 29@2x"
src="https://github.com/user-attachments/assets/a81e8b14-b2eb-402c-bf3b-affd0d4fde26"
/>
2026-02-14 21:26:25 +00:00
Iss 5d6085dcff docs: Realtime skipColumns and Bun indexing troubleshooting (#3054)
Documents the skipColumns option on useRealtimeRun and
useRealtimeRunsWithTag for status-only subscriptions (smaller payloads,
e.g. for progress/completion UI). Adds a troubleshooting section for the
“Failed to index deployment” source-map error when using the Bun
runtime, with a pnpm patch workaround and link to the GitHub issue
2026-02-14 14:02:00 -05:00
nicktrn 1d744fa3c6 fix(dashboard): apply time filter preset periods immediately on click (#3053)
Extract `applyPeriod` callback from `applySelection` so preset period
buttons ("Created in the last X") apply immediately when clicked,
instead of only updating the selection state and requiring a separate
apply step.

Also validates `maxPeriodDays` on instant-apply so the upgrade prompt
still works correctly for plan-limited periods.
2026-02-13 18:35:39 +00:00
Matt Aitken a3d3b17df4 TRQL: always add FINAL keyword (#3051)
For now we’re going to always add FINAL to TRQL queries for data
correctness.

In the future we will implement an automated optimization where we use
`SELECT argMax(column, _version)` and `WHERE _is_deleted = 0`. But this
is a more complex change and needs more investigation of downsides.
2026-02-13 17:34:43 +00:00
James Ritchie 35e11e09a3 Fix(webapp): Show an error to the user when their add-on upgrade payment fails (#3050)
A customer experienced a bug where their subscription downgraded to the
free plan unintentionally. This was due to a concurrency upgrade payment
attempt that failed a card check. We auto retry the payment across 2
weeks of attempts. When the final attempt failed, the whole subscription
downgraded.

Now we check if the payment is successful and if not, return an error
immediately so the subscription isn't modified until a successful
payment is made for an upgrade.
2026-02-13 16:52:04 +00:00
Matt Aitken 9030e94362 Metrics improvements (#3046)
Summary
- Remove LIMIT from built-in dashboard queries
- Make concurrency configurable per project via environment variables
- Fix widget fallback period to Metrics default (1d) instead of 7d
- Handle concurrency at the project level
- Sort series for graphs so largest is displayed at the bottom (legend
shows largest at top)
- Use average aggregation for some built-in charts
- Improve aggregation handling for the legend
- Only render chart points when there is data; render dots on line
charts
- Truncate legend items and show tooltip on hover
- Better preserve chart configuration when the underlying query changes
2026-02-13 15:57:11 +00:00
Eric Allam 2462c80c8a chore(vouch): vouch for capaj (#3048) 2026-02-13 15:56:05 +01:00
Mihai Popescu a8f9a90280 improv(webapp): Add new table for optimized logs search (#3036)
##  Checklist

- [X] I have followed every step in the [contributing
guide](https://github.com/triggerdotdev/trigger.dev/blob/main/CONTRIBUTING.md)
- [X] The PR title follows the convention.
- [X] I ran and tested the code works

---

## Testing

Tested the migration on test env and locally.
Tested query and merge performance.
Generated tasks and observed the ingested data and searches.

---

## Changelog

* New ClickHouse table & MV (task_events_search_v1): A search-optimized
materialized view that filters out debug events, partial spans, and
empty span events at ingestion time.
* ClickHouse client updates: New getLogsSearchListQueryBuilder and
taskEventsSearch accessor on the ClickHouse class.
* LogsListPresenter: Switches to the new search table, uses
triggered_timestamp for cursor pagination instead of unixTimestamp.
  *  Spans route: Also switches to the new search query builder.
* Seed spanSpammer task: Adds a 10s trace with events and metadata
operations for testing.
2026-02-13 16:28:14 +02:00
Oskar Otwinowski 4b7f67604a feat(webapp/deployments): Vercel improvements & fixes (#3037) 2026-02-13 14:06:22 +01:00
Matt Aitken bfe417f679 docs: add note about single-issue PRs to contributing guide (#3044)
Add a prominent notice at the top of CONTRIBUTING.md clarifying that we
only accept PRs addressing a single issue, not multiple fixes.

https://claude.ai/code/session_011ZNQd5zkf38piSMhWbgiWt

Co-authored-by: Claude <noreply@anthropic.com>
2026-02-13 10:07:06 +00:00
Eric Allam ba320a4bdb chore(vouch): vouch for gautamsi (#3039) 2026-02-12 22:27:14 +00:00
Matt Aitken bc0d1ff59a Metrics dashboards (#3019)
Summary
- Implemented metrics dashboards with a built-in dashboard and custom
dashboards
- Added a "Big number” display type

What changed
- New data format for metric layouts and saving/editing layouts
(editing, saving, cancel revert)
  - QueryWidget usable on Query page and Metrics dashboards
  - Time filtering, auto-reloading and timeBucket() auto-bin support
- Filters added to metrics; widget popover/improved history and blank
states
- Side menu:
- Metrics/Insights section with icons, colors, padding, collapsible
behavior and reordering of custom dashboards
- Move action logic into service for reuse and API querying; refactor
reordering for reuse
<!-- devin-review-badge-begin -->

---

<a
href="https://app.devin.ai/review/triggerdotdev/trigger.dev/pull/3019"
target="_blank">
  <picture>
<source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-open-in-devin-review-dark.svg?v=1">
<img
src="https://static.devin.ai/assets/gh-open-in-devin-review-light.svg?v=1"
alt="Open with Devin">
  </picture>
</a>
<!-- devin-review-badge-end -->

---------

Co-authored-by: James Ritchie <james@trigger.dev>
2026-02-12 17:48:02 +00:00
Eric Allam 062bcaece8 feat(mcp): add timeout parameter to wait_for_run_to_complete tool (#3035)
## Summary

- Adds an optional `timeoutInSeconds` parameter (default 60s) to the
`wait_for_run_to_complete` MCP tool
- If the run doesn't complete within the timeout, returns the current
run state instead of blocking indefinitely
- Uses `AbortSignal.timeout()` combined with the existing MCP signal

Fixes #3032
2026-02-12 16:14:25 +00:00
nicktrn c2085e6cc6 feat(dashboard): link git sha and ref to GitHub on settings page (#3034)
Make the git SHA and git ref in the org settings sidebar clickable links
to GitHub — SHA links to the commit, ref links to the branch/tag.
2026-02-12 15:31:52 +00:00
James Ritchie d7bc37fdc0 Feat(dashboard): show the Betterstack incident title in the dashboard (#3006)
When the incident panel is displayed, show the title added to
BetterStack as the contents of the incident panel.

I've also brightened the UI so it's more visible.

<img width="536" height="590" alt="CleanShot 2026-02-04 at 20 46 36@2x"
src="https://github.com/user-attachments/assets/040a04f8-5b52-40e8-8892-51c8efd6c08c"
/>

<!-- devin-review-badge-begin -->

---

<a
href="https://app.devin.ai/review/triggerdotdev/trigger.dev/pull/3006"
target="_blank">
  <picture>
<source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-open-in-devin-review-dark.svg?v=1">
<img
src="https://static.devin.ai/assets/gh-open-in-devin-review-light.svg?v=1"
alt="Open with Devin">
  </picture>
</a>
<!-- devin-review-badge-end -->
2026-02-12 08:58:21 +00:00
DKP 6e3ac8bd91 docs: cursor cli docs update (remove chmod workaround) (#3031)
<!-- devin-review-badge-begin -->

---

<a
href="https://app.devin.ai/review/triggerdotdev/trigger.dev/pull/3031"
target="_blank">
  <picture>
<source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-open-in-devin-review-dark.svg?v=1">
<img
src="https://static.devin.ai/assets/gh-open-in-devin-review-light.svg?v=1"
alt="Open with Devin">
  </picture>
</a>
<!-- devin-review-badge-end -->
2026-02-11 13:32:49 +00:00
Matt Aitken 170fde3498 Move vouch requirement to top of CONTRIBUTING.md (#3029)
Contributors need to be vouched before opening PRs, but this requirement
was buried far down in the document. This change:
- Adds mention of vouches in the intro paragraph
- Moves the "Getting vouched" section to right after the intro

This makes the requirement more visible to new contributors.

Slack thread:
https://triggerdotdev.slack.com/archives/C0A7Q6F62NS/p1770805895370749
https://claude.ai/code/session_01G6VVbgfUAeCpJfedELdqq1
<!-- devin-review-badge-begin -->

---

<a
href="https://app.devin.ai/review/triggerdotdev/trigger.dev/pull/3029"
target="_blank">
  <picture>
<source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-open-in-devin-review-dark.svg?v=1">
<img
src="https://static.devin.ai/assets/gh-open-in-devin-review-light.svg?v=1"
alt="Open with Devin">
  </picture>
</a>
<!-- devin-review-badge-end -->

Co-authored-by: Claude <noreply@anthropic.com>
2026-02-11 10:44:14 +00:00
Iss ddeb9c415e docs: heartbeats, Bun version, troubleshooting, and preview-branch cleanup (#3026)
Doc updates: 
- new Heartbeats page (yield, progress, external updates)
- Bun supported-version note
-  resource_exhausted troubleshooting with native builder link
- GitHub Actions preview-branch example with closed trigger so branches
archive when PRs close
<!-- devin-review-badge-begin -->

---

<a
href="https://app.devin.ai/review/triggerdotdev/trigger.dev/pull/3026"
target="_blank">
  <picture>
<source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-open-in-devin-review-dark.svg?v=1">
<img
src="https://static.devin.ai/assets/gh-open-in-devin-review-light.svg?v=1"
alt="Open with Devin">
  </picture>
</a>
<!-- devin-review-badge-end -->
2026-02-10 21:53:41 +00:00
Saadi Myftija 2feecece88 fix(api): skip external build creation for native builds (#3024)
Native builds don't use depot, but the `/deployments/:id/progress`
endpoint was unconditionally generating depot build tokens. This is now
fixed.

The initialize deployment endpoint was already doing this check.

<!-- devin-review-badge-begin -->

---

<a
href="https://app.devin.ai/review/triggerdotdev/trigger.dev/pull/3024"
target="_blank">
  <picture>
<source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-open-in-devin-review-dark.svg?v=1">
<img
src="https://static.devin.ai/assets/gh-open-in-devin-review-light.svg?v=1"
alt="Open with Devin">
  </picture>
</a>
<!-- devin-review-badge-end -->
2026-02-10 19:44:44 +01:00
Oskar Otwinowski 48a96efbdc chore(webapp): Expose Vercel errors (#3025) 2026-02-10 18:30:56 +01:00
DKP ebffa1039c docs: added Cursor background agent docs (#3023)
- Adds a new example project guide for running Cursor's headless CLI
agent as a Trigger.dev task with live Realtime Streams output
- New doc page at `guides/example-projects/cursor-background-agent.mdx`
- Added to sidebar nav and example projects table
<!-- devin-review-badge-begin -->

---

<a
href="https://app.devin.ai/review/triggerdotdev/trigger.dev/pull/3023"
target="_blank">
  <picture>
<source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-open-in-devin-review-dark.svg?v=1">
<img
src="https://static.devin.ai/assets/gh-open-in-devin-review-light.svg?v=1"
alt="Open with Devin">
  </picture>
</a>
<!-- devin-review-badge-end -->
2026-02-10 16:47:00 +00:00
Eric Allam bc63edd6bf chore(repo): adopt vouch with issue based workflow and require for PRs (#3022)
Adopting [https://github.com/mitchellh/vouch](vouch) so we can help
potential contributors by requiring a conversation before they can
submit a PR. Too many contributors have been skipping the conversation
part of contributing to an OSS repo and skipping right ahead to
submitting PRs
<!-- devin-review-badge-begin -->

---

<a
href="https://app.devin.ai/review/triggerdotdev/trigger.dev/pull/3022"
target="_blank">
  <picture>
<source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-open-in-devin-review-dark.svg?v=1">
<img
src="https://static.devin.ai/assets/gh-open-in-devin-review-light.svg?v=1"
alt="Open with Devin">
  </picture>
</a>
<!-- devin-review-badge-end -->
2026-02-10 12:03:24 +00:00
Mihai Popescu eaed7d0ba4 fix(webapp): UI/UX improvements for logs, query, and shortcuts (#2997)
##  Checklist

- [X] I have followed every step in the [contributing
guide](https://github.com/triggerdotdev/trigger.dev/blob/main/CONTRIBUTING.md)
- [X] The PR title follows the convention.
- [X] I ran and tested the code works

---

## Testing

Manually tested each implementation.

---

## Changelog

* Updated Logs Page with the new implementation in time filter component

* In TRQL editor users can now click on empty/blank spaces in the editor
and the cursor will appear

* Added CMD + / for line commenting in TRQL

* Activated proper undo/redo functionality in CodeMirror (TRQL editor)

* Added a check for new logs button, previously once the user got to the
end of the logs he could not check for newer logs

* Added showing MS in logs page Dates

* Removed LOG_INFO internal logs, they are available with Admin Debug
flag

* Added support for correct timezone render on server side.

* Increased CLICKHOUSE_LOGS_LIST_MAX_MEMORY_USAGE to 1GB

* Changed Previous run/ Next run to J/K, consistent with previous/next
page in Runs list
2026-02-10 12:19:26 +02:00
Oskar Otwinowski 9b21f8d322 feat(webapp): Vercel integration (#2994)
Vercel integration

Desc + Vid coming soon


For human reviewer:
- check the db schema
- check if posthog user attribution call is correct (telemetry.server.ts
& `referralSource`)
<!-- devin-review-badge-begin -->

---

<a
href="https://app.devin.ai/review/triggerdotdev/trigger.dev/pull/2994"
target="_blank">
  <picture>
<source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-open-in-devin-review-dark.svg?v=1">
<img
src="https://static.devin.ai/assets/gh-open-in-devin-review-light.svg?v=1"
alt="Open with Devin">
  </picture>
</a>
<!-- devin-review-badge-end -->
2026-02-10 10:37:09 +01:00
nicktrn e536d35b17 fix(ci): fix docker image publishing and worker builds (#3013)
## Summary
- **Fix Docker publish automation**: The `v.docker.*` tags pushed by the
release workflow using `GITHUB_TOKEN` don't trigger the publish workflow
(GitHub Actions limitation to prevent infinite loops). Added a
`workflow_call` to `publish.yml` directly from the release job so Docker
images are built automatically after npm publish. Tags are still pushed
for reference.
- **Fix worker Containerfiles**: The coordinator, docker-provider, and
kubernetes-provider builds have been failing since the superjson
vendoring change in `@trigger.dev/core` (#2949). The Containerfiles now
run `bundle-vendor` before `build:bundle` to generate the vendor files
that esbuild needs.

### Context
- Docker images on GHCR have been stuck at v4.3.0 — v4.3.1, v4.3.2,
v4.3.3 tags existed on GitHub but never triggered publish runs
- The worker builds (publish-worker) have been failing on every push to
main since Jan 30

## Test plan
- [x] Verified kubernetes-provider Containerfile builds locally with the
fix
- [x] Manually dispatched publish workflow for v4.3.1 — all jobs
succeeded
<!-- devin-review-badge-begin -->

---

<a
href="https://app.devin.ai/review/triggerdotdev/trigger.dev/pull/3013"
target="_blank">
  <picture>
<source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-open-in-devin-review-dark.svg?v=1">
<img
src="https://static.devin.ai/assets/gh-open-in-devin-review-light.svg?v=1"
alt="Open with Devin">
  </picture>
</a>
<!-- devin-review-badge-end -->
2026-02-06 13:56:39 +00:00
DKP b96a0b70d4 Docs: Clarify AI tool compatibility and expand context snippet (#3000)
This pull request overhauls the "Building with AI" documentation
section. It includes a comprehensive restructuring of the main
building-with-ai page with new setup guides and troubleshooting
sections, reorganizes the navigation hierarchy to elevate
mcp-agent-rules as a top-level page, and updates multiple documentation
pages to clarify the relationships between three AI tools: Skills, Agent
Rules, and MCP Server. Changes also include formatting improvements,
such as replacing italicized text with inline code formatting, and
consistent additions of explanatory Note blocks and CardGroup components
across related pages.
2026-02-05 15:22:44 -08:00
James Ritchie 3bb9aac014 Fix(webapp): Prevent big numbers on Queue page from jumping around when animating (#3007) 2026-02-05 07:45:40 -08:00
Saadi Myftija 283f88b203 feat(webapp): add triggered via field to deployment details page (#2850)
Display the deployment trigger source (CLI, CI/CD, Dashboard, GitHub
Integration) with appropriate icons on the deployment details page. The
triggeredVia field was already in the database but not displayed.

Co-authored-by: Claude <noreply@anthropic.com>
2026-02-05 15:24:23 +01:00
James Ritchie c55af7bead fix(webapp): ask ai button missing tooltip (#2964)
Fixes
- the tooltip not displaying on the AskAI button in the side menu
- incorrect AskAI button heights
- Small UI tweaks
<!-- devin-review-badge-begin -->

---

<a
href="https://app.devin.ai/review/triggerdotdev/trigger.dev/pull/2964">
  <picture>
<source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-open-in-devin-review-dark.svg?v=1">
<img
src="https://static.devin.ai/assets/gh-open-in-devin-review-light.svg?v=1"
alt="Open with Devin">
  </picture>
</a>
<!-- devin-review-badge-end -->

---------

Co-authored-by: Mihai Popescu <mihai.popescu.dev@gmail.com>
2026-02-05 03:17:18 +02:00
Iss db4fb9eeef docs: Add Hookdeck example (#3005)
Add documentation for integrating Hookdeck with Trigger.dev to receive
webhooks and forward them to Trigger.dev tasks.
<!-- devin-review-badge-begin -->

---

<a
href="https://app.devin.ai/review/triggerdotdev/trigger.dev/pull/3005">
  <picture>
<source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-open-in-devin-review-dark.svg?v=1">
<img
src="https://static.devin.ai/assets/gh-open-in-devin-review-light.svg?v=1"
alt="Open with Devin">
  </picture>
</a>
<!-- devin-review-badge-end -->
2026-02-04 16:50:59 -08:00
Iss c0595700f8 docs: clarify .env.local loading and idempotency key reset scope (#2996)
Document that .env.local is automatically loaded during dev, and clarify
that backend-triggered idempotency keys should be reset with global
scope
2026-02-04 09:37:35 -05:00
Iss 6a45f5623b docs: multi-tenant applications and concurrency limits (#2961)
Adds an example of multi-tenant applications as alternative to
project/limit increase, and more info about queue times and concurrency
2026-02-04 09:33:05 -05:00
Iss 104f720f6f docs(troubleshooting): add COULD_NOT_FIND_EXECUTOR error and IPv4 support (#2950)
Document COULD_NOT_FIND_EXECUTOR error with dynamic imports and IPv4
database connection limitation in troubleshooting guide.
2026-02-04 09:30:51 -05:00
Iss e017913021 docs(self-hosting): added graphile worker troubleshooting to docs (#2883)
Add troubleshooting documentation for graphile worker schema migration
failures and PostgreSQL SSL certificate issues that prevent worker
initialization.
2026-02-04 09:29:01 -05:00
Matt Aitken 7781e2aad1 docs: usage function examples were missing the imports (#2830)
Closes #2828
2026-02-04 14:05:59 +00:00
Saadi Myftija 8e0034484c feat(supervisor): project-based scheduling affinity for image cache locality (#2995)
⚒️ Publish Worker (v4) / build (supervisor) (push) Has been cancelled
Adds optional pod affinity so pods from the same project prefer
scheduling on the same node. This can help improve image cache hit
rates; subsequent pods benefit from already-pulled image layers,
reducing startup time.

Complements the built-in ImageLocality scheduler plugin by helping
during burst scheduling scenarios. Pod affinity sees scheduled pods
immediately, while ImageLocality only sees images after they're fully
pulled.

Configuration:
- `KUBERNETES_PROJECT_AFFINITY_ENABLED` - Enable/disable (default:
false)
- `KUBERNETES_PROJECT_AFFINITY_WEIGHT` - Scheduler weight 1-100
(default: 50)
- `KUBERNETES_PROJECT_AFFINITY_TOPOLOGY_KEY` - Topology key (default:
kubernetes.io/hostname)

Uses soft (preferred) affinity so pods always schedule even if preferred
node is full.

<!-- devin-review-badge-begin -->

---

<a
href="https://app.devin.ai/review/triggerdotdev/trigger.dev/pull/2995">
  <picture>
<source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-open-in-devin-review-dark.svg?v=1">
<img
src="https://static.devin.ai/assets/gh-open-in-devin-review-light.svg?v=1"
alt="Open with Devin">
  </picture>
</a>
<!-- devin-review-badge-end -->
2026-02-04 14:39:47 +01:00
Eric Allam b72cacc671 feat(debounce): add maxDelay option to limit total debounce time (#2984) 2026-02-02 20:15:06 +00:00
Mihai Popescu 1ccb8c186f changed schema id (#2983)
Fixed duplicate schema id for clickhouse
2026-02-02 02:43:39 +02:00
nicktrn 279102c17c fix(cli): reject execute() immediately when child process is dead (#2978)
## Summary
- When a child process crashes and a retry (`RETRY_IMMEDIATELY`) is
attempted on the same `TaskRunProcess`, `execute()` hangs forever
because the IPC send is silently skipped and the attempt promise can
never resolve
- This caused runner pods to stay up indefinitely with no heartbeats or
polls
- Fix: reject the attempt promise immediately when the child is not
connected, so the controller can proceed to warm start or exit

## Test plan
- [x] Added `taskRunProcess.test.ts` — verifies `execute()` rejects
promptly instead of hanging when the child process is dead
- [x] Deploy and verify no more stuck runner pods accumulate over time
2026-01-30 16:44:46 +00:00
Eric Allam b221719c09 chore(repo): fixed missing dependency in pnpm lockfile (#2976)
<!-- devin-review-badge-begin -->

---

<a
href="https://app.devin.ai/review/triggerdotdev/trigger.dev/pull/2976">
  <picture>
<source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-open-in-devin-review-dark.svg?v=1">
<img
src="https://static.devin.ai/assets/gh-open-in-devin-review-light.svg?v=1"
alt="Open with Devin">
  </picture>
</a>
<!-- devin-review-badge-end -->
2026-01-30 10:02:30 +00:00
Matt Aitken e6861f4fe4 Fix: run page logs keep refreshing when a run finishes (#2971)
Closes #2798

When a run finished the logs UI could get stuck and so be pending and
never update again. If you did a hard reload it would be correct.

This happened because when we insert a log/span we ping Redis which
causes a reload of the UI. However there was a race condition – the
insert into ClickHouse can take a while so we were refreshing the UI too
early. Then never refreshing it again.
  
Changes
- Send refresh pings every 5s to keep run page logs live
- Throttle updates so the run UI is never updated more than once per
second
- Stop auto-reloading when a run has been completed for >= 30s
- Add type inference improvements for the throttle function
<!-- devin-review-badge-begin -->

---

<a
href="https://app.devin.ai/review/triggerdotdev/trigger.dev/pull/2971">
  <picture>
<source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-open-in-devin-review-dark.svg?v=1">
<img
src="https://static.devin.ai/assets/gh-open-in-devin-review-light.svg?v=1"
alt="Open with Devin">
  </picture>
</a>
<!-- devin-review-badge-end -->
2026-01-30 09:47:56 +00:00
Matt Aitken bc7ce78103 fix(sdk): export AnyOnStartAttemptHookFunction type (#2966)
Export AnyOnStartAttemptHookFunction type to allow defining
onStartAttempt hooks for individual tasks.

https://claude.ai/code/session_018jgSVcFtKVyv65ktGNQFFq
<!-- devin-review-badge-begin -->

---

<a
href="https://app.devin.ai/review/triggerdotdev/trigger.dev/pull/2966">
  <picture>
<source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-open-in-devin-review-dark.svg?v=1">
<img
src="https://static.devin.ai/assets/gh-open-in-devin-review-light.svg?v=1"
alt="Open with Devin">
  </picture>
</a>
<!-- devin-review-badge-end -->

Co-authored-by: Claude <noreply@anthropic.com>
2026-01-30 09:17:37 +00:00
Eric Allam 9937823a7f Standardize @types/node to version 20.14.14 across monorepo (#2970)
##  Checklist

- [ ] I have followed every step in the [contributing
guide](https://github.com/triggerdotdev/trigger.dev/blob/main/CONTRIBUTING.md)
- [ ] The PR title follows the convention.
- [ ] I ran and tested the code works

---

## Description

This PR standardizes the `@types/node` dependency across the entire
monorepo to version `20.14.14`. Previously, different packages were
using different versions (ranging from 12.20.55 to 22.13.9), which could
cause type conflicts and inconsistencies.

### Changes Made

1. **tsconfig.json** - Added `"node"` to the `types` array in
`apps/webapp/tsconfig.json` to ensure Node.js types are properly
recognized
2. **package.json overrides** - Added `@types/node` version override to
`20.14.14` in the root `package.json`
3. **pnpm-lock.yaml** - Updated lock file to reflect the standardized
version across all packages and their dependencies
4. **Fixture package.json** - Updated
`packages/cli-v3/e2e/fixtures/emit-decorator-metadata/package.json` to
use the standardized version

This ensures consistent type definitions across the monorepo and
prevents version mismatches that could lead to type errors or unexpected
behavior.

---

## Testing

- Verified that all package references to `@types/node` now point to
version `20.14.14`
- Confirmed that the lock file properly reflects the override across all
transitive dependencies
- Ensured TypeScript configuration includes Node.js types for proper
type checking

---

## Changelog

- Standardized `@types/node` to version `20.14.14` across all packages
in the monorepo
- Added `"node"` to TypeScript compiler types in webapp configuration
- Updated all package dependencies to use the consistent version through
pnpm overrides

💯

https://claude.ai/code/session_018eqp2LvvErkFSN9oK5xBh1
<!-- devin-review-badge-begin -->

---

<a
href="https://app.devin.ai/review/triggerdotdev/trigger.dev/pull/2970">
  <picture>
<source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-open-in-devin-review-dark.svg?v=1">
<img
src="https://static.devin.ai/assets/gh-open-in-devin-review-light.svg?v=1"
alt="Open with Devin">
  </picture>
</a>
<!-- devin-review-badge-end -->

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Eric Allam <ericallam@users.noreply.github.com>
2026-01-30 09:15:35 +00:00
Eric Allam 3925f8cc49 fix(core): vendor superjson to fix ESM/CJS compatibility (#2949)
Bundle superjson and its dependency (copy-anything) during build to
avoid
ERR_REQUIRE_ESM errors on Node.js versions that don't support
require(ESM)
by default (< 22.12.0) and AWS Lambda which intentionally disables it.

- Add scripts/bundle-superjson.mjs to bundle superjson with esbuild
- Update build script to bundle vendor files before tshy compilation
- Move superjson from dependencies to devDependencies
- Update imports to use vendored bundles

Fixes #2937
<!-- devin-review-badge-begin -->

---

<a
href="https://app.devin.ai/review/triggerdotdev/trigger.dev/pull/2949">
  <picture>
<source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-open-in-devin-review-dark.svg?v=1">
<img
src="https://static.devin.ai/assets/gh-open-in-devin-review-light.svg?v=1"
alt="Open with Devin">
  </picture>
</a>
<!-- devin-review-badge-end -->

---------

Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Eric Allam <ericallam@users.noreply.github.com>
2026-01-30 09:15:02 +00:00
Iss 01208fde27 chore(docs): Add playwright workaround (#2973)
Adds a workaround to playwright to browser download failures based on
this GH issue: https://github.com/triggerdotdev/trigger.dev/issues/2440
<!-- devin-review-badge-begin -->

---

<a
href="https://app.devin.ai/review/triggerdotdev/trigger.dev/pull/2973">
  <picture>
<source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-open-in-devin-review-dark.svg?v=1">
<img
src="https://static.devin.ai/assets/gh-open-in-devin-review-light.svg?v=1"
alt="Open with Devin">
  </picture>
</a>
<!-- devin-review-badge-end -->
2026-01-30 08:43:23 +00:00
Eric Allam 5e049cde3a fix(run-engine): avoid NAPI string overflow in getExecutionSnapshotsSince by only fetching waitpoints for latest snapshot (#2972) 2026-01-29 19:03:54 +00:00
Matt Aitken 72c357125b Query enabled via feature flag (#2968)
- Renamed FeatureFlag functions to be singular where it makes sense.
- Added function to handle multiple feature flags
- canAccessQuery now checks the global feature flag and environment
variable as well
<!-- devin-review-badge-begin -->

---

<a
href="https://app.devin.ai/review/triggerdotdev/trigger.dev/pull/2968">
  <picture>
<source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-open-in-devin-review-dark.svg?v=1">
<img
src="https://static.devin.ai/assets/gh-open-in-devin-review-light.svg?v=1"
alt="Open with Devin">
  </picture>
</a>
<!-- devin-review-badge-end -->
2026-01-29 15:30:56 +00:00
DKP 0674d74bbb Added Trigger.dark theme to the docs (#2967)
<!-- devin-review-badge-begin -->

---

<a
href="https://app.devin.ai/review/triggerdotdev/trigger.dev/pull/2967">
  <picture>
<source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-open-in-devin-review-dark.svg?v=1">
<img
src="https://static.devin.ai/assets/gh-open-in-devin-review-light.svg?v=1"
alt="Open with Devin">
  </picture>
</a>
<!-- devin-review-badge-end -->
2026-01-29 13:59:27 +00:00
DKP 9e08712749 Add building with ai/skills pages and updated intro (#2962)
Closes #<issue>

##  Checklist

- [ ] I have followed every step in the [contributing
guide](https://github.com/triggerdotdev/trigger.dev/blob/main/CONTRIBUTING.md)
- [ ] The PR title follows the convention.
- [ ] I ran and tested the code works

---

## Testing

_[Describe the steps you took to test this change]_

---

## Changelog

_[Short description of what has changed]_

---

## Screenshots

_[Screenshots]_

💯

<!-- devin-review-badge-begin -->

---

<a
href="https://app.devin.ai/review/triggerdotdev/trigger.dev/pull/2962">
  <picture>
<source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-open-in-devin-review-dark.svg?v=1">
<img
src="https://static.devin.ai/assets/gh-open-in-devin-review-light.svg?v=1"
alt="Open with Devin">
  </picture>
</a>
<!-- devin-review-badge-end -->
2026-01-29 13:55:43 +00:00
Matt Aitken f53db6fd16 Query: time limits, performance improvements, styling (#2953)
Summary
- Query: add time limits, performance improvements, and styling updates

Changes
- Add ClickHouse output_text and error_text columns with indexes
- Automatically use _text columns for JSON based on query pattern;
support JSON column data prefixes
- Add idempotency key and scope columns
- Add enforcedWhereClause for tenant and time restrictions, instead of
the old tenant stuff.
- Implement basic time filter limiting and set default time period based
on plan; show message when results are clipped
- UX: resizable code area (including vertical splits), collapsible
sidebar, fix table/chart vertical sizing, max height for chart legend in
fullscreen
- Styling and UI tweaks: improved chart legend styling, more chart
colours, thinner line chart stroke, pricing callout color, improved
layout for callouts
- Features: generate and save AI titles
<!-- devin-review-badge-begin -->

---

<a
href="https://app.devin.ai/review/triggerdotdev/trigger.dev/pull/2953">
  <picture>
<source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-open-in-devin-review-dark.svg?v=1">
<img
src="https://static.devin.ai/assets/gh-open-in-devin-review-light.svg?v=1"
alt="Open with Devin">
  </picture>
</a>
<!-- devin-review-badge-end -->
2026-01-29 13:02:47 +00:00
Oskar Otwinowski c0b86efbd3 feat(webapp): Add MiddleTruncate component for long task names (#2946)
Closes #

##  Checklist

- [ ] I have followed every step in the [contributing
guide](https://github.com/triggerdotdev/trigger.dev/blob/main/CONTRIBUTING.md)
- [ ] The PR title follows the convention.
- [ ] I ran and tested the code works

---

## Testing

Tested the MiddleTruncate component in the TasksDropdown by:
1. Verifying that long task names (e.g.,
"namespace:category:subcategory:task-name") are truncated in the middle
2. Confirming the full text appears in a tooltip on hover
3. Testing responsive behavior - truncation adjusts when the container
is resized
4. Verifying that short task names that fit within the container are
displayed in full without truncation

---

## Changelog

Added a new `MiddleTruncate` primitive component that intelligently
truncates text in the middle while preserving the beginning and end
portions. This is particularly useful for long hierarchical identifiers
like task slugs.

**Key features:**
- Truncates text in the middle with an ellipsis (…) when it exceeds
available width
- Shows full text in a tooltip on hover when truncated
- Responsive - recalculates truncation on container resize using
ResizeObserver
- Maintains minimum character visibility (4 chars minimum on each side
for readability)
- Integrated into TasksDropdown to handle long task names

**Changes:**
- Created new `MiddleTruncate.tsx` component with binary search
algorithm for optimal character distribution
- Updated TasksDropdown to use MiddleTruncate for task slug display
- Increased TasksDropdown popover width from 240px to 360px to provide
better space for truncated text

---

## Screenshots
💯


https://github.com/user-attachments/assets/a7a2191a-2e36-437e-ab3f-517fe7620b93


<!-- devin-review-badge-begin -->

---

<a
href="https://app.devin.ai/review/triggerdotdev/trigger.dev/pull/2946">
  <picture>
<source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-open-in-devin-review-dark.svg?v=1">
<img
src="https://static.devin.ai/assets/gh-open-in-devin-review-light.svg?v=1"
alt="Open with Devin">
  </picture>
</a>
<!-- devin-review-badge-end -->

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-01-29 12:51:58 +01:00
Mihai Popescu e29e1c86d9 Fix/tri 7032 logs page feedback (#2947)
##  Changes
### UI & UX
- Normalized log level display across table and detail view
- Fixed table header scroll behavior and sidebar positioning
- Improved loading state with taller segment and disabled resizing
- Added "no more logs" message with count
- Enhanced keyboard shortcuts
### Filtering & Search
- Streamlined filters: RunId and Task only (removed run filters)
- Side panel closes when filters change
- Fixed logs from previous search remaining in table
- Fixed table scroll position when changing filters
### Backend
- Added performance indexes on message and attributes
(`014_add_task_runs_v2_search_indexes.sql`)
- Added DEBUG level logging by default
- Removed internal logs from display
- Fixed ServiceValidationError forwarding to frontend
  - Removed v1 logs API support
2026-01-29 12:47:59 +02:00
James Ritchie 34203d6a6f Fix(webapp) prevent incidents url being called every frame (#2956) 2026-01-27 18:33:36 +00:00
James Ritchie d4e4fbd7fc fix(webapp): Truncate long branch names to prevent breaking the onboarding layout (#2954)
### Before
<img width="1736" height="964" alt="CleanShot 2026-01-21 at 20 11 08@2x"
src="https://github.com/user-attachments/assets/76c6ed42-d2a0-4212-aaad-6fb848025c85"
/>

### After
<img width="1776" height="1032" alt="CleanShot 2026-01-27 at 17 32
47@2x"
src="https://github.com/user-attachments/assets/030ba26e-6aad-4232-a9ff-f2fde9931f8c"
/>

<!-- devin-review-badge-begin -->

---

<a
href="https://app.devin.ai/review/triggerdotdev/trigger.dev/pull/2954">
  <picture>
<source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-open-in-devin-review-dark.svg?v=1">
<img
src="https://static.devin.ai/assets/gh-open-in-devin-review-light.svg?v=1"
alt="Open with Devin">
  </picture>
</a>
<!-- devin-review-badge-end -->
2026-01-27 18:26:43 +00:00
James Ritchie 49de105862 Feat(webapp): collapsible side menu (#2939)
- Better separation between main feature pages and manage pages
- Toggle via keyboard shortcut Cmd/Ctrl + B or click collapse button
- Collapsed state persisted to database (survives page refresh)
- All menu items show tooltips when collapsed
- Smooth animations using Framer Motion and CSS transitions
- Impersonation-safe (preferences not modified when impersonating)


https://github.com/user-attachments/assets/35827922-b80a-418e-8341-eb22c9bb5ed4


<!-- devin-review-badge-begin -->

---

<a
href="https://app.devin.ai/review/triggerdotdev/trigger.dev/pull/2939">
  <picture>
<source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-open-in-devin-review-dark.svg?v=1">
<img
src="https://static.devin.ai/assets/gh-open-in-devin-review-light.svg?v=1"
alt="Open with Devin">
  </picture>
</a>
<!-- devin-review-badge-end -->
2026-01-27 17:13:21 +00:00
DKP 5fb9cc36bc fix(security): upgrade CLI deps and add overrides (#2952)
- Upgrade @modelcontextprotocol/sdk 1.24.0 → 1.25.2 (CVE-2026-0621
ReDoS)
- Upgrade tar 7.4.3 → 7.5.4+ (CVE-2026-23950 race condition)
- Add pnpm overrides for transitive deps:
  - qs <6.14.0 → 6.14.0 (CVE-2025-15284 DoS)
  - systeminformation <5.27.14 → 5.27.14 (CVE-2025-68154 cmd injection)
  - lodash <4.17.23 → 4.17.23 (CVE-2025-13465 prototype pollution)

---------

Co-authored-by: nicktrn <55853254+nicktrn@users.noreply.github.com>
2026-01-27 16:33:04 +00:00
nicktrn 70c8d6d14b fix(react-hooks): prevent onComplete from firing prematurely when stream disconnects (#2929)
## Summary

Fixes #2856 - The `onComplete` callback in `useRealtimeRun` was firing prematurely

## Root Cause

The callback was triggered when the long-poll stream ended, regardless
of whether the run had actually completed. Reverse proxies often close
idle connections, causing the stream to end prematurely. In this case it
was caused by fetch abort due to React strict mode.

## Fix

Changed the condition from checking if `run` exists to checking if
`run?.finishedAt` exists, ensuring `onComplete` only fires when the run
has reached a terminal state.

---------

Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: nicktrn <nicktrn@users.noreply.github.com>
2026-01-27 16:21:56 +00:00
Eric Allam eeab6bdeac fix(run-engine): fix queue cache memory leak and replace MemoryStore with LRU cache (#2945)
- Fix memory leak in RunAttemptSystem queue cache - was keying by runId
instead of queue identifier
- Replace `@unkey/cache` MemoryStore with new LRUMemoryStore for O(1)
operations and better memory bounds

## Problem

### Cache Key Bug
The queue cache in `#resolveTaskRunExecutionQueue` was keyed by `runId`,
creating one cache entry per run instead of per queue. With 1-2 hour
TTLs and 5000 entry soft cap, these accumulated causing memory growth.

### MemoryStore Performance
The `@unkey/cache` MemoryStore uses O(n) synchronous iteration for
eviction, blocking the event loop at high throughput.

## Solution

### Cache Key Fix
Changed cache key from `params.runId` to queue identifier:

```typescript
const cacheKey = params.lockedQueueId ?? `${params.runtimeEnvironmentId}:${params.queueName}`;
```

LRU Cache

Created LRUMemoryStore adapter using lru-cache package:
- O(1) get/set/delete operations
- Strict memory bounds (hard max vs soft cap)
- No event loop blocking

Test Results:

| Metric | Before Fix | After Fix |
|---|---|---|
| Queue cache entries (per 1000 runs) | ~1000 | 1 |
| Old space growth | 32.27 MB | 5.45 MB |
| Heap growth | 7.21 MB (3.9%) | 4.81 MB (2.6%) |


<!-- devin-review-badge-begin -->

---

<a
href="https://app.devin.ai/review/triggerdotdev/trigger.dev/pull/2945">
  <picture>
<source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-open-in-devin-review-dark.svg?v=1">
<img
src="https://static.devin.ai/assets/gh-open-in-devin-review-light.svg?v=1"
alt="Open with Devin">
  </picture>
</a>
<!-- devin-review-badge-end -->
2026-01-26 22:23:23 +00:00
Matt Aitken 825219a2f4 Fix: requeue a run if the DB is unavailable during dequeuing (#2938)
In the DequeueSystem if the database is unavailable we were dequeuing
from Redis and then failing to requeue in the error catcher – this was
because the requeuing required DB access.

Now if in the `catch` we encounter a DB error we requeue directly using
Redis, putting it back in the queue.
<!-- devin-review-badge-begin -->

---

<a
href="https://app.devin.ai/review/triggerdotdev/trigger.dev/pull/2938">
  <picture>
<source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-open-in-devin-review-dark.svg?v=1">
<img
src="https://static.devin.ai/assets/gh-open-in-devin-review-light.svg?v=1"
alt="Open with Devin">
  </picture>
</a>
<!-- devin-review-badge-end -->
2026-01-25 19:23:04 +00:00
bharath kumar b143027d95 Fix/sdk stream root fallback (#2874) 2026-01-25 06:53:26 +00:00
Eric Allam 409388365e fix(fair-queue): ensure concurrency is released when a message reaches visibility timeout to prevent concurrency leaks (#2907)
## Summary

Fixes a concurrency leak in the batch queue where visibility timeout
reclaims do not release concurrency slots.

**The bug:** When a message visibility timeout expires (60s),
`reclaimTimedOut` puts the message back in the queue but does NOT
release the concurrency slot. The messageId stays in the concurrency set
(`engine:batch:concurrency:tenant:{envId}`), counting against the tenant
limit even though the message is no longer in-flight.

This causes:
1. Tenant appears at capacity when checking `SCARD >= limit`
2. New messages get released back to queue instead of being processed
3. Messages stuck in infinite loop, master queue grows indefinitely

**The fix:**
- Modified `reclaimTimedOut` to capture message data (including
tenantId) BEFORE releasing from in-flight
- Returns `ReclaimedMessageInfo[]` with messageId, queueId, tenantId,
and metadata
- `#reclaimTimedOutMessages` now iterates over reclaimed messages and
calls `concurrencyManager.release()` for each

## Test plan

- [x] Added test: `should return reclaimed message info with tenantId
for concurrency release`
- [x] Added test: `should return empty array when no messages have timed
out`
- [x] Added test: `should reclaim multiple timed-out messages and return
all their info`
- [x] Updated `raceConditions.test.ts` for new return type
- [x] All tests passing
- [ ] Monitor production after deploy for concurrency leak recurrence

refs TRI-7049
<!-- devin-review-badge-begin -->

---

<a
href="https://app.devin.ai/review/triggerdotdev/trigger.dev/pull/2907">
  <picture>
<source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-open-in-devin-review-dark.svg?v=1">
<img
src="https://static.devin.ai/assets/gh-open-in-devin-review-light.svg?v=1"
alt="Open with Devin">
  </picture>
</a>
<!-- devin-review-badge-end -->
2026-01-24 13:06:35 +00:00
Gautam Singh fe5178f3e8 feat(webapp): add priority to task test and replay options (#2936)
Closes #2934

##  Checklist

- [x] I have followed every step in the [contributing
guide](https://github.com/triggerdotdev/trigger.dev/blob/main/CONTRIBUTING.md)
- [x] The PR title follows the convention.
- [x] I ran and tested the code works

---

## Testing

_[Describe the steps you took to test this change]_

1. make sure there are no worker connected or the queue is paused.
2. run hello world tasks and assign priority
3. notice the priority value in task context.
4. try running a replay for task with priority
5. notice the context priority
---

## Changelog

_[Short description of what has changed]_
Added option to set priority on task test and replay dialog
---

## Screenshots

_[Screenshots]_
<img width="953" height="804" alt="image"
src="https://github.com/user-attachments/assets/d452b872-6c05-4a9e-a1e3-f6a283bb363d"
/>
<img width="958" height="886" alt="image"
src="https://github.com/user-attachments/assets/8f9f3096-da99-4d81-a9c5-2f3004b89ceb"
/>
<img width="1022" height="736" alt="image"
src="https://github.com/user-attachments/assets/233e01ac-a42b-4b4f-ac3e-e9b18638b971"
/>


💯

<!-- devin-review-badge-begin -->

---

<a
href="https://app.devin.ai/review/triggerdotdev/trigger.dev/pull/2936">
  <picture>
<source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-open-in-devin-review-dark.svg?v=1">
<img
src="https://static.devin.ai/assets/gh-open-in-devin-review-light.svg?v=1"
alt="Open with Devin">
  </picture>
</a>
<!-- devin-review-badge-end -->
2026-01-24 09:10:14 +00:00
Iss a3f1eb2361 chore(docs): Admin requirements for creating worker groups (#2925)
Adds documentation for creating additional worker groups via the admin
API endpoint, including how to make users admin (new vs existing users),
and clarifies that ADMIN_EMAILS only applies on signup.
2026-01-24 09:00:50 +00:00
Iss ab4b50b95a chore(docs): Add clarifications for delay semantics and payload handling (#2928)
Adds documentation notes clarifying that:
- delayed runs execute on the current deployment version
- guidance for Date objects in payloads
- static IP availability
- version locking behavior for delayed runs
2026-01-24 08:59:55 +00:00
Eric Allam 1859fd0283 chore(docs): Improvements to the idempotency key docs for the user expored itempotency key changes (#2910) 2026-01-23 18:07:12 +00:00
github-actions[bot] 700fe91bec chore: release v4.3.3 (#2893)
🚀 Publish Trigger.dev Docker / typecheck (push) Failing after 0s
🚀 Publish Trigger.dev Docker / publish-webapp (push) Has been skipped
🚀 Publish Trigger.dev Docker / publish-worker (push) Has been skipped
🚀 Publish Trigger.dev Docker / publish-worker-v4 (push) Has been skipped
🚀 Publish Trigger.dev Docker / units (push) Failing after 0s
This PR was opened by the [Changesets
release](https://github.com/changesets/action) GitHub action. When
you're ready to do a release, you can merge this and publish to npm
yourself or [setup this action to publish
automatically](https://github.com/changesets/action#with-publishing). If
you're not ready to do a release yet, that's fine, whenever you add more
changesets to main, this PR will be updated.


# Releases
## @trigger.dev/build@4.3.3

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.3.3`

## trigger.dev@4.3.3

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.3.3`
    -   `@trigger.dev/build@4.3.3`
    -   `@trigger.dev/schema-to-json@4.3.3`

## @trigger.dev/core@4.3.3

### Patch Changes

- Add support for AI SDK v6 (Vercel AI SDK)
([#2919](https://github.com/triggerdotdev/trigger.dev/pull/2919))

    -   Updated peer dependency to allow `ai@^6.0.0` alongside v4 and v5
- Updated internal code to handle async validation from AI SDK v6's
Schema type

- Expose user-provided idempotency key and scope in task context.
`ctx.run.idempotencyKey` now returns the original key passed to
`idempotencyKeys.create()` instead of the hash, and
`ctx.run.idempotencyKeyScope` shows the scope ("run", "attempt", or
"global").
([#2903](https://github.com/triggerdotdev/trigger.dev/pull/2903))

- Fix batch trigger failing with "ReadableStream is locked" error when
network failures occur mid-stream. Added safe stream cancellation that
gracefully handles locked streams during retry attempts.
([#2917](https://github.com/triggerdotdev/trigger.dev/pull/2917))

- Add a maxDepth to flatten/unflattenAttributes to prevent possible
issues ([#2890](https://github.com/triggerdotdev/trigger.dev/pull/2890))

## @trigger.dev/python@4.3.3

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/sdk@4.3.3`
    -   `@trigger.dev/core@4.3.3`
    -   `@trigger.dev/build@4.3.3`

## @trigger.dev/react-hooks@4.3.3

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.3.3`

## @trigger.dev/redis-worker@4.3.3

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.3.3`

## @trigger.dev/rsc@4.3.3

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.3.3`

## @trigger.dev/schema-to-json@4.3.3

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.3.3`

## @trigger.dev/sdk@4.3.3

### Patch Changes

- Add support for AI SDK v6 (Vercel AI SDK)
([#2919](https://github.com/triggerdotdev/trigger.dev/pull/2919))

    -   Updated peer dependency to allow `ai@^6.0.0` alongside v4 and v5
- Updated internal code to handle async validation from AI SDK v6's
Schema type

- Expose user-provided idempotency key and scope in task context.
`ctx.run.idempotencyKey` now returns the original key passed to
`idempotencyKeys.create()` instead of the hash, and
`ctx.run.idempotencyKeyScope` shows the scope ("run", "attempt", or
"global").
([#2903](https://github.com/triggerdotdev/trigger.dev/pull/2903))

-   Updated dependencies:
    -   `@trigger.dev/core@4.3.3`

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2026-01-23 17:36:11 +00:00
Matt Aitken 6055c7d050 Recover runs that failed to dequeue (#2931)
There’s an edge case that means runs can end up in the
currentConcurrency set when they’re not in the correct state for
execution. This means they will be permanently stuck in queued.

Given an environmentId this will fix those runs.

This is a temporary fix while we permanently fix the issue.

---------

Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Eric Allam <ericallam@users.noreply.github.com>
2026-01-23 16:25:58 +00:00
Matt Aitken cf1c311dea Use the replica for API auth queries (#2932)
When we auth API keys we get the environment, project and org. This is a
very hot path so even though these queries are fast they contribute a
significant percentage of total load.

This moves them to use the read replica instead.
2026-01-23 16:23:51 +00:00
Eric Allam fb94e1741f fix(dashboard): display correct batch rate limit current token value (#2927) 2026-01-23 16:13:29 +00:00
Eric Allam 022f69c8c8 fix(v3): redesign of batch completion to prevent heavy row-level contention on the BatchTaskRun (#2930) 2026-01-23 15:52:52 +00:00
Eric Allam d893b26ed2 fix(engine): store costInCents and usageDurationMs on the TaskRun table via existing run engine updates (#2926)
Moving usage updates into the run engine to prevent inefficient &
additional incremental updates to the TaskRun table. Read/Modify/Write
pattern is safe inside of the run engine because of the run lock. We can
also now cap the usageDurationMs value from overflowing and causing an
error.

## Why?

This is preventing at least one update per TaskRun and instead updating
these values piggybacking on other updates.

## Aurora PostgreSQL Reader Consistency Notes

### TL;DR
Aurora readers share the same storage as the writer, but maintain
separate in-memory page caches. This means:
- **Storage is always consistent** - writes are synchronously committed
to shared storage
- **Page cache can lag** - typically <100ms, but can cause stale reads
if data is cached

### How It Works
1. Writer commits to shared storage (synchronous 4/6 quorum)
2. Writer sends cache invalidation messages to readers (asynchronous)
3. If reader has data in cache → returns cached (potentially stale)
value
4. If reader has cache miss → fetches from shared storage (always
current)

### Monitoring
```sql
SELECT server_id,
       CASE WHEN session_id = 'MASTER_SESSION_ID' THEN 'Writer' ELSE 'Reader' END AS role,
       replica_lag_in_msec
FROM aurora_replica_status();
```
2026-01-23 12:34:04 +00:00
Eric Allam 6f26acb581 fix(run-engine): use reader for pending version queries (#2924)
Move expensive findMany queries for PENDING_VERSION and
WAITING_FOR_DEPLOY
runs to read replicas to avoid blocking migrations on the primary
database.

Changes:
- Add readOnlyPrisma to SystemResources type
- Pass readOnlyPrisma to systems in RunEngine constructor  
- Update pendingVersionSystem to use readOnlyPrisma for findMany
- Update executeTasksWaitingForDeploy to use _replica for findMany
2026-01-21 15:37:21 +00:00
Eric Allam bd449f75dc fix(migrations): Add IF NOT EXISTS to 20260116154810_add_idempotency_key_options_to_task_run (#2923)
## Summary
- Adds `IF NOT EXISTS` to the migration that adds
`idempotencyKeyOptions` column to prevent errors if the column already
exists

## Migration Checksum Fix

If you've already applied the previous version of this migration, you'll
need to update the checksum in your `_prisma_migrations` table to match
the new migration file.

**Previous checksum:**
`f8876e274e3f7735312275eb24a9c4b40f512ac12a286b2de3add47f66df5b27`
**New checksum:**
`0620a914ddbaf01279576274432e51c41f41502cd4c8de38621625380750e397`

### Fix instructions

Run this SQL command against your database:

```sql
UPDATE "_prisma_migrations"
SET checksum = '0620a914ddbaf01279576274432e51c41f41502cd4c8de38621625380750e397'
WHERE migration_name = '20260116154810_add_idempotency_key_options_to_task_run';
```

This updates the stored checksum to match the modified migration file,
allowing future migrations to proceed without checksum mismatch errors.

## Test plan
- [x] Verified migration applies cleanly on fresh database
- [ ] Verified checksum update works on database with previous migration
applied

🤖 Generated with [Claude Code](https://claude.com/claude-code)
2026-01-21 14:43:37 +00:00
Saadi Myftija cd2f536620 feat(docker): enable skipping db migrations on container startup (#2922)
Adds support for skipping Postgres migrations on container startup via
the new `SKIP_POSTGRES_MIGRATIONS` environment variable.

Set `SKIP_POSTGRES_MIGRATIONS=1` to skip migrations, matching the
existing behavior of `SKIP_CLICKHOUSE_MIGRATIONS`.
2026-01-21 13:34:00 +00:00
Matt Aitken 3056a51b82 Query improvements (#2905)
What changed
- Upgraded recharts to 2.15.2
- Added multiple chart types and components: big number, line, stacked,
bar (including zoomable & reference line), big dataset bar, and usage
graph
- Implemented custom legend with animated values, tooltip showing x-axis
data, and hover/highlight behaviors for stacks and legend
- Added loading, no-data, and invalid chart states plus loading spinners
and improved loading animations/layout
- Storybook integration: initial charts setup, separate chart files,
alphabetized menu, chart state toggles, and story updates
- Interaction & UX improvements: zooming (drag/select), crosshair
pointer, show/select dates while zooming, prevent text selection on
drag, hide mouse wheel zoom, capped legend items, axis/legend styling
tweaks, better spacing, and min-height for charts
- Data & state handling: moved date data to route for unified zooming,
moved chartState to main Chart component, moved hard-coded/mock data out
of components, and set chart data when zooming to start/end dates
- Performance & animation: turned off/reduced chart animations, sped up
animated numbers, removed hover transitions for bars
- New UI primitives and layout: Card component, small card updates, SVG
icons, improved segmented control and popover variants, table
improvements (resizable columns, filtering, sorting, scrolling fixes)
- Various fixes and polish: tooltip style fixes, legend value updates,
hover/leave state resets, bar width fixes for small datasets,
type/import fixes, and numerous small style/typo tweaks

---------

Co-authored-by: James Ritchie <james@trigger.dev>
2026-01-21 13:07:07 +00:00
Eric Allam 23ec5ff8fa feat(sdk): add support for AI SDK v6 (#2919)
## Summary
- Add support for Vercel AI SDK v6 as a peer dependency
- Update internal code to handle async validation from AI SDK v6's
Schema type

Closes #2918

Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Eric Allam <ericallam@users.noreply.github.com>
2026-01-20 22:31:41 +00:00
Eric Allam 31e4753122 fix(sdk): handle locked ReadableStream when retrying batch trigger (#2917)
When fetch crashes mid-stream during batch item upload (e.g., connection
reset, timeout), the request stream may remain locked by fetch's
internal reader. Attempting to cancel a locked stream throws 'Invalid
state: ReadableStream is locked', causing the batch operation to fail.

Added safeStreamCancel() helper that gracefully handles locked streams
by catching and ignoring the locked error. The stream will be cleaned up
by garbage collection when fetch eventually releases the reader.

Fixes customer issue where batchTrigger failed with ReadableStream
locked error during network instability.
2026-01-20 15:55:23 +00:00
Eric Allam 8bc6b99285 fix(batch-queue): allow batch queue consumers to run independently from the run engine worker (#2916)
new environment variable `BATCH_QUEUE_WORKER_ENABLED` now can be used
independently from `RUN_ENGINE_WORKER_ENABLED`
2026-01-20 15:26:55 +00:00
Eric Allam 87167524cc fix(api): prevent null idempotency keys in responses, only undefined accepted (#2912) 2026-01-20 14:06:37 +00:00
Eric Allam 36168b3eb6 feat(sdk): expose user-provided idempotency key and scope in task context (#2903)
## Summary
- Store the original user-provided idempotency key and scope alongside
the hash
- Expose `ctx.run.idempotencyKey` as the user-provided key (not the
hash)
- Add `ctx.run.idempotencyKeyScope` to show the scope ("run", "attempt",
or "global")

<img width="539" height="450" alt="CleanShot 2026-01-19 at 11 40 46"
src="https://github.com/user-attachments/assets/b6f42991-697e-4314-a164-aef77b8fd25c"
/>

  ## Problem
Idempotency keys were hashed (SHA-256) before storage, making debugging
difficult since users couldn't see the value they originally set or
search for runs by idempotency key.

  ## Solution
Attach metadata to the `String` object returned by
`idempotencyKeys.create()` using a Symbol, extract it in the SDK before
the API call, and store it in the database alongside the hash.

  ```typescript
const key = await idempotencyKeys.create("my-key", { scope: "global" });
  await childTask.triggerAndWait(payload, { idempotencyKey: key });

  // In child task:
  ctx.run.idempotencyKey      // "my-key" (previously showed the hash)
  ctx.run.idempotencyKeyScope // "global"
```

  Test plan

  - Trigger task with idempotencyKeys.create() using different scopes (run, attempt, global)
  - Verify ctx.run.idempotencyKey returns user-provided key
  - Verify ctx.run.idempotencyKeyScope returns correct scope
  - Verify PostgreSQL stores idempotencyKeyOptions JSON
  - Verify ClickHouse receives idempotency_key_user and idempotency_key_scope via replication

---------

Co-authored-by: James Ritchie <james@trigger.dev>
2026-01-20 11:23:07 +00:00
mintlify[bot] c859be9c53 Document new Limits page in dashboard (#2908)
⚒️ Publish Worker (v4) / build (supervisor) (push) Has been cancelled
Added documentation for the new Limits page feature that allows users to
view their current limits, quotas, and rate limit usage in real-time
from the dashboard. The page displays rate limit token availability,
quota usage, and plan features for organizations.

## Files changed
- `docs/limits.mdx` - Added introductory paragraph about the new Limits
page in the dashboard

Generated from [feat(webapp): New limits
page](https://github.com/triggerdotdev/trigger.dev/pull/2885) @samejr

Co-authored-by: mintlify[bot] <109931778+mintlify[bot]@users.noreply.github.com>
Co-authored-by: James Ritchie <james@trigger.dev>
2026-01-19 11:30:02 +00:00
Saadi Myftija b7f7d88623 feat(supervisor): add per-machine-preset resource request ratios (#2906)
Adds support to configure CPU/memory request ratios per machine preset.
Falls back to the global request ratio configs if no specific override
is specified.

Runs across different machine presets have different usage patters, so
this enables use to manage the available capacity better.
2026-01-19 12:26:05 +01:00
Eric Allam 72594a46ee fix(dashboard): properly cleanup trace pubsub redis clients to redis/memory/elu leaks in the dashboard (#2901) 2026-01-16 15:20:30 +00:00
Matt Aitken 5504e7f8cc Fix schedule limit counting (#2899)
Fix schedule counting
- Inactive TaskSchedule not counted
- Inactive environments (archived preview branches) not counted

Clarified per-project limits descriptions.
2026-01-16 13:56:49 +00:00
James Ritchie 7a7c4b1a82 feat(webapp): New limits page (#2885)
<img width="1381" height="1362" alt="CleanShot 2026-01-14 at 13 41 02"
src="https://github.com/user-attachments/assets/0537dccf-60c7-4ab7-a0e4-3164eac1e97d"
/>

---------

Co-authored-by: Matt Aitken <matt@mattaitken.com>
2026-01-15 18:03:06 +00:00
Dan 733894bb4f Impersonation log (#2896)
Closes #<issue>

##  Checklist

- [ ] I have followed every step in the [contributing
guide](https://github.com/triggerdotdev/trigger.dev/blob/main/CONTRIBUTING.md)
- [ ] The PR title follows the convention.
- [ ] I ran and tested the code works

---

## Testing

_[Describe the steps you took to test this change]_

---

## Changelog

_[Short description of what has changed]_

---

## Screenshots

_[Screenshots]_

💯

---------

Co-authored-by: Matt Aitken <matt@mattaitken.com>
2026-01-15 16:54:45 +00:00
Matt Aitken b696bbb1df Add TaskScheduleInstance projectId (#2897)
Summary
- Add nullable projectId field to TaskScheduleInstance.
- Create an index for TaskScheduleInstance.environmentId (added only if
it doesn’t exist, handled concurrently).
- Ensure TaskScheduleInstance.projectId is set everywhere in the
codebase.

Backfilling projectIds, once this is live

```sql
UPDATE "TaskScheduleInstance" tsi
SET "projectId" = ts."projectId"
FROM "TaskSchedule" ts
WHERE tsi."taskScheduleId" = ts."id";
```
2026-01-15 15:06:52 +00:00
Eric Allam aa69b9027d fix(repo): undo node.js supervisor upgrades and use the multiplatform node.js digest in Dockerfile (#2895) 2026-01-15 11:49:37 +00:00
Mihai Popescu 0b0df071bf logs-page-fixes (#2889)
* Removed EVENT_REPOSITORY_CLICKHOUSE_ROLLOUT_PERCENT
* Added hasLogsPageAccess featureFlag for logs page
* Replaced attributes with attributes_text for logs to reduce memory
usage and improve query performance
* Added support for event_v1 for logs, now depending on the settings the
logs are fetched either from `task_events_v1` or `task_events_v2`
* Show an error in the interface in cast the repository store is
`postgres`
2026-01-15 11:07:01 +00:00
Eric Allam 936bddf198 fix: upgrade Node.js to 20.20.0 to address async_hooks DoS vulnerability (#2890)
## Summary

- Upgrades Node.js from 20.19.0 to 20.20.0 (and 22.12.0 to 22.22.0 for
supervisor) to address the async_hooks stack overflow DoS vulnerability
- Adds `maxDepth` parameter (default 128) to `flattenAttributes` and
`unflattenAttributes` to prevent stack overflow on maliciously deep
nested structures

## Details

The vulnerability (patched in Node.js 20.20.0, 22.22.0, 24.13.0, 25.3.0)
causes unrecoverable crashes (exit code 7) when stack overflow occurs
during async_hooks callbacks. Since the webapp uses `AsyncLocalStorage`,
it was theoretically vulnerable.

### Changes

**Node.js version updates:**
- `docker/Dockerfile`: 20.11.1 → 20.20.0
- `apps/supervisor/Containerfile`: 22-alpine → 22.22.0-alpine
- `.nvmrc`: 20.19.0 → 20.20.0
- `apps/supervisor/.nvmrc`: 22.12.0 → 22.22.0
- `references/prisma-7/.nvmrc`: 20.19.0 → 20.20.0
- All GitHub workflows: 20.19.0 → 20.20.0

**Defense in depth:**
- Added `maxDepth` parameter to `flattenAttributes()` and
`unflattenAttributes()` in `packages/core` to prevent stack overflow on
deeply nested user input

## Test plan

- [x] All existing `flattenAttributes` tests pass (50 tests)
- [x] New tests for depth limiting added
- [x] Verify Docker builds work with new base images
2026-01-15 10:47:44 +00:00
Eric Allam b1e21cf03b feat(api): add admin endpoint for updating org feature flags (#2891) 2026-01-15 10:41:34 +00:00
Eric Allam c3f2d07708 chore(repo): remove claude code review (keep claude bot) (#2887) 2026-01-15 09:32:30 +00:00
SHIVA REDDY VANJA 4dc504d9d0 docs: fix waitpoint token completion request body field (#2888)
Fixed documentation examples to use correct 'data' field instead of
'output' for the waitpoint token completion endpoint.

The API schema expects 'data' in the request body, but all code examples
(curl, Python, Ruby, Go) incorrectly showed 'output', causing waitpoints
to complete with empty/undefined output when users followed the docs.

Fixes #2872

Closes #<issue>

##  Checklist

- [x] I have followed every step in the [contributing
guide](https://github.com/triggerdotdev/trigger.dev/blob/main/CONTRIBUTING.md)
- [x] The PR title follows the convention.
- [x] I ran and tested the code works

---

## Testing

_[Describe the steps you took to test this change]_
This is simple doc fix
---

## Changelog

_[Short description of what has changed]_
Documentation example for the specific api route had a wrong field,
fixed that to have correct field

---

## Screenshots

_[Screenshots]_
<img width="780" height="252" alt="Screenshot 2026-01-14 at 10 49 50 PM"
src="https://github.com/user-attachments/assets/50a03f2c-edfb-4bd4-bb72-a0fd79e77216"
/>
The above image shows the correct request format, but docs previously
had incorrect payload.
<img width="723" height="307" alt="Screenshot 2026-01-14 at 11 24 01 PM"
src="https://github.com/user-attachments/assets/b9096225-3f7c-4511-b8c2-e8144c896900"
/>
This is the exact wrong field in docs, that was fixed
https://trigger.dev/docs/wait-for-token#from-another-language


💯

Co-authored-by: appdevelopers9a <appdeveloper@s9alabs.com>
2026-01-14 19:33:39 +00:00
Eric Allam a5339a1bac fix(dashboard): remove span ID to original span ID cache and use a different approach to link to the cached runs (#2886) 2026-01-14 14:36:29 +00:00
Matt Aitken 5b07bd11a0 chore: FeatureFlag add createdAt and updatedAt (#2880)
It’s useful to know when they were modified for debugging and auditing.

For existing rows createdAt and updatedAt are set to now() during the
migration, to avoid a nullable column.
2026-01-14 14:28:06 +00:00
nicktrn 260fb7cd70 fix(helm): support secrets.existingSecret for core secrets (#2860)
When users set `secrets.enabled=false` to use an external secret via
`secrets.existingSecret`, the environment variables `SESSION_SECRET`,
`MAGIC_LINK_SECRET`, `ENCRYPTION_KEY`, and `MANAGED_WORKER_SECRET` were
not being populated from the secret.

Fixes #2859

Also adds automatic helm prereleases for PRs

---------

Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: nicktrn <nicktrn@users.noreply.github.com>
2026-01-14 10:44:08 +00:00
Copilot 9934627c0b Revert "Set EVENT_REPOSITORY_DEFAULT_STORE default to clickhouse_v2" (#2881)
Reverts commit 495a2531f0.

Restores `EVENT_REPOSITORY_DEFAULT_STORE` default from `"clickhouse_v2"`
back to `"postgres"`.

##  Checklist

- [x] I have followed every step in the [contributing
guide](https://github.com/triggerdotdev/trigger.dev/blob/main/CONTRIBUTING.md)
- [x] The PR title follows the convention.
- [x] I ran and tested the code works

---

## Testing

Single-line env schema default change; no runtime testing required.

---

## Changelog

- Reverted `EVENT_REPOSITORY_DEFAULT_STORE` default value from
`"clickhouse_v2"` to `"postgres"` in `apps/webapp/app/env.server.ts`

---

## Screenshots

N/A - configuration change only

💯

<!-- START COPILOT CODING AGENT SUFFIX -->



<!-- START COPILOT ORIGINAL PROMPT -->



<details>

<summary>Original prompt</summary>

> Can you revert the latest commit
495a2531f0 and create a PR for the revert


</details>



<!-- START COPILOT CODING AGENT TIPS -->
---

 Let Copilot coding agent [set things up for
you](https://github.com/triggerdotdev/trigger.dev/issues/new?title=+Set+up+Copilot+instructions&body=Configure%20instructions%20for%20this%20repository%20as%20documented%20in%20%5BBest%20practices%20for%20Copilot%20coding%20agent%20in%20your%20repository%5D%28https://gh.io/copilot-coding-agent-tips%29%2E%0A%0A%3COnboard%20this%20repo%3E&assignees=copilot)
— coding agent works faster and does higher quality work when set up for
your repo.

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: ericallam <534+ericallam@users.noreply.github.com>
2026-01-14 10:38:08 +00:00
Matt Aitken 495a2531f0 Set EVENT_REPOSITORY_DEFAULT_STORE default to clickhouse_v2 (#2879)
Change EVENT_REPOSITORY_DEFAULT_STORE default from "postgres" to
"clickhouse_v2" so new deployments favor the ClickHouse v2 event store
by default. This updates runtime behavior to use the newer store
implementation unless explicitly overridden.
2026-01-13 18:06:00 +00:00
Matt Aitken 1bca378000 Query fixes (#2876)
Don’t allow aliased columns to be queried – it was actually safe but
confusing. We call `created_at` -> `triggered_at` but we still allowed
created_at which was confusing.

Now we have nice errors if you try select columns that aren’t
selectable.

Also removed a ClickHouse setting `allow_experimental_object_type` which
worked fine locally but stopped all queries working on ClickHouse Cloud
🤦‍♂️
2026-01-13 17:43:25 +00:00
nicktrn b042b0b2f9 fix(webapp): cancel button in API key regeneration modal (#2878)
The Cancel button was missing an onClick handler to close the modal
dialog. This caused confusing behavior where clicking Cancel would not
dismiss the dialog. Also added type="button" to prevent form submission
since the button is inside a form.

Co-authored-by: Claude <noreply@anthropic.com>
2026-01-13 18:26:13 +01:00
Dan 3be55db8f8 Add smart spreadsheet docs (#2877) 2026-01-13 17:21:00 +00:00
Eric Allam bb253400a2 perf(runs-replication): Improve the CPU efficiency and throughput of the runs replication to clickhouse (#2866)
## Summary

Optimizes the runs replication service for better CPU efficiency and
throughput when inserting task runs into ClickHouse.

### Key Changes

- **Switch to compact array format** - Uses
`JSONCompactEachRowWithNames` instead of `JSONEachRow` for ClickHouse
inserts, reducing JSON serialization overhead
- **Type-safe tuple arrays** - Introduces `TaskRunInsertArray` and
`PayloadInsertArray` tuple types with compile-time column order
validation
- **Pre-sorted batch inserts** - Sorts inserts by primary key before
flushing for better ClickHouse insert performance
- **Programmatic index generation** - `TASK_RUN_INDEX` and
`PAYLOAD_INDEX` are generated from column arrays to prevent manual
synchronization errors

### Files Changed

- `runsReplicationService.server.ts` - Core optimization to use compact
array inserts
- `@internal/clickhouse` - Added `insertCompactRaw` method and tuple
types
- `taskRuns.ts` - Column definitions, index constants, and insert
functions

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-13 12:18:30 +00:00
Mihai Popescu c8686b5f1c feat: tri-6738 Create aggregated logs page (#2862)
Closes #<issue>

##  Checklist

- [x] I have followed every step in the [contributing
guide](https://github.com/triggerdotdev/trigger.dev/blob/main/CONTRIBUTING.md)
- [x] The PR title follows the convention.
- [x] I ran and tested the code works

---

## Testing
- Verified log detail view displays correctly with message, metadata,
and attributes
- Tested search highlighting functionality in log messages (escapes
special regex characters)
- Confirmed tabs (Details/Run) switch properly with keyboard shortcuts
(d/r)
  - Verified run information loads via async fetcher in Run tab
  - Tested close button and Escape key for dismissing the panel
- Verified log details display correct information: level badges, kind
badges, timestamps, trace IDs, span IDs
  - Confirmed links to parent spans and run pages work correctly
- Tested with various log levels (ERROR, WARN, INFO, DEBUG, TRACE) and
kinds (SPAN, SPAN_EVENT, LOG_*)
- Verified admin-only fields display correctly when user has admin
access
- Tested data loading states and error states (log not found, run not
found)


---

## Changelog

Created new Logs page. 
The information shown is gathered from the spans from each run.
The feature supports all run filters with two new filters for level and
logs text search.


---

## Screenshots

<img width="2059" height="1196" alt="Logs page preview"
src="https://github.com/user-attachments/assets/70b667b4-98cc-4728-855a-2766dd5c1aa5"
/>

💯

---------

Co-authored-by: James Ritchie <james@trigger.dev>
2026-01-13 11:21:13 +00:00
Iss dfb46d8847 chore(docs): add WASM external configuration documentation (#2871)
Adds a WASM block to the External section documenting that WASM packages
must be added to the `external` array in `trigger.config.ts`.
2026-01-13 11:12:54 +00:00
Matt Aitken 9942518e49 TRQL/Query improvements (#2870)
Summary
- Improve query experience and safety across ClickHouse and TSQL.

Changes
- Display JSON columns when in non-pretty mode (no longer show [Object
Object]).
- Sanitize ClickHouse errors originating from TSQL.
- Remove tenant details from errors.
- Add AI-assisted error-fixing for queries.
- Improve code quality and readability.
- Provide autocomplete support for enum values.
- Enforce limits on ClickHouse queries (10s query limit).
- Add org-level and global concurrency limits.
- Warn and train AI to avoid SELECT *; when used, only return core
columns and show info.
- If AI suggests no time range, default to past 7 days.
- Format the default query for readability.
- Add an admin-only EXPLAIN button.
- Prevent impersonation queries from being saved to history.
2026-01-13 11:12:15 +00:00
Saadi Myftija a3c387697e feat(supervisor): add node affinity rules for large machine worker pool scheduling (#2869)
⚒️ Publish Worker (v4) / build (supervisor) (push) Has been cancelled
**Background**
Runs with `large-1x` or `large-2x` machine presets are disproportionally
affected by scheduling delays during peak times. This is in part caused
by the fact that the worker pool is shared for all runs, meaning large
runs compete with smaller runs for available capacity. Because large
runs require significantly more CPU and memory, they are harder for the
scheduler to bin-pack onto existing nodes, often requiring a node with a
significant amount of free resources or waiting for a new node to spin
up entirely. This effect is amplified during peak times when nodes are
already densely packed with smaller workloads, leaving insufficient
contiguous resources for large runs. Also, large runs make up a small
percentage of the total runs.

**Changes**

This PR adds Kubernetes node affinity settings to separate large and
standard machine workloads across node pools.

- Controlled via `KUBERNETES_LARGE_MACHINE_POOL_LABEL` env var (disabled
when not set)
- Large machine presets (large-*) get a soft preference to schedule on
the large pool, with fallback to standard nodes
- Non-large machines are excluded from the large pool via required
anti-affinity
- This ensures the large machine pool is reserved for large workloads
while allowing large workloads to spill over to standard nodes if needed
2026-01-13 10:54:47 +01:00
James Ritchie 7a9490893c fix(webapp): Fix for missing table rows divider lines in safari (#2873) 2026-01-12 21:14:11 +00:00
James Ritchie 768206c91b feat(dashboard): Upgrade to the dateTime filter UI (#2864)
UI/UX improvement to the date/time picker:

- You can now choose a custom duration
- Adds a new DateTimePicker.tsx component, using a new shadcn
Calendar.tsx component
- Clear UI separation between the 2 actions, choosing a duration or
choosing date range
- Adds new quick select options for picking a date range quickly


https://github.com/user-attachments/assets/6b59b49d-2a56-4354-ad72-d8426437e56e

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Eric Allam <ericallam@users.noreply.github.com>
2026-01-12 13:01:17 +00:00
Eric Allam 2c30c2defb chore(claude): add CLAUDE.md and claude skill for writing trigger.dev tasks + add 4.3.0 rule set (#2867)
- Add CLAUDE.md providing Claude Code guidance and documenting the
Claude Code skill
- Add trigger-dev-tasks skill to assist writing Trigger.dev tasks
- Add SDK rules version 4.3.0 including batch trigger v2 and debouncing
features
2026-01-12 11:01:06 +00:00
Matt Aitken 839d5e8eba Allow query access when impersonating (#2863)
Co-authored-by: Mihai Popescu <mihai.popescu.dev@gmail.com>
2026-01-09 14:57:09 +00:00
Eric Allam 42dfb87ebb chore(ci): pre-pull test container images to prevent dockerhub rate limiting when running tests (#2861) 2026-01-09 12:15:53 +00:00
Matt Aitken 49df40cb11 TRQL and the Query page (#2843)
TRQL (pronounced Treacle like the delicious British dark sweet syrup) is
the TRiggerQueryLanguage. It allows users to safely write queries on
their data. The queries are safely turned into ClickHouse queries which
are tenant-safe and not SQL injectable.


https://github.com/user-attachments/assets/bbfca473-b3fc-4150-8fe6-79e8840a2d29

This started out as a translation of HogQL by PostHog from Python to
TypeScript.

Features
- Tenant safe queries.
- Many underlying ClickHouse features including functions and
aggregations.
- Virtual columns, which are exposed to users as real columns but are
actually expressions.
- Transformations of data types and where clauses.
- Simple JSON path querying.
- Limits on execution time.
- Reporting of query statistics.

## Query page

There’s a new Query page (currently behind a feature flag) where you can
write TRQL queries and execute them against your environment, project or
organization.

Features
- Executing TRQL queries
- Syntax highlighting and errors
- Autocomplete
- AI generation/editing of queries
- Help and examples
- Table with auto-inferred data types from the table schema
- Table cell renderers for our special types like Run ids, environments,
machines, tasks, queues, etc.
- Copy/export as CSV/JSON
- Line and bar graphs with grouping and stacking
- History of queries
2026-01-09 11:39:36 +00:00
Eric Allam cf0aa9b3ca core(claude): correctly setup repo and deps for claude code agent (#2858) 2026-01-09 10:27:23 +00:00
Eric Allam a8024afd0a chore(claude): add some allowed tools to the claude github workflow agent (#2857) 2026-01-09 10:17:29 +00:00
Eric Allam 57ba2528b2 feat(runs): use metrics instead of spans in the Runs Replication service (#2851) 2026-01-08 15:56:44 +00:00
Eric Allam 7d29f5aac9 fix(webapp): use useFetcher for env var edits to preserve scroll/toggle state (#2847)
Replaces `redirectDocument` with `useFetcher` for editing environment
variables. This allows background form submission without full page
reload, which preserves:

- Scroll position in the env vars list
- "Reveal values" toggle state
- Search filter state

Fixes #2845

Generated with [Claude Code](https://claude.ai/code)

Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Eric Allam <ericallam@users.noreply.github.com>
2026-01-08 12:51:07 +00:00
Eric Allam 36b0762100 feat(metrics): add observable gauge for batch queue worker length (#2848) 2026-01-08 12:50:46 +00:00
Eric Allam f5b2ccb494 chore(docs): add docs for handling batch trigger errors (#2838) 2026-01-08 11:33:43 +00:00
github-actions[bot] f73138fe6c chore: release v4.3.2 (#2811)
🚀 Publish Trigger.dev Docker / typecheck (push) Failing after 4s
🚀 Publish Trigger.dev Docker / units (push) Failing after 0s
🚀 Publish Trigger.dev Docker / publish-webapp (push) Has been skipped
🚀 Publish Trigger.dev Docker / publish-worker (push) Has been skipped
🚀 Publish Trigger.dev Docker / publish-worker-v4 (push) Has been skipped
This PR was opened by the [Changesets
release](https://github.com/changesets/action) GitHub action. When
you're ready to do a release, you can merge this and publish to npm
yourself or [setup this action to publish
automatically](https://github.com/changesets/action#with-publishing). If
you're not ready to do a release yet, that's fine, whenever you add more
changesets to main, this PR will be updated.


# Releases
## @trigger.dev/build@4.3.2

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.3.2`

## trigger.dev@4.3.2

### Patch Changes

- fix(cli): update command should preserve existing package.json order
([#2810](https://github.com/triggerdotdev/trigger.dev/pull/2810))
-   Updated dependencies:
    -   `@trigger.dev/build@4.3.2`
    -   `@trigger.dev/core@4.3.2`
    -   `@trigger.dev/schema-to-json@4.3.2`

## @trigger.dev/python@4.3.2

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/sdk@4.3.2`
    -   `@trigger.dev/build@4.3.2`
    -   `@trigger.dev/core@4.3.2`

## @trigger.dev/react-hooks@4.3.2

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.3.2`

## @trigger.dev/redis-worker@4.3.2

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.3.2`

## @trigger.dev/rsc@4.3.2

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.3.2`

## @trigger.dev/schema-to-json@4.3.2

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.3.2`

## @trigger.dev/sdk@4.3.2

### Patch Changes

- Improve batch trigger error messages, especially when rate limited
([#2837](https://github.com/triggerdotdev/trigger.dev/pull/2837))
-   Updated dependencies:
    -   `@trigger.dev/core@4.3.2`

## @trigger.dev/core@4.3.2

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2026-01-08 11:26:59 +00:00
Eric Allam 7c2e78c9de fix(batch): more high cardinality metric attribute fixes (#2846) 2026-01-08 10:07:49 +00:00
mintlify[bot] 6dfbe1d762 Document spike alerts feature (#2844)
Updated the billing alerts documentation to reflect the new spike alerts
feature added in PR #2829. The documentation now explains both standard
alerts (75%, 90%, 100%, 200%, 500%) and spike alerts (10x, 20x, 50x,
100x) that help catch runaway usage from bugs or errors.

## Files changed
- `docs/how-to-reduce-your-spend.mdx` - Added section explaining the two
types of billing alerts

Generated from [Chore(webapp): Adds additional billing
alerts](https://github.com/triggerdotdev/trigger.dev/pull/2829) @samejr

Co-authored-by: mintlify[bot] <109931778+mintlify[bot]@users.noreply.github.com>
2026-01-07 16:40:04 +00:00
James Ritchie 3f19b98f9f Fix(webapp): Dynamically load spline on 404 on page (#2834)
Remove the spline dependency and load it dynamically on the 404 page.

Uses a web component with a workaround for React.
2026-01-07 15:50:49 +00:00
Eric Allam 09460ab37d feat: add event loop block notifications and env flag (#2842)
Add colored console warnings when the event loop is blocked and wire
a feature flag to enable/disable notifications- Introduce
notifyEventLoopBlocked() in eventLoopMonitor.server.ts to
  log a colored warning with blocked and async type.
- Call notifyEventLoopBlocked() when an event-loop stall is detected.
- Add EVENT_LOOP_MONITOR_NOTIFY_ENABLED to env schema with a default of
  "0" so notifications are off by default.
- Will notify when over the `EVENT_LOOP_MONITOR_THRESHOLD_MS` env var

This makes it easier to spot long event-loop stalls during development
or when notifications are explicitly enabled.

<img width="840" height="132" alt="CleanShot 2026-01-07 at 15 03 24@2x"
src="https://github.com/user-attachments/assets/be20fa6a-be2b-46a1-aa89-d0913ed8b5b3"
/>
2026-01-07 15:04:50 +00:00
Eric Allam 062766e974 fix(batch): optimize processing batch trigger v2 (#2841)
This PR fixes some issues with the new BatchQueue by implementing the
full two-phase dequeue process in the FairQueue, and moving the
responsibility of consuming the worker queue to the BatchQueue and
independently enabling it via the `BATCH_QUEUE_WORKER_QUEUE_ENABLED` env
var. We've also introduced the `BATCH_QUEUE_SHARD_COUNT` env var to
control the count of master queue shards in the FairQueue. We can also
control how many queues are considered in each iteration of the master
queue consumer via the `BATCH_QUEUE_MASTER_QUEUE_LIMIT` env var.

This PR will also now skip trying to dequeue from tenants that are at
concurrency capacity, which should lead to fewer issues with low
concurrency tenants blocking higher concurrency tenants from processing.
2026-01-07 15:00:09 +00:00
James Ritchie c68eb2d951 fix(webapp): concurrency limits modal cancels and resets limit on enter (#2804)
The Override concurrency limit modal has 2 type="submit" buttons. The
first one in the DOM was firing when the "enter" key is hit which
canceled and reset the limit instead which is a bad UX.

### The fix
This fix adds a hidden button above in the DOM order which mirrors the
Update Override button. Having a double submit button is rare in our
modals so feels safe to add this to the specific modal that needs it.

### Alternative solution
Switching the order of the buttons in the main FormButton component,
then using `flex-row-reverse` to flip them back in CSS works, but it
reverses the tab order. Adding a `tabIndex` to fix that issue didn't
seem to work reliably.
2026-01-07 14:14:23 +00:00
James Ritchie 583d299891 chore(webapp): Concurrency page UI improvements (#2825)
### UI Improvements to the Concurrency page: 

- Truncates long branch names and includes a tooltip
- The Tables have a new variant if you don't want the rows to highlight
on hover
- Small fix to pluralize some words in the purchase modal
- Fix to prevent tooltip buttons being `type=submit`
- Updates the /limits docs page to include purchasing more concurrency
- Adds a clear banner when you have a positive balance of unallocated
concurrency



https://github.com/user-attachments/assets/54d927c3-84e3-4d55-8f42-726098f4daf0
2026-01-07 14:13:36 +00:00
James Ritchie 17338050b1 Chore(webapp): Adds additional billing alerts (#2829)
- Adds 4 additional alert thresholds to ensure customers are emailed if
they have runaway usage.
- Separated these into a new section called "Spike alerts" with a
tooltip so it's clear what they are.
- Tooltip message is: "Catch runaway usage from bugs or errors. We
recommend keeping these enabled as a safety net."
- A billing service PR now returns all orgs to populate the email list,
rather than oldest 5.
- Adds `defaultChecked` logic to honour existing orgs who have
configured alerts in the DB. New orgs get all alerts checked on by
default.

<img width="1316" height="1560" alt="CleanShot 2026-01-05 at 09 43
56@2x"
src="https://github.com/user-attachments/assets/ce749407-2b7f-4864-9c09-9333c5ac495a"
/>
2026-01-07 14:10:10 +00:00
James Ritchie c7a238c664 Chore(webapp): Queues page UI improvements (#2826)
Very small UI improvement to increase the spacing between the right hand
table columns. Also a tooltip wording improvement.
2026-01-07 14:07:50 +00:00
Eric Allam 8ba7526d51 fix(batch): rate limiting by token bucket no longer incorrectly goes negative (#2837)
Also improves the BatchTriggerError when a result of getting rate
limited.
2026-01-07 14:02:19 +00:00
nicktrn 47bed15255 docs: fix self-hosting section in deploy docs (#2836)
Fixes #2835

There were still some flags in here we removed, deploying is a lot
simpler now for self-hosters.

Also updates the github actions guide.

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-01-06 20:20:20 +00:00
Eric Allam 752ad32293 Add Claude Code GitHub Workflow (#2839) 2026-01-06 19:26:31 +00:00
James Ritchie db0df17a6a chore(webapp): Run navigation UI improvements (#2802)
**Improvements to the run ID copy button and run navigation buttons for
consistency**

- Adds some x-padding and layout adjustment to the copy ID button.
<img width="664" height="114" alt="CleanShot 2025-12-19 at 16 05 21@2x"
src="https://github.com/user-attachments/assets/ebc8e0de-011b-419c-bdcc-eb4157553d1c"
/>

- New custom navigation icons that work better at tiny sizes
<img width="330" height="196" alt="CleanShot 2025-12-19 at 16 06 38@2x"
src="https://github.com/user-attachments/assets/bfd8d6b8-8a65-4eac-9ce1-d70acf0ad265"
/>

Some other small improvements/fixes:
- Fixes a browser html error where there was a <button> inside a
<button>
- Updates the shortcut description to match the tooltip text for
consistency
- Made the hover states more consistent
- The shortcut bar at the bottom snaps to the list sooner because there
are more items now
2026-01-06 18:20:54 +00:00
Eric Allam edf5b142fc fix(fair-queue): Prevent unbounded memory growth from metrics cardinality explosion (#2819) 2025-12-25 08:51:40 +00:00
Eric Allam 4c3dfac223 fix(fair-queue): prevent unbounded cooloff states growth (#2818) 2025-12-24 16:33:38 +00:00
nicktrn 7ccbbdb368 docs: tidy up sync env vars note blocks (#2817) 2025-12-24 13:02:26 +00:00
Eric Allam 71279a7b12 fix(fair-queue): prevent unbounded memory growth by cleaning up queue descriptor and cooloff state cache (#2816) 2025-12-24 10:40:32 +00:00
Eric Allam 2eba36c086 chore(redis-worker): add otel spans to fair queue processing pipeline (#2815) 2025-12-24 00:37:24 +00:00
Eric Allam 29827e96f2 fix(batch): add batch queue back into master queue after visibility timeout (#2814) 2025-12-24 00:25:46 +00:00
Dan d416f340ad Added example projects link (#2812)
Closes #<issue>

##  Checklist

- [ ] I have followed every step in the [contributing
guide](https://github.com/triggerdotdev/trigger.dev/blob/main/CONTRIBUTING.md)
- [ ] The PR title follows the convention.
- [ ] I ran and tested the code works

---

## Testing

_[Describe the steps you took to test this change]_

---

## Changelog

_[Short description of what has changed]_

---

## Screenshots

_[Screenshots]_

💯
2025-12-23 18:37:19 +00:00
nicktrn 7a54a843e2 fix(cli): update command should preserve existing package.json order (#2810)
This fixes a regression introduced in #2778 - stable sort is required
for deterministic builds, but we can safely preserve order for the user
package.json during package updates
2025-12-23 12:52:06 +00:00
nicktrn 52e9baede5 fix(webapp): don't rate limit deployment finalization (#2809)
turns out we also had v2 and v3 routes
2025-12-23 09:22:00 +00:00
Eric Allam deb80890fe chore(otel): add spans to the batch queue processing pipeline (#2808) 2025-12-23 08:40:33 +00:00
Eric Allam d82089686c fix(batch): extract the queue name out of an already nested queue option (#2807) 2025-12-22 21:40:14 +00:00
Oskar Otwinowski acc10e847c chore(docs): add more cost saving tips (#2806)
Co-authored-by: Eric Allam <eallam@icloud.com>
2025-12-22 15:36:54 +00:00
Eric Allam f1a83cffc4 chore(docs): debounce options (#2797) 2025-12-22 15:35:51 +00:00
Eric Allam 61fee91830 chore(docs): upgrade for new batch trigger limits and functionality (#2787) 2025-12-22 15:35:43 +00:00
github-actions[bot] caa40ce925 chore: release v4.3.1 (#2788)
🚀 Publish Trigger.dev Docker / typecheck (push) Failing after 0s
🚀 Publish Trigger.dev Docker / publish-webapp (push) Has been skipped
🚀 Publish Trigger.dev Docker / publish-worker (push) Has been skipped
🚀 Publish Trigger.dev Docker / publish-worker-v4 (push) Has been skipped
🚀 Publish Trigger.dev Docker / units (push) Failing after 4s
This PR was opened by the [Changesets
release](https://github.com/changesets/action) GitHub action. When
you're ready to do a release, you can merge this and publish to npm
yourself or [setup this action to publish
automatically](https://github.com/changesets/action#with-publishing). If
you're not ready to do a release yet, that's fine, whenever you add more
changesets to main, this PR will be updated.


# Releases
## @trigger.dev/build@4.3.1

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.3.1`

## trigger.dev@4.3.1

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.3.1`
    -   `@trigger.dev/build@4.3.1`
    -   `@trigger.dev/schema-to-json@4.3.1`

## @trigger.dev/core@4.3.1

### Patch Changes

- Added support for idempotency reset
([#2777](https://github.com/triggerdotdev/trigger.dev/pull/2777))

## @trigger.dev/python@4.3.1

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/sdk@4.3.1`
    -   `@trigger.dev/core@4.3.1`
    -   `@trigger.dev/build@4.3.1`

## @trigger.dev/react-hooks@4.3.1

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.3.1`

## @trigger.dev/redis-worker@4.3.1

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.3.1`

## @trigger.dev/rsc@4.3.1

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.3.1`

## @trigger.dev/schema-to-json@4.3.1

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.3.1`

## @trigger.dev/sdk@4.3.1

### Patch Changes

- feat: Support for new batch trigger system
([#2779](https://github.com/triggerdotdev/trigger.dev/pull/2779))
- feat(sdk): Support debouncing runs when triggering with new debounce
options
([#2794](https://github.com/triggerdotdev/trigger.dev/pull/2794))
- Added support for idempotency reset
([#2777](https://github.com/triggerdotdev/trigger.dev/pull/2777))
-   Updated dependencies:
    -   `@trigger.dev/core@4.3.1`

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2025-12-22 15:29:18 +00:00
Dan ba9b0e17c1 Added Claude Agent SDK guide and examples (#2803) 2025-12-19 17:10:32 +00:00
Oskar Otwinowski 469808cf09 fix(webapp): Make spans fluid again while task is executing (#2801)
Closes #<issue>

##  Checklist

- [x] I have followed every step in the [contributing
guide](https://github.com/triggerdotdev/trigger.dev/blob/main/CONTRIBUTING.md)
- [x] The PR title follows the convention.
- [x] I ran and tested the code works

---

## Testing

Tested on executing tasks and on moving between tasks

---

## Changelog

Small UI fix for spans on the task runs page

---

## Screenshots



https://github.com/user-attachments/assets/72095b03-c74f-4472-afde-9c63e6e7c224



💯
2025-12-19 15:45:54 +01:00
Mihai Popescu 7574c69c2d feat(webapp): Add support for resetting idempotency keys (#2777)
Add support for resetting idempotency keys both from ui and sdk

##  Checklist

- [x] I have followed every step in the [contributing
guide](https://github.com/triggerdotdev/trigger.dev/blob/main/CONTRIBUTING.md)
- [x] The PR title follows the convention.
- [x] I ran and tested the code works

---

## Testing
- Created a new run with a idempotency idempotencyKey.
- Started a new run with the same task and got redirected to the first
run.
- Deleted the key from the UI on the run details
- Started a new run with the same task  and it created a new one
- Did the above steps using the SDK


---

## Changelog

- Add new action route for resetting idempotency keys via UI
- Add reset button in Idempotency section of run detail view
- Added API and SDK for resetting imdepotency
- Updated docs page for this feature


---

## Screenshots

_[Screenshots]_
<img width="438" height="363" alt="Screenshot 2025-12-11 at 11 56 37"
src="https://github.com/user-attachments/assets/30b8ef5e-8aac-4d04-b57a-9bf30d085dcb"
/>
2025-12-19 11:55:04 +00:00
Eric Allam 06cbe6e3ca chore(docs): upgrade sentry guide to use new global init and global hooks (#2799) 2025-12-19 00:00:02 +00:00
Eric Allam 3875bb292a feat(engine): run debounce system (#2794)
Adds support for **debounced task runs** - when triggering a task with a
debounce key, subsequent triggers with the same key will reschedule the
existing delayed run instead of creating new runs. This continues until
no new triggers occur within the delay window.

## Usage

```typescript
await myTask.trigger({ userId: "123" }, {
  debounce: {
    key: "user-123-update",
    delay: "5s",
    mode: "leading", // default
  }
});
```

- **key**: Scoped to the task identifier
- **delay**: How long to wait before executing (supports duration
strings like `"5s"`, `"1m"`)
- **mode**: Either `"leading"` or `"trailing"`. Leading debounce will
use the payload and options from the first run created with the debounce
key. Trailing will use payload and options from the last run.

### "trailing" mode overrides

When using `mode: "trailing"` with debounce, the following options are
updated from the **last** trigger:

- **`payload`** - The task input data
- **`metadata`** - Run metadata
- **`tags`** - Run tags (replaces existing tags)
- **`maxAttempts`** - Maximum retry attempts
- **`maxDuration`** - Maximum compute time
- **`machine`**  - Machine preset (cpu/memory)

## Behavior

- **First run wins**: The first trigger creates the run, subsequent
triggers push its execution time later
- **Idempotency keys take precedence**: If both are specified,
idempotency is checked first
- **Max duration**: Configurable via `DEBOUNCE_MAX_DURATION_MS` env var
(default: 10 minutes)

Works with `triggerAndWait` - parent runs correctly block on the
debounced run.
2025-12-18 16:04:43 +00:00
Saadi Myftija ff80742ab7 fix(ci): use workflow_dispatch instead of repository_dispatch for publish.yml (#2791)
Mixed up the trigger in #2790
2025-12-16 16:53:22 +01:00
Saadi Myftija 11366e658c chore(ci): add manual trigger to the image publishing workflow (#2790)
Useful to retry failures manually.
2025-12-16 16:34:57 +01:00
Eric Allam e751f8832e fix(app): incorrectly duplicated env vars for new batch trigger system (#2789) 2025-12-16 15:27:13 +00:00
Eric Allam a999d9ea3f feat(engine): Batch trigger reloaded (#2779)
New batch trigger system with larger payloads, streaming ingestion,
larger batch sizes, and a fair processing system.

This PR introduces a new `FairQueue` abstraction inspired by our own
`RunQueue` that enables multi-tenant fair queueing with concurrency
limits. The new `BatchQueue` is built on top of the `FairQueue`, and
handles processing Batch triggers in a fair manner with per-environment
concurrency limits defined per-org. Additionally, there is a global
concurrency limit to prevent the BatchQueue system from creating too
many runs too quickly, which can cause downstream issues.

For this new BatchQueue system we have a completely new batch trigger
creation and ingestion system. Previously this was a single endpoint
with a single JSON body that defined details about the batch as well as
all the items in the batch.

We're introducing a two-phase batch trigger ingestion system. In the
first phase, the BatchTaskRun record is created (and possibly rate
limited). The second phase is another endpoint that accepts an NDJSON
body with each line being a single item/run with payload and options.

At ingestion time all items are added to a queue, in order, and then
processed by the BatchQueue system.

## New batch trigger rate limits

This PR implements a new batch trigger specific rate limit, configured
on the `Organization.batchRateLimitConfig` column, and defaults using
these environment variables:

- `BATCH_RATE_LIMIT_REFILL_RATE` defaults to 10
- `BATCH_RATE_LIMIT_REFILL_INTERVAL` the duration interval, defaults to
`"10s"`
- `BATCH_RATE_LIMIT_MAX` defaults to 1200

This rate limiter is scoped to the environment ID and controls how many
runs can be submitted via batch triggers per interval. The SDK handles
the retrying side.

## Batch queue concurrency limits

The new column `Organization.batchQueueConcurrencyConfig` now defines an
org specific `processingConcurrency` value, with a backup of the env var
`BATCH_CONCURRENCY_LIMIT_DEFAULT` which defaults to 10. This controls
how many batch queue items are processed concurrently per environment.

There is also a global rate limit for the batch queue set via the
`BATCH_QUEUE_GLOBAL_RATE_LIMIT` which defaults to being disabled. If
set, the entire batch queue system won't process more than
`BATCH_QUEUE_GLOBAL_RATE_LIMIT` items per second. This allows
controlling the maximum number of runs created per second via batch
triggers.

## Batch trigger settings

- `STREAMING_BATCH_MAX_ITEMS` controls the maximum number of items in a
single batch
- `STREAMING_BATCH_ITEM_MAXIMUM_SIZE` controls the maximum size of each
item in a batch
- `BATCH_CONCURRENCY_DEFAULT_CONCURRENCY` controls the default
environment concurrency
- `BATCH_QUEUE_DRR_QUANTUM` how many credits each environment gets each
round for the DRR scheduler
- `BATCH_QUEUE_MAX_DEFICIT` the maximum deficit for the DRR scheduler
- `BATCH_QUEUE_CONSUMER_COUNT` how many queue consumers to run
- `BATCH_QUEUE_CONSUMER_INTERVAL_MS` how frequently they poll for items
in the queue

### Configuration Recommendations by Use Case

**High-throughput priority (fairness acceptable at 0.98+):**

```env
BATCH_QUEUE_DRR_QUANTUM=25
BATCH_QUEUE_MAX_DEFICIT=100
BATCH_QUEUE_CONSUMER_COUNT=10
BATCH_QUEUE_CONSUMER_INTERVAL_MS=50
BATCH_CONCURRENCY_DEFAULT_CONCURRENCY=25
```

**Strict fairness priority (throughput can be lower):**

```env
BATCH_QUEUE_DRR_QUANTUM=5
BATCH_QUEUE_MAX_DEFICIT=25
BATCH_QUEUE_CONSUMER_COUNT=3
BATCH_QUEUE_CONSUMER_INTERVAL_MS=100
BATCH_CONCURRENCY_DEFAULT_CONCURRENCY=5
```
2025-12-16 14:32:49 +00:00
Oskar Otwinowski 28a66ac021 fix(ui): respect rootOnlyDefault, disable adjacent run navigation in inspector views (#2781)
🚀 Publish Trigger.dev Docker / typecheck (push) Failing after 0s
🚀 Publish Trigger.dev Docker / publish-webapp (push) Has been skipped
🚀 Publish Trigger.dev Docker / publish-worker (push) Has been skipped
🚀 Publish Trigger.dev Docker / publish-worker-v4 (push) Has been skipped
🚀 Publish Trigger.dev Docker / units (push) Failing after 0s
## Changelog

- Add disableAdjacentRows prop to TaskRunsTable component to control
table state encoding
- Pass rootOnlyDefault prop from loader to TaskRunsTable for proper
state management
- Disable adjacent run navigation in schedule, waitpoint, and other
inspector views
- Preserve adjacent run navigation on main runs list page with rootOnly
filter support
2025-12-12 12:16:29 +00:00
github-actions[bot] 7d34817473 chore: release v4.3.0 (#2770)
This PR was opened by the [Changesets
release](https://github.com/changesets/action) GitHub action. When
you're ready to do a release, you can merge this and publish to npm
yourself or [setup this action to publish
automatically](https://github.com/changesets/action#with-publishing). If
you're not ready to do a release yet, that's fine, whenever you add more
changesets to main, this PR will be updated.


# Releases
## trigger.dev@4.3.0

### Minor Changes

- feat(cli): deterministic image builds for deployments
([#2778](https://github.com/triggerdotdev/trigger.dev/pull/2778))
- feat(cli): enable zstd compression for deployment images
([#2773](https://github.com/triggerdotdev/trigger.dev/pull/2773))

### Patch Changes

- The new `triggeredVia` field is now populated in deployments via the
CLI. ([#2767](https://github.com/triggerdotdev/trigger.dev/pull/2767))
- fix(dev): stop max listeners exceeded warning messages when running
more than 10 runs concurrently
([#2771](https://github.com/triggerdotdev/trigger.dev/pull/2771))
- Upgrade @modelcontextprotocol/sdk to 1.24.3
([#2768](https://github.com/triggerdotdev/trigger.dev/pull/2768))
-   Updated dependencies:
    -   `@trigger.dev/core@4.3.0`
    -   `@trigger.dev/build@4.3.0`
    -   `@trigger.dev/schema-to-json@4.3.0`

## @trigger.dev/core@4.3.0

### Minor Changes

- feat(cli): deterministic image builds for deployments
([#2778](https://github.com/triggerdotdev/trigger.dev/pull/2778))

### Patch Changes

- The new `triggeredVia` field is now populated in deployments via the
CLI. ([#2767](https://github.com/triggerdotdev/trigger.dev/pull/2767))

## @trigger.dev/build@4.3.0

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.3.0`

## @trigger.dev/python@4.3.0

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.3.0`
    -   `@trigger.dev/build@4.3.0`
    -   `@trigger.dev/sdk@4.3.0`

## @trigger.dev/react-hooks@4.3.0

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.3.0`

## @trigger.dev/redis-worker@4.3.0

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.3.0`

## @trigger.dev/rsc@4.3.0

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.3.0`

## @trigger.dev/schema-to-json@4.3.0

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.3.0`

## @trigger.dev/sdk@4.3.0

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.3.0`

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2025-12-12 13:11:56 +01:00
Saadi Myftija 6d6ed471d1 feat(cli): deterministic image builds for deployments (#2778)
⚒️ Publish Worker (v4) / build (supervisor) (push) Has been cancelled
This PR makes our image builds deterministic and reproducible by
ensuring that identical source code always produces the same image
layers and image digest. This means that deployments where nothing has
changed will no longer invalidate the image cache in our worker cluster
nodes, thus avoid making the cold starts for runs worse.

**Context**
New deployments currently increase the cold start times for runs, as
they generate a new image which needs to be pulled in the worker cluster
where runs are executed. It happens also when the source code for the
deployment has not changed due to non-deterministic steps in our build
system. This addresses the latter issue by making builds reproducible.

**Main changes**
- Avoided baking `TRIGGER_DEPLOYMENT_ID` and
`TRIGGER_DEPLOYMENT_VERSION` in the image, we now pass these via the
supervisor instead.
- Used `json-stable-stringify` for consistent key ordering in the files
we generate for the build, e.g., `package.json`, `build.json`,
`index.json`.
- Removed `metafile.json` from the image contents as it is not actually
used in the container. This is only relevant for the `analyze` command.
- Added `SOURCE_DATE_EPOCH=0` and `rewrite-timestamp=true` to Docker
builds to normalize file timestamps.
- Removed some `timings` and `outputHashes` from build outputs and
manifests.

The builds are now reproducible for both native build server and Depot
paths. This should also lead to better image layer cache reuse in
general.
2025-12-12 09:48:04 +01:00
Oskar Otwinowski 7f7f993587 feat(webapp): improve adjacent runs navigation and timeline performance (#2776)
- Add replace prop to LinkButton to use history replacement for adjacent
run navigation
- Preserve span and tab params when navigating between adjacent runs
- Disable animations for completed spans in timeline to improve
performance
- Include spanId in runs list navigation for better context preservation
- Direct link to task test page when filtering by single task with no
runs
- Fix minor styling issue with run friendlyId display padding
2025-12-11 17:26:15 +00:00
Oskar Otwinowski d28707826c feat(webapp): add GitHub onboarding flow to empty Tasks and Deployments pages (#2775)
##  Checklist

- [x] I have followed every step in the [contributing
guide](https://github.com/triggerdotdev/trigger.dev/blob/main/CONTRIBUTING.md)
- [x] The PR title follows the convention.
- [x] I ran and tested the code works

---

## Testing

Set up the local github application, and tested its connection with
trigger.dev

Checked:
- Change is backwards compatible
- Actions/ux are uniform across the pages (empty Tasks, Deployments,
project settings)
- Connecting GH, Connecting Repo, disconnecting Repo, modifying settings

---

## Changelog

- Create new resource route for GitHub settings management with loader
and actions
- Add GitHubSettingsPresenter to fetch connected repos and installations
- Implement GitHubSettingsPanel component for reusable GitHub
configuration UI
- Refactor project settings page to use shared GitHubSettingsPanel
component
- Integrate GitHub connection flow into empty state onboarding for Tasks
and Deployments
- Add support for GitHub repo connection, disconnection, and branch
tracking settings
- Include redirect URL support for seamless navigation after GitHub
actions
- Remove duplicate GitHub connection code from project settings route

---

## Screenshots


https://github.com/user-attachments/assets/8fc24699-640b-4f9e-afd8-b26edc945218

🐐

---------

Co-authored-by: James Ritchie <james@trigger.dev>
2025-12-10 17:03:16 +00:00
Max Strübing 8b00198f99 docs(kubernetes): use v4 helm chart instead of beta (#2671) 2025-12-10 14:52:32 +00:00
nicktrn 74e9246bfa feat(cli): enable zstd compression for deployment images (#2773)
This will speed up ice cold starts (*) for two reasons:
- better compression ratio
- faster decompression

This is a minor release because zstd compression will now be enabled by
default for all deployments.

(*) ice cold starts happen when deploy images are not cached on the
worker node yet. These cold start durations are highly dependent on
image size and as it turns out, also the type of compression used.
2025-12-10 14:49:44 +00:00
Mihai Popescu 07a31d3732 fix(webapp) : fixed Cmd+Left Arrow was intercepted by the TreeView component (#2772)
Fixed the issue where Cmd+Left Arrow was being intercepted by the
TreeView component on task runs screen.

Solution:
Added a check in the getTreeProps keyboard handler to detect when
metaKey (Cmd on macOS) is pressed with Left Arrow. When detected, the
handler returns early without preventing the default browser behavior,
allowing Chrome's native back navigation to work.


Closes #<issue>

##  Checklist

- [ x] I have followed every step in the [contributing
guide](https://github.com/triggerdotdev/trigger.dev/blob/main/CONTRIBUTING.md)
- [ x] The PR title follows the convention.
- [ x] I ran and tested the code works

---

## Testing

Went to the task details page and confirmed that Cmd + Left Arrow will
navigate back

---

## Changelog

Modified TreeView.tsx to check for e.metaKey before handling Left Arrow
key events
When Cmd+Left is pressed, the event is no longer prevented, allowing
browser default behavior
---

Co-authored-by: Mihai Popescu <mihaipopescu@Mihais-MacBook-Pro.local>
2025-12-10 12:05:16 +00:00
Saadi Myftija da111e220f fix(api): whitelist deployment endpoints from the general API rate limits (#2774)
Deployments are affected by general API rate limits, this is just a
quick fix by whitelisting the deployment related endpoints. In a follow
up PR we'll add a separate rate limiter for this group of endpoints.
2025-12-10 12:03:56 +01:00
Oskar Otwinowski 2c3cb4a43a feat(webapp): UX improvements for TaskRun page and TaskRun table (#2760)
##  Checklist

- [x] I have followed every step in the [contributing
guide](https://github.com/triggerdotdev/trigger.dev/blob/main/CONTRIBUTING.md)
- [x] The PR title follows the convention.
- [x] I ran and tested the code works

---

## Testing

Manual testing of the task run pages

---

## Changelog

- Add previous/next run navigation buttons to run detail page header
- Support [ and ] keyboard shortcuts to jump between adjacent runs
- Preserve runs table state (filters, pagination) when navigating
- Preload adjacent page runs at boundaries for seamless navigation
- Add actions prop to PageTitle component
- Document shortcut in keyboard shortcuts panel

- Store current filter state from runs table as `tableState` search
param when navigating to individual run pages
- Restore filters when navigating back from run detail view to runs list
- Update `v3RunPath` and `v3RunSpanPath` helpers to accept optional
searchParams
- Use `useOptimisticLocation` to capture current search params in
TaskRunsTable
- Parse `tableState` param in run detail route and pass filters to back
button

- This improves UX by remembering filter selections (task, status, date
range, etc.) when users click into a run and then navigate back to the
runs list

- Add new text-below variant that shows "Click to copy" tooltip on hover
and "Copied" on click. Also add controlled open/onOpenChange props to
SimpleTooltip for managing tooltip visibility.

---

## Screenshots


https://github.com/user-attachments/assets/5067bbe0-1bcd-4e75-80a7-f56dabd5ed69
2025-12-09 13:10:11 +00:00
Saadi Myftija b71bf89444 chore(releases): minor improvements to the release workflow (#2764)
Changes in this PR:
- Arbitrary refs are now allowed when triggering the release workflow
manually (ref must be on the main branch).
- Release summary is now displayed in the GH job output; makes for a
nicer experience when approving the release workflow.
2025-12-09 11:29:48 +01:00
Eric Allam 28c0c78257 fix(api): triggering a batch on a v4 project via the v3 SDK no longer results in errors (#2752)
<!-- CURSOR_SUMMARY -->
> [!NOTE]
> Updates `BatchTriggerV3Service` to send `batch.id` (not `friendlyId`)
as `batchId` to `TriggerTaskService.call`.
> 
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
68bf8df4a29bdcac5bfb2806bc417541f69ffd6e. This will update automatically
on new commits. Configure
[here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->
2025-12-09 09:54:39 +00:00
Eric Allam c021d1db63 fix(dev): stop max listeners exceeded warning messages when running more than 10 runs concurrently (#2771)
<!-- CURSOR_SUMMARY -->
> [!NOTE]
> Centralizes SIGTERM handling in `DevSupervisor` and removes per-run
SIGTERM listeners in `DevRunController` to avoid
MaxListenersExceededWarning under high concurrency.
> 
> - **Dev runtime**:
> - **SIGTERM handling**: Add centralized handler in
`packages/cli-v3/src/dev/devSupervisor.ts` to gracefully stop all run
controllers; unregisters on `shutdown()`.
> - **Cleanup**: Remove per-controller `SIGTERM` listener and handler
from `packages/cli-v3/src/entryPoints/dev-run-controller.ts` to reduce
event listeners and warnings.
> - **Changeset**: Add patch note in
`.changeset/fuzzy-ghosts-admire.md`.
> 
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
5ad2f5341829cebf6fd37a3c616a2db5e4ad936a. This will update automatically
on new commits. Configure
[here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->
2025-12-09 09:54:28 +00:00
Eric Allam f62cdfe00e feat(dashboard): login with google and "last used" indicator (#2746)
<img width="568" height="513" alt="CleanShot 2025-12-05 at 14 27 16"
src="https://github.com/user-attachments/assets/1f44d8b9-8791-4b44-96d5-4a0960a1ab36"
/>

<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> Adds Google OAuth login and a cookie-based “last used” indicator on
the login page, with supporting backend, routes, and schema updates.
> 
> - **Auth/Backend**:
> - **Google OAuth**: Integrates `remix-auth-google` via new
`addGoogleStrategy` and enables when `AUTH_GOOGLE_CLIENT_ID/SECRET` are
set (`services/googleAuth.server.ts`, `services/auth.server.ts`).
> - **User handling**: Implements `findOrCreateGoogleUser` with
linking/upsert logic and conflict logging (`models/user.server.ts`).
> - **MFA + session**: Google/GitHub/Magic callbacks now set session,
handle MFA, and set a "last-auth-method" cookie
(`routes/auth.google*.tsx`, `routes/auth.github.callback.tsx`,
`routes/magic.tsx`, `services/lastAuthMethod.server.ts`).
> - **GitHub strategy**: Safer email check
(`services/gitHubAuth.server.ts`).
> - **Routes/UI**:
> - **Login page**: Adds "Continue with Google" button and animated
"Last used" badge based on cookie; keeps GitHub/Email options
(`routes/login._index/route.tsx`).
> - **Redirect safety**: Sanitize redirect paths and persist redirect
via cookies in auth actions (`routes/auth.github.ts`,
`routes/auth.google.ts`).
>   - **Assets**: Adds `GoogleLogo` SVG.
>   - **Avatar**: Set `referrerPolicy="no-referrer"` on profile image.
> - **Config/Schema**:
> - **Env**: Adds `AUTH_GOOGLE_CLIENT_ID`/`AUTH_GOOGLE_CLIENT_SECRET`
(`env.server.ts`).
> - **DB**: Extends `AuthenticationMethod` enum with `GOOGLE` (Prisma
schema + migration).
> - **Dependencies**:
>   - Adds `remix-auth-google` in `package.json`.
> 
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
9f84f974bd6f21f1699c4f69a6aa91616842d1b1. This will update automatically
on new commits. Configure
[here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->

---------

Co-authored-by: James Ritchie <james@trigger.dev>
2025-12-09 09:51:42 +00:00
nicktrn 66c6da7114 security: dependabot alert triage (#2768)
- Upgrade @modelcontextprotocol/sdk to 1.24.3
- Override jws to 3.2.3
2025-12-08 21:02:13 +00:00
Saadi Myftija 7fba9e9f6b feat(deployments): add build server meta and trigger source info (#2767)
This PR applies a small change to the deployments table to keep track
of:
- where the deployment was triggered from
- build server metadata, if the build server was involved
2025-12-08 16:29:26 +00:00
Saadi Myftija cf63fc9cd2 chore: update version requirement hint in build settings (#2757) 2025-12-08 09:27:28 +01:00
Saadi Myftija d1c3bfb9c9 fix(deployments): ecr repo exists check (#2762)
We recently upgraded the ECR sdk version. Our ECR repo exists check
relies on the type of the error thrown and the new ECR sdk version seems
to have broken that behavior. This PR adds a workaround to the issue.
2025-12-05 19:16:28 +01:00
Saadi Myftija d8f5853457 fix(releases): use contents: write permission in the release workflow (#2761)
Needed to push tags
2025-12-05 17:22:24 +01:00
Saadi Myftija a52566d9cc feat(releases): add a manual trigger for the package release workflow (#2759)
The manual trigger is currently enabled only for the prerelease job.
This PR adds it for the normal release flow too, as it is useful to
retrigger failed release workflows.
2025-12-05 17:00:02 +01:00
Saadi Myftija 07a1d04d52 fix(releases): add missing npm upgrade step (#2758)
Accidentally removed this in a previous PR.
2025-12-05 16:41:45 +01:00
github-actions[bot] 185f4ecaf9 chore: release v4.2.0 (#2713)
🚀 Publish Trigger.dev Docker / units (push) Failing after 0s
🚀 Publish Trigger.dev Docker / typecheck (push) Failing after 19s
🚀 Publish Trigger.dev Docker / publish-webapp (push) Has been skipped
🚀 Publish Trigger.dev Docker / publish-worker (push) Has been skipped
🚀 Publish Trigger.dev Docker / publish-worker-v4 (push) Has been skipped
This PR was opened by the [Changesets
release](https://github.com/changesets/action) GitHub action. When
you're ready to do a release, you can merge this and publish to npm
yourself or [setup this action to publish
automatically](https://github.com/changesets/action#with-publishing). If
you're not ready to do a release yet, that's fine, whenever you add more
changesets to main, this PR will be updated.


# Releases
## trigger.dev@4.2.0

### Minor Changes

- feat(cli): upgrade bun deployments to v1.3.3
([#2756](https://github.com/triggerdotdev/trigger.dev/pull/2756))

### Patch Changes

- fix(otel): exported logs and spans will now have matching trace IDs
([#2724](https://github.com/triggerdotdev/trigger.dev/pull/2724))
- The `--force-local-build` flag is now renamed to just `--local-build`
([#2702](https://github.com/triggerdotdev/trigger.dev/pull/2702))
- fix(cli): header will always print the correct profile
([#2728](https://github.com/triggerdotdev/trigger.dev/pull/2728))
- feat: add ability to set custom resource properties through
trigger.config.ts or via the OTEL_RESOURCE_ATTRIBUTES env var
([#2704](https://github.com/triggerdotdev/trigger.dev/pull/2704))
- feat(cli): implements content-addressable store for the dev CLI build
outputs, reducing disk usage
([#2725](https://github.com/triggerdotdev/trigger.dev/pull/2725))
- Added support for native build server builds in the deploy command
(`--native-build-server`)
([#2702](https://github.com/triggerdotdev/trigger.dev/pull/2702))
-   Updated dependencies:
    -   `@trigger.dev/build@4.2.0`
    -   `@trigger.dev/core@4.2.0`
    -   `@trigger.dev/schema-to-json@4.2.0`

## @trigger.dev/build@4.2.0

### Patch Changes

- syncVercelEnvVars to skip API and read env vars directly from
env.process for Vercel build environments. New syncNeonEnvVars build
extension for syncing environment variablesfrom Neon database projects
to Trigger.dev. The extension automatically detects branches and builds
appropriate PostgreSQL connection strings for non-production, non-dev
environments (staging, preview).
([#2729](https://github.com/triggerdotdev/trigger.dev/pull/2729))
-   Updated dependencies:
    -   `@trigger.dev/core@4.2.0`

## @trigger.dev/core@4.2.0

### Patch Changes

- fix: prevent ERR_IPC_CHANNEL_CLOSED errors from causing an unhandled
exception on TaskRunProcess
([#2743](https://github.com/triggerdotdev/trigger.dev/pull/2743))
- Added support for native build server builds in the deploy command
(`--native-build-server`)
([#2702](https://github.com/triggerdotdev/trigger.dev/pull/2702))

## @trigger.dev/python@4.2.0

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/build@4.2.0`
    -   `@trigger.dev/sdk@4.2.0`
    -   `@trigger.dev/core@4.2.0`

## @trigger.dev/react-hooks@4.2.0

### Patch Changes

- fix: prevent infinite useEffect when passing an array of tags to
useRealtimeRunsWithTag
([#2705](https://github.com/triggerdotdev/trigger.dev/pull/2705))
-   Updated dependencies:
    -   `@trigger.dev/core@4.2.0`

## @trigger.dev/redis-worker@4.2.0

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.2.0`

## @trigger.dev/rsc@4.2.0

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.2.0`

## @trigger.dev/schema-to-json@4.2.0

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.2.0`

## @trigger.dev/sdk@4.2.0

### Patch Changes

- fix(sdk): Re-export schemaTask types to prevent the TypeScript error
TS2742: The inferred type of 'task' cannot be named without a reference
to '@trigger.dev/core/v3'. This is likely not portable.
([#2735](https://github.com/triggerdotdev/trigger.dev/pull/2735))
- feat: add ability to set custom resource properties through
trigger.config.ts or via the OTEL_RESOURCE_ATTRIBUTES env var
([#2704](https://github.com/triggerdotdev/trigger.dev/pull/2704))
-   Updated dependencies:
    -   `@trigger.dev/core@4.2.0`

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2025-12-05 16:27:21 +01:00
nicktrn 0d764d4c46 feat(cli): upgrade bun deployments to v1.3.3 (#2756)
New deployments with `runtime: "bun"` will now use Bun v1.3.3

Link to Bun release: https://bun.com/blog/bun-v1.3.3
2025-12-05 13:31:27 +00:00
Saadi Myftija beb52b9800 chore(releases): adjust gh workflow title (#2755)
Just a tiny workflow label change
2025-12-05 13:33:35 +01:00
Saadi Myftija 3401a1d0a9 feat(releases): require approval for package releases (#2753)
**Background**
Currently the changeset PR creation and the publishing is handled by the
same workflow. This is not ideal:
- The build steps are executed on every run of the pipeline, even though
they're only needed for the publish case.
- The PR creation workflow does not need permissions to publish to npm,
only the release path needs them.
- Adding an approval step is painful as we'd need to also approve each
changeset PR creation workflow run.

**Changes in this PR**
- Separated the changeset PR creation into its own workflow and minimum
permission set.
- Added a GH environment with an approval step for the package
publishing workflow (also for prereleases).
- New publish workflow runs will not cancel in-progress runs; helps
avoid partial failures in publishing.

These changes also enable hardening the npm OIDC setup by tying it to a
GH environment that requires approval.
2025-12-05 13:25:25 +01:00
nicktrn 3f982ed366 docs: update email regex examples (#2749)
The previous examples lacked start and end anchors
2025-12-04 17:58:25 +00:00
Saadi Myftija 4dc956470d fix(deployments): misc fixes for the native build server deployment flow (#2748)
Improved a couple of error messages.

Also fixed an issue with the s2 token caching.
2025-12-04 17:01:47 +01:00
Oskar Otwinowski 7fddadcce9 chore: Improve Vercel/Neon syncEnvVars build extensions docs (#2747)
chore: Improve Vercel/Neon `syncEnvVars` build extensions docs
2025-12-04 16:34:57 +01:00
Oskar 6e038d4d1e chore: Improve Vercel/Neon syncEnvVars extensions docs 2025-12-04 16:32:15 +01:00
Oskar Otwinowski 05b53967ea feat(build): Add NeonDB branch resolution for Vercel preview environments (#2729)
Vercel's NeonDB integration renders database connection environment
variables at runtime, which means Trigger.dev cannot directly sync these
values during the build process. This change adds support for fetching
branch-specific NeonDB connection strings via the Neon API.

feat(build): Add syncNeonEnvVars extension and improve Vercel env var
syncing

Add a new `syncNeonEnvVars` build extension for syncing environment
variables
from Neon database projects to Trigger.dev. The extension automatically
detects
branches and builds appropriate PostgreSQL connection strings for
non-production
environments (staging, dev, preview).

Features of `syncNeonEnvVars`:
- Fetches branch-specific database credentials from Neon API
- Generates all standard Postgres connection strings (DATABASE_URL,
POSTGRES_URL,
  POSTGRES_PRISMA_URL, etc.) with both pooled and unpooled variants
- Supports custom database name, role name, and env var prefix options
- Skips automatically in Vercel environments (Neon's Vercel integration
handles this)
- Skips for production environments (designed for preview/staging/dev
branches)

Improvements to `syncVercelEnvVars`:
- When running in a Vercel build environment (detected via VERCEL env
var),
values are now read from process.env instead of the Vercel API response
- This ensures the build uses the actual runtime values Vercel provides
- Removed embedded Neon-specific logic (now handled by separate
extension)
- Simplified and cleaned up the extension code

Documentation updates for both extensions with usage examples and
configuration
options.

Closes #2714

##  Checklist

- [x] I have followed every step in the [contributing
guide](https://github.com/triggerdotdev/trigger.dev/blob/main/CONTRIBUTING.md)
- [x] The PR title follows the convention.
- [x] I ran and tested the code works

---

## Testing

Set up Vercel + Trigger.dev envs, used Vercel's chat-bot-ai template.
2025-12-04 16:26:35 +01:00
Oskar 702f3b4bca chore: Improve changeset message 2025-12-04 16:26:24 +01:00
nicktrn 8fcd93001d chore: insert missing spaces in changeset 2025-12-04 13:42:32 +00:00
Eric Allam b01b8740cc fix(engine): prevent overriding a queue concurrency limit resuming a paused queue (#2745) 2025-12-04 13:33:12 +00:00
Oskar 357aa99309 Add changeset 2025-12-04 14:12:42 +01:00
Oskar 7cbf82a4ae fix: PR feedback, doc improvements for neon/vercel syncEnvVars 2025-12-04 14:07:31 +01:00
Eric Allam e7fec4097f fix(dev): CLI now properly cleans up the store dir on dev CLI exit (#2744) 2025-12-04 13:13:01 +01:00
Eric Allam d279988e38 fix(workers): prevent ERR_IPC_CHANNEL_CLOSED errors from causing an unhandled exception on TaskRunProcess (#2743) 2025-12-04 12:02:30 +00:00
Saadi Myftija 748ae658f7 fix(releases): use npm 11.x for OIDC support in the release workflow (#2742)
Support for OIDC requires npm v11.5.1 or newer.
2025-12-04 12:21:21 +01:00
Oskar Otwinowski 249878ed92 feat(webapp): link to task-specific test page from filtered runs table (#2741)
When viewing runs filtered to a single task, the "Create a test run" and
"Run a test" buttons now navigate directly to the task-specific test
page instead of the generic test page.

This improves UX by pre-populating the test form with the filtered task,
saving users from having to manually select it again.


##  Checklist

- [x] I have followed every step in the [contributing
guide](https://github.com/triggerdotdev/trigger.dev/blob/main/CONTRIBUTING.md)
- [x] The PR title follows the convention.
- [x] I ran and tested the code works

## Screenshots

<img width="1706" height="1392" alt="image"
src="https://github.com/user-attachments/assets/d8b5d445-73b5-426c-83a4-90ac2a95b955"
/>
2025-12-04 10:56:24 +00:00
Eric Allam 117b1d5a53 chore(dependabot): upgrade next.js for CVE-2025-66478 in d3-chat example project (#2740) 2025-12-04 10:30:37 +00:00
Eric Allam 04173a93b9 fix(replication): detect misconfigered run replication publication and output helpful error messages (#2736)
Add validation for logical replication publication configuration. Helps
diagnose an issue where runs are no longer replicated to clickhouse
because of a configuration issue with the replication publication.

## Problem

The `LogicalReplicationClient` only checked if a publication existed,
not if it was correctly configured. This caused a silent failure where:

- Replication would start successfully
- Transaction boundaries (begin/commit) were received
- **But no actual data changes were replicated**

This happened when a publication existed but:
1. Had no tables associated with it
2. Was missing required actions (e.g., `delete`)

## Solution

Added `#validatePublicationConfiguration()` method that validates:
-  Publication includes the expected table
-  Publication has all required actions configured

When validation fails, error messages include the exact SQL command to
fix the issue:

**Missing table:**
```
Publication 'task_runs_to_clickhouse_v1_publication' exists but has NO TABLES configured. 
Expected table: "public.TaskRun". 
Run: ALTER PUBLICATION task_runs_to_clickhouse_v1_publication ADD TABLE "TaskRun";
```

**Missing actions:**
```
Publication 'task_runs_to_clickhouse_v1_publication' is missing required actions. 
Expected: [insert, update, delete], Current: [insert, update], Missing: [delete]. 
Run: ALTER PUBLICATION task_runs_to_clickhouse_v1_publication SET (publish = 'insert, update, delete');
```

This prevents silent data loss and makes debugging configuration issues
much easier.
2025-12-04 10:29:13 +00:00
Saadi Myftija da4c753b68 fix(releases): add missing GH token in prerelease workflow (#2739)
The `changeset version` command needs the GH token too.
2025-12-04 11:27:11 +01:00
Eric Allam 6ae3b69745 chore(cursor): add deslop command (#2722) 2025-12-04 10:24:09 +00:00
Saadi Myftija 652d95c7eb feat(releases): add a prerelease workflow (#2737)
Adds a manual trigger to the `release.yml` workflow for publishing
prerelease versions. Needs to be in the same workflow file due to a NPM
limitation on how OIDC claims are checked.

Currently there is a validation step on the ref for the prerelease: it
must be merged to the main branch. We can revisit this in the future in
case we find it too limiting.
2025-12-04 10:17:05 +01:00
Saadi Myftija 341e27d213 chore(deployments): lazily update ECR repo cache settings (#2734)
To avoid doing a migration for the ECR repo cache settings, we lazily do
it on the next deployment for that project. Failures to update the repo
settings are just logged and will not cause the deployment to fail.
2025-12-03 17:15:59 +01:00
Eric Allam 9821d02af7 fix(sdk): Re-export schemaTask types to prevent the TypeScript error TS2742 (#2735)
Fixes this type of error when exporting a `schemaTask` in a monorepo:

```
error TS2742: The inferred type of 'helloWorldSchema' cannot be named without a reference to '@trigger.dev/core/v3'. This is likely not portable.
```
2025-12-03 16:12:23 +00:00
Saadi Myftija 255a73a2fe feat(deployments): --native-build-server support for the deploy command (#2702)
This PR adds support for CLI deployments using the native build server.

**Background**

The deployment command currently does the following:
- bundles the code
- submits the build context to our external build provider and waits for
the build
- triggers deployment state transitions using the platform API

Upstream build provider outages cause issue with deployments,
potentially blocking deployments entirely. We recently introduced the
`--force-local-build` flag as a fallback to enable deployment without a
dependency on the upstream build provider, though it requires users to
have docker in their systems. This PR continues that work by providing a
remote build path which uses our own build server and does not rely on
the external provider.

**Changes in this PR**

Introduced the new `--native-build-server` flag, which does the
following:
- scans all files relevant for the Trigger deployment and evaluates
ignore rules
- packages it up in an archive and uploads it as a deployment artifact
- queues the deployment and triggers the build
- streams logs from the build server

This no longer relies on external build services. Also deployment state
transitions happen on the server-side, giving us more flexibility to
evolve the flow and schemas of related deployment API endpoints. In
general it gives us better control of the whole build and deployment
process. This path will eventually become the default.

The `--detach` flag is also new, allowing to trigger deployments without
waiting for the result.

The deployment artifacts are uploaded via pre-signed URLs to avoid
unnecessary load on the platform. The new `/artifacts` endpoint
generates the pre-signed URLs; size limits are enforced on s3. This
endpoint is deliberately generic, we could extend it in the future to
upload other artifacts client-side in a similar way, e.g., large payload
packets.
2025-12-03 16:40:21 +01:00
Saadi Myftija c5f7a8daf7 fix(releases): explicitly use no frozen lockfile in the install step (#2733) 2025-12-03 16:28:06 +01:00
Saadi Myftija 8b0f51b317 chore(releases): automatically update the lockfile in changeset PRs (#2732)
This is a step which we currently need to do manually and it's rather
painful. The lockfile update is necessary due to cross references in our
packages.

Added it as a separate job instead of a step to start from fresh
workspace, as the state that the `changeset` step leaves the workdir is
not explicitly clear to the reader.
2025-12-03 16:04:30 +01:00
Oskar 53f21e1330 feat(build): Add syncNeonEnvVars extension and improve Vercel env var syncing
Add a new `syncNeonEnvVars` build extension for syncing environment variables
from Neon database projects to Trigger.dev. The extension automatically detects
branches and builds appropriate PostgreSQL connection strings for non-production
environments (staging, dev, preview).

Features of `syncNeonEnvVars`:
- Fetches branch-specific database credentials from Neon API
- Generates all standard Postgres connection strings (DATABASE_URL, POSTGRES_URL,
  POSTGRES_PRISMA_URL, etc.) with both pooled and unpooled variants
- Supports custom database name, role name, and env var prefix options
- Skips automatically in Vercel environments (Neon's Vercel integration handles this)
- Skips for production environments (designed for preview/staging/dev branches)

Improvements to `syncVercelEnvVars`:
- When running in a Vercel build environment (detected via VERCEL env var),
  values are now read from process.env instead of the Vercel API response
- This ensures the build uses the actual runtime values Vercel provides
- Removed embedded Neon-specific logic (now handled by separate extension)
- Simplified and cleaned up the extension code

Documentation updates for both extensions with usage examples and configuration
options.
2025-12-03 16:03:41 +01:00
Saadi Myftija 331882f59c chore(releases): add package version in changeset PRs (#2730)
* Add the release version to changeset PRs

* Add missing id-token permission, needed for oidc

* Remove a couple of unnecesary steps

* Reference the `changeset-release/main` branch explicitly
2025-12-03 14:42:40 +01:00
nicktrn 1276491a83 docs: clarify slack channel on pro (#2731) 2025-12-03 13:23:38 +00:00
Eric Allam 5b7dfe23b5 feat(cli): implements content-addressable store for the dev CLI build outputs, reducing disk usage (#2725)
* feat(cli): implements content-addressable store for the dev CLI build outputs, reducing disk usage

* fix a few things
2025-12-03 10:17:35 +00:00
Oskar 49b2f683f4 feat(build): Add NeonDB branch resolution for Vercel preview environments
Vercel's NeonDB integration renders database connection environment
variables at runtime, which means Trigger.dev cannot directly sync
these values during the build process. This change adds support for
fetching branch-specific NeonDB connection strings via the Neon API.

Changes:
- Discover NEON_PROJECT_ID from incoming Vercel environment variables
- Call NeonDB API to search for branches matching the git branch name
- Filter branches to find exact matches with Vercel environment prefix
  (e.g., "preview/branch-name") to avoid false positives from partial
  string matches
- Retrieve branch endpoints and select the write endpoint (or first
  available)
- Build connection strings (DATABASE_URL, POSTGRES_URL, etc.) using
  the branch endpoint host while preserving user/password credentials

Safety measures for non-production environments:
- Filter out all Neon-related env vars (DATABASE_URL, PGHOST, etc.)
  before calling the Neon API to prevent accidental use of production
  database credentials
- Only add branch-specific database env vars if a matching Neon branch
  is found and the API call succeeds
- If neonDbAccessToken is not provided or the API fails, non-production
  environments will not receive any database connection env vars

Usage:
Users must provide a NEON_ACCESS_TOKEN (via options or env var) to
enable automatic branch resolution for preview deployments. Production
environments continue to use Vercel's standard env var sync without
modification.
2025-12-02 19:52:45 +01:00
nicktrn af9b3e1c99 fix(cli): header will always print the correct profile (#2728) 2025-12-02 17:45:35 +00:00
Eric Allam df4ab97d59 fix(otel): fix broken schedule run spans (#2727)
schedule spans can sometimes show as generic spans when using the 
task_events_v2 table because of the inserted_at filter. Increasing the 
buffer for the start time does the trick and doesn’t cause any perf 
Issues (and is in general just more robust)
2025-12-02 17:16:08 +00:00
Eric Allam 9f27422472 fix(otel): exported logs and spans will now have matching trace IDs (#2724)
When using custom OTLP exporters via `telemetry.exporters` and 

This occurred when tasks were triggered **without** a parent trace 
context (e.g., via API or dashboard). In this scenario: - Spans were 
correctly rewritten to use the generated `externalTraceId` - Logs kept 
their original internal trace ID due to a bug in the early return logic

### Root Cause

In `ExternalLogRecordExporterWrapper.transformLogRecord()`, the early 
return condition incorrectly included `!this.externalTraceContext`:

```typescript
if (!logRecord.spanContext || !this.externalTraceId || 
!this.externalTraceContext) {   return logRecord;  // Bug: Returns early
when externalTraceContext is undefined }

// This fallback logic was never reached:
const externalTraceId = this.externalTraceContext
  ? this.externalTraceContext.traceId
  : this.externalTraceId;
```

### Fix

1. **Reordered logic in `transformLogRecord()`**: Move the 
1. `externalTraceId` calculation before the early return, and check the 
1. culated value instead of `this.externalTraceContext`:

```typescript
const externalTraceId = this.externalTraceContext
  ? this.externalTraceContext.traceId
  : this.externalTraceId;

if (!logRecord.spanContext || !externalTraceId) {
  return logRecord;
}
```

2. **Clarified `_isExternallySampled` logic**: Updated both 
2. `ExternalSpanExporterWrapper` and `ExternalLogRecordExporterWrapper` 
2. explicitly handle the case where there's no external trace context 
2. a generated `externalTraceId` exists:

```typescript
this._isExternallySampled = externalTraceContext
  ? isTraceFlagSampled(externalTraceContext.traceFlags)
  : !!externalTraceId;
```

### Impact

Logs and spans from the same task run will now have matching trace IDs 
when exported to external observability tools, enabling proper trace correlation regardless of whether the task was triggered with or without a parent trace context. 
`telemetry.logExporters` in `trigger.config.ts`, logs and spans were 
exported with **different trace IDs**, breaking trace correlation in
external observability tools like Datadog.
2025-12-02 14:16:06 +00:00
nicktrn 2f1a72b109 security: remedy dependabot alerts (#2723)
* security: override js-yaml

* security: upgrade vite

* security: update nodemailer
2025-12-02 12:02:48 +00:00
Eric Allam 3c326a4b4a fix(clickhouse): ensure start_time is never older than X ms to prevent old partition merge issues (#2721) 2025-12-01 15:43:10 +00:00
Saadi Myftija 6ae1317b69 chore(clickhouse): enable dropping run debug events (#2720)
Added a new env var (`EVENT_REPOSITORY_DEBUG_LOGS_DISABLED`) that allows disabling writing run debug logs in the event repository.
2025-12-01 15:42:35 +01:00
Eric Allam 2e1c4f6df6 fix(clickhouse): partition by insertion date to prevent "Too many parts" errors when partitioning by start time (#2719) 2025-12-01 12:01:06 +00:00
nicktrn 2bf86dc20e fix(supervisor): image builds with pnpm v10 (#2718) 2025-12-01 11:28:05 +00:00
nicktrn 485782cae1 feat(ch): optionally disable migrations (#2715) 2025-11-28 19:39:29 +00:00
Eric Allam 61b338bea7 chore(repo): upgrade repo to pnpm@10 to prevent executing dep scripts on install (#2712)
* chore: migrate pnpm lockfile to v9 format via pnpm@9

* Upgrade to pnpm 10.23.0

* update the dockerfile and added a few deps to bundle in remix app
2025-11-27 16:26:19 +00:00
Kim Hallberg 83ddf721a4 docs: update Firecrawl example (#2652)
Update the example to use Firecrawl v2 API
2025-11-24 14:54:51 +00:00
Kim Hallberg a4dd2562d2 docs: format embedding example (#2656) 2025-11-24 14:51:53 +00:00
Felipe Martinez Albeche 5ff21a758c fix(docs): tooltip color contrast accessibility (#2662)
* fix(docs): color contrast accessibility on tooltip

* add changeset

* Delete .changeset/tough-feet-accept.md

---------

Co-authored-by: Eric Allam <eric@trigger.dev>
2025-11-24 14:39:14 +00:00
Lindsey 47a64c0335 docs: improve env var documentation with .env upload + ctx environment details (#2680) 2025-11-24 14:37:04 +00:00
Eric Allam fc351cb6c4 chore(repo): remove GITHUB_TOKEN requirement for publishing prerelease packages (#2679) 2025-11-24 14:33:40 +00:00
Eric Allam c4f2a9d065 chore(references): added prisma-generator-ts-enums to prisma ref project as an example (#2701) 2025-11-24 14:33:22 +00:00
Eric Allam 2762c542c2 chore(repo): remove format script to AI stops calling it (#2703) 2025-11-24 14:33:14 +00:00
Eric Allam 72e286af2f feat(otel): support for custom resource attributes via config#telemetry.resource and OTEL_RESOURCE_ATTRIBUTES env var (#2704) 2025-11-24 14:33:07 +00:00
Eric Allam f7240a99e7 fix(react): prevent infinite useEffect when passing an array of tags to useRealtimeRunsWithTag (#2705) 2025-11-24 14:32:57 +00:00
github-actions[bot] 6e47377766 chore: Update version for release (#2700)
🚀 Publish Trigger.dev Docker / typecheck (push) Failing after 0s
🚀 Publish Trigger.dev Docker / publish-webapp (push) Has been skipped
🚀 Publish Trigger.dev Docker / publish-worker (push) Has been skipped
🚀 Publish Trigger.dev Docker / publish-worker-v4 (push) Has been skipped
🚀 Publish Trigger.dev Docker / units (push) Failing after 21s
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2025-11-24 10:30:01 +00:00
nicktrn a8563ca534 fix(docker): support the latest docker version (#2686) 2025-11-21 15:24:31 +00:00
Eric Allam 5e5c97ea4c fix(cli): stop dev runs stuck in dequeued status (fix #2639) (#2699)
* stop deleting the first dev version files on the first change, prevents system failures

* prevent dev runs getting stuck in dequeued status by deleting workers

* add changeset
2025-11-21 15:24:07 +00:00
Matt Aitken abd99aa27f Don't show (or count) archived branches/projects against allocated concurrency (#2698)
Archived preview branches were being counted. Also archived projects.
2025-11-20 11:41:12 +00:00
Eric Allam 4347499799 fix(api): Fix preview branch targeting in environment variable API routes (#2697)
This PR fixes the `x-trigger-branch` header support for targeting specific preview branches when managing environment variables. The header was documented but not actually being extracted or used in the environment variable API routes. Additionally, the query logic in `authenticatedEnvironmentForAuthentication` was fundamentally broken—it searched for environments with both `slug: "preview"` (parent environment property) AND a specific `branchName` (child environment property), which no environment could satisfy simultaneously. The fix extracts the branch name using `branchNameFromRequest()` and correctly queries for child branch environments using `type: "PREVIEW"` and the specific `branchName`. This ensures that environment variable operations (create, update, get, list) properly target individual preview branches instead of affecting all preview environments.
2025-11-19 18:01:12 +00:00
Eric Allam abee783d3f chore(logs): remove unnecessary debug logs (#2696) 2025-11-19 13:56:15 +00:00
Eric Allam 6464eeed53 fix(webapp): correctly generate JWT tokens for preview branches after triggering a run (fix #2678) (#2695) 2025-11-19 13:42:12 +00:00
Matt Aitken bee59de3a0 Concurrency self serve (#2681)
* Don't use the organization max concurrency anymore

* Early draft of the concurrency page

* WIP adding a new stepper input component

* Move stepper to be alphabetical

* When max value is reached, disabled the + button

* Show placeholder if you delete all numbers

* Make all the html input values available to the component

* Adds size variants

* Move stepper into its own component

* Work on showing the extra concurrency

* The purchase form styling and functionality (minus actually purchasing)

* New style for outline input fields

* Concurrency purchasing working

* Purchasing concurrency and quota emails working

* Improvements to the modal

* Show cost breakdown in the modal

* Fix for allocated concurrency including DEV

* Improved types

* Allocating concurrency is working

* Live updates total env concurrency

* Implemented reset

* Fix for concurrency allocation editing across multiple projects

* Tabular numbers

* Added an error from allocating concurrency

* Fixes for allocating concurrency where it didn't calculate correctly

* "Increase limit" link to concurrency page

* Indent environments

* Added Preview limit when updating concurrency for an org

* Show error when changing plan fails

* Added maximumProjectCount column to Org

* Limit project count and display a rich error toast (with title and button now)

* Added title and button to toasts. Use it for new project error

* @trigger.dev/platform 1.0.20

* Allow submitting zero concurrency so you can downgrade back to nothing

* Use the server as the truth for omitted environments

* Updated the pricing panels

---------

Co-authored-by: James Ritchie <james@trigger.dev>
2025-11-19 11:22:23 +00:00
Eric Allam 3af7303156 feat(docs): update prismaExtension for new mode functionality in 4.1.1 (#2690) 2025-11-19 11:16:35 +00:00
github-actions[bot] 6231ddc67a Release v4.1.1
* chore: Update version for release

* chore: Update pnpm-lock.yaml (#2694)

* Initial plan

* Update pnpm-lock.yaml

Co-authored-by: ericallam <534+ericallam@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: ericallam <534+ericallam@users.noreply.github.com>

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: ericallam <534+ericallam@users.noreply.github.com>
2025-11-19 11:09:26 +00:00
Eric Allam 15fef916f6 feat(build): update prisma extension to work with generated clients and rust-free clients (#2689)
* prisma extension fixes WIP

* More prisma stuff

* more prisma stuff

* remove changelog

* upgrade github workflows to use node 20.19 because installing prisma@7 breaks with lower versions

* Don't use generate for the prisma reference projects

* make sure it works if no mode is passed in
2025-11-19 10:48:15 +00:00
Eric Allam e2a703bfa8 fix(dashboard): continuously apply log filter during live run (#2692) 2025-11-18 17:00:29 +00:00
Saadi Myftija 1a7ee24b9e fix(ui): refresh deployments logs stream on status change (#2688)
The logs stream is created after the deployment moves from `PENDING` status to `INSTALLING`.
2025-11-17 14:50:05 +01:00
Saadi Myftija bb99af52cb feat(deployments): expose native build server option in build settings (#2685)
Adds a build setting about using our build server for remote builds.
2025-11-17 12:46:57 +01:00
Saadi Myftija 01797a1668 feat(deployments): ECR repo adaptations to enable external build cache (#2684)
* Update aws sdk ecr client to the latest version

* Exlude the cache tag from the immutability enforcement

* Attach a policy to ECR repos to expire untagged images

* Fix filterType
2025-11-17 12:34:54 +01:00
Eric Allam 19fa669318 Release v4.1.0 (#2683) 2025-11-14 09:55:01 +00:00
github-actions[bot] 78fcba518a chore: Update version for release (#2666)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2025-11-14 09:52:22 +00:00
Saadi Myftija 53047ab648 feat(deployments): external cache support for local builds (#2682)
* Add registry cache support for local builds

* Add changeset
2025-11-14 10:46:31 +01:00
Eric Allam bb5cefa92c docs: realtime streams v2 (#2673) 2025-11-14 09:42:20 +00:00
Eric Allam b8b198579c chore(rules): upgrade rules for 4.1.0 (#2676) 2025-11-14 09:42:15 +00:00
Eric Allam 892bed8c4c Upgrade to electricsql 1.2.4 (#2668) 2025-11-13 15:19:59 +00:00
Eric Allam a94a11f44d feat(sdk): replace onStart lifecycle hook with onStartAttempt (#2515)
* fix(sdk): prevent uncaught errors thrown onSuccess, onComplete, and onFailure hooks to fail attempts & in some cases runs

* Add onStartAttempt hook and deprecate onSuccess

* Add onStartAttempt hook and deprecate onStart hook

* Fix onStartAttempt overload types

* Update lifecycle functions diagram
2025-11-13 14:51:13 +00:00
Matt Aitken f116e93e01 Docs: tags can be up to 128 chars (#2678) 2025-11-13 14:43:31 +00:00
Eric Allam 6137338da9 feat(streams): make v2 streams the default when using 4.1.0+ if they are supported (#2677) 2025-11-13 13:53:42 +00:00
Lindsey 8cec3b763b docs: add section on using .env.production and dotenvx for environmen… (#2674)
* docs: add section on using .env.production and dotenvx for environment variables

* fix: correct dotenvx API usage in documentation example
2025-11-13 11:28:26 +00:00
Eric Allam 343ba54c69 fix(streams): restore realtime stream writing for v3 tasks (#2675) 2025-11-13 11:21:48 +00:00
Eric Allam a70ab10809 fix(streams): fixed broken wrapping in streams inspector (#2672) 2025-11-12 16:00:54 +00:00
Eric Allam f7cb637b32 fix(streams): scope s2 access token to environment and fix streams v1 appends (#2670)
* fix(streams): scope s2 access token to environment and fix streams v1 appends

* Less stale time
2025-11-12 12:34:15 +00:00
Eric Allam 668559ec1a fix(streams): buffer v1 streams on read to prevent split chunks (#2669) 2025-11-11 21:08:49 +00:00
Eric Allam d0ad38d684 chore(docker): remove unused seed copy from dockerfile (#2667) 2025-11-11 15:13:10 +00:00
Eric Allam 536d9fa217 feat(realtime): Realtime streams v2 (#2632) 2025-11-11 14:54:00 +00:00
github-actions[bot] d75c3aeadd chore: Update version for release (#2665)
* chore: Update version for release

* Release 4.0.7

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Matt Aitken <matt@mattaitken.com>
2025-11-11 14:29:24 +00:00
Matt Aitken a342332146 Fix for the MCP tool that gets logs for debugging runs (#2653)
* Fix for the MCP tool that gets logs for debugging runs

This was broken when we changed the data on the backend that returns 
log/span data from runs. We changed the data structured and the internal
API that the MCP client uses was failing to parse with the Zod schema

* add changeset

* Revert "add changeset"

This reverts commit 86eca836d5907fa0d0f8ac595d4d5ebade140514.

---------

Co-authored-by: nicktrn <55853254+nicktrn@users.noreply.github.com>
2025-11-11 13:26:07 +00:00
Saadi Myftija 9624465ee2 fix: error handling issue with s2 streams (#2664)
* Fix logger import

*old man yells at auto-import*

* Fix s2 error handling for non-existing streams
2025-11-11 09:44:54 +01:00
Saadi Myftija 42f53b12b7 feat(deployments): stream build server logs (#2663)
* Use read-only project-scoped s2 tokens for streaming deployment logs

* Add http2 to remix polyfills

Needed for using s2 client-side.

* Stream build-server logs in the deployment details page

* Disable 12-hour format in the DateTime component

* Enable collapsing the logs panel

* Auto-collapse logs for succesful/timedout/queued deployments

* Make S2 env vars optional

* Show the logs section only for gh-triggered deployments

* Cache s2 access tokens in redis

* Reset streaming state

* Expose 12h format as a param for the Datetime components
2025-11-10 12:42:40 +01:00
github-actions[bot] 9fdf91a1c4 Release v4.0.6 (#2647)
* chore: Update version for release

* Update lock file

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: myftija <saadi.myftija@gmail.com>
2025-11-03 14:42:02 +01:00
nicktrn 9593a46364 fix(extensions): prevent audiowaveform binary removal (#2643)
* fix(extensions): prevent audiowaveform binary removal

* add changeset
2025-11-03 14:17:38 +01:00
Saadi Myftija 27376df903 fix(cli): show a useful error message when config file is missing (#2650)
* fix(cli): show a useful error message when config file is missing

* Add changeset
2025-10-31 16:34:48 +01:00
Saadi Myftija e0cece72ae docs: deployments with local builds (#2645)
* docs: deployments with local builds

Adds a section to the deployment docs page about the option to deploy using a locally built image.

* Update installation link
2025-10-30 18:10:23 +01:00
Saadi Myftija 2f3f82f3a9 feat(deployments): show local build hint during depot outages (#2646)
* Add an API endpoint to query remote build provider status

* Show local build hint for failed deployments when Depot is down

* Show the local build flag in the help output

* Add changeset

* Fix import

* Fix docs link
2025-10-30 18:09:54 +01:00
Dan 4264bdf429 Added use cases section in the docs (#2641)
* Started adding use cases pages

* Updates

* More improvements

* Improved diagrams

* Added overview page

* More copy + diagram updates

* Improved diagram titles

* Further diagram improvements

* Corrected workflow

* Updated copy

* Typos

* Updated intro file

* Reverted aiRunFilterService.server.ts
2025-10-30 09:21:31 +00:00
nicktrn bf6735bd56 fix(webapp): concurrency override upper bound should be env not org (#2642) 2025-10-29 19:16:42 +00:00
James Ritchie dacd53b906 Fix for showing the incident UI in the side menu (#2638) 2025-10-28 21:13:23 +00:00
2260 changed files with 402995 additions and 39555 deletions
+16
View File
@@ -0,0 +1,16 @@
---
"@trigger.dev/sdk": patch
"@trigger.dev/core": patch
"@trigger.dev/build": patch
"trigger.dev": patch
---
Add Agent Skills for `chat.agent`. Drop a folder with a `SKILL.md` and any helper scripts/references next to your task code, register it with `skills.define({ id, path })`, and the CLI bundles it into the deploy image automatically — no `trigger.config.ts` changes. The agent gets a one-line summary in its system prompt and discovers full instructions on demand via `loadSkill`, with `bash` and `readFile` tools scoped per-skill (path-traversal guards, output caps, abort-signal propagation).
```ts
const pdfSkill = skills.define({ id: "pdf-extract", path: "./skills/pdf-extract" });
chat.skills.set([await pdfSkill.local()]);
```
Built on the [AI SDK cookbook pattern](https://ai-sdk.dev/cookbook/guides/agent-skills) — portable across providers. SDK + CLI only for now; dashboard-editable `SKILL.md` text is on the roadmap.
+52
View File
@@ -0,0 +1,52 @@
---
"@trigger.dev/sdk": minor
---
**AI Prompts** — define prompt templates as code alongside your tasks, version them on deploy, and override the text or model from the dashboard without redeploying. Prompts integrate with the Vercel AI SDK via `toAISDKTelemetry()` (links every generation span back to the prompt) and with `chat.agent` via `chat.prompt.set()` + `chat.toStreamTextOptions()`.
```ts
import { prompts } from "@trigger.dev/sdk";
import { generateText } from "ai";
import { openai } from "@ai-sdk/openai";
import { z } from "zod";
export const supportPrompt = prompts.define({
id: "customer-support",
model: "gpt-4o",
config: { temperature: 0.7 },
variables: z.object({
customerName: z.string(),
plan: z.string(),
issue: z.string(),
}),
content: `You are a support agent for Acme.
Customer: {{customerName}} ({{plan}} plan)
Issue: {{issue}}`,
});
const resolved = await supportPrompt.resolve({
customerName: "Alice",
plan: "Pro",
issue: "Can't access billing",
});
const result = await generateText({
model: openai(resolved.model ?? "gpt-4o"),
system: resolved.text,
prompt: "Can't access billing",
...resolved.toAISDKTelemetry(),
});
```
**What you get:**
- **Code-defined, deploy-versioned templates** — define with `prompts.define({ id, model, config, variables, content })`. Every deploy creates a new version visible in the dashboard. Mustache-style placeholders (`{{var}}`, `{{#cond}}...{{/cond}}`) with Zod / ArkType / Valibot-typed variables.
- **Dashboard overrides** — change a prompt's text or model from the dashboard without redeploying. Overrides take priority over the deployed "current" version and are environment-scoped (dev / staging / production independent).
- **Resolve API** — `prompt.resolve(vars, { version?, label? })` returns the compiled `text`, resolved `model`, `version`, and labels. Standalone `prompts.resolve<typeof handle>(slug, vars)` for cross-file resolution with full type inference on slug and variable shape.
- **AI SDK integration** — spread `resolved.toAISDKTelemetry({ ...extra })` into any `generateText` / `streamText` call and every generation span links to the prompt in the dashboard alongside its input variables, model, tokens, and cost.
- **`chat.agent` integration** — `chat.prompt.set(resolved)` stores the resolved prompt run-scoped; `chat.toStreamTextOptions({ registry })` pulls `system`, `model` (resolved via the AI SDK provider registry), `temperature` / `maxTokens` / etc., and telemetry into a single spread for `streamText`.
- **Management SDK** — `prompts.list()`, `prompts.versions(slug)`, `prompts.promote(slug, version)`, `prompts.createOverride(slug, body)`, `prompts.updateOverride(slug, body)`, `prompts.removeOverride(slug)`, `prompts.reactivateOverride(slug, version)`.
- **Dashboard** — prompts list with per-prompt usage sparklines; per-prompt detail with Template / Details / Versions / Generations / Metrics tabs. AI generation spans get a custom inspector showing the linked prompt's metadata, input variables, and template content alongside model, tokens, cost, and the message thread.
See [/docs/ai/prompts](https://trigger.dev/docs/ai/prompts) for the full reference — template syntax, version resolution order, override workflow, and type utilities (`PromptHandle`, `PromptIdentifier`, `PromptVariables`).
+15
View File
@@ -0,0 +1,15 @@
---
"@trigger.dev/sdk": patch
---
Add `ai.toolExecute(task)` so you can wire a Trigger subtask in as the `execute` handler of an AI SDK `tool()` while defining `description` and `inputSchema` yourself — useful when you want full control over the tool surface and just need Trigger's subtask machinery for the body.
```ts
const myTool = tool({
description: "...",
inputSchema: z.object({ ... }),
execute: ai.toolExecute(mySubtask),
});
```
`ai.tool(task)` (`toolFromTask`) keeps doing the all-in-one wrap and now aligns its return type with AI SDK's `ToolSet`. Minimum `ai` peer raised to `^6.0.116` to avoid cross-version `ToolSet` mismatches in monorepos.
+5
View File
@@ -0,0 +1,5 @@
---
"trigger.dev": patch
---
Fix `chat.agent` skills silently missing in `trigger dev` for projects whose task files read `process.env` at module top level (e.g. a third-party SDK client initialized at import). Skill folders now bundle into `.trigger/skills/` reliably regardless of which env vars are set when the CLI launches.
+5
View File
@@ -0,0 +1,5 @@
---
"@trigger.dev/core": patch
---
Reject overlong `idempotencyKey` values at the API boundary so they no longer trip an internal size limit on the underlying unique index and surface as a generic 500. Inputs are capped at 2048 characters — well above what `idempotencyKeys.create()` produces (a 64-character hash) and above any realistic raw key. Applies to `tasks.trigger`, `tasks.batchTrigger`, `batch.create` (Phase 1 streaming batches), `wait.createToken`, `wait.forDuration`, and the input/session stream waitpoint endpoints. Over-limit requests now return a structured 400 instead.
+21
View File
@@ -0,0 +1,21 @@
---
"@trigger.dev/sdk": minor
---
Adds `onBoot` to `chat.agent` — a lifecycle hook that fires once per worker process picking up the chat. Runs for the initial run, preloaded runs, AND reactive continuation runs (post-cancel, crash, `endRun`, `requestUpgrade`, OOM retry), before any other hook. Use it to initialize `chat.local`, open per-process resources, or re-hydrate state from your DB on continuation — anywhere the SAME run picking up after suspend/resume isn't enough.
```ts
const userContext = chat.local<{ name: string; plan: string }>({ id: "userContext" });
export const myChat = chat.agent({
id: "my-chat",
onBoot: async ({ clientData, continuation }) => {
const user = await db.user.findUnique({ where: { id: clientData.userId } });
userContext.init({ name: user.name, plan: user.plan });
},
run: async ({ messages, signal }) =>
streamText({ model: openai("gpt-4o"), messages, abortSignal: signal }),
});
```
Use `onBoot` (not `onChatStart`) for state setup that must run every time a worker picks up the chat — `onChatStart` fires once per chat and won't run on continuation, leaving `chat.local` uninitialized when `run()` tries to use it.
+15
View File
@@ -0,0 +1,15 @@
---
"@trigger.dev/sdk": patch
---
Add a `tools` option to `chat.agent`. Declaring your tools here threads them into the SDK's internal `convertToModelMessages`, so each tool's `toModelOutput` is re-applied when prior-turn history is re-converted.
```ts
chat.agent({
tools: { readFile, search },
run: async ({ messages, tools, signal }) =>
streamText({ model, messages, tools, abortSignal: signal }),
});
```
Also exports `InferChatUIMessageFromTools<typeof tools>` to derive the chat `UIMessage` type (typed tool parts) directly from a tool set.
+44
View File
@@ -0,0 +1,44 @@
---
"@trigger.dev/sdk": minor
"@trigger.dev/core": patch
---
**AI Agents** — run AI SDK chat completions as durable Trigger.dev agents instead of fragile API routes. Define an agent in one function, point `useChat` at it from React, and the conversation survives page refreshes, network blips, and process restarts.
```ts
import { chat } from "@trigger.dev/sdk/ai";
import { streamText } from "ai";
import { openai } from "@ai-sdk/openai";
export const myChat = chat.agent({
id: "my-chat",
run: async ({ messages, signal }) =>
streamText({ model: openai("gpt-4o"), messages, abortSignal: signal }),
});
```
```tsx
import { useChat } from "@ai-sdk/react";
import { useTriggerChatTransport } from "@trigger.dev/sdk/chat/react";
const transport = useTriggerChatTransport({ task: "my-chat", accessToken, startSession });
const { messages, sendMessage } = useChat({ transport });
```
**What you get:**
- **AI SDK `useChat` integration** — a custom [`ChatTransport`](https://sdk.vercel.ai/docs/ai-sdk-ui/transport) (`useTriggerChatTransport`) plugs straight into Vercel AI SDK's `useChat` hook. Text streaming, tool calls, reasoning, and `data-*` parts all work natively over Trigger.dev's realtime streams. No custom API routes needed.
- **First-turn fast path (`chat.headStart`)** — opt-in handler that runs the first turn's `streamText` step in your warm server process while the agent run boots in parallel, cutting cold-start TTFC by roughly half (measured 2801ms → 1218ms on `claude-sonnet-4-6`). The agent owns step 2+ (tool execution, persistence, hooks) so heavy deps stay where they belong. Web Fetch handler works natively in Next.js, Hono, SvelteKit, Remix, Workers, etc.; bridge to Express/Fastify/Koa via `chat.toNodeListener`. New `@trigger.dev/sdk/chat-server` subpath.
- **Multi-turn durability via Sessions** — every chat is backed by a durable Session that outlives any individual run. Conversations resume across page refreshes, idle timeout, crashes, and deploys; `resume: true` reconnects via `lastEventId` so clients only see new chunks. `sessions.list` enumerates chats for inbox-style UIs.
- **Auto-accumulated history, delta-only wire** — the backend accumulates the full conversation across turns; clients only ship the new message each turn. Long chats never hit the 512 KiB body cap. Register `hydrateMessages` to be the source of truth yourself.
- **Lifecycle hooks** — `onPreload`, `onChatStart`, `onValidateMessages`, `hydrateMessages`, `onTurnStart`, `onBeforeTurnComplete`, `onTurnComplete`, `onChatSuspend`, `onChatResume` — for persistence, validation, and post-turn work.
- **Stop generation** — client-driven `transport.stopGeneration(chatId)` aborts mid-stream; the run stays alive for the next message, partial response is captured, and aborted parts (stuck `partial-call` tools, in-progress reasoning) are auto-cleaned.
- **Tool approvals (HITL)** — tools with `needsApproval: true` pause until the user approves or denies via `addToolApprovalResponse`. The runtime reconciles the updated assistant message by ID and continues `streamText`.
- **Steering and background injection** — `pendingMessages` injects user messages between tool-call steps so users can steer the agent mid-execution; `chat.inject()` + `chat.defer()` adds context from background work (self-review, RAG, safety checks) between turns.
- **Actions** — non-turn frontend commands (undo, rollback, regenerate, edit) sent via `transport.sendAction`. Fire `hydrateMessages` + `onAction` only — no turn hooks, no `run()`. `onAction` can return a `StreamTextResult` for a model response, or `void` for side-effect-only.
- **Typed state primitives** — `chat.local<T>` for per-run state accessible from hooks, `run()`, tools, and subtasks (auto-serialized through `ai.toolExecute`); `chat.store` for typed shared data between agent and client; `chat.history` for reading and mutating the message chain; `clientDataSchema` for typed `clientData` in every hook.
- **`chat.toStreamTextOptions()`** — one spread into `streamText` wires up versioned system [Prompts](https://trigger.dev/docs/ai/prompts), model resolution, telemetry metadata, compaction, steering, and background injection.
- **Multi-tab coordination** — `multiTab: true` + `useMultiTabChat` prevents duplicate sends and syncs state across browser tabs via `BroadcastChannel`. Non-active tabs go read-only with live updates.
- **Network resilience** — built-in indefinite retry with bounded backoff, reconnect on `online` / tab refocus / bfcache restore, `Last-Event-ID` mid-stream resume. No app code needed.
See [/docs/ai-chat](https://trigger.dev/docs/ai-chat/overview) for the full surface — quick start, three backend approaches (`chat.agent`, `chat.createSession`, raw task), persistence and code-sandbox patterns, type-level guides, and API reference.
@@ -0,0 +1,21 @@
---
"@trigger.dev/sdk": minor
---
Add read primitives to `chat.history` for HITL flows: `getPendingToolCalls()`, `getResolvedToolCalls()`, `extractNewToolResults(message)`, `getChain()`, and `findMessage(messageId)`. These lift the accumulator-walking logic that customers building human-in-the-loop tools were re-implementing into the SDK.
Use `getPendingToolCalls()` to gate fresh user turns while a tool call is awaiting an answer. Use `extractNewToolResults(message)` to dedup tool results when persisting to your own store — the helper returns only the parts whose `toolCallId` is not already resolved on the chain.
```ts
const pending = chat.history.getPendingToolCalls();
if (pending.length > 0) {
// an addToolOutput is expected before a new user message
}
onTurnComplete: async ({ responseMessage }) => {
const newResults = chat.history.extractNewToolResults(responseMessage);
for (const r of newResults) {
await db.toolResults.upsert({ id: r.toolCallId, output: r.output, errorText: r.errorText });
}
};
```
+6
View File
@@ -0,0 +1,6 @@
---
"@trigger.dev/sdk": patch
"@trigger.dev/core": patch
---
Stamp `gen_ai.conversation.id` (the chat id) on every span and metric emitted from inside a `chat.task` or `chat.agent` run. Lets you filter dashboard spans, runs, and metrics by the chat conversation that produced them — independent of the run boundary, so multi-run chats correlate cleanly. No code changes required on the user side.
+31
View File
@@ -0,0 +1,31 @@
---
"@trigger.dev/sdk": patch
---
Fix `chat.agent` HITL continuations on reasoning-heavy turns. Two changes that work together:
- The per-turn merge now overlays the wire copy's tool-part state advancement onto the agent's existing chain — `state` + the matching resolution field (`output` / `errorText` / `approval`) come from the wire, everything else (text, reasoning, tool `input`, provider metadata) stays whatever the snapshot or `hydrateMessages` returned. Previously a full-message replace overwrote those fields with whatever the client shipped, so a slimmed wire copy landed a tool call with no `arguments` on the next LLM call. Covers `output-available` / `output-error` (HITL `addToolOutput`) and `approval-responded` / `output-denied` (approval flow).
- `TriggerChatTransport.sendMessages` and `AgentChat.sendRaw` now slim assistant messages that carry advanced tool parts. The wire payload is just `{ id, role, parts: [<state + resolution field>] }` for `submit-message` continuations; everything else passes through. Reasoning blobs and full tool inputs no longer ride the wire on every `addToolOutput` / `addToolApproveResponse`, so continuation payloads stay well under the `.in/append` cap on long agent loops.
Note: `onValidateMessages` receives the slim wire on HITL turns. If you call `validateUIMessages` from `ai` against the full `messages` array it will reject the slim assistant; filter to user messages (or skip on HITL turns) — see the updated docstring on `onValidateMessages` for the recommended pattern.
For `hydrateMessages` hooks that persist the chain, this release also adds a small helper to the `@trigger.dev/sdk/ai` surface:
```ts
import { chat, upsertIncomingMessage } from "@trigger.dev/sdk/ai";
chat.agent({
hydrateMessages: async ({ chatId, trigger, incomingMessages }) => {
const record = await db.chat.findUnique({ where: { id: chatId } });
const stored = record?.messages ?? [];
if (upsertIncomingMessage(stored, { trigger, incomingMessages })) {
await db.chat.update({ where: { id: chatId }, data: { messages: stored } });
}
return stored;
},
});
```
It pushes fresh user messages by id, no-ops on HITL continuations (the incoming shares an id with the existing assistant — the runtime overlays the new tool-state advance), and skips on non-`submit-message` triggers. Returns `true` if it mutated `stored` so the caller knows whether to persist.
Net effect: `chat.addToolOutput(...)` / `chat.addToolApproveResponse(...)` on multi-step reasoning agents (OpenAI Responses with `store: false`, Anthropic extended thinking, etc.) no longer blows the cap and no longer corrupts the LLM input.
@@ -0,0 +1,22 @@
---
"@trigger.dev/sdk": patch
---
Type `chat.createStartSessionAction` against your chat agent so `clientData` is typed end-to-end on the first turn:
```ts
import { chat } from "@trigger.dev/sdk/ai";
import type { myChat } from "@/trigger/chat";
export const startChatSession = chat.createStartSessionAction<typeof myChat>("my-chat");
// In the browser, threaded from the transport's typed startSession callback:
const transport = useTriggerChatTransport<typeof myChat>({
task: "my-chat",
startSession: ({ chatId, clientData }) =>
startChatSession({ chatId, clientData }),
// ...
});
```
`ChatStartSessionParams` gains a typed `clientData` field — folded into the first run's `payload.metadata` so `onPreload` / `onChatStart` see the same shape per-turn `metadata` carries via the transport. The opaque session-level `metadata` field is unchanged.
@@ -0,0 +1,5 @@
---
"trigger.dev": patch
---
Add `TRIGGER_BUILD_SKIP_REWRITE_TIMESTAMP=1` escape hatch for local self-hosted builds whose buildx driver doesn't support `rewrite-timestamp` alongside push (e.g. orbstack's default `docker` driver).
@@ -0,0 +1,5 @@
---
"@trigger.dev/core": patch
---
Coerce numeric `concurrencyKey` values to string at the API boundary across `tasks.trigger`, `tasks.batchTrigger`, and the Phase-2 streaming batch endpoint.
@@ -0,0 +1,5 @@
---
"@trigger.dev/core": patch
---
Fix `LocalsKey<T>` type incompatibility across dual-package builds. The phantom value-type brand no longer uses a module-level `unique symbol`, so a single TypeScript compilation that resolves the type from both the ESM and CJS outputs (which can happen under certain pnpm hoisting layouts) no longer sees two structurally-incompatible variants of the same type.
+5
View File
@@ -0,0 +1,5 @@
---
"trigger.dev": patch
---
The CLI MCP server's agent-chat tools (`start_agent_chat`, `send_agent_message`, `close_agent_chat`) now run on the new Sessions primitive, so AI assistants driving a `chat.agent` get the same idempotent-by-`chatId`, durable-across-runs behavior the browser transport gets. Required PAT scopes go from `write:inputStreams` to `read:sessions` + `write:sessions`.
+5
View File
@@ -0,0 +1,5 @@
---
"trigger.dev": patch
---
MCP `list_runs` tool: add a `region` filter input and surface each run's executing region in the formatted summary.
@@ -0,0 +1,8 @@
---
"@trigger.dev/sdk": patch
"@trigger.dev/core": patch
---
Unit-test `chat.agent` definitions offline with `mockChatAgent` from `@trigger.dev/sdk/ai/test`. Drives a real agent's turn loop in-process — no network, no task runtime — so you can send messages, actions, and stop signals via driver methods, inspect captured output chunks, and verify hooks fire. Pairs with `MockLanguageModelV3` from `ai/test` for model mocking. `setupLocals` lets you pre-seed `locals` (DB clients, service stubs) before `run()` starts.
The broader `runInMockTaskContext` harness it's built on lives at `@trigger.dev/core/v3/test` — useful for unit-testing any task code, not just chat.
@@ -0,0 +1,5 @@
---
"@trigger.dev/redis-worker": minor
---
Mollifier buffer extensions: idempotency dedup, an atomic `mutateSnapshot` API, metadata CAS, claim primitives, and a `MollifierSnapshot` type. The buffer's Redis client now reconnects with jittered backoff so a fleet of clients doesn't stampede Redis in lockstep after a blip.
@@ -0,0 +1,5 @@
---
"@trigger.dev/redis-worker": patch
---
Pipeline the per-entry `HGETALL` fetches in `MollifierBuffer.listEntriesForEnv`. The previous serial implementation issued one Redis round-trip per runId returned by `LRANGE`, which dominated stale-sweep wall-time at any meaningful backlog (at the sweep's default maxCount=1000, this is ~1000 RTTs per env per pass). Behaviour is unchanged — entries are still skipped when the entry hash has been torn down by a concurrent drainer ack/fail between the LRANGE and the HGETALL.
@@ -0,0 +1,5 @@
---
"@trigger.dev/redis-worker": minor
---
Add `onTerminalFailure` callback to `MollifierDrainerOptions` so the customer's run lands a SYSTEM_FAILURE PG row even when the drainer exhausts `maxAttempts` on a retryable PG error. Previously, retryable-error exhaustion called `buffer.fail()` directly, which atomically marks FAILED + DELs the entry hash with no PG write — silent data loss when PG was unreachable across the full retry budget. The callback fires before `buffer.fail()` on any terminal path (`cause: "non-retryable"` or `"max-attempts-exhausted"`); throwing a retryable error from the callback causes the drainer to requeue rather than fail.
@@ -0,0 +1,9 @@
---
"@trigger.dev/redis-worker": patch
---
Add MollifierBuffer and MollifierDrainer primitives for trigger burst smoothing.
MollifierBuffer (`accept`, `pop`, `ack`, `requeue`, `fail`, `evaluateTrip`) is a per-env FIFO over Redis with atomic Lua transitions for status tracking. `evaluateTrip` is a sliding-window trip evaluator the webapp gate uses to detect per-env trigger bursts.
MollifierDrainer pops entries through a polling loop with a user-supplied handler. The loop survives transient Redis errors via capped exponential backoff (up to 5s), and per-env pop failures don't poison the rest of the batch — one env's blip is logged and counted as failed for that tick. Rotation is two-level: orgs at the top, envs within each org. The buffer maintains `mollifier:orgs` and `mollifier:org-envs:${orgId}` atomically with per-env queues, so the drainer walks orgs → envs directly without an in-memory cache. The `maxOrgsPerTick` option (default 500) caps how many orgs are scheduled per tick; for each picked org, one env is popped (rotating round-robin within the org). An org with N envs gets the same per-tick scheduling slot as an org with 1 env, so tenant-level drainage throughput is determined by org count rather than env count.
+5
View File
@@ -0,0 +1,5 @@
---
"@trigger.dev/redis-worker": patch
---
Mollifier `mutateSnapshot` now enforces a tag cap: an `append_tags` patch carrying `maxTags` returns `"limit_exceeded"` (writing nothing) when the deduped tag count would exceed the limit, so a buffered run can't accumulate more tags via the tags API than the trigger validator allows at creation.
+5
View File
@@ -0,0 +1,5 @@
---
"@trigger.dev/plugins": patch
---
The public interfaces for a plugin system. Initially consolidated authentication and authorization interfaces.
+56
View File
@@ -0,0 +1,56 @@
{
"mode": "pre",
"tag": "rc",
"initialVersions": {
"coordinator": "0.0.1",
"docker-provider": "0.0.1",
"kubernetes-provider": "0.0.1",
"supervisor": "0.0.1",
"webapp": "1.0.0",
"@trigger.dev/build": "4.4.6",
"trigger.dev": "4.4.6",
"@trigger.dev/core": "4.4.6",
"@trigger.dev/plugins": "4.4.6",
"@trigger.dev/python": "4.4.6",
"@trigger.dev/react-hooks": "4.4.6",
"@trigger.dev/redis-worker": "4.4.6",
"@trigger.dev/rsc": "4.4.6",
"@trigger.dev/schema-to-json": "4.4.6",
"@trigger.dev/sdk": "4.4.6"
},
"changesets": [
"agent-skills",
"ai-prompts",
"ai-tool-helpers",
"bundle-skills-single-pass",
"cap-idempotency-key-length",
"chat-agent-on-boot-hook",
"chat-agent-tools",
"chat-agent",
"chat-history-read-primitives",
"chat-session-attributes",
"chat-slim-wire-merge",
"chat-start-session-action-typed-client-data",
"cli-deploy-skip-rewrite-timestamp",
"coerce-concurrency-key-to-string",
"locals-key-dual-package-fix",
"mcp-agent-chat-sessions",
"mcp-list-runs-region",
"mock-chat-agent-test-harness",
"mollifier-buffer-extensions",
"mollifier-buffer-pipeline-list-entries",
"mollifier-drainer-terminal-failure-callback",
"mollifier-redis-worker-primitives",
"mollifier-tag-cap",
"plugin-auth-path",
"resource-catalog-runtime-registration",
"retry-middleware-errors",
"retry-sigsegv",
"runs-list-region-filter",
"s2-batch-transform-linger-fix",
"sessions-primitive",
"trigger-client",
"unflatten-attributes-conflict",
"warm-start-external-trace-context-leak"
]
}
@@ -0,0 +1,6 @@
---
"@trigger.dev/core": patch
"trigger.dev": patch
---
Fix `COULD_NOT_FIND_EXECUTOR` when a task's definition is loaded via `await import(...)` from inside another task's `run()`. The runtime workers now register such tasks with a sentinel file context, and the catalog logs a one-time warning per task id.
+5
View File
@@ -0,0 +1,5 @@
---
"@trigger.dev/core": patch
---
Retry `TASK_MIDDLEWARE_ERROR` under the task's retry policy instead of failing the run on the first attempt. The error was already classified as retryable by `shouldRetryError`, but `shouldLookupRetrySettings` did not include it, so the retry flow fell through to `fail_run`. Fixes #3231.
+5
View File
@@ -0,0 +1,5 @@
---
"@trigger.dev/core": patch
---
Retry `TASK_PROCESS_SIGSEGV` task crashes under the user's retry policy instead of failing the run on the first segfault. SIGSEGV in Node tasks is frequently non-deterministic (native addon races, JIT/GC interaction, near-OOM in native code, host issues), so retrying on a fresh process often succeeds. The retry is gated by the task's existing `retry` config + `maxAttempts` — same path `TASK_PROCESS_SIGTERM` and uncaught exceptions already use — so tasks without a retry policy still fail fast.
+6
View File
@@ -0,0 +1,6 @@
---
"@trigger.dev/core": patch
"@trigger.dev/sdk": patch
---
Add `region` to the runs list / retrieve API: filter runs by region (`runs.list({ region: "..." })` / `filter[region]=<masterQueue>`) and read each run's executing region from the new `region` field on the response.
@@ -0,0 +1,6 @@
---
"@trigger.dev/core": patch
"trigger.dev": patch
---
Bump `@s2-dev/streamstore` to `0.22.10` to fix a `TASK_RUN_UNCAUGHT_EXCEPTION` ("Invalid state: Unable to enqueue") when a `chat.agent` turn is aborted mid-stream.
+26
View File
@@ -0,0 +1,26 @@
---
"@trigger.dev/sdk": minor
"@trigger.dev/core": patch
---
**Sessions** — a durable, run-aware stream channel keyed on a stable `externalId`. A Session is the unit of state that owns a multi-run conversation: messages flow through `.in`, responses through `.out`, both survive run boundaries. Sessions back the new `chat.agent` runtime, and you can build on them directly for any pattern that needs durable bi-directional streaming across runs.
```ts
import { sessions, tasks } from "@trigger.dev/sdk";
// Trigger a task and subscribe to its session output in one call
const { runId, stream } = await tasks.triggerAndSubscribe("my-task", payload, {
externalId: "user-456",
});
for await (const chunk of stream) {
// ...
}
// Enumerate existing sessions (powers inbox-style UIs without a separate index)
for await (const s of sessions.list({ type: "chat.agent", tag: "user:user-456" })) {
console.log(s.id, s.externalId, s.createdAt, s.closedAt);
}
```
See [/docs/ai-chat/overview](https://trigger.dev/docs/ai-chat/overview) for the full surface — Sessions powers the durable, resumable chat runtime described there.
+18
View File
@@ -0,0 +1,18 @@
---
"@trigger.dev/sdk": patch
---
Add `TriggerClient` for running multiple SDK clients side-by-side, each with its own auth, preview branch, and baseURL. Useful when a single process needs to trigger tasks or read runs across multiple projects, environments, or preview branches without mutating shared global state.
```ts
import { TriggerClient } from "@trigger.dev/sdk";
const prod = new TriggerClient({ accessToken: process.env.TRIGGER_PROD_KEY });
const preview = new TriggerClient({
accessToken: process.env.TRIGGER_PREVIEW_KEY,
previewBranch: "signup-flow",
});
await prod.tasks.trigger("send-email", payload);
await preview.runs.list({ status: ["COMPLETED"] });
```
@@ -0,0 +1,5 @@
---
"@trigger.dev/core": patch
---
Fix `TypeError` in `unflattenAttributes` when the input attribute map contains conflicting dotted key paths (e.g. both `a.b` set to a scalar and `a.b.c` set to a value). The path-walk loop now applies last-write-wins when a prior key wrote a primitive, null, or array at an intermediate slot, matching the existing precedent in `AttributeFlattener.addAttribute`. Callers no longer crash when handed malformed external attribute inputs.
@@ -0,0 +1,5 @@
---
"@trigger.dev/core": patch
---
Fix external trace context leaking across runs on warm-started workers with `processKeepAlive` enabled. Every subsequent run's attempt span was being exported with the first run's `traceId` and `parentSpanId`, breaking causal-chain navigation in external APM tools. Runs without an external trace context are unaffected.
+50
View File
@@ -0,0 +1,50 @@
# REVIEW.md — Trigger.dev OSS
Repo-specific signal for anyone (human or agent) reviewing a PR in this codebase. Calibrates what counts as critical, what to always check, and what to skip.
## What makes a 🔴 Important finding here
Reserve 🔴 for things that would page someone or block a rollback. In this codebase, that means:
- **Rolling-deploy breakage.** Old and new versions of the webapp/supervisor run side-by-side during deploys. A change is broken if:
- A Lua script's behavior changes for a given key set without versioning (rename the script with a behavior-descriptive suffix like `Tracked` rather than `V2` — both versions must coexist safely).
- A Redis data shape used by both versions changes in place. New shapes need a new key namespace.
- A migration is not backward-compatible with the prior image.
- **Schema / migration safety.** Prisma migrations must be backward-compatible with the prior deploy. Adding NOT NULL without a default, dropping a column an old image still reads, renaming a column — all 🔴.
- **ClickHouse migration ordering + idempotency.** Goose runs in strict mode in the deploy pipeline and refuses to apply a missing version below the current version — slotting a new file in below the latest already-applied version blocks the deploy. New ClickHouse migration files MUST use the next available number (`max(files in internal-packages/clickhouse/schema/) + 1`); if main has added migrations while you've been on a branch, renumber yours. DDL must also be idempotent (`ADD COLUMN IF NOT EXISTS`, `DROP COLUMN IF EXISTS`, `CREATE TABLE IF NOT EXISTS`, `ADD INDEX IF NOT EXISTS`) so a partial / `--allow-missing` apply elsewhere doesn't fail on retry. Either fault is 🔴 — both break test/prod deploys. Rules live in `internal-packages/clickhouse/CLAUDE.md`.
- **Queue / concurrency correctness.** RunQueue, MarQS (V1, legacy), redis-worker — any change to enqueue / dequeue / locking semantics. Re-derive the invariant on paper before flagging or accepting.
- **Missing index on a hot table.** New Prisma queries against `TaskRun`, `TaskRunExecutionSnapshot`, `JobRun`, `Project`, etc. must use an existing index. Check `internal-packages/database/prisma/schema.prisma` for the relevant `@@index` lines — don't guess and don't propose `EXPLAIN`.
- **Recovery-path queries.** Any `TaskRun.findFirst` / `findMany` added to a schedule, run-recovery, or restart loop. Recovery fan-outs (Redis crash, restart storms) turn "rare indexed query" into a DB incident. 🔴 even if indexed.
- **Aggregations on hot tables.** No `COUNT` / `GROUP BY` on `TaskRun` or other multi-million-row tables. Use Redis or ClickHouse for counts.
- **Prod Redis blast-radius.** New code paths that `SCAN` with broad patterns (`*foo*`) on prod-shaped Redis, or `EVAL` Lua with `SCAN` loops inside. Both are 🔴.
- **`@trigger.dev/core` direct import** from anywhere outside the SDK package. Always import from `@trigger.dev/sdk`. Core direct imports are 🔴 — they break the public API contract.
- **Heavy execute-deps imported into request-handler bundles.** Specifically `chat.handover` and similar split-bundle entry points must not transitively import the agent task's execute path. Watch for new imports added at module top-level of route files.
- **V1 engine code modified in a "V2 only" PR.** The `apps/webapp/app/v3/` directory contains both. If the PR description says V2-only but it touches `triggerTaskV1`, `cancelTaskRunV1`, `MarQS`, etc. — 🔴.
## Always check
- **Tests use testcontainers, not mocks.** Vitest with `redisTest` / `postgresTest` / `containerTest` from `@internal/testcontainers`. Any new `vi.mock(...)` on Redis, Postgres, BullMQ, or other infra is wrong here — 🔴 if added in production-path tests, 🟡 if isolated unit test.
- **Public-package changes have a changeset.** `pnpm run changeset:add` produces `.changeset/*.md`. Required for any edit under `packages/*`. Missing → 🟡; missing on a breaking change → 🔴.
- **Server-only changes have `.server-changes/*.md`.** Required for `apps/webapp/`, `apps/supervisor/` edits with no public-package change. Body should be 1-2 sentences (it has to fit as one bullet in a future changelog). Missing → 🟡.
- **Lua script naming.** Coexisting scripts use behavior-descriptive suffixes (`Tracked`), never `V2`. Old name must keep working until the next deploy clears it.
- **RunQueue payload shape.** V2 run-queue payload's `projectId` is consumed by `workerQueueResolver` for override matching. If a PR drops it from the payload, 🔴.
- **`safeSend` scope.** Defensive IPC wrappers belong on loop / interval / handler contexts, not one-shot terminal sends. If the PR adds `safeSend` to a single terminal call for consistency, 🟡 with a "remove this" suggestion.
- **Zod version.** Pinned to `3.25.76` monorepo-wide. New package adding zod with a different version or range — 🔴.
## Skip (do NOT flag)
- Anything Prettier / ESLint catches. CI runs both.
- TypeScript style preferences (`type` vs `interface`) — already covered by repo standards.
- Test coverage exhortations as a generic suggestion. Only flag missing tests when a specific code path is genuinely untested and the path has prior incidents.
- `agentcrumbs` markers (`// @crumbs`, `// #region @crumbs`) and `agentcrumbs` imports — these are temporary debug instrumentation stripped before merge.
- `// removed comments for removed code`, renamed `_unused` vars, re-exported types as "backwards compatibility shims" — also covered by repo standards.
- Suggestions to "add error handling" without naming a specific scenario that breaks.
- Documentation prose nitpicks in `docs/*` MDX files unless factually wrong.
## Things V1/legacy that should NOT block a PR
The `apps/webapp/app/v3/` directory name is misleading — most code there is V2. Only specific files are V1-only legacy: `MarQS` queue, `triggerTaskV1`, `cancelTaskRunV1`, and a handful of others (see `apps/webapp/CLAUDE.md` for the exact list). Don't flag "you should refactor this to use V2" on those — they're frozen.
## Confidence calibration for this repo
The most common false-positive pattern: speculating about race conditions in code paths the agent doesn't have runtime visibility into. If the only evidence is "this *could* race", drop it. If you can point to a specific interleaving with file:line for each step, surface it.
@@ -0,0 +1,287 @@
# Review guide — chat.agent on Sessions, row-agnostic addressing
Scope: the 12 uncommitted files. **No new behaviour beyond the public surface
already on this branch** — this is plumbing cleanup that:
1. Eliminates the transport's session-creation step
2. Makes `chatId` the universal addressing string everywhere
3. Makes the server-side stream/append/wait routes row-agnostic
## The two design moves
**Move 1 — agent owns session lifecycle.** `chat.agent` and
`chat.customAgent` upsert the backing `Session` row at bind, fire-and-forget,
keyed on `externalId = payload.chatId`. The transport, server-side
`AgentChat`, and `chat.createTriggerAction` no longer create sessions at all.
Browsers cannot mint sessions either (`POST /api/v1/sessions` is now
secret-key-only). One owner, one path.
**Move 2 — `chatId` is the only address.** The transport, server-side
`AgentChat`, JWT scopes, and S2 stream paths all use `chatId` directly. The
Session's friendlyId is informational. To make this safe, the three stream
routes (`.in/.out` PUT, GET, POST append, plus the run-engine `wait`
endpoint) became "row-optional" and derive a *canonical addressing key*
(`row.externalId ?? row.friendlyId`, fallback to the URL param when the row
hasn't been upserted yet). Same canonical key is used to build the S2 stream
path, the waitpoint cache key, and the JWT resource set — so any caller
addressing by either form converges on the same physical stream.
Together these remove an entire class of "did the row land yet?" races. The
transport can subscribe to `/sessions/{chatId}/out` before the agent boots,
the agent's `void sessions.create({externalId: chatId})` lands a moment
later, and any earlier reads/writes are already on the right S2 key.
---
## Read in this order
### 1. `apps/webapp/app/services/realtime/sessions.server.ts` (+34 lines)
The new primitive. Two helpers:
- `isSessionFriendlyIdForm(value)``value.startsWith("session_")`. Used to
decide whether a missing row is a hard 404 (opaque friendlyId) or a soft
"row will land later" (externalId form).
- `canonicalSessionAddressingKey(row, paramSession)` — `row.externalId ??
row.friendlyId` if the row exists, else `paramSession`. **This is the load-
bearing function.** Read its docstring.
**Question to ask:** can two callers addressing the "same" session ever get
different canonical keys? Only if the row exists for one and not the other,
*and* the URL forms differ — but in that case the row-less caller used the
externalId form (friendlyId-form would have 404'd earlier), and the row-ful
caller computes `row.externalId ?? row.friendlyId`. If the row's externalId
matches the URL, they converge. If it doesn't, there's no row to find by
that string anyway. The interesting edge is "row exists with no externalId",
addressed via friendlyId — both sides read `row.friendlyId`. ✓
### 2. `apps/webapp/app/routes/realtime.v1.sessions.$session.$io.ts` (+47/-12)
PUT initialize + GET subscribe (SSE). Both use the helper. The interesting
part is the loader's `findResource` + `authorization.resource`:
```ts
findResource: async (params, auth) => {
const row = await resolveSessionByIdOrExternalId(...);
if (!row && isSessionFriendlyIdForm(params.session)) return undefined; // 404
return { row, addressingKey: canonicalSessionAddressingKey(row, params.session) };
},
authorization: {
resource: ({ row, addressingKey }) => {
const ids = new Set<string>([addressingKey]);
if (row) {
ids.add(row.friendlyId);
if (row.externalId) ids.add(row.externalId);
}
return { sessions: [...ids] };
},
superScopes: ["read:sessions", "read:all", "admin"],
},
```
**Why three IDs in the resource set?** `checkAuthorization` is "any-match"
across the resource values. We want a JWT scoped to *either* form to
authorize *either* URL form. Smoke test verified the 4-cell matrix passes.
**The PUT path** (action handler) is simpler — it just resolves the row,
builds an addressing key, and hands it to `initializeSessionStream`. Worth
noting the `closedAt` check is now `maybeSession?.closedAt` — no row means
no closedAt to enforce.
### 3. `apps/webapp/app/routes/realtime.v1.sessions.$session.$io.append.ts` (+22/-13)
POST append (browser writes a record to `.in` or server writes to `.out`).
Same row-optional pattern. Both the S2 append and the waitpoint drain use
`addressingKey`.
**Question to ask:** what fires the waitpoint? An agent's
`session.in.wait()` registers a waitpoint keyed on `(addressingKey, io)` via
the wait endpoint (file 4). The append handler drains by the *same* key —
even if the agent registered with externalId form and the transport
appended via friendlyId form, both compute the same canonical key, so they
converge. ✓
### 4. `apps/webapp/app/routes/api.v1.runs.$runFriendlyId.session-streams.wait.ts` (+18/-13)
The agent's `.in.wait()` endpoint. Run-engine creates the waitpoint, then
registers it in Redis under `(addressingKey, io)`. The race-check that runs
right after creation reads from S2 by the same key. Three call sites —
`addSessionStreamWaitpoint`, `readSessionStreamRecords`,
`removeSessionStreamWaitpoint` — all consistent.
### 5. `apps/webapp/app/routes/api.v1.sessions.ts` (+4/-2)
**Security tightening.** Removed `allowJWT: true` and `corsStrategy: "all"`
from the `POST /api/v1/sessions` action — secret-key only now.
**Question to ask:** was the JWT path actually used? Until this branch, the
transport called it via `ensureSession` (now deleted). After this branch,
nobody reaches it from the browser. `chat.createTriggerAction` (server
secret key) is the only browser-adjacent path.
### 6. `packages/trigger-sdk/src/v3/ai.ts` (+62/-39)
Two near-identical edits — one in `chatAgent`, one in `chatCustomAgent`.
Both bind on `payload.chatId` and fire-and-forget the upsert:
```ts
locals.set(chatSessionHandleKey, sessions.open(payload.chatId));
void sessions
.create({ type: "chat.agent", externalId: payload.chatId })
.catch(() => { /* best effort */ });
```
**Question to ask:** why `void`-and-`catch`? Awaiting the upsert would gate
the agent's bind on a network round-trip that doesn't unblock anything
user-visible — `.in/.out` routes are row-agnostic and the waitpoint cache
is keyed on the addressing string, not the row id. If the upsert genuinely
fails, the next bind retries the same idempotent call (`sessions.create`
upserts on `externalId`, so concurrent triggers on one chatId converge to
one row). The row matters for downstream metadata + listing, not for live
addressing.
The PAT scope minting in `chatAgent` (two call sites — preload and
sendMessage) now uses `payload.chatId` for the `sessions:` resource. That
matches what the transport/AgentChat use as the JWT resource and what the
JWT's resource set in the loader includes. Cross-form addressing works
either way (smoke-tested), but using `chatId` keeps the chain tight.
`createChatTriggerAction` is the most visibly trimmed: no pre-create, no
threading `sessionId` into payload, scope mint uses `chatId`. Return type
no longer carries `sessionId` — note `TriggerChatTaskResult.sessionId` was
already declared optional, so this isn't a public-API break.
**Stale docstring to flag:** `chat.ts:59` and `chat.ts:112` still describe
PAT scopes as `read:sessions:{sessionId}` and
`write:sessions:{sessionId}`. Functionally either ID works (row lookup
canonicalises), but the doc text is now out of date — it should say
`{chatId}`. Worth a tidy-up before merge but not blocking.
### 7. `packages/trigger-sdk/src/v3/chat.ts` (+63/-117)
**The biggest mechanical edit.** Net -54 lines from deleting `ensureSession`
and untangling its callers.
What disappeared:
- `private async ensureSession(chatId)` — gone
- The "lazy upsert from the browser if no triggerTask callback" branch in
`sendMessages` and `preload` — gone
- The "throw if neither path surfaced a sessionId" guard — gone
- All `state.sessionId` URL params replaced with `chatId`
- `subscribeToSessionStream`'s `chatId?` (optional) is now `chatId` (required)
What stayed:
- `state.sessionId` in `ChatSessionState` — optional, informational
- The `restore from external storage` branch in the constructor still
hydrates `sessionId` if persisted, just doesn't *require* it
- `notifySessionChange` still surfaces `sessionId` if known
**Question to ask:** does the transport ever still need the friendlyId? The
only place is the `onSessionChange` callback's payload (so consumers
persisting state can save it for later display). The transport itself never
puts it in a URL or a waitpoint key.
The `sendMessages` path is worth re-reading: when state.runId is set, it
appends to `.in/append` and subscribes to `.out`. If the append fails with
a non-auth error, it falls through to triggering a new run (legacy "run is
dead" detection — unchanged from pre-Sessions, doesn't depend on
addressing).
### 8. `packages/trigger-sdk/src/v3/chat-client.ts` (+34/-33)
Server-side `AgentChat`. Mirrors the transport changes — every URL uses
`this.chatId`. `triggerNewRun` no longer pre-creates a session. `ChatSession`
and internal `SessionState` types now have optional `sessionId`.
The shape of the diff is identical to the transport: delete the upsert,
swap addressing identifiers, optionalise the friendlyId. If you've read
`chat.ts` carefully, this one is mostly mechanical confirmation that both
client surfaces (browser transport + server-side AgentChat) speak the same
addressing protocol.
### 9. Test infrastructure — `sessions.ts` (+18) + `mock-chat-agent.ts` (+25)
`__setSessionCreateImplForTests` mirrors the existing
`__setSessionOpenImplForTests`. `mockChatAgent` installs a no-op create stub
returning a synthetic `CreatedSessionResponseBody` so the agent's bind-time
`void sessions.create(...)` doesn't try to hit a real API. Cleanup runs in
the same `.finally` as the open override.
**Question to ask:** is the synthetic response shape correct? It mirrors
`CreatedSessionResponseBody` — `id`, `externalId`, `type`, `tags`,
`metadata`, `closedAt`, `closedReason`, `expiresAt`, `createdAt`,
`updatedAt`, `isCached`. Tests don't currently assert on this object, so
the bar is "doesn't crash + matches the type". Met.
### 10. `packages/trigger-sdk/src/v3/chat.test.ts` (+13/-12)
Three classes of test edits, all consequences:
- Stream URL assertion: `chat-1` (the chatId) instead of
`session_streamurl` (the friendlyId)
- `renewRunAccessToken` callback: `sessionId: undefined` (was
`DEFAULT_SESSION_ID` because the mocked trigger doesn't surface it)
- Token resolve count: `1` (was `2` — second resolve was for `ensureSession`)
- One `onSessionChange` matchObject loses `sessionId`
### 11. `apps/webapp/app/routes/_app.../playground/.../route.tsx` (1 line)
`sessionId: string` → `sessionId?: string` in the playground sidebar prop
to track the transport type change.
---
## Edge cases I checked, so you don't have to
- **Cross-form JWT auth (curl matrix).** JWT scoped to externalId can call
externalId URL ✓ and friendlyId URL ✓. JWT scoped to friendlyId can call
externalId URL ✓ and friendlyId URL ✓. Smoke-tested.
- **Row materialises after subscribe.** Transport opens
`GET /sessions/{chatId}/out` before agent's bind upsert lands → 200 OK,
`addressingKey = chatId` (paramSession fallback). Once the row lands
with `externalId = chatId`, addressingKey resolves to the same value via
`row.externalId`. Same S2 key throughout.
- **Concurrent triggers on one chatId.** Two browser tabs trigger two runs
→ two binds → two `sessions.create({externalId: chatId})` calls. Upsert
semantics: both return the same row.
- **Closed session enforcement.** Still enforced when a row exists.
`maybeSession?.closedAt` is null-safe; no row = no close-state to honour.
- **Agent run cancellation.** Frontend doesn't auto-detect — unchanged from
pre-Sessions; messages sit in S2 until the next trigger (the existing
run-PAT auth-error path is the only reaper). Out of scope for this branch.
- **Idle timeout in dev.** Runs stay `EXECUTING_WITH_WAITPOINTS` past the
configured idle because dev runs don't snapshot/restore; the in-process
idle clock advances locally without touching the row. Expected, not a
regression.
## Things explicitly **not** in this branch
- Run-state subscription on the transport side (the "run died, re-trigger
silently" UX gap)
- Session auto-close on agent exit (still client-driven by design)
- Any change to `Session` schema, `sessions.create` semantics, or
`chatAccessTokenTTL`
- Docstring updates for `read:sessions:{sessionId}` / `write:sessions:{sessionId}`
in `chat.ts:59` and `chat.ts:112` (functional but textually stale —
follow-up nit)
---
## What I'd be ready to answer cold
- Why fire-and-forget upsert (vs. `await`) in the agent's bind step
- Why the route's authorization resource set has three IDs (cross-form JWT
auth)
- Why `POST /api/v1/sessions` lost `allowJWT` (security tightening — no
caller needs it after the transport's `ensureSession` is gone)
- What converges two callers using different URL forms onto the same S2
stream (`canonicalSessionAddressingKey`, identical computation on both
sides for any given row)
- What makes `sessions.create` race-safe under concurrent triggers
(`externalId` upsert)
- Why `state.sessionId` stayed on `ChatSessionState` at all (pure
informational, surfaced via `onSessionChange` for consumer persistence;
zero addressing role)
- Why the chat-client (server-side AgentChat) and chat (transport) edits
look near-identical (they implement the same client protocol against the
same row-agnostic routes)
+13
View File
@@ -0,0 +1,13 @@
---
paths:
- "internal-packages/database/**"
---
# Database Migration Safety
- When adding indexes to **existing tables**, use `CREATE INDEX CONCURRENTLY IF NOT EXISTS` to avoid table locks. These must be in their own separate migration file (one index per file).
- Indexes on **newly created tables** (same migration as `CREATE TABLE`) do not need CONCURRENTLY.
- When indexing a **new column on an existing table**, split into two migrations: first `ADD COLUMN IF NOT EXISTS`, then `CREATE INDEX CONCURRENTLY IF NOT EXISTS` in a separate file.
- After generating a migration with Prisma, remove extraneous lines for: `_BackgroundWorkerToBackgroundWorkerFile`, `_BackgroundWorkerToTaskQueue`, `_TaskRunToTaskRunTag`, `_WaitpointRunConnections`, `_completedWaitpoints`, `SecretStore_key_idx`, and unrelated TaskRun indexes.
- Never drop columns or tables without explicit approval.
- New code should target `RunEngineVersion.V2` only.
+14
View File
@@ -0,0 +1,14 @@
---
paths:
- "docs/**"
---
# Documentation Writing Rules
- Use Mintlify MDX format. Frontmatter: `title`, `description`, `sidebarTitle` (optional).
- After creating a new page, add it to `docs.json` navigation under the correct group.
- Use Mintlify components: `<Note>`, `<Warning>`, `<Info>`, `<Tip>`, `<CodeGroup>`, `<Expandable>`, `<Steps>`/`<Step>`.
- Code examples should be complete and runnable where possible.
- Always import from `@trigger.dev/sdk`, never `@trigger.dev/sdk/v3`.
- Keep paragraphs short. Use headers to break up content.
- Link to related pages using relative paths (e.g., `[Tasks](/tasks/overview)`).
+33
View File
@@ -0,0 +1,33 @@
---
paths:
- "apps/webapp/app/v3/**"
---
# Legacy V1 Engine Code in `app/v3/`
The `v3/` directory name is misleading - most code here is actively used by the current V2 engine. Only the specific files below are legacy V1-only code.
## V1-Only Files - Never Modify
- `marqs/` directory (entire MarQS queue system: sharedQueueConsumer, devQueueConsumer, fairDequeuingStrategy, devPubSub)
- `legacyRunEngineWorker.server.ts` (V1 background job worker)
- `services/triggerTaskV1.server.ts` (deprecated V1 task triggering)
- `services/cancelTaskRunV1.server.ts` (deprecated V1 cancellation)
- `authenticatedSocketConnection.server.ts` (V1 dev WebSocket using DevQueueConsumer)
- `sharedSocketConnection.ts` (V1 shared queue socket using SharedQueueConsumer)
## V1/V2 Branching Pattern
Some services act as routers that branch on `RunEngineVersion`:
- `services/cancelTaskRun.server.ts` - calls V1 service or `engine.cancelRun()` for V2
- `services/batchTriggerV3.server.ts` - uses marqs for V1 path, run-engine for V2
When editing these shared services, only modify V2 code paths.
## V2 Modern Stack
- **Run lifecycle**: `@internal/run-engine` (internal-packages/run-engine)
- **Background jobs**: `@trigger.dev/redis-worker` (not graphile-worker/zodworker)
- **Queue operations**: RunQueue inside run-engine (not MarQS)
- **V2 engine singleton**: `runEngine.server.ts`, `runEngineHandlers.server.ts`
- **V2 workers**: `commonWorker.server.ts`, `alertsWorker.server.ts`, `batchTriggerWorker.server.ts`
+22
View File
@@ -0,0 +1,22 @@
---
paths:
- "**/package.json"
---
# Installing Packages
When adding a new dependency to any package.json in the monorepo:
1. **Look up the latest version** on npm before adding:
```bash
pnpm view <package-name> version
```
If unsure which version to use (e.g. major version compatibility), confirm with the user.
2. **Edit the package.json directly** — do NOT use `pnpm add` as it can cause issues in the monorepo. Add the dependency with the correct version range (typically `^x.y.z`).
3. **Run `pnpm i` from the repo root** after editing to install and update the lockfile:
```bash
pnpm i
```
Always run from the repo root, not from the package directory.
+12
View File
@@ -0,0 +1,12 @@
---
paths:
- "packages/**"
---
# Public Package Rules
- Changes to `packages/` are **customer-facing**. Always add a changeset: `pnpm run changeset:add`
- Default to **patch**. Get maintainer approval for minor. Never select major without explicit approval.
- `@trigger.dev/core`: **Never import the root**. Always use subpath imports (e.g., `@trigger.dev/core/v3`).
- Do NOT update `rules/` or `.claude/skills/trigger-dev-tasks/` unless explicitly asked. These are maintained in separate dedicated passes.
- Test changes using `references/hello-world` reference project.
+23
View File
@@ -0,0 +1,23 @@
---
paths:
- "apps/**"
---
# Server App Changes
When modifying server apps (webapp, supervisor, coordinator, etc.) with **no package changes**, add a `.server-changes/` file instead of a changeset:
```bash
cat > .server-changes/descriptive-name.md << 'EOF'
---
area: webapp
type: fix
---
Brief description of what changed and why.
EOF
```
- **area**: `webapp` | `supervisor` | `coordinator` | `kubernetes-provider` | `docker-provider`
- **type**: `feature` | `fix` | `improvement` | `breaking`
- If the PR also touches `packages/`, just the changeset is sufficient (no `.server-changes/` needed).
@@ -0,0 +1,78 @@
---
name: span-timeline-events
description: Use when adding, modifying, or debugging OTel span timeline events in the trace view. Covers event structure, ClickHouse storage constraints, rendering in SpanTimeline component, admin visibility, and the step-by-step process for adding new events.
allowed-tools: Read, Write, Edit, Glob, Grep, Bash
---
# Span Timeline Events
The trace view's right panel shows a timeline of events for the selected span. These are OTel span events rendered by `app/utils/timelineSpanEvents.ts` and the `SpanTimeline` component.
## How They Work
1. **Span events** in OTel are attached to a parent span. In ClickHouse, they're stored as separate rows with `kind: "SPAN_EVENT"` sharing the parent span's `span_id`. The `#mergeRecordsIntoSpanDetail` method reassembles them into the span's `events` array at query time.
2. The timeline only renders events whose `name` starts with `trigger.dev/` - all others are silently filtered out.
3. The **display name** comes from `properties.event` (not the span event name), mapped through `getFriendlyNameForEvent()`.
4. Events are shown on the **span they belong to** - events on one span don't appear in another span's timeline.
## ClickHouse Storage Constraint
When events are written to ClickHouse, `spanEventsToTaskEventV1Input()` filters out events whose `start_time` is not greater than the parent span's `startTime`. Events at or before the span start are silently dropped. This means span events must have timestamps strictly after the span's own `startTimeUnixNano`.
## Timeline Rendering (SpanTimeline component)
The `SpanTimeline` component in `app/components/run/RunTimeline.tsx` renders:
1. **Events** (thin 1px line with hollow dots) - all events from `createTimelineSpanEventsFromSpanEvents()`
2. **"Started"** marker (thick cap) - at the span's `startTime`
3. **Duration bar** (thick 7px line) - from "Started" to "Finished"
4. **"Finished"** marker (thick cap) - at `startTime + duration`
The thin line before "Started" only appears when there are events with timestamps between the span start and the first child span. For the Attempt span this works well (Dequeued -> Pod scheduled -> Launched -> etc. all happen before execution starts). Events all get `lineVariant: "light"` (thin) while the execution bar gets `variant: "normal"` (thick).
## Trace View Sort Order
Sibling spans (same parent) are sorted by `start_time ASC` from the ClickHouse query. The `createTreeFromFlatItems` function preserves this order. Event timestamps don't affect sort order - only the span's own `start_time`.
## Event Structure
```typescript
// OTel span event format
{
name: "trigger.dev/run", // Must start with "trigger.dev/" to render
timeUnixNano: "1711200000000000000",
attributes: [
{ key: "event", value: { stringValue: "dequeue" } }, // The actual event type
{ key: "duration", value: { intValue: 150 } }, // Optional: duration in ms
]
}
```
## Admin-Only Events
`getAdminOnlyForEvent()` controls visibility. Events default to **admin-only** (`true`).
| Event | Admin-only | Friendly name |
|-------|-----------|---------------|
| `dequeue` | No | Dequeued |
| `fork` | No | Launched |
| `import` | No (if no fork event) | Importing task file |
| `create_attempt` | Yes | Attempt created |
| `lazy_payload` | Yes | Lazy attempt initialized |
| `pod_scheduled` | Yes | Pod scheduled |
| (default) | Yes | (raw event name) |
## Adding New Timeline Events
1. Add OTLP span event with `name: "trigger.dev/<scope>"` and `properties.event: "<type>"`
2. Event timestamp must be strictly after the parent span's `startTimeUnixNano` (ClickHouse drops earlier events)
3. Add friendly name in `getFriendlyNameForEvent()` in `app/utils/timelineSpanEvents.ts`
4. Set admin visibility in `getAdminOnlyForEvent()`
5. Optionally add help text in `getHelpTextForEvent()`
## Key Files
- `app/utils/timelineSpanEvents.ts` - filtering, naming, admin logic
- `app/components/run/RunTimeline.tsx` - `SpanTimeline` component (thin line + thick bar rendering)
- `app/presenters/v3/SpanPresenter.server.ts` - loads span data including events
- `app/v3/eventRepository/clickhouseEventRepository.server.ts` - `spanEventsToTaskEventV1Input()` (storage filter), `#mergeRecordsIntoSpanDetail` (reassembly)
+200
View File
@@ -0,0 +1,200 @@
---
name: trigger-dev-tasks
description: Use this skill when writing, designing, or optimizing Trigger.dev background tasks and workflows. This includes creating reliable async tasks, implementing AI workflows, setting up scheduled jobs, structuring complex task hierarchies with subtasks, configuring build extensions for tools like ffmpeg or Puppeteer/Playwright, and handling task schemas with Zod validation.
allowed-tools: Read, Write, Edit, Glob, Grep, Bash
---
# Trigger.dev Task Expert
You are an expert Trigger.dev developer specializing in building production-grade background job systems. Tasks deployed to Trigger.dev run in Node.js 21+ and use the `@trigger.dev/sdk` package.
## Critical Rules
1. **Always use `@trigger.dev/sdk`** - Never use `@trigger.dev/sdk/v3` or deprecated `client.defineJob` pattern
2. **Never use `node-fetch`** - Use the built-in `fetch` function
3. **Export all tasks** - Every task must be exported, including subtasks
4. **Never wrap wait/trigger calls in Promise.all** - `triggerAndWait`, `batchTriggerAndWait`, and `wait.*` calls cannot be wrapped in `Promise.all` or `Promise.allSettled`
## Basic Task Pattern
```ts
import { task } from "@trigger.dev/sdk";
export const processData = task({
id: "process-data",
retry: {
maxAttempts: 10,
factor: 1.8,
minTimeoutInMs: 500,
maxTimeoutInMs: 30_000,
},
run: async (payload: { userId: string; data: any[] }) => {
console.log(`Processing ${payload.data.length} items`);
return { processed: payload.data.length };
},
});
```
## Schema Task (with validation)
```ts
import { schemaTask } from "@trigger.dev/sdk";
import { z } from "zod";
export const validatedTask = schemaTask({
id: "validated-task",
schema: z.object({
name: z.string(),
email: z.string().email(),
}),
run: async (payload) => {
// Payload is automatically validated and typed
return { message: `Hello ${payload.name}` };
},
});
```
## Triggering Tasks
### From Backend Code (type-only import to prevent dependency leakage)
```ts
import { tasks } from "@trigger.dev/sdk";
import type { processData } from "./trigger/tasks";
const handle = await tasks.trigger<typeof processData>("process-data", {
userId: "123",
data: [{ id: 1 }],
});
```
### From Inside Tasks
```ts
export const parentTask = task({
id: "parent-task",
run: async (payload) => {
// Trigger and wait - returns Result object, NOT direct output
const result = await childTask.triggerAndWait({ data: "value" });
if (result.ok) {
console.log("Output:", result.output);
} else {
console.error("Failed:", result.error);
}
// Or unwrap directly (throws on error)
const output = await childTask.triggerAndWait({ data: "value" }).unwrap();
},
});
```
## Idempotency (Critical for Retries)
Always use idempotency keys when triggering tasks from inside other tasks:
```ts
import { idempotencyKeys } from "@trigger.dev/sdk";
export const paymentTask = task({
id: "process-payment",
run: async (payload: { orderId: string }) => {
// Scoped to current run - survives retries
const key = await idempotencyKeys.create(`payment-${payload.orderId}`);
await chargeCustomer.trigger(payload, {
idempotencyKey: key,
idempotencyKeyTTL: "24h",
});
},
});
```
## Trigger Options
```ts
await myTask.trigger(payload, {
delay: "1h", // Delay execution
ttl: "10m", // Cancel if not started within TTL
idempotencyKey: key,
queue: "my-queue",
machine: "large-1x", // micro, small-1x, small-2x, medium-1x, medium-2x, large-1x, large-2x
maxAttempts: 3,
tags: ["user_123"], // Max 10 tags
debounce: { // Consolidate rapid triggers
key: "unique-key",
delay: "5s",
mode: "trailing", // "leading" (default) or "trailing"
},
});
```
## Debouncing
Consolidate multiple triggers into a single execution:
```ts
// Rapid triggers with same key = single execution
await myTask.trigger({ userId: "123" }, {
debounce: {
key: "user-123-update",
delay: "5s",
},
});
// Trailing mode: use payload from LAST trigger
await myTask.trigger({ data: "latest" }, {
debounce: {
key: "my-key",
delay: "10s",
mode: "trailing",
},
});
```
Use cases: user activity updates, webhook deduplication, search indexing, notification batching.
## Batch Triggering
Up to 1,000 items per batch, 3MB per payload:
```ts
const results = await myTask.batchTriggerAndWait([
{ payload: { userId: "1" } },
{ payload: { userId: "2" } },
]);
for (const result of results) {
if (result.ok) console.log(result.output);
}
```
## Machine Presets
| Preset | vCPU | Memory |
|-------------|------|--------|
| micro | 0.25 | 0.25GB |
| small-1x | 0.5 | 0.5GB |
| small-2x | 1 | 1GB |
| medium-1x | 1 | 2GB |
| medium-2x | 2 | 4GB |
| large-1x | 4 | 8GB |
| large-2x | 8 | 16GB |
## Design Principles
1. **Break complex workflows into subtasks** that can be independently retried and made idempotent
2. **Don't over-complicate** - Sometimes `Promise.allSettled` inside a single task is better than many subtasks (each task has dedicated process and is charged by millisecond)
3. **Always configure retries** - Set appropriate `maxAttempts` based on the operation
4. **Use idempotency keys** - Especially for payment/critical operations
5. **Group related subtasks** - Keep subtasks only used by one parent in the same file, don't export them
6. **Use logger** - Log at key execution points with `logger.info()`, `logger.error()`, etc.
## Reference Documentation
For detailed documentation on specific topics, read these files:
- `basic-tasks.md` - Task basics, triggering, waits
- `advanced-tasks.md` - Tags, queues, concurrency, metadata, error handling
- `scheduled-tasks.md` - Cron schedules, declarative and imperative
- `realtime.md` - Real-time subscriptions, streams, React hooks
- `config.md` - trigger.config.ts, build extensions (Prisma, Playwright, FFmpeg, etc.)
@@ -0,0 +1,485 @@
# Trigger.dev Advanced Tasks (v4)
**Advanced patterns and features for writing tasks**
## Tags & Organization
```ts
import { task, tags } from "@trigger.dev/sdk";
export const processUser = task({
id: "process-user",
run: async (payload: { userId: string; orgId: string }, { ctx }) => {
// Add tags during execution
await tags.add(`user_${payload.userId}`);
await tags.add(`org_${payload.orgId}`);
return { processed: true };
},
});
// Trigger with tags
await processUser.trigger(
{ userId: "123", orgId: "abc" },
{ tags: ["priority", "user_123", "org_abc"] } // Max 10 tags per run
);
// Subscribe to tagged runs
for await (const run of runs.subscribeToRunsWithTag("user_123")) {
console.log(`User task ${run.id}: ${run.status}`);
}
```
**Tag Best Practices:**
- Use prefixes: `user_123`, `org_abc`, `video:456`
- Max 10 tags per run, 1-64 characters each
- Tags don't propagate to child tasks automatically
## Batch Triggering v2
Enhanced batch triggering with larger payloads and streaming ingestion.
### Limits
- **Maximum batch size**: 1,000 items (increased from 500)
- **Payload per item**: 3MB each (increased from 1MB combined)
- Payloads > 512KB automatically offload to object storage
### Rate Limiting (per environment)
| Tier | Bucket Size | Refill Rate |
|------|-------------|-------------|
| Free | 1,200 runs | 100 runs/10 sec |
| Hobby | 5,000 runs | 500 runs/5 sec |
| Pro | 5,000 runs | 500 runs/5 sec |
### Concurrent Batch Processing
| Tier | Concurrent Batches |
|------|-------------------|
| Free | 1 |
| Hobby | 10 |
| Pro | 10 |
### Usage
```ts
import { myTask } from "./trigger/myTask";
// Basic batch trigger (up to 1,000 items)
const runs = await myTask.batchTrigger([
{ payload: { userId: "user-1" } },
{ payload: { userId: "user-2" } },
{ payload: { userId: "user-3" } },
]);
// Batch trigger with wait
const results = await myTask.batchTriggerAndWait([
{ payload: { userId: "user-1" } },
{ payload: { userId: "user-2" } },
]);
for (const result of results) {
if (result.ok) {
console.log("Result:", result.output);
}
}
// With per-item options
const batchHandle = await myTask.batchTrigger([
{
payload: { userId: "123" },
options: {
idempotencyKey: "user-123-batch",
tags: ["priority"],
},
},
{
payload: { userId: "456" },
options: {
idempotencyKey: "user-456-batch",
},
},
]);
```
## Debouncing
Consolidate multiple triggers into a single execution by debouncing task runs with a unique key and delay window.
### Use Cases
- **User activity updates**: Batch rapid user actions into a single run
- **Webhook deduplication**: Handle webhook bursts without redundant processing
- **Search indexing**: Combine document updates instead of processing individually
- **Notification batching**: Group notifications to prevent user spam
### Basic Usage
```ts
await myTask.trigger(
{ userId: "123" },
{
debounce: {
key: "user-123-update", // Unique identifier for debounce group
delay: "5s", // Wait duration ("5s", "1m", or milliseconds)
},
}
);
```
### Execution Modes
**Leading Mode** (default): Uses payload/options from the first trigger; subsequent triggers only reschedule execution time.
```ts
// First trigger sets the payload
await myTask.trigger({ action: "first" }, {
debounce: { key: "my-key", delay: "10s" }
});
// Second trigger only reschedules - payload remains "first"
await myTask.trigger({ action: "second" }, {
debounce: { key: "my-key", delay: "10s" }
});
// Task executes with { action: "first" }
```
**Trailing Mode**: Uses payload/options from the most recent trigger.
```ts
await myTask.trigger(
{ data: "latest-value" },
{
debounce: {
key: "trailing-example",
delay: "10s",
mode: "trailing",
},
}
);
```
In trailing mode, these options update with each trigger:
- `payload` — task input data
- `metadata` — run metadata
- `tags` — run tags (replaces existing)
- `maxAttempts` — retry attempts
- `maxDuration` — maximum compute time
- `machine` — machine preset
### Important Notes
- Idempotency keys take precedence over debounce settings
- Compatible with `triggerAndWait()` — parent runs block correctly on debounced execution
- Debounce key is scoped to the task
## Concurrency & Queues
```ts
import { task, queue } from "@trigger.dev/sdk";
// Shared queue for related tasks
const emailQueue = queue({
name: "email-processing",
concurrencyLimit: 5, // Max 5 emails processing simultaneously
});
// Task-level concurrency
export const oneAtATime = task({
id: "sequential-task",
queue: { concurrencyLimit: 1 }, // Process one at a time
run: async (payload) => {
// Critical section - only one instance runs
},
});
// Per-user concurrency
export const processUserData = task({
id: "process-user-data",
run: async (payload: { userId: string }) => {
// Override queue with user-specific concurrency
await childTask.trigger(payload, {
queue: {
name: `user-${payload.userId}`,
concurrencyLimit: 2,
},
});
},
});
export const emailTask = task({
id: "send-email",
queue: emailQueue, // Use shared queue
run: async (payload: { to: string }) => {
// Send email logic
},
});
```
## Error Handling & Retries
```ts
import { task, retry, AbortTaskRunError } from "@trigger.dev/sdk";
export const resilientTask = task({
id: "resilient-task",
retry: {
maxAttempts: 10,
factor: 1.8, // Exponential backoff multiplier
minTimeoutInMs: 500,
maxTimeoutInMs: 30_000,
randomize: false,
},
catchError: async ({ error, ctx }) => {
// Custom error handling
if (error.code === "FATAL_ERROR") {
throw new AbortTaskRunError("Cannot retry this error");
}
// Log error details
console.error(`Task ${ctx.task.id} failed:`, error);
// Allow retry by returning nothing
return { retryAt: new Date(Date.now() + 60000) }; // Retry in 1 minute
},
run: async (payload) => {
// Retry specific operations
const result = await retry.onThrow(
async () => {
return await unstableApiCall(payload);
},
{ maxAttempts: 3 }
);
// Conditional HTTP retries
const response = await retry.fetch("https://api.example.com", {
retry: {
maxAttempts: 5,
condition: (response, error) => {
return response?.status === 429 || response?.status >= 500;
},
},
});
return result;
},
});
```
## Machines & Performance
```ts
export const heavyTask = task({
id: "heavy-computation",
machine: { preset: "large-2x" }, // 8 vCPU, 16 GB RAM
maxDuration: 1800, // 30 minutes timeout
run: async (payload, { ctx }) => {
// Resource-intensive computation
if (ctx.machine.preset === "large-2x") {
// Use all available cores
return await parallelProcessing(payload);
}
return await standardProcessing(payload);
},
});
// Override machine when triggering
await heavyTask.trigger(payload, {
machine: { preset: "medium-1x" }, // Override for this run
});
```
**Machine Presets:**
- `micro`: 0.25 vCPU, 0.25 GB RAM
- `small-1x`: 0.5 vCPU, 0.5 GB RAM (default)
- `small-2x`: 1 vCPU, 1 GB RAM
- `medium-1x`: 1 vCPU, 2 GB RAM
- `medium-2x`: 2 vCPU, 4 GB RAM
- `large-1x`: 4 vCPU, 8 GB RAM
- `large-2x`: 8 vCPU, 16 GB RAM
## Idempotency
```ts
import { task, idempotencyKeys } from "@trigger.dev/sdk";
export const paymentTask = task({
id: "process-payment",
retry: {
maxAttempts: 3,
},
run: async (payload: { orderId: string; amount: number }) => {
// Automatically scoped to this task run, so if the task is retried, the idempotency key will be the same
const idempotencyKey = await idempotencyKeys.create(`payment-${payload.orderId}`);
// Ensure payment is processed only once
await chargeCustomer.trigger(payload, {
idempotencyKey,
idempotencyKeyTTL: "24h", // Key expires in 24 hours
});
},
});
// Payload-based idempotency
import { createHash } from "node:crypto";
function createPayloadHash(payload: any): string {
const hash = createHash("sha256");
hash.update(JSON.stringify(payload));
return hash.digest("hex");
}
export const deduplicatedTask = task({
id: "deduplicated-task",
run: async (payload) => {
const payloadHash = createPayloadHash(payload);
const idempotencyKey = await idempotencyKeys.create(payloadHash);
await processData.trigger(payload, { idempotencyKey });
},
});
```
## Metadata & Progress Tracking
```ts
import { task, metadata } from "@trigger.dev/sdk";
export const batchProcessor = task({
id: "batch-processor",
run: async (payload: { items: any[] }, { ctx }) => {
const totalItems = payload.items.length;
// Initialize progress metadata
metadata
.set("progress", 0)
.set("totalItems", totalItems)
.set("processedItems", 0)
.set("status", "starting");
const results = [];
for (let i = 0; i < payload.items.length; i++) {
const item = payload.items[i];
// Process item
const result = await processItem(item);
results.push(result);
// Update progress
const progress = ((i + 1) / totalItems) * 100;
metadata
.set("progress", progress)
.increment("processedItems", 1)
.append("logs", `Processed item ${i + 1}/${totalItems}`)
.set("currentItem", item.id);
}
// Final status
metadata.set("status", "completed");
return { results, totalProcessed: results.length };
},
});
// Update parent metadata from child task
export const childTask = task({
id: "child-task",
run: async (payload, { ctx }) => {
// Update parent task metadata
metadata.parent.set("childStatus", "processing");
metadata.root.increment("childrenCompleted", 1);
return { processed: true };
},
});
```
## Logging & Tracing
```ts
import { task, logger } from "@trigger.dev/sdk";
export const tracedTask = task({
id: "traced-task",
run: async (payload, { ctx }) => {
logger.info("Task started", { userId: payload.userId });
// Custom trace with attributes
const user = await logger.trace(
"fetch-user",
async (span) => {
span.setAttribute("user.id", payload.userId);
span.setAttribute("operation", "database-fetch");
const userData = await database.findUser(payload.userId);
span.setAttribute("user.found", !!userData);
return userData;
},
{ userId: payload.userId }
);
logger.debug("User fetched", { user: user.id });
try {
const result = await processUser(user);
logger.info("Processing completed", { result });
return result;
} catch (error) {
logger.error("Processing failed", {
error: error.message,
userId: payload.userId,
});
throw error;
}
},
});
```
## Hidden Tasks
```ts
// Hidden task - not exported, only used internally
const internalProcessor = task({
id: "internal-processor",
run: async (payload: { data: string }) => {
return { processed: payload.data.toUpperCase() };
},
});
// Public task that uses hidden task
export const publicWorkflow = task({
id: "public-workflow",
run: async (payload: { input: string }) => {
// Use hidden task internally
const result = await internalProcessor.triggerAndWait({
data: payload.input,
});
if (result.ok) {
return { output: result.output.processed };
}
throw new Error("Internal processing failed");
},
});
```
## Best Practices
- **Concurrency**: Use queues to prevent overwhelming external services
- **Retries**: Configure exponential backoff for transient failures
- **Idempotency**: Always use for payment/critical operations
- **Metadata**: Track progress for long-running tasks
- **Machines**: Match machine size to computational requirements
- **Tags**: Use consistent naming patterns for filtering
- **Debouncing**: Use for user activity, webhooks, and notification batching
- **Batch triggering**: Use for bulk operations up to 1,000 items
- **Error Handling**: Distinguish between retryable and fatal errors
Design tasks to be stateless, idempotent, and resilient to failures. Use metadata for state tracking and queues for resource management.
@@ -0,0 +1,199 @@
# Trigger.dev Basic Tasks (v4)
**MUST use `@trigger.dev/sdk`, NEVER `client.defineJob`**
## Basic Task
```ts
import { task } from "@trigger.dev/sdk";
export const processData = task({
id: "process-data",
retry: {
maxAttempts: 10,
factor: 1.8,
minTimeoutInMs: 500,
maxTimeoutInMs: 30_000,
randomize: false,
},
run: async (payload: { userId: string; data: any[] }) => {
// Task logic - runs for long time, no timeouts
console.log(`Processing ${payload.data.length} items for user ${payload.userId}`);
return { processed: payload.data.length };
},
});
```
## Schema Task (with validation)
```ts
import { schemaTask } from "@trigger.dev/sdk";
import { z } from "zod";
export const validatedTask = schemaTask({
id: "validated-task",
schema: z.object({
name: z.string(),
age: z.number(),
email: z.string().email(),
}),
run: async (payload) => {
// Payload is automatically validated and typed
return { message: `Hello ${payload.name}, age ${payload.age}` };
},
});
```
## Triggering Tasks
### From Backend Code
```ts
import { tasks } from "@trigger.dev/sdk";
import type { processData } from "./trigger/tasks";
// Single trigger
const handle = await tasks.trigger<typeof processData>("process-data", {
userId: "123",
data: [{ id: 1 }, { id: 2 }],
});
// Batch trigger (up to 1,000 items, 3MB per payload)
const batchHandle = await tasks.batchTrigger<typeof processData>("process-data", [
{ payload: { userId: "123", data: [{ id: 1 }] } },
{ payload: { userId: "456", data: [{ id: 2 }] } },
]);
```
### Debounced Triggering
Consolidate multiple triggers into a single execution:
```ts
// Multiple rapid triggers with same key = single execution
await myTask.trigger(
{ userId: "123" },
{
debounce: {
key: "user-123-update", // Unique key for debounce group
delay: "5s", // Wait before executing
},
}
);
// Trailing mode: use payload from LAST trigger
await myTask.trigger(
{ data: "latest-value" },
{
debounce: {
key: "trailing-example",
delay: "10s",
mode: "trailing", // Default is "leading" (first payload)
},
}
);
```
**Debounce modes:**
- `leading` (default): Uses payload from first trigger, subsequent triggers only reschedule
- `trailing`: Uses payload from most recent trigger
### From Inside Tasks (with Result handling)
```ts
export const parentTask = task({
id: "parent-task",
run: async (payload) => {
// Trigger and continue
const handle = await childTask.trigger({ data: "value" });
// Trigger and wait - returns Result object, NOT task output
const result = await childTask.triggerAndWait({ data: "value" });
if (result.ok) {
console.log("Task output:", result.output); // Actual task return value
} else {
console.error("Task failed:", result.error);
}
// Quick unwrap (throws on error)
const output = await childTask.triggerAndWait({ data: "value" }).unwrap();
// Batch trigger and wait
const results = await childTask.batchTriggerAndWait([
{ payload: { data: "item1" } },
{ payload: { data: "item2" } },
]);
for (const run of results) {
if (run.ok) {
console.log("Success:", run.output);
} else {
console.log("Failed:", run.error);
}
}
},
});
export const childTask = task({
id: "child-task",
run: async (payload: { data: string }) => {
return { processed: payload.data };
},
});
```
> Never wrap triggerAndWait or batchTriggerAndWait calls in a Promise.all or Promise.allSettled as this is not supported in Trigger.dev tasks.
## Waits
```ts
import { task, wait } from "@trigger.dev/sdk";
export const taskWithWaits = task({
id: "task-with-waits",
run: async (payload) => {
console.log("Starting task");
// Wait for specific duration
await wait.for({ seconds: 30 });
await wait.for({ minutes: 5 });
await wait.for({ hours: 1 });
await wait.for({ days: 1 });
// Wait until specific date
await wait.until({ date: new Date("2024-12-25") });
// Wait for token (from external system)
await wait.forToken({
token: "user-approval-token",
timeoutInSeconds: 3600, // 1 hour timeout
});
console.log("All waits completed");
return { status: "completed" };
},
});
```
> Never wrap wait calls in a Promise.all or Promise.allSettled as this is not supported in Trigger.dev tasks.
## Key Points
- **Result vs Output**: `triggerAndWait()` returns a `Result` object with `ok`, `output`, `error` properties - NOT the direct task output
- **Type safety**: Use `import type` for task references when triggering from backend
- **Waits > 5 seconds**: Automatically checkpointed, don't count toward compute usage
- **Debounce + idempotency**: Idempotency keys take precedence over debounce settings
## NEVER Use (v2 deprecated)
```ts
// BREAKS APPLICATION
client.defineJob({
id: "job-id",
run: async (payload, io) => {
/* ... */
},
});
```
Use SDK (`@trigger.dev/sdk`), check `result.ok` before accessing `result.output`
+346
View File
@@ -0,0 +1,346 @@
# Trigger.dev Configuration
**Complete guide to configuring `trigger.config.ts` with build extensions**
## Basic Configuration
```ts
import { defineConfig } from "@trigger.dev/sdk";
export default defineConfig({
project: "<project-ref>", // Required: Your project reference
dirs: ["./trigger"], // Task directories
runtime: "node", // "node", "node-22", or "bun"
logLevel: "info", // "debug", "info", "warn", "error"
// Default retry settings
retries: {
enabledInDev: false,
default: {
maxAttempts: 3,
minTimeoutInMs: 1000,
maxTimeoutInMs: 10000,
factor: 2,
randomize: true,
},
},
// Build configuration
build: {
autoDetectExternal: true,
keepNames: true,
minify: false,
extensions: [], // Build extensions go here
},
// Global lifecycle hooks
onStartAttempt: async ({ payload, ctx }) => {
console.log("Global task start");
},
onSuccess: async ({ payload, output, ctx }) => {
console.log("Global task success");
},
onFailure: async ({ payload, error, ctx }) => {
console.log("Global task failure");
},
});
```
## Build Extensions
### Database & ORM
#### Prisma
```ts
import { prismaExtension } from "@trigger.dev/build/extensions/prisma";
extensions: [
prismaExtension({
schema: "prisma/schema.prisma",
version: "5.19.0", // Optional: specify version
migrate: true, // Run migrations during build
directUrlEnvVarName: "DIRECT_DATABASE_URL",
typedSql: true, // Enable TypedSQL support
}),
];
```
#### TypeScript Decorators (for TypeORM)
```ts
import { emitDecoratorMetadata } from "@trigger.dev/build/extensions/typescript";
extensions: [
emitDecoratorMetadata(), // Enables decorator metadata
];
```
### Scripting Languages
#### Python
```ts
import { pythonExtension } from "@trigger.dev/build/extensions/python";
extensions: [
pythonExtension({
scripts: ["./python/**/*.py"], // Copy Python files
requirementsFile: "./requirements.txt", // Install packages
devPythonBinaryPath: ".venv/bin/python", // Dev mode binary
}),
];
// Usage in tasks
const result = await python.runInline(`print("Hello, world!")`);
const output = await python.runScript("./python/script.py", ["arg1"]);
```
### Browser Automation
#### Playwright
```ts
import { playwright } from "@trigger.dev/build/extensions/playwright";
extensions: [
playwright({
browsers: ["chromium", "firefox", "webkit"], // Default: ["chromium"]
headless: true, // Default: true
}),
];
```
#### Puppeteer
```ts
import { puppeteer } from "@trigger.dev/build/extensions/puppeteer";
extensions: [puppeteer()];
// Environment variable needed:
// PUPPETEER_EXECUTABLE_PATH: "/usr/bin/google-chrome-stable"
```
#### Lightpanda
```ts
import { lightpanda } from "@trigger.dev/build/extensions/lightpanda";
extensions: [
lightpanda({
version: "latest", // or "nightly"
disableTelemetry: false,
}),
];
```
### Media Processing
#### FFmpeg
```ts
import { ffmpeg } from "@trigger.dev/build/extensions/core";
extensions: [
ffmpeg({ version: "7" }), // Static build, or omit for Debian version
];
// Automatically sets FFMPEG_PATH and FFPROBE_PATH
// Add fluent-ffmpeg to external packages if using
```
#### Audio Waveform
```ts
import { audioWaveform } from "@trigger.dev/build/extensions/audioWaveform";
extensions: [
audioWaveform(), // Installs Audio Waveform 1.1.0
];
```
### System & Package Management
#### System Packages (apt-get)
```ts
import { aptGet } from "@trigger.dev/build/extensions/core";
extensions: [
aptGet({
packages: ["ffmpeg", "imagemagick", "curl=7.68.0-1"], // Can specify versions
}),
];
```
#### Additional NPM Packages
Only use this for installing CLI tools, NOT packages you import in your code.
```ts
import { additionalPackages } from "@trigger.dev/build/extensions/core";
extensions: [
additionalPackages({
packages: ["wrangler"], // CLI tools and specific versions
}),
];
```
#### Additional Files
```ts
import { additionalFiles } from "@trigger.dev/build/extensions/core";
extensions: [
additionalFiles({
files: ["wrangler.toml", "./assets/**", "./fonts/**"], // Glob patterns supported
}),
];
```
### Environment & Build Tools
#### Environment Variable Sync
```ts
import { syncEnvVars } from "@trigger.dev/build/extensions/core";
extensions: [
syncEnvVars(async (ctx) => {
// ctx contains: environment, projectRef, env
return [
{ name: "SECRET_KEY", value: await getSecret(ctx.environment) },
{ name: "API_URL", value: ctx.environment === "prod" ? "api.prod.com" : "api.dev.com" },
];
}),
];
```
#### ESBuild Plugins
```ts
import { esbuildPlugin } from "@trigger.dev/build/extensions";
import { sentryEsbuildPlugin } from "@sentry/esbuild-plugin";
extensions: [
esbuildPlugin(
sentryEsbuildPlugin({
org: process.env.SENTRY_ORG,
project: process.env.SENTRY_PROJECT,
authToken: process.env.SENTRY_AUTH_TOKEN,
}),
{ placement: "last", target: "deploy" } // Optional config
),
];
```
## Custom Build Extensions
```ts
import { defineConfig } from "@trigger.dev/sdk";
const customExtension = {
name: "my-custom-extension",
externalsForTarget: (target) => {
return ["some-native-module"]; // Add external dependencies
},
onBuildStart: async (context) => {
console.log(`Build starting for ${context.target}`);
// Register esbuild plugins, modify build context
},
onBuildComplete: async (context, manifest) => {
console.log("Build complete, adding layers");
// Add build layers, modify deployment
context.addLayer({
id: "my-layer",
files: [{ source: "./custom-file", destination: "/app/custom" }],
commands: ["chmod +x /app/custom"],
});
},
};
export default defineConfig({
project: "my-project",
build: {
extensions: [customExtension],
},
});
```
## Advanced Configuration
### Telemetry
```ts
import { PrismaInstrumentation } from "@prisma/instrumentation";
import { OpenAIInstrumentation } from "@langfuse/openai";
export default defineConfig({
// ... other config
telemetry: {
instrumentations: [new PrismaInstrumentation(), new OpenAIInstrumentation()],
exporters: [customExporter], // Optional custom exporters
},
});
```
### Machine & Performance
```ts
export default defineConfig({
// ... other config
defaultMachine: "large-1x", // Default machine for all tasks
maxDuration: 300, // Default max duration (seconds)
enableConsoleLogging: true, // Console logging in development
});
```
## Common Extension Combinations
### Full-Stack Web App
```ts
extensions: [
prismaExtension({ schema: "prisma/schema.prisma", migrate: true }),
additionalFiles({ files: ["./public/**", "./assets/**"] }),
syncEnvVars(async (ctx) => [...envVars]),
];
```
### AI/ML Processing
```ts
extensions: [
pythonExtension({
scripts: ["./ai/**/*.py"],
requirementsFile: "./requirements.txt",
}),
ffmpeg({ version: "7" }),
additionalPackages({ packages: ["wrangler"] }),
];
```
### Web Scraping
```ts
extensions: [
playwright({ browsers: ["chromium"] }),
puppeteer(),
additionalFiles({ files: ["./selectors.json", "./proxies.txt"] }),
];
```
## Best Practices
- **Use specific versions**: Pin extension versions for reproducible builds
- **External packages**: Add modules with native addons to the `build.external` array
- **Environment sync**: Use `syncEnvVars` for dynamic secrets
- **File paths**: Use glob patterns for flexible file inclusion
- **Debug builds**: Use `--log-level debug --dry-run` for troubleshooting
Extensions only affect deployment, not local development. Use `external` array for packages that shouldn't be bundled.
@@ -0,0 +1,244 @@
# Trigger.dev Realtime
**Real-time monitoring and updates for runs**
## Core Concepts
Realtime allows you to:
- Subscribe to run status changes, metadata updates, and streams
- Build real-time dashboards and UI updates
- Monitor task progress from frontend and backend
## Authentication
### Public Access Tokens
```ts
import { auth } from "@trigger.dev/sdk";
// Read-only token for specific runs
const publicToken = await auth.createPublicToken({
scopes: {
read: {
runs: ["run_123", "run_456"],
tasks: ["my-task-1", "my-task-2"],
},
},
expirationTime: "1h", // Default: 15 minutes
});
```
### Trigger Tokens (Frontend only)
```ts
// Single-use token for triggering tasks
const triggerToken = await auth.createTriggerPublicToken("my-task", {
expirationTime: "30m",
});
```
## Backend Usage
### Subscribe to Runs
```ts
import { runs, tasks } from "@trigger.dev/sdk";
// Trigger and subscribe
const handle = await tasks.trigger("my-task", { data: "value" });
// Subscribe to specific run
for await (const run of runs.subscribeToRun<typeof myTask>(handle.id)) {
console.log(`Status: ${run.status}, Progress: ${run.metadata?.progress}`);
if (run.status === "COMPLETED") break;
}
// Subscribe to runs with tag
for await (const run of runs.subscribeToRunsWithTag("user-123")) {
console.log(`Tagged run ${run.id}: ${run.status}`);
}
// Subscribe to batch
for await (const run of runs.subscribeToBatch(batchId)) {
console.log(`Batch run ${run.id}: ${run.status}`);
}
```
### Realtime Streams v2
```ts
import { streams, InferStreamType } from "@trigger.dev/sdk";
// 1. Define streams (shared location)
export const aiStream = streams.define<string>({
id: "ai-output",
});
export type AIStreamPart = InferStreamType<typeof aiStream>;
// 2. Pipe from task
export const streamingTask = task({
id: "streaming-task",
run: async (payload) => {
const completion = await openai.chat.completions.create({
model: "gpt-4",
messages: [{ role: "user", content: payload.prompt }],
stream: true,
});
const { waitUntilComplete } = aiStream.pipe(completion);
await waitUntilComplete();
},
});
// 3. Read from backend
const stream = await aiStream.read(runId, {
timeoutInSeconds: 300,
startIndex: 0, // Resume from specific chunk
});
for await (const chunk of stream) {
console.log("Chunk:", chunk); // Fully typed
}
```
## React Frontend Usage
### Installation
```bash
npm add @trigger.dev/react-hooks
```
### Triggering Tasks
```tsx
"use client";
import { useTaskTrigger, useRealtimeTaskTrigger } from "@trigger.dev/react-hooks";
import type { myTask } from "../trigger/tasks";
function TriggerComponent({ accessToken }: { accessToken: string }) {
// Basic trigger
const { submit, handle, isLoading } = useTaskTrigger<typeof myTask>("my-task", {
accessToken,
});
// Trigger with realtime updates
const {
submit: realtimeSubmit,
run,
isLoading: isRealtimeLoading,
} = useRealtimeTaskTrigger<typeof myTask>("my-task", { accessToken });
return (
<div>
<button onClick={() => submit({ data: "value" })} disabled={isLoading}>
Trigger Task
</button>
<button onClick={() => realtimeSubmit({ data: "realtime" })} disabled={isRealtimeLoading}>
Trigger with Realtime
</button>
{run && <div>Status: {run.status}</div>}
</div>
);
}
```
### Subscribing to Runs
```tsx
"use client";
import { useRealtimeRun, useRealtimeRunsWithTag } from "@trigger.dev/react-hooks";
import type { myTask } from "../trigger/tasks";
function SubscribeComponent({ runId, accessToken }: { runId: string; accessToken: string }) {
// Subscribe to specific run
const { run, error } = useRealtimeRun<typeof myTask>(runId, {
accessToken,
onComplete: (run) => {
console.log("Task completed:", run.output);
},
});
// Subscribe to tagged runs
const { runs } = useRealtimeRunsWithTag("user-123", { accessToken });
if (error) return <div>Error: {error.message}</div>;
if (!run) return <div>Loading...</div>;
return (
<div>
<div>Status: {run.status}</div>
<div>Progress: {run.metadata?.progress || 0}%</div>
{run.output && <div>Result: {JSON.stringify(run.output)}</div>}
<h3>Tagged Runs:</h3>
{runs.map((r) => (
<div key={r.id}>
{r.id}: {r.status}
</div>
))}
</div>
);
}
```
### Realtime Streams with React
```tsx
"use client";
import { useRealtimeStream } from "@trigger.dev/react-hooks";
import { aiStream } from "../trigger/streams";
function StreamComponent({ runId, accessToken }: { runId: string; accessToken: string }) {
// Pass defined stream directly for type safety
const { parts, error } = useRealtimeStream(aiStream, runId, {
accessToken,
timeoutInSeconds: 300,
throttleInMs: 50, // Control re-render frequency
});
if (error) return <div>Error: {error.message}</div>;
if (!parts) return <div>Loading...</div>;
const text = parts.join(""); // parts is typed as AIStreamPart[]
return <div>Streamed Text: {text}</div>;
}
```
### Wait Tokens
```tsx
"use client";
import { useWaitToken } from "@trigger.dev/react-hooks";
function WaitTokenComponent({ tokenId, accessToken }: { tokenId: string; accessToken: string }) {
const { complete } = useWaitToken(tokenId, { accessToken });
return <button onClick={() => complete({ approved: true })}>Approve Task</button>;
}
```
## Run Object Properties
Key properties available in run subscriptions:
- `id`: Unique run identifier
- `status`: `QUEUED`, `EXECUTING`, `COMPLETED`, `FAILED`, `CANCELED`, etc.
- `payload`: Task input data (typed)
- `output`: Task result (typed, when completed)
- `metadata`: Real-time updatable data
- `createdAt`, `updatedAt`: Timestamps
- `costInCents`: Execution cost
## Best Practices
- **Use Realtime over SWR**: Recommended for most use cases due to rate limits
- **Scope tokens properly**: Only grant necessary read/trigger permissions
- **Handle errors**: Always check for errors in hooks and subscriptions
- **Type safety**: Use task types for proper payload/output typing
- **Cleanup subscriptions**: Backend subscriptions auto-complete, frontend hooks auto-cleanup
@@ -0,0 +1,113 @@
# Scheduled Tasks (Cron)
Recurring tasks using cron. For one-off future runs, use the **delay** option.
## Define a Scheduled Task
```ts
import { schedules } from "@trigger.dev/sdk";
export const task = schedules.task({
id: "first-scheduled-task",
run: async (payload) => {
payload.timestamp; // Date (scheduled time, UTC)
payload.lastTimestamp; // Date | undefined
payload.timezone; // IANA, e.g. "America/New_York" (default "UTC")
payload.scheduleId; // string
payload.externalId; // string | undefined
payload.upcoming; // Date[]
payload.timestamp.toLocaleString("en-US", { timeZone: payload.timezone });
},
});
```
> Scheduled tasks need at least one schedule attached to run.
## Attach Schedules
**Declarative (sync on dev/deploy):**
```ts
schedules.task({
id: "every-2h",
cron: "0 */2 * * *", // UTC
run: async () => {},
});
schedules.task({
id: "tokyo-5am",
cron: { pattern: "0 5 * * *", timezone: "Asia/Tokyo", environments: ["PRODUCTION", "STAGING"] },
run: async () => {},
});
```
**Imperative (SDK or dashboard):**
```ts
await schedules.create({
task: task.id,
cron: "0 0 * * *",
timezone: "America/New_York", // DST-aware
externalId: "user_123",
deduplicationKey: "user_123-daily", // updates if reused
});
```
### Dynamic / Multi-tenant Example
```ts
// /trigger/reminder.ts
export const reminderTask = schedules.task({
id: "todo-reminder",
run: async (p) => {
if (!p.externalId) throw new Error("externalId is required");
const user = await db.getUser(p.externalId);
await sendReminderEmail(user);
},
});
```
```ts
// app/reminders/route.ts
export async function POST(req: Request) {
const data = await req.json();
return Response.json(
await schedules.create({
task: reminderTask.id,
cron: "0 8 * * *",
timezone: data.timezone,
externalId: data.userId,
deduplicationKey: `${data.userId}-reminder`,
})
);
}
```
## Cron Syntax (no seconds)
```
* * * * *
| | | | └ day of week (07 or 1L7L; 0/7=Sun; L=last)
| | | └── month (112)
| | └──── day of month (131 or L)
| └────── hour (023)
└──────── minute (059)
```
## When Schedules Won't Trigger
- **Dev:** only when the dev CLI is running.
- **Staging/Production:** only for tasks in the **latest deployment**.
## SDK Management
```ts
await schedules.retrieve(id);
await schedules.list();
await schedules.update(id, { cron: "0 0 1 * *", externalId: "ext", deduplicationKey: "key" });
await schedules.deactivate(id);
await schedules.activate(id);
await schedules.del(id);
await schedules.timezones(); // list of IANA timezones
```
+11
View File
@@ -0,0 +1,11 @@
# Remove AI code slop
Check the diff against main, and remove all AI generated slop introduced in this branch.
This includes:
- Extra comments that a human wouldn't add or is inconsistent with the rest of the file
- Extra defensive checks or try/catch blocks that are abnormal for that area of the codebase (especially if called by trusted / validated codepaths)
- Casts to any to get around type issues
- Any other style that is inconsistent with the file
Report at the end with only a 1-3 sentence summary of what you changed
+5 -1
View File
@@ -1,3 +1,7 @@
{
"mcpServers": {}
"mcpServers": {
"linear": {
"url": "https://mcp.linear.app/mcp"
}
}
}
+6
View File
@@ -0,0 +1,6 @@
---
description: how to create and apply database migrations
alwaysApply: false
---
Follow our [migrations.md](mdc:ai/references/migrations.md) guide for how to create and apply database migrations.
+66
View File
@@ -0,0 +1,66 @@
---
description: Guidelines for creating OpenTelemetry metrics to avoid cardinality issues
globs:
- "**/*.ts"
---
# OpenTelemetry Metrics Guidelines
When creating or editing OTEL metrics (counters, histograms, gauges), always ensure metric attributes have **low cardinality**.
## What is Cardinality?
Cardinality refers to the number of unique values an attribute can have. Each unique combination of attribute values creates a new time series, which consumes memory and storage in your metrics backend.
## Rules
### DO use low-cardinality attributes:
- **Enums**: `environment_type` (PRODUCTION, STAGING, DEVELOPMENT, PREVIEW)
- **Booleans**: `hasFailures`, `streaming`, `success`
- **Bounded error codes**: A finite, controlled set of error types
- **Shard IDs**: When sharding is bounded (e.g., 0-15)
### DO NOT use high-cardinality attributes:
- **UUIDs/IDs**: `envId`, `userId`, `runId`, `projectId`, `organizationId`
- **Unbounded integers**: `itemCount`, `batchSize`, `retryCount`
- **Timestamps**: `createdAt`, `startTime`
- **Free-form strings**: `errorMessage`, `taskName`, `queueName`
## Example
```typescript
// BAD - High cardinality
this.counter.add(1, {
envId: options.environmentId, // UUID - unbounded
itemCount: options.runCount, // Integer - unbounded
});
// GOOD - Low cardinality
this.counter.add(1, {
environment_type: options.environmentType, // Enum - 4 values
streaming: true, // Boolean - 2 values
});
```
## Prometheus Metric Naming
When metrics are exported via OTLP to Prometheus, the exporter automatically adds unit suffixes to metric names:
| OTel Metric Name | Unit | Prometheus Name |
|------------------|------|-----------------|
| `my_duration_ms` | `ms` | `my_duration_ms_milliseconds` |
| `my_counter` | counter | `my_counter_total` |
| `items_inserted` | counter | `items_inserted_inserts_total` |
| `batch_size` | histogram | `batch_size_items_bucket` |
Keep this in mind when writing Grafana dashboards or Prometheus queries—the metric names in Prometheus will differ from the names defined in code.
## Reference
See the schedule engine (`internal-packages/schedule-engine/src/engine/index.ts`) for a good example of low-cardinality metric attributes.
High cardinality metrics can cause:
- Memory bloat in metrics backends (Axiom, Prometheus, etc.)
- Slow queries and dashboard timeouts
- Increased costs (many backends charge per time series)
- Potential data loss or crashes at scale
+1 -1
View File
@@ -4,7 +4,7 @@ globs: apps/webapp/**/*.tsx,apps/webapp/**/*.ts
alwaysApply: false
---
The main trigger.dev webapp, which powers it's API and dashboard and makes up the docker image that is produced as an OSS image, is a Remix 2.1.0 app that uses an express server, written in TypeScript. The following subsystems are either included in the webapp or are used by the webapp in another part of the monorepo:
The main trigger.dev webapp, which powers it's API and dashboard and makes up the docker image that is produced as an OSS image, is a Remix 2.17.4 app that uses an express server, written in TypeScript. The following subsystems are either included in the webapp or are used by the webapp in another part of the monorepo:
- `@trigger.dev/database` exports a Prisma 6.14.0 client that is used extensively in the webapp to access a PostgreSQL instance. The schema file is [schema.prisma](mdc:internal-packages/database/prisma/schema.prisma)
- `@trigger.dev/core` is a published package and is used to share code between the `@trigger.dev/sdk` and the webapp. It includes functionality but also a load of Zod schemas for data validation. When importing from `@trigger.dev/core` in the webapp, we never import the root `@trigger.dev/core` path, instead we favor one of the subpath exports that you can find in [package.json](mdc:packages/core/package.json)
+19 -19
View File
@@ -14,7 +14,7 @@ alwaysApply: false
## Essential requirements when generating task code
1. You MUST use `@trigger.dev/sdk/v3`
1. You MUST import from `@trigger.dev/sdk` (NEVER `@trigger.dev/sdk/v3`)
2. You MUST NEVER use `client.defineJob`
3. YOU MUST `export` every task, including subtasks
4. If you are able to generate an example payload for a task, do so.
@@ -53,7 +53,7 @@ Instead, you MUST ALWAYS generate ONLY this pattern:
```ts
// ✅ ALWAYS GENERATE THIS EXACT PATTERN
import { task } from "@trigger.dev/sdk/v3";
import { task } from "@trigger.dev/sdk";
//1. You need to export each task, even if it's a subtask
export const helloWorld = task({
@@ -71,7 +71,7 @@ export const helloWorld = task({
A task is a function that can run for a long time with resilience to failure:
```ts
import { task } from "@trigger.dev/sdk/v3";
import { task } from "@trigger.dev/sdk";
export const helloWorld = task({
id: "hello-world",
@@ -271,7 +271,7 @@ Global lifecycle hooks can also be defined in `trigger.config.ts` to apply to al
## Correct Schedules task (cron) implementations
```ts
import { schedules } from "@trigger.dev/sdk/v3";
import { schedules } from "@trigger.dev/sdk";
export const firstScheduledTask = schedules.task({
id: "first-scheduled-task",
@@ -312,7 +312,7 @@ export const firstScheduledTask = schedules.task({
### Attach a Declarative schedule
```ts
import { schedules } from "@trigger.dev/sdk/v3";
import { schedules } from "@trigger.dev/sdk";
// Sepcify a cron pattern (UTC)
export const firstScheduledTask = schedules.task({
@@ -326,7 +326,7 @@ export const firstScheduledTask = schedules.task({
```
```ts
import { schedules } from "@trigger.dev/sdk/v3";
import { schedules } from "@trigger.dev/sdk";
// Specify a specific timezone like this:
export const secondScheduledTask = schedules.task({
@@ -375,7 +375,7 @@ const createdSchedule = await schedules.create({
Schema tasks validate payloads against a schema before execution:
```ts
import { schemaTask } from "@trigger.dev/sdk/v3";
import { schemaTask } from "@trigger.dev/sdk";
import { z } from "zod";
const myTask = schemaTask({
@@ -400,7 +400,7 @@ When you trigger a task from your backend code, you need to set the `TRIGGER_SEC
Triggers a single run of a task with specified payload and options without importing the task. Use type-only imports for full type checking.
```ts
import { tasks } from "@trigger.dev/sdk/v3";
import { tasks } from "@trigger.dev/sdk";
import type { emailSequence } from "~/trigger/emails";
export async function POST(request: Request) {
@@ -418,7 +418,7 @@ export async function POST(request: Request) {
Triggers multiple runs of a single task with different payloads without importing the task.
```ts
import { tasks } from "@trigger.dev/sdk/v3";
import { tasks } from "@trigger.dev/sdk";
import type { emailSequence } from "~/trigger/emails";
export async function POST(request: Request) {
@@ -436,7 +436,7 @@ export async function POST(request: Request) {
Triggers multiple runs of different tasks at once, useful when you need to execute multiple tasks simultaneously.
```ts
import { batch } from "@trigger.dev/sdk/v3";
import { batch } from "@trigger.dev/sdk";
import type { myTask1, myTask2 } from "~/trigger/myTasks";
export async function POST(request: Request) {
@@ -621,7 +621,7 @@ const handle = await myTask.trigger(
Access metadata inside a run:
```ts
import { task, metadata } from "@trigger.dev/sdk/v3";
import { task, metadata } from "@trigger.dev/sdk";
export const myTask = task({
id: "my-task",
@@ -713,7 +713,7 @@ Trigger.dev Realtime enables subscribing to runs for real-time updates on run st
Subscribe to a run after triggering a task:
```ts
import { runs, tasks } from "@trigger.dev/sdk/v3";
import { runs, tasks } from "@trigger.dev/sdk";
async function myBackend() {
const handle = await tasks.trigger("my-task", { some: "data" });
@@ -735,7 +735,7 @@ async function myBackend() {
You can infer types of run's payload and output by passing the task type:
```ts
import { runs } from "@trigger.dev/sdk/v3";
import { runs } from "@trigger.dev/sdk";
import type { myTask } from "./trigger/my-task";
for await (const run of runs.subscribeToRun<typeof myTask>(handle.id)) {
@@ -752,7 +752,7 @@ for await (const run of runs.subscribeToRun<typeof myTask>(handle.id)) {
Stream data in realtime from inside your tasks using the metadata system:
```ts
import { task, metadata } from "@trigger.dev/sdk/v3";
import { task, metadata } from "@trigger.dev/sdk";
import OpenAI from "openai";
export type STREAMS = {
@@ -947,7 +947,7 @@ For most use cases, Realtime hooks are preferred over SWR hooks with polling due
For client-side usage, generate a public access token with appropriate scopes:
```ts
import { auth } from "@trigger.dev/sdk/v3";
import { auth } from "@trigger.dev/sdk";
const publicToken = await auth.createPublicToken({
scopes: {
@@ -967,7 +967,7 @@ Idempotency ensures that an operation produces the same result when called multi
Provide an `idempotencyKey` when triggering a task to ensure it runs only once with that key:
```ts
import { idempotencyKeys, task } from "@trigger.dev/sdk/v3";
import { idempotencyKeys, task } from "@trigger.dev/sdk";
export const myTask = task({
id: "my-task",
@@ -1058,7 +1058,7 @@ function hash(payload: any): string {
```ts
// onFailure executes after all retries are exhausted; use for notifications, logging, or side effects on final failure:
import { task, logger } from "@trigger.dev/sdk/v3";
import { task, logger } from "@trigger.dev/sdk";
export const loggingExample = task({
id: "logging-example",
@@ -1078,7 +1078,7 @@ export const loggingExample = task({
The `trigger.config.ts` file configures your Trigger.dev project, specifying task locations, retry settings, telemetry, and build options.
```ts
import { defineConfig } from "@trigger.dev/sdk/v3";
import { defineConfig } from "@trigger.dev/sdk";
export default defineConfig({
project: "<project ref>",
@@ -1226,7 +1226,7 @@ await myTask.trigger({ name: "Alice", age: 30 });
Before generating any code, you MUST verify:
1. Are you importing from `@trigger.dev/sdk/v3`? If not, STOP and FIX.
1. Are you importing from `@trigger.dev/sdk` (NOT `@trigger.dev/sdk/v3`)? If not, STOP and FIX.
2. Have you exported every task? If not, STOP and FIX.
3. Have you generated any DEPRECATED code patterns? If yes, STOP and FIX.
+77 -6
View File
@@ -29,14 +29,60 @@ REDIS_TLS_DISABLED="true"
DEV_OTEL_EXPORTER_OTLP_ENDPOINT="http://localhost:3030/otel"
DEV_OTEL_BATCH_PROCESSING_ENABLED="0"
# Realtime streams v2 (Sessions, chat.agent, large stream backfills) backed
# by S2 (https://s2.dev). The `s2` service in docker/docker-compose.yml runs
# the open-source s2-lite binary and pre-creates a basin named `trigger-local`
# (see docker/config/s2-spec.json). Comment these out to fall back to v1
# (Redis-only) streams; Sessions and chat.agent then become unavailable.
REALTIME_STREAMS_S2_BASIN=trigger-local
REALTIME_STREAMS_S2_ACCESS_TOKEN=ignored
REALTIME_STREAMS_S2_ENDPOINT=http://localhost:4566/v1
REALTIME_STREAMS_S2_SKIP_ACCESS_TOKENS=true
REALTIME_STREAMS_DEFAULT_VERSION=v2
# Running multiple instances side by side (worktrees, branch experiments)
#
# Every host port in docker/docker-compose.yml is `${VAR:-default}` and the
# project name comes from `COMPOSE_PROJECT_NAME`. To stand up a second stack
# alongside the default one, uncomment the block below in this clone's `.env`
# (pick any offset that doesn't clash with anything else running), then update
# the URL/PORT vars further up to match. Default values are commented for
# reference.
#
# --- core (pnpm run docker) ---
# COMPOSE_PROJECT_NAME=triggerdotdev-docker-alt
# CONTAINER_PREFIX=alt-
# POSTGRES_HOST_PORT=15432 # default 5432
# REDIS_HOST_PORT=16379 # default 6379
# ELECTRIC_HOST_PORT=13060 # default 3060
# MINIO_API_HOST_PORT=19005 # default 9005
# MINIO_CONSOLE_HOST_PORT=19006 # default 9006
# CLICKHOUSE_HTTP_HOST_PORT=18123 # default 8123
# CLICKHOUSE_TCP_HOST_PORT=19000 # default 9000
# S2_HOST_PORT=14566 # default 4566
# REMIX_APP_PORT=13030 # default 3030
# --- extras (only needed if you also run `pnpm run docker:full`) ---
# ELECTRIC_SHARD_1_HOST_PORT=13061 # default 3061
# CH_UI_HOST_PORT=15521 # default 5521
# TOXIPROXY_PROXY_HOST_PORT=40303 # default 30303
# TOXIPROXY_API_HOST_PORT=18474 # default 8474
# NGINX_H2_HOST_PORT=18443 # default 8443
# OTEL_GRPC_HOST_PORT=14317 # default 4317
# OTEL_HTTP_HOST_PORT=14318 # default 4318
# OTEL_PROMETHEUS_HOST_PORT=18889 # default 8889
# PROMETHEUS_HOST_PORT=19090 # default 9090
# GRAFANA_HOST_PORT=13001 # default 3001
# (and update DATABASE_URL / CLICKHOUSE_URL / REDIS_PORT / APP_ORIGIN /
# LOGIN_ORIGIN / ELECTRIC_ORIGIN / REALTIME_STREAMS_S2_ENDPOINT to match)
# When the domain is set to `localhost` the CLI deploy command will only --load the image by default and not --push it
DEPLOY_REGISTRY_HOST=localhost:5000
# OPTIONAL VARIABLES
# This is used for validating emails that are allowed to log in. Every email that do not match this regex will be rejected.
# WHITELISTED_EMAILS="authorized@yahoo\.com|authorized@gmail\.com"
# WHITELISTED_EMAILS="^(authorized@yahoo\.com|authorized@gmail\.com)$"
# Accounts with these emails will get global admin rights. This grants access to the admin UI.
# ADMIN_EMAILS="admin@example\.com|another-admin@example\.com"
# ADMIN_EMAILS="^(admin@example\.com|another-admin@example\.com)$"
# This is used for logging in via GitHub. You can leave these commented out if you don't want to use GitHub for authentication.
# AUTH_GITHUB_CLIENT_ID=
# AUTH_GITHUB_CLIENT_SECRET=
@@ -77,12 +123,37 @@ POSTHOG_PROJECT_KEY=
# DEPOT_TOKEN=<Depot org token>
# DEV_OTEL_EXPORTER_OTLP_ENDPOINT="http://0.0.0.0:4318"
# These are needed for the object store (for handling large payloads/outputs)
# OBJECT_STORE_BASE_URL="https://{bucket}.{accountId}.r2.cloudflarestorage.com"
# OBJECT_STORE_ACCESS_KEY_ID=
# OBJECT_STORE_SECRET_ACCESS_KEY=
#
# Default provider
# OBJECT_STORE_BASE_URL=http://localhost:9005
# OBJECT_STORE_BUCKET=packets
# OBJECT_STORE_ACCESS_KEY_ID=minioadmin
# OBJECT_STORE_SECRET_ACCESS_KEY=minioadmin
# OBJECT_STORE_REGION=us-east-1
# OBJECT_STORE_SERVICE=s3
#
# OBJECT_STORE_DEFAULT_PROTOCOL=s3 # Only specify this if you're going to migrate object storage and set protocol values below
# Named providers (protocol-prefixed data) - optional for multi-provider support
# OBJECT_STORE_S3_BASE_URL=https://s3.amazonaws.com
# OBJECT_STORE_S3_ACCESS_KEY_ID=
# OBJECT_STORE_S3_SECRET_ACCESS_KEY=
# OBJECT_STORE_S3_REGION=us-east-1
# OBJECT_STORE_S3_SERVICE=s3
#
# OBJECT_STORE_R2_BASE_URL=https://{bucket}.{accountId}.r2.cloudflarestorage.com
# OBJECT_STORE_R2_ACCESS_KEY_ID=
# OBJECT_STORE_R2_SECRET_ACCESS_KEY=
# OBJECT_STORE_R2_REGION=auto
# OBJECT_STORE_R2_SERVICE=s3
# CHECKPOINT_THRESHOLD_IN_MS=10000
# These control the server-side internal telemetry
# INTERNAL_OTEL_TRACE_EXPORTER_URL=<URL to send traces to>
# INTERNAL_OTEL_TRACE_LOGGING_ENABLED=1
# INTERNAL_OTEL_TRACE_INSTRUMENT_PRISMA_ENABLED=0,
# INTERNAL_OTEL_TRACE_INSTRUMENT_PRISMA_ENABLED=0
# Enable local observability stack (requires `pnpm run docker:full` to bring up otel-collector + prometheus + grafana)
# Uncomment these to send metrics to the local Prometheus via OTEL Collector:
# INTERNAL_OTEL_METRIC_EXPORTER_ENABLED=1
# INTERNAL_OTEL_METRIC_EXPORTER_URL=http://localhost:4318/v1/metrics
# INTERNAL_OTEL_METRIC_EXPORTER_INTERVAL_MS=15000
+28
View File
@@ -0,0 +1,28 @@
name: Vouch Request
description: Request to be vouched as a contributor
labels: ["vouch-request"]
body:
- type: markdown
attributes:
value: |
## Vouch Request
We use [vouch](https://github.com/mitchellh/vouch) to manage contributor trust. PRs from unvouched users are automatically closed.
To get vouched, fill out this form. A maintainer will review your request and vouch for you by commenting on this issue.
- type: textarea
id: context
attributes:
label: Why do you want to contribute?
description: Tell us a bit about yourself and what you'd like to work on.
placeholder: "I'd like to fix a bug I found in..."
validations:
required: true
- type: textarea
id: prior-work
attributes:
label: Prior contributions or relevant experience
description: Links to previous open source work, relevant projects, or anything that helps us understand your background.
placeholder: "https://github.com/..."
validations:
required: false
+25
View File
@@ -0,0 +1,25 @@
# Vouched contributors for Trigger.dev
# See: https://github.com/mitchellh/vouch
#
# Org members
0ski
D-K-P
ericallam
matt-aitken
mpcgrid
myftija
nicktrn
samejr
isshaddad
# Bots
devin-ai-integration[bot]
dependabot[bot]
# Outside contributors
gautamsi
capaj
chengzp
bharathkumar39293
bhekanik
jrossi
ThullyoCunha
ConProgramming
+16 -14
View File
@@ -23,35 +23,37 @@ runs:
id: get_tag
shell: bash
run: |
if [[ -n "${{ inputs.tag }}" ]]; then
tag="${{ inputs.tag }}"
elif [[ "${{ github.ref_type }}" == "tag" ]]; then
if [[ "${{ github.ref_name }}" == infra-*-* ]]; then
env=$(echo ${{ github.ref_name }} | cut -d- -f2)
sha=$(echo ${{ github.sha }} | head -c7)
if [[ -n "${INPUTS_TAG}" ]]; then
tag="${INPUTS_TAG}"
elif [[ "${GITHUB_REF_TYPE}" == "tag" ]]; then
if [[ "${GITHUB_REF_NAME}" == infra-*-* ]]; then
env=$(echo ${GITHUB_REF_NAME} | cut -d- -f2)
sha=$(echo "${GITHUB_SHA}" | head -c7)
ts=$(date +%s)
tag=${env}-${sha}-${ts}
elif [[ "${{ github.ref_name }}" == re2-*-* ]]; then
env=$(echo ${{ github.ref_name }} | cut -d- -f2)
sha=$(echo ${{ github.sha }} | head -c7)
elif [[ "${GITHUB_REF_NAME}" == re2-*-* ]]; then
env=$(echo ${GITHUB_REF_NAME} | cut -d- -f2)
sha=$(echo "${GITHUB_SHA}" | head -c7)
ts=$(date +%s)
tag=${env}-${sha}-${ts}
elif [[ "${{ github.ref_name }}" == v.docker.* ]]; then
elif [[ "${GITHUB_REF_NAME}" == v.docker.* ]]; then
version="${GITHUB_REF_NAME#v.docker.}"
tag="v${version}"
elif [[ "${{ github.ref_name }}" == build-* ]]; then
elif [[ "${GITHUB_REF_NAME}" == build-* ]]; then
tag="${GITHUB_REF_NAME#build-}"
else
echo "Invalid git tag: ${{ github.ref_name }}"
echo "Invalid git tag: ${GITHUB_REF_NAME}"
exit 1
fi
elif [[ "${{ github.ref_name }}" == "main" ]]; then
elif [[ "${GITHUB_REF_NAME}" == "main" ]]; then
tag="main"
else
echo "Invalid git ref: ${{ github.ref }}"
echo "Invalid git ref: ${GITHUB_REF}"
exit 1
fi
echo "tag=${tag}" >> "$GITHUB_OUTPUT"
env:
INPUTS_TAG: ${{ inputs.tag }}
- name: 🔍 Check for validity
id: check_validity
+12
View File
@@ -0,0 +1,12 @@
version: 2
updates:
- package-ecosystem: github-actions
directory: /
schedule:
interval: weekly
cooldown:
default-days: 7
groups:
github-actions:
patterns:
- "*"
+99
View File
@@ -0,0 +1,99 @@
name: 🦋 Changesets PR
on:
push:
branches:
- main
paths:
- "packages/**"
- ".changeset/**"
- ".server-changes/**"
- "package.json"
- "pnpm-lock.yaml"
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
release-pr:
name: Create Release PR
runs-on: ubuntu-latest
permissions:
contents: write
pull-requests: write
checks: write
if: github.repository == 'triggerdotdev/trigger.dev'
steps:
- name: Checkout
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 # zizmor: ignore[artipacked] changesets/action pushes the release branch; no artifact upload here so no leak path
with:
fetch-depth: 0
- name: Setup pnpm
uses: pnpm/action-setup@fc06bc1257f339d1d5d8b3a19a8cae5388b55320 # v5.0.0
- name: Setup node
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: 20.20.0
cache: "pnpm"
- name: Install dependencies
run: pnpm install --frozen-lockfile
- name: Create release PR
id: changesets
uses: changesets/action@6a0a831ff30acef54f2c6aa1cbbc1096b066edaf # v1.7.0
with:
version: pnpm run changeset:version
commit: "chore: release"
title: "chore: release"
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: Update PR title and enhance body
if: steps.changesets.outputs.published != 'true'
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
PR_NUMBER=$(gh pr list --head changeset-release/main --json number --jq '.[0].number')
if [ -n "$PR_NUMBER" ]; then
git fetch origin changeset-release/main
# we arbitrarily reference the version of the cli package here; it is the same for all package releases
VERSION=$(git show origin/changeset-release/main:packages/cli-v3/package.json | jq -r '.version')
gh pr edit "$PR_NUMBER" --title "chore: release v$VERSION"
# Enhance the PR body with a clean, deduplicated summary
RAW_BODY=$(gh pr view "$PR_NUMBER" --json body --jq '.body')
ENHANCED_BODY=$(CHANGESET_PR_BODY="$RAW_BODY" node scripts/enhance-release-pr.mjs "$VERSION")
if [ -n "$ENHANCED_BODY" ]; then
gh api repos/triggerdotdev/trigger.dev/pulls/"$PR_NUMBER" \
-X PATCH \
-f body="$ENHANCED_BODY"
fi
fi
# The changesets bot authors release PRs with GITHUB_TOKEN, which by GitHub
# design cannot trigger downstream workflows. That leaves the required
# "All PR Checks" status permanently Expected and the PR unmergeable.
# The release PR only bumps package.json + lockfile + CHANGELOGs from
# changesets already on main, so we self-report the required check as
# success. If a human ever pushes to changeset-release/main, the real
# pr_checks.yml fires and its result overwrites this one (last write wins
# for the same context on the same SHA).
- name: Self-report "All PR Checks" success on release PR
if: steps.changesets.outputs.published != 'true'
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
PR_NUMBER=$(gh pr list --head changeset-release/main --json number --jq '.[0].number')
if [ -z "$PR_NUMBER" ]; then exit 0; fi
HEAD_SHA=$(gh pr view "$PR_NUMBER" --json headRefOid --jq '.headRefOid')
gh api -X POST repos/${{ github.repository }}/check-runs \
-f name="All PR Checks" \
-f head_sha="$HEAD_SHA" \
-f status=completed \
-f conclusion=success \
-f 'output[title]=Auto-pass for changeset release PR' \
-f 'output[summary]=Required check auto-satisfied for changeset-release/main PRs. Full CI ran on the underlying commits before they landed on main.'
+93
View File
@@ -0,0 +1,93 @@
name: 🔎 REVIEW.md Drift Audit
on:
pull_request:
types: [opened, ready_for_review, synchronize]
paths-ignore:
- "docs/**"
- ".changeset/**"
- ".server-changes/**"
- "references/**"
concurrency:
group: review-md-drift-${{ github.event.pull_request.number }}
cancel-in-progress: true
jobs:
audit:
if: >-
github.event.pull_request.draft == false &&
github.event.pull_request.head.repo.full_name == github.repository
runs-on: ubuntu-latest
permissions:
contents: read
pull-requests: write
id-token: write
steps:
- name: Checkout repository
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
fetch-depth: 0
persist-credentials: false
- name: Run Claude Code
id: claude
uses: anthropics/claude-code-action@fefa07e9c665b7320f08c3b525980457f22f58aa # v1.0.111
with:
anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
use_sticky_comment: true
allowed_bots: "devin-ai-integration[bot]"
claude_args: |
--max-turns 30
--allowedTools "Read,Glob,Grep,Bash(git diff:*)"
prompt: |
You are auditing this PR for drift against `.claude/REVIEW.md`.
## Context
`.claude/REVIEW.md` is the repo's source of truth for what AI / agent code reviewers should treat as critical findings (rolling-deploy safety, hot-table indexes, recovery-path queries, testcontainers usage, Lua versioning, etc.). It is consumed by review agents to calibrate severity. If REVIEW.md goes stale, every future agent review degrades.
## Strategy — read this first
You have a hard turn budget. Spend it on signal, not coverage. The audit is allowed to miss things; it is NOT allowed to time out.
1. Read `.claude/REVIEW.md` once, in full.
2. Run `git diff origin/main...HEAD --name-only` to get the list of changed files. Do NOT read the diff content yet.
3. Scan the file-list for relevance to REVIEW.md scope. Relevance signals: changes to Prisma schema, Redis / queue / Lua code, hot tables, recovery / restart loops, new packages, deletions of paths REVIEW.md cites. Skim everything else.
4. Open at most **5 files** total — only the ones most likely to surface a real signal. If nothing in the file-list looks relevant to any REVIEW.md rule, do NOT read any files; go straight to the verdict.
5. Form a verdict and stop. Do not exhaust the turn budget exploring.
Large PRs (>50 files changed) are a strong signal to be MORE selective, not more thorough. Pick 3-5 files at most.
## What to look for
- **Stale references** — does any REVIEW.md rule cite a file, directory, function, table, Prisma model, or package name that has been removed or renamed in this PR (or is already gone from `main`)?
- **Contradictions** — does code in this PR clearly violate a current REVIEW.md rule? (Don't re-review the PR. Only flag if REVIEW.md and the PR plainly disagree.)
- **Missing rules** — does this PR introduce a new pattern future reviewers should know about? Examples: a new hot table, a new Lua-script versioning convention, a new safety wrapper, a new "must always check" invariant.
- **Obsolete rules** — has the repo moved past a constraint REVIEW.md still asserts? (e.g. a deprecated path is gone, a pattern is now linted, V1 code is deleted.)
## Response format
If nothing needs changing:
✅ REVIEW.md looks current for this PR.
Otherwise:
📝 **REVIEW.md updates suggested:**
- **[stale]** `<rule excerpt>` — <what's stale and why>
- **[contradiction]** `<rule excerpt>` — <what in this PR disagrees>
- **[missing]** under `## <section>` — <one-sentence draft rule>
- **[obsolete]** `<rule excerpt>` — <why this rule no longer applies>
## Rules
- Maximum 3 suggestions per audit. Pick the highest-signal ones.
- Only flag things that would actually mislead a future reviewer. Style and wording do not count.
- Do NOT review the PR itself. Do NOT propose rules outside REVIEW.md's existing sections.
- Do NOT propose rules for one-off PR specifics that don't generalize to future PRs.
- If REVIEW.md does not exist in the repo, respond with `(skip)` and stop.
- When in doubt between "one more file read" and "finish now" — finish now.
+73
View File
@@ -0,0 +1,73 @@
name: 📝 CLAUDE.md Audit
on:
pull_request:
types: [opened, ready_for_review, synchronize]
paths-ignore:
- "docs/**"
- ".changeset/**"
- ".server-changes/**"
- "**/*.md"
- "references/**"
concurrency:
group: claude-md-audit-${{ github.event.pull_request.number }}
cancel-in-progress: true
jobs:
audit:
if: >-
github.event.pull_request.draft == false &&
github.event.pull_request.head.repo.full_name == github.repository
runs-on: ubuntu-latest
permissions:
contents: read
pull-requests: write
issues: write
id-token: write
steps:
- name: Checkout repository
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
fetch-depth: 0
persist-credentials: false
- name: Run Claude Code
id: claude
uses: anthropics/claude-code-action@fefa07e9c665b7320f08c3b525980457f22f58aa # v1.0.111
with:
anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
use_sticky_comment: true
allowed_bots: "devin-ai-integration[bot]"
claude_args: |
--max-turns 15
--allowedTools "Read,Glob,Grep,Bash(git diff:*)"
prompt: |
You are reviewing a PR to check whether any CLAUDE.md files or .claude/rules/ files need updating.
## Your task
1. Run `git diff origin/main...HEAD --name-only` to see which files changed in this PR.
2. For each changed directory, check if there's a CLAUDE.md in that directory or a parent directory.
3. Determine if any CLAUDE.md or .claude/rules/ file should be updated based on the changes. Consider:
- New files/directories that aren't covered by existing documentation
- Changed architecture or patterns that contradict current CLAUDE.md guidance
- New dependencies, services, or infrastructure that Claude should know about
- Renamed or moved files that are referenced in CLAUDE.md
- Changes to build commands, test patterns, or development workflows
## Response format
If NO updates are needed, respond with exactly:
✅ CLAUDE.md files look current for this PR.
If updates ARE needed, respond with a short list:
📝 **CLAUDE.md updates suggested:**
- `path/to/CLAUDE.md`: [what should be added/changed]
- `.claude/rules/file.md`: [what should be added/changed]
Keep suggestions specific and brief. Only flag things that would actually mislead Claude in future sessions.
Do NOT suggest updates for trivial changes (bug fixes, small refactors within existing patterns).
Do NOT suggest creating new CLAUDE.md files - only updates to existing ones.
+71
View File
@@ -0,0 +1,71 @@
name: Claude Code
on:
issue_comment:
types: [created]
pull_request_review_comment:
types: [created]
issues:
types: [opened, assigned]
pull_request_review:
types: [submitted]
jobs:
claude:
if: |
(github.event_name == 'issue_comment' && contains(github.event.comment.body, '@claude')) ||
(github.event_name == 'pull_request_review_comment' && contains(github.event.comment.body, '@claude')) ||
(github.event_name == 'pull_request_review' && contains(github.event.review.body, '@claude')) ||
(github.event_name == 'issues' && (contains(github.event.issue.body, '@claude') || contains(github.event.issue.title, '@claude')))
runs-on: ubuntu-latest
permissions:
contents: write
pull-requests: write
issues: write
id-token: write
actions: read # Required for Claude to read CI results on PRs
steps:
- name: Checkout repository
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
fetch-depth: 1
persist-credentials: false
- name: ⎔ Setup pnpm
uses: pnpm/action-setup@fc06bc1257f339d1d5d8b3a19a8cae5388b55320 # v5.0.0
with:
version: 10.33.2
- name: ⎔ Setup node
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: 20.20.0
cache: "pnpm"
- name: 📥 Download deps
run: pnpm install --frozen-lockfile
- name: 📀 Generate Prisma Client
run: pnpm run generate
- name: Run Claude Code
id: claude
uses: anthropics/claude-code-action@fefa07e9c665b7320f08c3b525980457f22f58aa # v1.0.111
with:
anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
# This is an optional setting that allows Claude to read CI results on PRs
additional_permissions: |
actions: read
claude_args: |
--model claude-opus-4-5-20251101
--allowedTools "Bash(pnpm:*),Bash(turbo:*),Bash(git:*),Bash(gh:*),Bash(npx:*),Bash(docker:*),Edit,MultiEdit,Read,Write,Glob,Grep,LS,Task"
# Optional: Give a custom prompt to Claude. If this is not specified, Claude will perform the instructions specified in the comment that tagged it.
# prompt: 'Update the pull request description to include a summary of changes.'
# Optional: Add claude_args to customize behavior and configuration
# See https://github.com/anthropics/claude-code-action/blob/main/docs/usage.md
# or https://code.claude.com/docs/en/cli-reference for available options
# claude_args: '--allowed-tools Bash(gh pr:*)'
@@ -0,0 +1,83 @@
name: Dependabot Critical Alerts
on:
schedule:
- cron: "0 8 * * *" # Daily 08:00 UTC
workflow_dispatch:
inputs:
severity:
description: "Severity to alert on"
type: choice
options:
- critical
- high
- medium
- low
default: critical
concurrency:
group: ${{ github.workflow }}
cancel-in-progress: false
permissions:
contents: read
jobs:
alert:
name: Post critical alerts
runs-on: ubuntu-latest
environment: dependabot-summary
env:
SEVERITY: ${{ inputs.severity || 'critical' }}
steps:
- name: Fetch alerts
id: alerts
env:
GH_TOKEN: ${{ secrets.DEPENDABOT_ALERTS_TOKEN }}
REPO: ${{ github.repository }}
run: |
set -euo pipefail
gh api -X GET "/repos/$REPO/dependabot/alerts" \
-F state=open -F severity="$SEVERITY" --paginate > pages.json
jq -s 'add' pages.json > alerts.json
TOTAL=$(jq 'length' alerts.json)
echo "total=$TOTAL" >> "$GITHUB_OUTPUT"
if [ "$TOTAL" = "0" ]; then
exit 0
fi
LIST=$(jq -r '
map("• <\(.html_url)|#\(.number)> *\(.dependency.package.name)* - \(.security_advisory.summary)")
| join("\n")
' alerts.json)
{
echo "list<<EOF"
echo "$LIST"
echo "EOF"
} >> "$GITHUB_OUTPUT"
- name: Build Slack payload
if: steps.alerts.outputs.total != '0'
env:
REPO: ${{ github.repository }}
CHANNEL: ${{ vars.SLACK_CHANNEL_ID }}
TOTAL: ${{ steps.alerts.outputs.total }}
LIST: ${{ steps.alerts.outputs.list }}
run: |
jq -n \
--arg channel "$CHANNEL" \
--arg repo "$REPO" \
--arg total "$TOTAL" \
--arg list "$LIST" \
--arg severity "$SEVERITY" \
'{
channel: $channel,
text: ":bufo-alarma: `\($repo)` - *\($total) open \($severity) alert(s)*\n\($list)\n\n<https://github.com/\($repo)/security/dependabot?q=is%3Aopen+severity%3A\($severity)|View \($severity) alerts>"
}' > payload.json
- name: Post Slack alert
if: steps.alerts.outputs.total != '0'
uses: slackapi/slack-github-action@45a88b9581bfab2566dc881e2cd66d334e621e2c # v3.0.3
with:
method: chat.postMessage
token: ${{ secrets.SLACK_BOT_TOKEN }}
payload-file-path: payload.json
@@ -0,0 +1,206 @@
name: Dependabot Weekly Summary
on:
schedule:
- cron: "0 8 * * 1" # Mon 08:00 UTC
workflow_dispatch:
# Single-purpose monitoring workflow; serialise on workflow name only - we never
# want two concurrent summary runs racing to post the same digest.
concurrency:
group: ${{ github.workflow }}
cancel-in-progress: false
permissions:
contents: read # gh CLI baseline
pull-requests: read # gh pr list (open dependabot PRs)
actions: read # gh run list / view (parse latest dependabot run logs)
jobs:
summary:
name: Post weekly Dependabot summary
runs-on: ubuntu-latest
environment: dependabot-summary
env:
# Severities surface in the actions list when their remaining TTR drops
# below this many days. Override via repo/env var ACTION_THRESHOLD_DAYS.
THRESHOLD_DAYS: ${{ vars.ACTION_THRESHOLD_DAYS || '7' }}
steps:
- name: Fetch alerts and compute summaries
id: alerts
env:
GH_TOKEN: ${{ secrets.DEPENDABOT_ALERTS_TOKEN }}
REPO: ${{ github.repository }}
run: |
if ! gh api -X GET "/repos/$REPO/dependabot/alerts" --paginate > pages.json 2> err.txt; then
echo "total=?" >> "$GITHUB_OUTPUT"
ERR=$(head -c 200 err.txt | tr '\n' ' ')
echo "by_severity=:x: _failed to fetch alerts: ${ERR}_" >> "$GITHUB_OUTPUT"
echo "actions=:x: _alerts unavailable_" >> "$GITHUB_OUTPUT"
exit 0
fi
jq -s '[.[][] | select(.state == "open")]' pages.json > open.json
TOTAL=$(jq 'length' open.json)
echo "total=$TOTAL" >> "$GITHUB_OUTPUT"
if [ "$TOTAL" = "0" ]; then
echo "by_severity=:white_check_mark: No open alerts." >> "$GITHUB_OUTPUT"
echo "actions=_None_" >> "$GITHUB_OUTPUT"
exit 0
fi
# Severity breakdown - real newlines so jq --arg in the payload
# builder encodes them as proper \n in JSON (Slack renders as breaks).
BY_SEV=$(jq -r '
group_by(.security_advisory.severity)
| map({sev: .[0].security_advisory.severity,
count: length,
weight: ({"critical":0,"high":1,"medium":2,"low":3}[.[0].security_advisory.severity])})
| sort_by(.weight)
| map("• *\(.count)* \(.sev)")
| join("\n")
' open.json)
{
echo "by_severity<<EOF"
echo "$BY_SEV"
echo "EOF"
} >> "$GITHUB_OUTPUT"
# Actions: alerts within THRESHOLD_DAYS of their TTR (P0=7d, P1=30d, P2=90d, P3=no deadline)
# Grouped by (package, severity); shows earliest deadline per group.
ACTIONS=$(jq -r --argjson threshold "$THRESHOLD_DAYS" '
[.[]
| (.security_advisory.severity) as $sev
| ({"critical":7,"high":30,"medium":90,"low":null}[$sev]) as $ttr
| select($ttr != null)
| ((now - (.created_at | fromdateiso8601)) / 86400 | floor) as $age
| {pkg: .dependency.package.name, sev: $sev, remaining: ($ttr - $age)}
]
| group_by([.pkg, .sev])
| map({pkg: .[0].pkg, sev: .[0].sev, count: length, min_remaining: ([.[].remaining] | min)})
| map(select(.min_remaining < $threshold))
| sort_by(.min_remaining)
| if length == 0 then "_None_"
else (map(
"• *\(.pkg)* (\(.sev))" +
(if .count > 1 then " ×\(.count)" else "" end) + " - " +
(if .min_remaining < 0 then "*OVERDUE* by \(-.min_remaining)d"
else "\(.min_remaining)d remaining" end)
) | join("\n"))
end
' open.json)
{
echo "actions<<EOF"
echo "$ACTIONS"
echo "EOF"
} >> "$GITHUB_OUTPUT"
- name: Fetch open dependabot PRs
id: prs
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
REPO: ${{ github.repository }}
REPO_URL: https://github.com/${{ github.repository }}
run: |
if ! PR_JSON=$(gh pr list --repo "$REPO" --state open --author "app/dependabot" --json number,title 2> err.txt); then
ERR=$(head -c 200 err.txt | tr '\n' ' ')
echo "list=:x: _failed to fetch PRs: ${ERR}_" >> "$GITHUB_OUTPUT"
exit 0
fi
LIST=$(echo "$PR_JSON" | jq -r --arg url "$REPO_URL" '
if length == 0 then "_None_"
else (map("• <\($url)/pull/\(.number)|#\(.number)> \(.title)") | join("\n"))
end
')
{
echo "list<<EOF"
echo "$LIST"
echo "EOF"
} >> "$GITHUB_OUTPUT"
- name: Find latest npm dependabot run
id: latest
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
REPO: ${{ github.repository }}
run: |
# Repos without a dependabot.yml have no "Dependabot Updates" workflow;
# treat the lookup failure as "no recent run found" rather than failing.
if ! RUN_ID=$(gh run list --repo "$REPO" --workflow "Dependabot Updates" --status success --limit 30 --json databaseId,name --jq 'first(.[] | select(.name | startswith("npm_and_yarn")) | .databaseId) // empty' 2>/dev/null); then
RUN_ID=""
fi
echo "run_id=$RUN_ID" >> "$GITHUB_OUTPUT"
- name: Extract stuck deps (only if actions pending)
id: stuck
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
REPO: ${{ github.repository }}
RUN_ID: ${{ steps.latest.outputs.run_id }}
ACTIONS: ${{ steps.alerts.outputs.actions }}
run: |
# Skip the stuck section entirely when nothing in the actions list
# - keeps the digest tidy when there's nothing to actually act on.
if [ "$ACTIONS" = "_None_" ]; then
echo "section=" >> "$GITHUB_OUTPUT"
exit 0
fi
HEADER=$'\n\n*Couldn\'t auto-fix (need manual `pnpm.overrides`):*\n'
if [ -z "$RUN_ID" ]; then
{
echo "section<<EOF"
echo "${HEADER}_(no recent npm run found)_"
echo "EOF"
} >> "$GITHUB_OUTPUT"
exit 0
fi
gh run view "$RUN_ID" --repo "$REPO" --log > log.txt 2>&1 || true
STUCK=$(grep -oE "No update possible for [^[:space:]]+ [0-9][^[:space:]]*" log.txt | sed 's/No update possible for //' | sort -u || true)
if [ -z "$STUCK" ]; then
{
echo "section<<EOF"
echo "${HEADER}_None_"
echo "EOF"
} >> "$GITHUB_OUTPUT"
exit 0
fi
LIST=$(echo "$STUCK" | awk 'NR>1{printf "\n"} {printf "• *%s* %s", $1, $2}')
{
echo "section<<EOF"
echo "${HEADER}${LIST}"
echo "EOF"
} >> "$GITHUB_OUTPUT"
- name: Build Slack payload
env:
REPO: ${{ github.repository }}
CHANNEL: ${{ vars.SLACK_CHANNEL_ID }}
TOTAL: ${{ steps.alerts.outputs.total }}
BY_SEVERITY: ${{ steps.alerts.outputs.by_severity }}
PRS_LIST: ${{ steps.prs.outputs.list }}
ACTIONS: ${{ steps.alerts.outputs.actions }}
STUCK: ${{ steps.stuck.outputs.section }}
run: |
# Build payload via jq so PR titles or error strings containing
# quotes/backslashes/newlines can't break the JSON.
jq -n \
--arg channel "$CHANNEL" \
--arg repo "$REPO" \
--arg total "$TOTAL" \
--arg by_severity "$BY_SEVERITY" \
--arg prs_list "$PRS_LIST" \
--arg actions "$ACTIONS" \
--arg stuck "$STUCK" \
--arg threshold "$THRESHOLD_DAYS" \
'{
channel: $channel,
text: ":calendar: *Weekly Dependabot summary* - `\($repo)`\n\n*Open alerts (\($total)):*\n\($by_severity)\n\n*Open Dependabot PRs:*\n\($prs_list)\n\n*Actions needed (<\($threshold)d remaining):*\n\($actions)\($stuck)\n\n<https://github.com/\($repo)/security/dependabot|Dependabot alerts>"
}' > payload.json
- name: Post Slack summary
uses: slackapi/slack-github-action@45a88b9581bfab2566dc881e2cd66d334e621e2c # v3.0.3
with:
method: chat.postMessage
token: ${{ secrets.SLACK_BOT_TOKEN }}
payload-file-path: payload.json
+4 -2
View File
@@ -26,10 +26,12 @@ jobs:
working-directory: ./docs
steps:
- name: 📥 Checkout repository
uses: actions/checkout@v4
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
persist-credentials: false
- name: 📦 Cache npm
uses: actions/cache@v4
uses: actions/cache@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5
with:
path: |
~/.npm
+120
View File
@@ -0,0 +1,120 @@
name: "🛡️ E2E Tests: Webapp Auth (full)"
# Comprehensive RBAC auth test suite — see TRI-8731. Runs separately from
# the smoke e2e-webapp.yml because it covers every route family with a
# pass/fail matrix and would otherwise dominate per-PR CI time.
#
# Triggered:
# - Manually via workflow_dispatch.
# - Nightly via schedule.
# - On pull requests touching auth-relevant files only (paths filter).
permissions:
contents: read
on:
workflow_dispatch:
schedule:
- cron: "0 4 * * *" # 04:00 UTC daily
pull_request:
paths:
- "apps/webapp/app/services/routeBuilders/**"
- "apps/webapp/app/services/rbac.server.ts"
- "apps/webapp/app/services/apiAuth.server.ts"
- "apps/webapp/app/services/personalAccessToken.server.ts"
- "apps/webapp/app/services/sessionStorage.server.ts"
- "apps/webapp/app/routes/api.v*.**"
- "apps/webapp/app/routes/realtime.v*.**"
- "apps/webapp/test/**/*.e2e.full.test.ts"
- "apps/webapp/test/setup/global-e2e-full-setup.ts"
- "apps/webapp/test/helpers/sharedTestServer.ts"
- "apps/webapp/test/helpers/seedTestSession.ts"
- "apps/webapp/vitest.e2e.full.config.ts"
- "internal-packages/rbac/**"
- "packages/plugins/**"
- ".github/workflows/e2e-webapp-auth-full.yml"
jobs:
e2eAuthFull:
name: "🛡️ E2E Auth Tests (full)"
runs-on: ubuntu-latest
timeout-minutes: 30
env:
DOCKERHUB_USERNAME: ${{ secrets.DOCKERHUB_USERNAME }}
steps:
- name: 🔧 Disable IPv6
run: |
sudo sysctl -w net.ipv6.conf.all.disable_ipv6=1
sudo sysctl -w net.ipv6.conf.default.disable_ipv6=1
sudo sysctl -w net.ipv6.conf.lo.disable_ipv6=1
- name: 🔧 Configure docker address pool
run: |
CONFIG='{
"default-address-pools" : [
{
"base" : "172.17.0.0/12",
"size" : 20
},
{
"base" : "192.168.0.0/16",
"size" : 24
}
]
}'
mkdir -p /etc/docker
echo "$CONFIG" | sudo tee /etc/docker/daemon.json
- name: 🔧 Restart docker daemon
run: sudo systemctl restart docker
- name: ⬇️ Checkout repo
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
fetch-depth: 0
# Don't leave the GITHUB_TOKEN in .git/config — this job
# doesn't need to push and the persisted creds would be
# readable from any subsequent step (zizmor/artipacked).
persist-credentials: false
- name: ⎔ Setup pnpm
uses: pnpm/action-setup@fc06bc1257f339d1d5d8b3a19a8cae5388b55320 # v5.0.0
with:
version: 10.33.2
- name: ⎔ Setup node
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: 20.20.0
cache: "pnpm"
- name: 🐳 Login to DockerHub
if: ${{ env.DOCKERHUB_USERNAME }}
uses: docker/login-action@4907a6ddec9925e35a0a9e82d7399ccc52663121 # v4.1.0
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: 🐳 Skipping DockerHub login (no secrets available)
if: ${{ !env.DOCKERHUB_USERNAME }}
run: echo "DockerHub login skipped because secrets are not available."
- name: 🐳 Pre-pull testcontainer images
if: ${{ env.DOCKERHUB_USERNAME }}
run: |
docker pull postgres:14
docker pull redis:7.2
docker pull testcontainers/ryuk:0.11.0
- name: 📥 Download deps
run: pnpm install --frozen-lockfile
- name: 📀 Generate Prisma Client
run: pnpm run generate
- name: 🏗️ Build Webapp
run: pnpm run build --filter webapp
- name: 🛡️ Run Webapp Full Auth E2E Tests
run: cd apps/webapp && pnpm exec vitest run --config vitest.e2e.full.config.ts --reporter=default
env:
WEBAPP_TEST_VERBOSE: "1"
+97
View File
@@ -0,0 +1,97 @@
name: "🧪 E2E Tests: Webapp"
permissions:
contents: read
on:
workflow_call:
secrets:
DOCKERHUB_USERNAME:
required: false
DOCKERHUB_TOKEN:
required: false
jobs:
e2eTests:
name: "🧪 E2E Tests: Webapp"
runs-on: ubuntu-latest
timeout-minutes: 20
env:
DOCKERHUB_USERNAME: ${{ secrets.DOCKERHUB_USERNAME }}
steps:
- name: 🔧 Disable IPv6
run: |
sudo sysctl -w net.ipv6.conf.all.disable_ipv6=1
sudo sysctl -w net.ipv6.conf.default.disable_ipv6=1
sudo sysctl -w net.ipv6.conf.lo.disable_ipv6=1
- name: 🔧 Configure docker address pool
run: |
CONFIG='{
"default-address-pools" : [
{
"base" : "172.17.0.0/12",
"size" : 20
},
{
"base" : "192.168.0.0/16",
"size" : 24
}
]
}'
mkdir -p /etc/docker
echo "$CONFIG" | sudo tee /etc/docker/daemon.json
- name: 🔧 Restart docker daemon
run: sudo systemctl restart docker
- name: ⬇️ Checkout repo
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
fetch-depth: 0
persist-credentials: false
- name: ⎔ Setup pnpm
uses: pnpm/action-setup@fc06bc1257f339d1d5d8b3a19a8cae5388b55320 # v5.0.0
with:
version: 10.33.2
- name: ⎔ Setup node
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: 20.20.0
cache: "pnpm"
# ..to avoid rate limits when pulling images
- name: 🐳 Login to DockerHub
if: ${{ env.DOCKERHUB_USERNAME }}
uses: docker/login-action@4907a6ddec9925e35a0a9e82d7399ccc52663121 # v4.1.0
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: 🐳 Skipping DockerHub login (no secrets available)
if: ${{ !env.DOCKERHUB_USERNAME }}
run: echo "DockerHub login skipped because secrets are not available."
- name: 🐳 Pre-pull testcontainer images
if: ${{ env.DOCKERHUB_USERNAME }}
run: |
echo "Pre-pulling Docker images with authenticated session..."
docker pull postgres:14
docker pull redis:7.2
docker pull testcontainers/ryuk:0.11.0
echo "Image pre-pull complete"
- name: 📥 Download deps
run: pnpm install --frozen-lockfile
- name: 📀 Generate Prisma Client
run: pnpm run generate
- name: 🏗️ Build Webapp
run: pnpm run build --filter webapp
- name: 🧪 Run Webapp E2E Tests
run: cd apps/webapp && pnpm exec vitest run --config vitest.e2e.config.ts --reporter=default
env:
WEBAPP_TEST_VERBOSE: "1"
+7 -6
View File
@@ -24,19 +24,20 @@ jobs:
package-manager: ["npm", "pnpm"]
steps:
- name: ⬇️ Checkout repo
uses: actions/checkout@v4
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
fetch-depth: 0
persist-credentials: false
- name: ⎔ Setup pnpm
uses: pnpm/action-setup@v4
uses: pnpm/action-setup@fc06bc1257f339d1d5d8b3a19a8cae5388b55320 # v5.0.0
with:
version: 8.15.5
version: 10.33.2
- name: ⎔ Setup node
uses: buildjet/setup-node@v4
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: 20.11.1
node-version: 20.20.0
- name: 📥 Download deps
run: pnpm install --frozen-lockfile --filter trigger.dev...
@@ -48,7 +49,7 @@ jobs:
run: pnpm run build --filter trigger.dev^...
- name: 🔧 Build worker template files
run: pnpm --filter trigger.dev run build:workers
run: pnpm --filter trigger.dev run --if-present build:workers
- name: Enable corepack
run: corepack enable
+200
View File
@@ -0,0 +1,200 @@
name: 🧭 Helm Chart Prerelease
on:
pull_request:
types: [opened, synchronize, reopened]
paths:
- "hosting/k8s/helm/**"
push:
branches:
- main
paths:
- "hosting/k8s/helm/**"
workflow_dispatch:
inputs:
app_version:
description: "Override appVersion (e.g. 'main', 'v4.4.4'). Leave empty to keep Chart.yaml value."
required: false
type: string
default: ""
concurrency:
group: helm-prerelease-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
env:
REGISTRY: ghcr.io
CHART_NAME: trigger
jobs:
lint-and-test:
runs-on: ubuntu-latest
permissions:
contents: read
steps:
- name: Checkout
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
persist-credentials: false
- name: Set up Helm
uses: azure/setup-helm@dda3372f752e03dde6b3237bc9431cdc2f7a02a2 # v5.0.0
with:
version: "3.18.3"
- name: Build dependencies
run: helm dependency build ./hosting/k8s/helm/
- name: Extract dependency charts
run: |
cd ./hosting/k8s/helm/
for file in ./charts/*.tgz; do echo "Extracting $file"; tar -xzf "$file" -C ./charts; done
- name: Lint Helm Chart
run: |
helm lint ./hosting/k8s/helm/
- name: Render templates
run: |
helm template test-release ./hosting/k8s/helm/ \
--values ./hosting/k8s/helm/values.yaml \
--output-dir ./helm-output
- name: Validate manifests
uses: docker://ghcr.io/yannh/kubeconform:v0.7.0@sha256:85dbef6b4b312b99133decc9c6fc9495e9fc5f92293d4ff3b7e1b30f5611823c
with:
entrypoint: "/kubeconform"
args: "-summary -output json ./helm-output"
prerelease:
needs: lint-and-test
if: |
(github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == github.repository) ||
github.event_name == 'push' ||
github.event_name == 'workflow_dispatch'
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
pull-requests: write
steps:
- name: Checkout
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
persist-credentials: false
- name: Set up Helm
uses: azure/setup-helm@dda3372f752e03dde6b3237bc9431cdc2f7a02a2 # v5.0.0
with:
version: "3.18.3"
- name: Build dependencies
run: helm dependency build ./hosting/k8s/helm/
- name: Extract dependency charts
run: |
cd ./hosting/k8s/helm/
for file in ./charts/*.tgz; do echo "Extracting $file"; tar -xzf "$file" -C ./charts; done
- name: Log in to Container Registry
uses: docker/login-action@4907a6ddec9925e35a0a9e82d7399ccc52663121 # v4.1.0
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Generate prerelease version
id: version
run: |
BASE_VERSION=$(grep '^version:' ./hosting/k8s/helm/Chart.yaml | awk '{print $2}')
if [[ "${{ github.event_name }}" == "pull_request" ]]; then
PR_NUMBER=${{ github.event.pull_request.number }}
SHORT_SHA=$(echo "${{ github.event.pull_request.head.sha }}" | cut -c1-7)
PRERELEASE_VERSION="${BASE_VERSION}-pr${PR_NUMBER}.${SHORT_SHA}"
elif [[ "${{ github.event_name }}" == "push" ]]; then
SHORT_SHA=$(echo "${GITHUB_SHA}" | cut -c1-7)
PRERELEASE_VERSION="${BASE_VERSION}-main.${SHORT_SHA}"
else
SHORT_SHA=$(echo "${GITHUB_SHA}" | cut -c1-7)
REF_SLUG=$(echo "${GITHUB_REF_NAME}" | tr '/' '-' | tr -cd 'a-zA-Z0-9-')
if [[ -z "$REF_SLUG" ]]; then
REF_SLUG="manual"
fi
PRERELEASE_VERSION="${BASE_VERSION}-${REF_SLUG}.${SHORT_SHA}"
fi
echo "version=$PRERELEASE_VERSION" >> "$GITHUB_OUTPUT"
echo "Prerelease version: $PRERELEASE_VERSION"
- name: Update Chart.yaml with prerelease version
run: |
sed -i "s/^version:.*/version: ${STEPS_VERSION_OUTPUTS_VERSION}/" ./hosting/k8s/helm/Chart.yaml
env:
STEPS_VERSION_OUTPUTS_VERSION: ${{ steps.version.outputs.version }}
- name: Override appVersion
if: github.event_name == 'workflow_dispatch' && inputs.app_version != ''
env:
APP_VERSION: ${{ inputs.app_version }}
run: |
yq -i '.appVersion = strenv(APP_VERSION)' ./hosting/k8s/helm/Chart.yaml
- name: Package Helm Chart
run: |
helm package ./hosting/k8s/helm/ --destination /tmp/
- name: Push Helm Chart to GHCR
run: |
VERSION="${STEPS_VERSION_OUTPUTS_VERSION}"
CHART_PACKAGE="/tmp/${{ env.CHART_NAME }}-${VERSION}.tgz"
# Push to GHCR OCI registry
helm push "$CHART_PACKAGE" "oci://${{ env.REGISTRY }}/${{ github.repository_owner }}/charts"
env:
STEPS_VERSION_OUTPUTS_VERSION: ${{ steps.version.outputs.version }}
- name: Write run summary
run: |
{
echo "### 🧭 Helm Chart Prerelease Published"
echo ""
echo "**Version:** \`${STEPS_VERSION_OUTPUTS_VERSION}\`"
echo ""
echo "**Install:**"
echo '```bash'
echo "helm upgrade --install trigger \\"
echo " oci://${{ env.REGISTRY }}/${{ github.repository_owner }}/charts/${{ env.CHART_NAME }} \\"
echo " --version \"${STEPS_VERSION_OUTPUTS_VERSION}\""
echo '```'
} >> "$GITHUB_STEP_SUMMARY"
env:
STEPS_VERSION_OUTPUTS_VERSION: ${{ steps.version.outputs.version }}
- name: Find existing comment
if: github.event_name == 'pull_request'
uses: peter-evans/find-comment@b30e6a3c0ed37e7c023ccd3f1db5c6c0b0c23aad # v4.0.0
id: find-comment
with:
issue-number: ${{ github.event.pull_request.number }}
comment-author: "github-actions[bot]"
body-includes: "Helm Chart Prerelease Published"
- name: Create or update PR comment
if: github.event_name == 'pull_request'
uses: peter-evans/create-or-update-comment@e8674b075228eee787fea43ef493e45ece1004c9 # v5.0.0
with:
comment-id: ${{ steps.find-comment.outputs.comment-id }}
issue-number: ${{ github.event.pull_request.number }}
body: |
### 🧭 Helm Chart Prerelease Published
**Version:** `${{ steps.version.outputs.version }}`
**Install:**
```bash
helm upgrade --install trigger \
oci://ghcr.io/${{ github.repository_owner }}/charts/trigger \
--version "${{ steps.version.outputs.version }}"
```
> ⚠️ This is a prerelease for testing. Do not use in production.
edit-mode: replace
+159 -11
View File
@@ -3,9 +3,6 @@ name: 🤖 PR Checks
on:
pull_request:
types: [opened, synchronize, reopened]
paths-ignore:
- "docs/**"
- ".changeset/**"
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
@@ -13,19 +10,170 @@ concurrency:
permissions:
contents: read
id-token: write
pull-requests: read
jobs:
typecheck:
uses: ./.github/workflows/typecheck.yml
secrets: inherit
changes:
name: Detect changes
runs-on: ubuntu-latest
outputs:
code: ${{ steps.code_filter.outputs.code }}
typecheck_self: ${{ steps.filter.outputs.typecheck_self }}
webapp: ${{ steps.filter.outputs.webapp }}
packages: ${{ steps.filter.outputs.packages }}
internal: ${{ steps.filter.outputs.internal }}
cli: ${{ steps.filter.outputs.cli }}
sdk: ${{ steps.filter.outputs.sdk }}
steps:
# `code` uses `every` semantics so the negation patterns actually subtract.
# With the default `some` quantifier, `**` matches every file and the
# subsequent `!...` patterns are no-ops (each pattern is OR'd, not AND'd).
- uses: dorny/paths-filter@fbd0ab8f3e69293af611ebaee6363fc25e6d187d # v4.0.1
id: code_filter
with:
predicate-quantifier: every
filters: |
code:
- '**'
- '!docs/**'
- '!.changeset/**'
- '!hosting/**'
- '!.github/**'
- '!references/**'
- '!**/*.md'
- '!**/.env.example'
- uses: dorny/paths-filter@fbd0ab8f3e69293af611ebaee6363fc25e6d187d # v4.0.1
id: filter
with:
filters: |
typecheck_self:
- '.github/workflows/pr_checks.yml'
- '.github/workflows/typecheck.yml'
webapp:
- 'apps/webapp/**'
- 'packages/**'
- 'internal-packages/**'
- '.github/workflows/pr_checks.yml'
- '.github/workflows/unit-tests-webapp.yml'
- '.github/workflows/e2e-webapp.yml'
- '.configs/**'
- 'package.json'
- 'pnpm-lock.yaml'
- 'pnpm-workspace.yaml'
- 'turbo.json'
packages:
- 'packages/**'
- '.github/workflows/pr_checks.yml'
- '.github/workflows/unit-tests-packages.yml'
- '.configs/**'
- 'package.json'
- 'pnpm-lock.yaml'
- 'pnpm-workspace.yaml'
- 'turbo.json'
internal:
- 'internal-packages/**'
- 'packages/**'
- '.github/workflows/pr_checks.yml'
- '.github/workflows/unit-tests-internal.yml'
- '.configs/**'
- 'package.json'
- 'pnpm-lock.yaml'
- 'pnpm-workspace.yaml'
- 'turbo.json'
cli:
- 'packages/cli-v3/**'
- 'packages/build/**'
- 'packages/core/**'
- 'packages/schema-to-json/**'
- '.github/workflows/pr_checks.yml'
- '.github/workflows/e2e.yml'
- '.configs/**'
- 'package.json'
- 'pnpm-lock.yaml'
- 'pnpm-workspace.yaml'
- 'turbo.json'
sdk:
- 'packages/trigger-sdk/**'
- 'packages/core/**'
- '.github/workflows/pr_checks.yml'
- '.github/workflows/sdk-compat.yml'
- '.configs/**'
- 'package.json'
- 'pnpm-lock.yaml'
- 'pnpm-workspace.yaml'
- 'turbo.json'
units:
uses: ./.github/workflows/unit-tests.yml
secrets: inherit
typecheck:
needs: changes
if: needs.changes.outputs.code == 'true' || needs.changes.outputs.typecheck_self == 'true'
uses: ./.github/workflows/typecheck.yml
webapp:
needs: changes
if: needs.changes.outputs.webapp == 'true'
uses: ./.github/workflows/unit-tests-webapp.yml
secrets:
DOCKERHUB_USERNAME: ${{ secrets.DOCKERHUB_USERNAME }}
DOCKERHUB_TOKEN: ${{ secrets.DOCKERHUB_TOKEN }}
e2e-webapp:
needs: changes
if: needs.changes.outputs.webapp == 'true'
uses: ./.github/workflows/e2e-webapp.yml
secrets:
DOCKERHUB_USERNAME: ${{ secrets.DOCKERHUB_USERNAME }}
DOCKERHUB_TOKEN: ${{ secrets.DOCKERHUB_TOKEN }}
packages:
needs: changes
if: needs.changes.outputs.packages == 'true'
uses: ./.github/workflows/unit-tests-packages.yml
secrets:
DOCKERHUB_USERNAME: ${{ secrets.DOCKERHUB_USERNAME }}
DOCKERHUB_TOKEN: ${{ secrets.DOCKERHUB_TOKEN }}
internal:
needs: changes
if: needs.changes.outputs.internal == 'true'
uses: ./.github/workflows/unit-tests-internal.yml
secrets:
DOCKERHUB_USERNAME: ${{ secrets.DOCKERHUB_USERNAME }}
DOCKERHUB_TOKEN: ${{ secrets.DOCKERHUB_TOKEN }}
e2e:
needs: changes
if: needs.changes.outputs.cli == 'true'
uses: ./.github/workflows/e2e.yml
with:
package: cli-v3
secrets: inherit
sdk-compat:
needs: changes
if: needs.changes.outputs.sdk == 'true'
uses: ./.github/workflows/sdk-compat.yml
all-checks:
name: All PR Checks
needs:
- changes
- typecheck
- webapp
- e2e-webapp
- packages
- internal
- e2e
- sdk-compat
if: always()
runs-on: ubuntu-latest
steps:
- name: Verify all checks
run: |
if [[ "${{ contains(needs.*.result, 'failure') }}" == "true" ]]; then
echo "One or more checks failed"
exit 1
fi
if [[ "${{ contains(needs.*.result, 'cancelled') }}" == "true" ]]; then
echo "One or more checks were cancelled"
exit 1
fi
echo "All checks passed or were skipped due to path filters"
+76
View File
@@ -0,0 +1,76 @@
name: 🌱 Preview environment dispatch
# Opt-in per-PR preview environments
on:
pull_request:
types: [opened, reopened, synchronize, closed, labeled, unlabeled]
# Serialize a PR's events so dispatches arrive in order. Cloud-side concurrency
# collapses by branch but can't fix out-of-order arrival — e.g. a push racing a
# close could cancel the in-flight destroy and leak the preview. One short API
# call, so queuing is cheap; cancel-in-progress: false lets an in-flight
# dispatch finish (GitHub keeps only the latest pending, the desired behavior).
concurrency:
group: preview-dispatch-${{ github.event.pull_request.number }}
cancel-in-progress: false
permissions: {}
jobs:
dispatch:
name: Dispatch preview-deploy to cloud
runs-on: ubuntu-latest
# label added -> create
# new commit while labeled -> update
# label removed / PR closed -> destroy
if: >-
github.event.pull_request.head.repo.full_name == github.repository &&
(
(github.event.action == 'labeled' && github.event.label.name == 'preview') ||
(github.event.action == 'unlabeled' && github.event.label.name == 'preview') ||
(
contains(github.event.pull_request.labels.*.name, 'preview') &&
contains(fromJSON('["opened","reopened","synchronize","closed"]'), github.event.action)
)
)
steps:
- name: Build dispatch payload
id: payload
env:
ACTION: ${{ github.event.action }}
BRANCH: ${{ github.event.pull_request.head.ref }}
COMMIT: ${{ github.event.pull_request.head.sha }}
run: |
set -euo pipefail
# Map the GitHub PR action to the cloud pipeline's lifecycle event.
case "$ACTION" in
labeled | opened | reopened) EVENT=opened ;;
synchronize) EVENT=synchronize ;;
unlabeled | closed) EVENT=closed ;;
*) echo "unexpected action: $ACTION" >&2; exit 1 ;;
esac
# jq --arg JSON-escapes every value, so a branch name containing
# quotes/braces can't break or inject into the client payload.
payload=$(jq -nc \
--arg b "$BRANCH" \
--arg c "$COMMIT" \
--arg e "$EVENT" \
'{branch_name: $b, commit: $c, pull_request_event: $e}')
{
echo "client_payload=$payload"
echo "summary=$EVENT for $BRANCH @ ${COMMIT:0:7}"
} >> "$GITHUB_OUTPUT"
- name: Log dispatch
env:
SUMMARY: ${{ steps.payload.outputs.summary }}
run: echo "Dispatching preview-deploy event ($SUMMARY)"
- name: Send repository_dispatch
uses: peter-evans/repository-dispatch@28959ce8df70de7be546dd1250a005dd32156697 # v4.0.1
with:
token: ${{ secrets.CROSS_REPO_PAT }}
repository: triggerdotdev/cloud
event-type: preview-deploy
client-payload: ${{ steps.payload.outputs.client_payload }}
+44 -17
View File
@@ -4,6 +4,7 @@ permissions:
contents: read
packages: write
id-token: write
attestations: write
on:
workflow_call:
@@ -13,6 +14,9 @@ on:
type: string
required: false
default: ""
secrets:
SENTRY_AUTH_TOKEN:
required: false
jobs:
publish:
@@ -24,12 +28,13 @@ jobs:
short_sha: ${{ steps.get_commit.outputs.sha_short }}
steps:
- name: 🏭 Setup Depot CLI
uses: depot/setup-action@v1
uses: depot/setup-action@15c09a5f77a0840ad4bce955686522a257853461 # v1.7.1
- name: ⬇️ Checkout repo
uses: actions/checkout@v4
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
submodules: recursive
persist-credentials: false
- name: "#️⃣ Get the image tag"
id: get_tag
@@ -40,42 +45,52 @@ jobs:
- name: 🔢 Get the commit hash
id: get_commit
run: |
echo "sha_short=$(echo ${{ github.sha }} | cut -c1-7)" >> "$GITHUB_OUTPUT"
echo "sha_short=$(echo "${GITHUB_SHA}" | cut -c1-7)" >> "$GITHUB_OUTPUT"
- name: 📛 Set the tags
id: set_tags
run: |
ref_without_tag=ghcr.io/triggerdotdev/trigger.dev
image_tags=$ref_without_tag:${{ steps.get_tag.outputs.tag }}
image_tags=$ref_without_tag:${STEPS_GET_TAG_OUTPUTS_TAG}
# if tag is a semver, also tag it as v4
if [[ "${{ steps.get_tag.outputs.is_semver }}" == true ]]; then
# TODO: switch to v4 tag on GA
image_tags=$image_tags,$ref_without_tag:v4-beta
# when pushing the mutable main tag, also push an immutable-by-convention
# full-commit-sha tag so a commit can be resolved to a specific digest
if [[ "${STEPS_GET_TAG_OUTPUTS_TAG}" == "main" ]]; then
image_tags=$image_tags,$ref_without_tag:${GITHUB_SHA}
fi
echo "image_tags=${image_tags}" >> "$GITHUB_OUTPUT"
env:
STEPS_GET_TAG_OUTPUTS_TAG: ${{ steps.get_tag.outputs.tag }}
STEPS_GET_TAG_OUTPUTS_IS_SEMVER: ${{ steps.get_tag.outputs.is_semver }}
- name: 📝 Set the build info
id: set_build_info
run: |
tag=${{ steps.get_tag.outputs.tag }}
if [[ "${{ steps.get_tag.outputs.is_semver }}" == true ]]; then
echo "BUILD_APP_VERSION=${tag}" >> "$GITHUB_OUTPUT"
fi
echo "BUILD_GIT_SHA=${{ github.sha }}" >> "$GITHUB_OUTPUT"
echo "BUILD_GIT_REF_NAME=${{ github.ref_name }}" >> "$GITHUB_OUTPUT"
echo "BUILD_TIMESTAMP_SECONDS=$(date +%s)" >> "$GITHUB_OUTPUT"
{
tag="${STEPS_GET_TAG_OUTPUTS_TAG}"
if [[ "${STEPS_GET_TAG_OUTPUTS_IS_SEMVER}" == true ]]; then
echo "BUILD_APP_VERSION=${tag}"
fi
echo "BUILD_GIT_SHA=${GITHUB_SHA}"
echo "BUILD_GIT_REF_NAME=${GITHUB_REF_NAME}"
echo "BUILD_TIMESTAMP_SECONDS=$(date +%s)"
echo "BUILD_TIMESTAMP_RFC3339=$(date -u +%Y-%m-%dT%H:%M:%SZ)"
} >> "$GITHUB_OUTPUT"
env:
STEPS_GET_TAG_OUTPUTS_TAG: ${{ steps.get_tag.outputs.tag }}
STEPS_GET_TAG_OUTPUTS_IS_SEMVER: ${{ steps.get_tag.outputs.is_semver }}
- name: 🐙 Login to GitHub Container Registry
uses: docker/login-action@v3
uses: docker/login-action@4907a6ddec9925e35a0a9e82d7399ccc52663121 # v4.1.0
with:
registry: ghcr.io
username: ${{ github.repository_owner }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: 🐳 Build image and push to GitHub Container Registry
uses: depot/build-push-action@v1
id: build_push
uses: depot/build-push-action@5f3b3c2e5a00f0093de47f657aeaefcedff27d18 # v1.17.0
with:
file: ./docker/Dockerfile
platforms: linux/amd64,linux/arm64
@@ -86,8 +101,20 @@ jobs:
BUILD_GIT_SHA=${{ steps.set_build_info.outputs.BUILD_GIT_SHA }}
BUILD_GIT_REF_NAME=${{ steps.set_build_info.outputs.BUILD_GIT_REF_NAME }}
BUILD_TIMESTAMP_SECONDS=${{ steps.set_build_info.outputs.BUILD_TIMESTAMP_SECONDS }}
BUILD_TIMESTAMP_RFC3339=${{ steps.set_build_info.outputs.BUILD_TIMESTAMP_RFC3339 }}
SENTRY_RELEASE=${{ steps.set_build_info.outputs.BUILD_GIT_SHA }}
SENTRY_ORG=triggerdev
SENTRY_PROJECT=trigger-cloud
secrets: |
sentry_auth_token=${{ secrets.SENTRY_AUTH_TOKEN }}
- name: 🪪 Attest build provenance
# Image is already pushed by this point — don't fail releases (and the
# downstream publish-helm job) on a Sigstore/GHCR-referrer hiccup. Real
# config errors still surface as a step warning in the workflow run.
continue-on-error: true
uses: actions/attest-build-provenance@a2bbfa25375fe432b6a289bc6b6cd05ecd0c4c32 # v4.1.0
with:
subject-name: ghcr.io/triggerdotdev/trigger.dev
subject-digest: ${{ steps.build_push.outputs.digest }}
push-to-registry: true
+17 -16
View File
@@ -37,19 +37,22 @@ jobs:
DOCKER_BUILDKIT: "1"
steps:
- name: 🏭 Setup Depot CLI
uses: depot/setup-action@v1
uses: depot/setup-action@15c09a5f77a0840ad4bce955686522a257853461 # v1.7.1
- name: ⬇️ Checkout git repo
uses: actions/checkout@v4
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
persist-credentials: false
- name: 📦 Get image repo
id: get_repository
env:
PACKAGE: ${{ matrix.package }}
run: |
if [[ "${{ matrix.package }}" == *-provider ]]; then
provider_type=$(echo "${{ matrix.package }}" | cut -d- -f1)
repo=provider/${provider_type}
if [[ "$PACKAGE" == *-provider ]]; then
repo="provider/${PACKAGE%-provider}"
else
repo="${{ matrix.package }}"
repo="$PACKAGE"
fi
echo "repo=${repo}" >> "$GITHUB_OUTPUT"
@@ -62,26 +65,24 @@ jobs:
- name: 📛 Set tags to push
id: set_tags
run: |
ref_without_tag=ghcr.io/triggerdotdev/${{ steps.get_repository.outputs.repo }}
image_tags=$ref_without_tag:${{ steps.get_tag.outputs.tag }}
# if tag is a semver, also tag it as v4
if [[ "${{ steps.get_tag.outputs.is_semver }}" == true ]]; then
# TODO: switch to v4 tag on GA
image_tags=$image_tags,$ref_without_tag:v4-beta
fi
ref_without_tag=ghcr.io/triggerdotdev/${STEPS_GET_REPOSITORY_OUTPUTS_REPO}
image_tags=$ref_without_tag:${STEPS_GET_TAG_OUTPUTS_TAG}
echo "image_tags=${image_tags}" >> "$GITHUB_OUTPUT"
env:
STEPS_GET_REPOSITORY_OUTPUTS_REPO: ${{ steps.get_repository.outputs.repo }}
STEPS_GET_TAG_OUTPUTS_TAG: ${{ steps.get_tag.outputs.tag }}
STEPS_GET_TAG_OUTPUTS_IS_SEMVER: ${{ steps.get_tag.outputs.is_semver }}
- name: 🐙 Login to GitHub Container Registry
uses: docker/login-action@v3
uses: docker/login-action@4907a6ddec9925e35a0a9e82d7399ccc52663121 # v4.1.0
with:
registry: ghcr.io
username: ${{ github.repository_owner }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: 🐳 Build image and push to GitHub Container Registry
uses: depot/build-push-action@v1
uses: depot/build-push-action@5f3b3c2e5a00f0093de47f657aeaefcedff27d18 # v1.17.0
with:
file: ./apps/${{ matrix.package }}/Containerfile
platforms: linux/amd64,linux/arm64
+18 -8
View File
@@ -8,6 +8,11 @@ on:
type: string
required: false
default: ""
secrets:
DOCKERHUB_USERNAME:
required: false
DOCKERHUB_TOKEN:
required: false
push:
tags:
- "infra-dev-*"
@@ -26,18 +31,22 @@ jobs:
runs-on: ubuntu-latest
env:
DOCKER_BUILDKIT: "1"
DOCKERHUB_USERNAME: ${{ secrets.DOCKERHUB_USERNAME }}
steps:
- name: ⬇️ Checkout git repo
uses: actions/checkout@v4
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
persist-credentials: false
- name: 📦 Get image repo
id: get_repository
env:
PACKAGE: ${{ matrix.package }}
run: |
if [[ "${{ matrix.package }}" == *-provider ]]; then
provider_type=$(echo "${{ matrix.package }}" | cut -d- -f1)
repo=provider/${provider_type}
if [[ "$PACKAGE" == *-provider ]]; then
repo="provider/${PACKAGE%-provider}"
else
repo="${{ matrix.package }}"
repo="$PACKAGE"
fi
echo "repo=${repo}" >> "$GITHUB_OUTPUT"
@@ -47,11 +56,12 @@ jobs:
tag: ${{ inputs.image_tag }}
- name: 🐋 Set up Docker Buildx
uses: docker/setup-buildx-action@v3
uses: docker/setup-buildx-action@4d04d5d9486b7bd6fa91e7baf45bbb4f8b9deedd # v4.0.0
# ..to avoid rate limits when pulling images
- name: 🐳 Login to DockerHub
uses: docker/login-action@v3
if: ${{ env.DOCKERHUB_USERNAME }}
uses: docker/login-action@4907a6ddec9925e35a0a9e82d7399ccc52663121 # v4.1.0
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
@@ -62,7 +72,7 @@ jobs:
# ..to push image
- name: 🐙 Login to GitHub Container Registry
uses: docker/login-action@v3
uses: docker/login-action@4907a6ddec9925e35a0a9e82d7399ccc52663121 # v4.1.0
with:
registry: ghcr.io
username: ${{ github.repository_owner }}
+28 -7
View File
@@ -1,12 +1,20 @@
name: 🚀 Publish Trigger.dev Docker
on:
workflow_dispatch:
workflow_call:
inputs:
image_tag:
description: The image tag to publish
required: true
type: string
secrets:
DOCKERHUB_USERNAME:
required: false
DOCKERHUB_TOKEN:
required: false
SENTRY_AUTH_TOKEN:
required: false
push:
branches:
- main
@@ -36,8 +44,6 @@ on:
- "tests/**"
permissions:
id-token: write
packages: write
contents: read
concurrency:
@@ -49,29 +55,44 @@ env:
jobs:
typecheck:
uses: ./.github/workflows/typecheck.yml
secrets: inherit
units:
uses: ./.github/workflows/unit-tests.yml
secrets: inherit
secrets:
DOCKERHUB_USERNAME: ${{ secrets.DOCKERHUB_USERNAME }}
DOCKERHUB_TOKEN: ${{ secrets.DOCKERHUB_TOKEN }}
publish-webapp:
needs: [typecheck]
permissions:
contents: read
packages: write
id-token: write
attestations: write
uses: ./.github/workflows/publish-webapp.yml
secrets: inherit
secrets:
SENTRY_AUTH_TOKEN: ${{ secrets.SENTRY_AUTH_TOKEN }}
with:
image_tag: ${{ inputs.image_tag }}
publish-worker:
needs: [typecheck]
permissions:
contents: read
packages: write
uses: ./.github/workflows/publish-worker.yml
secrets: inherit
secrets:
DOCKERHUB_USERNAME: ${{ secrets.DOCKERHUB_USERNAME }}
DOCKERHUB_TOKEN: ${{ secrets.DOCKERHUB_TOKEN }}
with:
image_tag: ${{ inputs.image_tag }}
publish-worker-v4:
needs: [typecheck]
permissions:
contents: read
packages: write
id-token: write
uses: ./.github/workflows/publish-worker-v4.yml
secrets: inherit
with:
image_tag: ${{ inputs.image_tag }}
+30 -15
View File
@@ -4,6 +4,12 @@ on:
push:
tags:
- 'helm-v*'
workflow_call:
inputs:
chart_version:
description: 'Chart version to release'
required: true
type: string
workflow_dispatch:
inputs:
chart_version:
@@ -22,10 +28,12 @@ jobs:
contents: read
steps:
- name: Checkout
uses: actions/checkout@v4
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
persist-credentials: false
- name: Set up Helm
uses: azure/setup-helm@v4
uses: azure/setup-helm@dda3372f752e03dde6b3237bc9431cdc2f7a02a2 # v5.0.0
with:
version: "3.18.3"
@@ -48,7 +56,7 @@ jobs:
--output-dir ./helm-output
- name: Validate manifests
uses: docker://ghcr.io/yannh/kubeconform:v0.7.0
uses: docker://ghcr.io/yannh/kubeconform:v0.7.0@sha256:85dbef6b4b312b99133decc9c6fc9495e9fc5f92293d4ff3b7e1b30f5611823c
with:
entrypoint: '/kubeconform'
args: "-summary -output json ./helm-output"
@@ -61,10 +69,12 @@ jobs:
packages: write
steps:
- name: Checkout
uses: actions/checkout@v4
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
persist-credentials: false
- name: Set up Helm
uses: azure/setup-helm@v4
uses: azure/setup-helm@dda3372f752e03dde6b3237bc9431cdc2f7a02a2 # v5.0.0
with:
version: "3.18.3"
@@ -77,7 +87,7 @@ jobs:
for file in ./charts/*.tgz; do echo "Extracting $file"; tar -xzf "$file" -C ./charts; done
- name: Log in to Container Registry
uses: docker/login-action@v3
uses: docker/login-action@4907a6ddec9925e35a0a9e82d7399ccc52663121 # v4.1.0
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
@@ -86,18 +96,20 @@ jobs:
- name: Extract version from tag or input
id: version
run: |
if [ "${{ github.event_name }}" = "workflow_dispatch" ]; then
VERSION="${{ github.event.inputs.chart_version }}"
if [ -n "${INPUTS_CHART_VERSION}" ]; then
VERSION="${INPUTS_CHART_VERSION}"
else
VERSION="${{ github.ref_name }}"
VERSION="${GITHUB_REF_NAME}"
VERSION="${VERSION#helm-v}"
fi
echo "version=$VERSION" >> $GITHUB_OUTPUT
echo "version=$VERSION" >> "$GITHUB_OUTPUT"
echo "Releasing version: $VERSION"
env:
INPUTS_CHART_VERSION: ${{ inputs.chart_version }}
- name: Check Chart.yaml version matches release version
run: |
VERSION="${{ steps.version.outputs.version }}"
VERSION="${STEPS_VERSION_OUTPUTS_VERSION}"
CHART_VERSION=$(grep '^version:' ./hosting/k8s/helm/Chart.yaml | awk '{print $2}')
echo "Chart.yaml version: $CHART_VERSION"
echo "Release version: $VERSION"
@@ -106,6 +118,8 @@ jobs:
exit 1
fi
echo "✅ Chart.yaml version matches release version."
env:
STEPS_VERSION_OUTPUTS_VERSION: ${{ steps.version.outputs.version }}
- name: Package Helm Chart
run: |
@@ -113,18 +127,19 @@ jobs:
- name: Push Helm Chart to GHCR
run: |
VERSION="${{ steps.version.outputs.version }}"
VERSION="${STEPS_VERSION_OUTPUTS_VERSION}"
CHART_PACKAGE="/tmp/${{ env.CHART_NAME }}-${VERSION}.tgz"
# Push to GHCR OCI registry
helm push "$CHART_PACKAGE" "oci://${{ env.REGISTRY }}/${{ github.repository_owner }}/charts"
env:
STEPS_VERSION_OUTPUTS_VERSION: ${{ steps.version.outputs.version }}
- name: Create GitHub Release
id: release
uses: softprops/action-gh-release@v1
if: github.event_name == 'push'
uses: softprops/action-gh-release@b4309332981a82ec1c5618f44dd2e27cc8bfbfda # v3.0.0
with:
tag_name: ${{ github.ref_name }}
tag_name: helm-v${{ steps.version.outputs.version }}
name: "Helm Chart ${{ steps.version.outputs.version }}"
body: |
### Installation
+285 -55
View File
@@ -1,98 +1,328 @@
name: 🦋 Changesets Release
permissions:
contents: write
on:
push:
pull_request:
types: [closed]
branches:
- main
paths-ignore:
- "docs/**"
- "**.md"
- ".github/CODEOWNERS"
- ".github/ISSUE_TEMPLATE/**"
workflow_dispatch:
inputs:
type:
description: "Select release type"
required: true
type: choice
options:
- release
- prerelease
default: "prerelease"
ref:
description: "The ref (branch, tag, or SHA) to checkout and release from"
required: true
type: string
prerelease_tag:
description: "The npm dist-tag for the prerelease (e.g., 'v4-prerelease')"
required: false
type: string
default: "prerelease"
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
group: ${{ github.workflow }}
cancel-in-progress: false
jobs:
release:
name: 🦋 Changesets Release
show-release-summary:
name: 📋 Release Summary
runs-on: ubuntu-latest
permissions: {}
if: |
github.repository == 'triggerdotdev/trigger.dev' &&
github.event_name == 'pull_request' &&
github.event.pull_request.merged == true &&
github.event.pull_request.head.ref == 'changeset-release/main'
steps:
- name: Show release summary
env:
PR_BODY: ${{ github.event.pull_request.body }}
run: |
echo "$PR_BODY" | sed -n '/^# Releases/,$p' >> "$GITHUB_STEP_SUMMARY"
release:
name: 🚀 Release npm packages
runs-on: ubuntu-latest
environment: npm-publish
permissions:
contents: write
packages: write
pull-requests: write
if: github.repository == 'triggerdotdev/trigger.dev'
id-token: write
if: |
github.repository == 'triggerdotdev/trigger.dev' &&
(
(github.event_name == 'workflow_dispatch' && github.event.inputs.type == 'release') ||
(github.event_name == 'pull_request' && github.event.pull_request.merged == true && github.event.pull_request.head.ref == 'changeset-release/main')
)
outputs:
published: ${{ steps.changesets.outputs.published }}
published_packages: ${{ steps.changesets.outputs.publishedPackages }}
published_package_version: ${{ steps.get_version.outputs.package_version }}
is_prerelease: ${{ steps.get_version.outputs.is_prerelease }}
steps:
- name: ⬇️ Checkout repo
uses: actions/checkout@v4
- name: Checkout repo
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 # zizmor: ignore[artipacked] needs persisted git creds for tag push; no artifact upload here so no leak path
with:
fetch-depth: 0
ref: ${{ github.event_name == 'workflow_dispatch' && github.event.inputs.ref || github.sha }}
- name: ⎔ Setup pnpm
uses: pnpm/action-setup@v4
with:
version: 8.15.5
- name: Verify ref is on main
if: github.event_name == 'workflow_dispatch'
run: |
if ! git merge-base --is-ancestor "${GITHUB_EVENT_INPUTS_REF}" origin/main; then
echo "Error: ref must be an ancestor of main (i.e., already merged)"
exit 1
fi
env:
GITHUB_EVENT_INPUTS_REF: ${{ github.event.inputs.ref }}
- name: Setup node
uses: buildjet/setup-node@v4
- name: Setup pnpm
uses: pnpm/action-setup@fc06bc1257f339d1d5d8b3a19a8cae5388b55320 # v5.0.0
with:
node-version: 20.11.1
version: 10.33.2
- name: Setup node
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: 20.20.0
cache: "pnpm"
- name: 📥 Download deps
# npm v11.5.1 or newer is required for OIDC support
# https://github.blog/changelog/2025-07-31-npm-trusted-publishing-with-oidc-is-generally-available/#whats-new
- name: Setup npm 11.x for OIDC
run: npm install -g npm@11.6.4
- name: Install dependencies
run: pnpm install --frozen-lockfile
- name: 📀 Generate Prisma Client
- name: Generate Prisma client
run: pnpm run generate
- name: 🏗️ Build
- name: Build
run: pnpm run build --filter "@trigger.dev/*" --filter "trigger.dev"
- name: 🔎 Type check
- name: Type check
run: pnpm run typecheck --filter "@trigger.dev/*" --filter "trigger.dev"
- name: 🔐 Setup npm auth
run: |
echo "registry=https://registry.npmjs.org" >> ~/.npmrc
echo "//registry.npmjs.org/:_authToken=${{ secrets.NPM_TOKEN }}" >> ~/.npmrc
# This action has two responsibilities. The first time the workflow runs
# (initial push to the `main` branch) it will create a new branch and
# then open a PR with the related changes for the new version. After the
# PR is merged, the workflow will run again and this action will build +
# publish to npm.
- name: 🚀 PR / Publish
if: ${{ !env.ACT }}
- name: Publish
id: changesets
uses: changesets/action@v1
uses: changesets/action@6a0a831ff30acef54f2c6aa1cbbc1096b066edaf # v1.7.0
with:
version: pnpm run changeset:version
commit: "chore: Update version for release"
title: "chore: Update version for release"
publish: pnpm run changeset:release
createGithubReleases: true
createGithubReleases: false
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
NPM_TOKEN: ${{ secrets.NPM_TOKEN }}
# - name: 🚀 PR / Publish (mock)
# if: ${{ env.ACT }}
# id: changesets
# run: |
# echo "published=true" >> "$GITHUB_OUTPUT"
# echo "publishedPackages=[{\"name\": \"@xx/xx\", \"version\": \"1.2.0\"}, {\"name\": \"@xx/xy\", \"version\": \"0.8.9\"}]" >> "$GITHUB_OUTPUT"
- name: 📦 Get package version
- name: Show package version
if: steps.changesets.outputs.published == 'true'
id: get_version
run: |
package_version=$(echo '${{ steps.changesets.outputs.publishedPackages }}' | jq -r '.[0].version')
package_version=$(echo "${STEPS_CHANGESETS_OUTPUTS_PUBLISHEDPACKAGES}" | jq -r '.[0].version')
echo "package_version=${package_version}" >> "$GITHUB_OUTPUT"
# Any semver with a hyphen is a prerelease (e.g. 4.5.0-rc.0, 0.0.0-snapshot-...)
if [[ "${package_version}" == *-* ]]; then
echo "is_prerelease=true" >> "$GITHUB_OUTPUT"
else
echo "is_prerelease=false" >> "$GITHUB_OUTPUT"
fi
env:
STEPS_CHANGESETS_OUTPUTS_PUBLISHEDPACKAGES: ${{ steps.changesets.outputs.publishedPackages }}
- name: Create unified GitHub release
if: steps.changesets.outputs.published == 'true'
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
RELEASE_PR_BODY: ${{ github.event.pull_request.body }}
STEPS_GET_VERSION_OUTPUTS_PACKAGE_VERSION: ${{ steps.get_version.outputs.package_version }}
STEPS_GET_VERSION_OUTPUTS_IS_PRERELEASE: ${{ steps.get_version.outputs.is_prerelease }}
run: |
VERSION="${STEPS_GET_VERSION_OUTPUTS_PACKAGE_VERSION}"
node scripts/generate-github-release.mjs "$VERSION" > /tmp/release-body.md
PRERELEASE_FLAG=""
if [ "${STEPS_GET_VERSION_OUTPUTS_IS_PRERELEASE}" = "true" ]; then
PRERELEASE_FLAG="--prerelease"
fi
gh release create "v${VERSION}" \
--title "trigger.dev v${VERSION}" \
--notes-file /tmp/release-body.md \
--target main \
$PRERELEASE_FLAG
- name: Create and push Docker tag
if: steps.changesets.outputs.published == 'true'
run: |
set -e
git tag "v.docker.${STEPS_GET_VERSION_OUTPUTS_PACKAGE_VERSION}"
git push origin "v.docker.${STEPS_GET_VERSION_OUTPUTS_PACKAGE_VERSION}"
env:
STEPS_GET_VERSION_OUTPUTS_PACKAGE_VERSION: ${{ steps.get_version.outputs.package_version }}
- name: Create and push Helm chart tag
if: steps.changesets.outputs.published == 'true'
run: |
set -e
git tag "helm-v${STEPS_GET_VERSION_OUTPUTS_PACKAGE_VERSION}"
git push origin "helm-v${STEPS_GET_VERSION_OUTPUTS_PACKAGE_VERSION}"
env:
STEPS_GET_VERSION_OUTPUTS_PACKAGE_VERSION: ${{ steps.get_version.outputs.package_version }}
# Trigger Docker builds directly via workflow_call since tags pushed with
# GITHUB_TOKEN don't trigger other workflows (GitHub Actions limitation).
publish-docker:
name: 🐳 Publish Docker images
needs: release
if: needs.release.outputs.published == 'true'
permissions:
contents: read
packages: write
id-token: write
attestations: write
uses: ./.github/workflows/publish.yml
secrets:
DOCKERHUB_USERNAME: ${{ secrets.DOCKERHUB_USERNAME }}
DOCKERHUB_TOKEN: ${{ secrets.DOCKERHUB_TOKEN }}
SENTRY_AUTH_TOKEN: ${{ secrets.SENTRY_AUTH_TOKEN }}
with:
image_tag: v${{ needs.release.outputs.published_package_version }}
# Trigger Helm chart release directly via workflow_call (same GITHUB_TOKEN
# limitation as the Docker path). Runs after Docker images are published so
# the chart never references images that don't exist yet.
publish-helm:
name: 🧭 Publish Helm chart
needs: [release, publish-docker]
if: needs.release.outputs.published == 'true'
permissions:
contents: write
packages: write
uses: ./.github/workflows/release-helm.yml
with:
chart_version: ${{ needs.release.outputs.published_package_version }}
# After Docker images are published, update the GitHub release with the exact GHCR tag URL.
# The GHCR package version ID is only known after the image is pushed, so we query for it here.
update-release:
name: 🔗 Update release Docker link
needs: [release, publish-docker]
if: needs.release.outputs.published == 'true'
runs-on: ubuntu-latest
permissions:
contents: write
packages: read
steps:
- name: Update GitHub release with Docker image link
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
NEEDS_RELEASE_OUTPUTS_PUBLISHED_PACKAGE_VERSION: ${{ needs.release.outputs.published_package_version }}
run: |
set -e
VERSION="${NEEDS_RELEASE_OUTPUTS_PUBLISHED_PACKAGE_VERSION}"
TAG="v${VERSION}"
# Query GHCR for the version ID matching this tag
VERSION_ID=$(gh api --paginate -H "Accept: application/vnd.github+json" \
/orgs/triggerdotdev/packages/container/trigger.dev/versions \
--jq ".[] | select(.metadata.container.tags[] == \"${TAG}\") | .id" \
| head -1)
if [ -z "$VERSION_ID" ]; then
echo "Warning: Could not find GHCR version ID for tag ${TAG}, skipping update"
exit 0
fi
DOCKER_URL="https://github.com/triggerdotdev/trigger.dev/pkgs/container/trigger.dev/${VERSION_ID}?tag=${TAG}"
GENERIC_URL="https://github.com/triggerdotdev/trigger.dev/pkgs/container/trigger.dev"
# Get current release body and replace the generic link with the tag-specific one.
# Use word boundary after GENERIC_URL (closing paren) to avoid matching URLs that
# already have a version ID appended (idempotent on re-runs).
gh release view "${TAG}" --repo triggerdotdev/trigger.dev --json body --jq '.body' > /tmp/release-body.md
sed -i "s|${GENERIC_URL})|${DOCKER_URL})|g" /tmp/release-body.md
gh release edit "${TAG}" --repo triggerdotdev/trigger.dev --notes-file /tmp/release-body.md
# Dispatch changelog entry creation to the marketing site repo.
# Runs after update-release so the GitHub release body already has the exact Docker image URL.
dispatch-changelog:
name: 📝 Dispatch changelog PR
needs: [release, update-release]
if: needs.release.outputs.published == 'true' && needs.release.outputs.is_prerelease != 'true'
runs-on: ubuntu-latest
permissions: {}
steps:
- uses: peter-evans/repository-dispatch@28959ce8df70de7be546dd1250a005dd32156697 # v4.0.1
with:
token: ${{ secrets.CROSS_REPO_PAT }}
repository: triggerdotdev/trigger.dev-site-v3
event-type: new-release
client-payload: '{"version": "${{ needs.release.outputs.published_package_version }}"}'
# The prerelease job needs to be on the same workflow file due to a limitation related to how npm verifies OIDC claims.
prerelease:
name: 🧪 Prerelease
runs-on: ubuntu-latest
environment: npm-publish
permissions:
contents: read
id-token: write
if: github.repository == 'triggerdotdev/trigger.dev' && github.event_name == 'workflow_dispatch' && github.event.inputs.type == 'prerelease'
steps:
- name: Checkout repo
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
fetch-depth: 0
ref: ${{ github.event.inputs.ref }}
persist-credentials: false
- name: Setup pnpm
uses: pnpm/action-setup@fc06bc1257f339d1d5d8b3a19a8cae5388b55320 # v5.0.0
with:
version: 10.33.2
- name: Setup node
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: 20.20.0
cache: "pnpm"
# npm v11.5.1 or newer is required for OIDC support
# https://github.blog/changelog/2025-07-31-npm-trusted-publishing-with-oidc-is-generally-available/#whats-new
- name: Setup npm 11.x for OIDC
run: npm install -g npm@11.6.4
- name: Download deps
run: pnpm install --frozen-lockfile
- name: Generate Prisma Client
run: pnpm run generate
- name: Exit changeset pre mode (if active)
run: |
if [ -f .changeset/pre.json ]; then
echo "Repo is in changeset pre mode; exiting so snapshot release can run"
pnpm exec changeset pre exit
fi
- name: Snapshot version
run: pnpm exec changeset version --snapshot "${GITHUB_EVENT_INPUTS_PRERELEASE_TAG}"
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
GITHUB_EVENT_INPUTS_PRERELEASE_TAG: ${{ github.event.inputs.prerelease_tag }}
- name: Clean
run: pnpm run clean --filter "@trigger.dev/*" --filter "trigger.dev"
- name: Build
run: pnpm run build --filter "@trigger.dev/*" --filter "trigger.dev"
- name: Publish prerelease
run: pnpm exec changeset publish --no-git-tag --snapshot --tag "${GITHUB_EVENT_INPUTS_PRERELEASE_TAG}"
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
GITHUB_EVENT_INPUTS_PRERELEASE_TAG: ${{ github.event.inputs.prerelease_tag }}
+182
View File
@@ -0,0 +1,182 @@
name: "🔌 SDK Compatibility Tests"
permissions:
contents: read
on:
workflow_call:
jobs:
node-compat:
name: "Node.js ${{ matrix.node }} (${{ matrix.os }})"
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest]
node: ["20.20", "22.12"]
steps:
- name: ⬇️ Checkout repo
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
fetch-depth: 0
persist-credentials: false
- name: ⎔ Setup pnpm
uses: pnpm/action-setup@fc06bc1257f339d1d5d8b3a19a8cae5388b55320 # v5.0.0
with:
version: 10.33.2
- name: ⎔ Setup node
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: ${{ matrix.node }}
cache: "pnpm"
- name: 📥 Download deps
run: pnpm install --frozen-lockfile
- name: 📀 Generate Prisma Client
run: pnpm run generate
- name: 🔨 Build SDK dependencies
shell: bash
run: pnpm run build --filter '@trigger.dev/sdk^...'
- name: 🔨 Build SDK
shell: bash
run: pnpm run build --filter '@trigger.dev/sdk'
- name: 🧪 Run SDK Compatibility Tests
shell: bash
run: pnpm --filter @internal/sdk-compat-tests test
bun-compat:
name: "Bun Runtime"
runs-on: ubuntu-latest
steps:
- name: ⬇️ Checkout repo
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
fetch-depth: 0
persist-credentials: false
- name: ⎔ Setup pnpm
uses: pnpm/action-setup@fc06bc1257f339d1d5d8b3a19a8cae5388b55320 # v5.0.0
with:
version: 10.33.2
- name: ⎔ Setup node
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: 20.20.0
cache: "pnpm"
- name: 🥟 Setup Bun
uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2.2.0
with:
bun-version: latest
- name: 📥 Download deps
run: pnpm install --frozen-lockfile
- name: 📀 Generate Prisma Client
run: pnpm run generate
- name: 🔨 Build SDK dependencies
run: pnpm run build --filter @trigger.dev/sdk^...
- name: 🔨 Build SDK
run: pnpm run build --filter @trigger.dev/sdk
- name: 🧪 Run Bun Compatibility Test
working-directory: internal-packages/sdk-compat-tests/src/fixtures/bun
run: bun run test.ts
deno-compat:
name: "Deno Runtime"
runs-on: ubuntu-latest
steps:
- name: ⬇️ Checkout repo
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
fetch-depth: 0
persist-credentials: false
- name: ⎔ Setup pnpm
uses: pnpm/action-setup@fc06bc1257f339d1d5d8b3a19a8cae5388b55320 # v5.0.0
with:
version: 10.33.2
- name: ⎔ Setup node
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: 20.20.0
cache: "pnpm"
- name: 🦕 Setup Deno
uses: denoland/setup-deno@667a34cdef165d8d2b2e98dde39547c9daac7282 # v2.0.4
with:
deno-version: v2.x
- name: 📥 Download deps
run: pnpm install --frozen-lockfile
- name: 📀 Generate Prisma Client
run: pnpm run generate
- name: 🔨 Build SDK dependencies
run: pnpm run build --filter @trigger.dev/sdk^...
- name: 🔨 Build SDK
run: pnpm run build --filter @trigger.dev/sdk
- name: 🔗 Link node_modules for Deno fixture
working-directory: internal-packages/sdk-compat-tests/src/fixtures/deno
run: ln -s ../../../../../node_modules node_modules
- name: 🧪 Run Deno Compatibility Test
working-directory: internal-packages/sdk-compat-tests/src/fixtures/deno
run: deno run --allow-read --allow-env --allow-sys test.ts
cloudflare-compat:
name: "Cloudflare Workers"
runs-on: ubuntu-latest
steps:
- name: ⬇️ Checkout repo
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
fetch-depth: 0
persist-credentials: false
- name: ⎔ Setup pnpm
uses: pnpm/action-setup@fc06bc1257f339d1d5d8b3a19a8cae5388b55320 # v5.0.0
with:
version: 10.33.2
- name: ⎔ Setup node
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: 20.20.0
cache: "pnpm"
- name: 📥 Download deps
run: pnpm install --frozen-lockfile
- name: 📀 Generate Prisma Client
run: pnpm run generate
- name: 🔨 Build SDK dependencies
run: pnpm run build --filter @trigger.dev/sdk^...
- name: 🔨 Build SDK
run: pnpm run build --filter @trigger.dev/sdk
- name: 📥 Install Cloudflare fixture deps
working-directory: internal-packages/sdk-compat-tests/src/fixtures/cloudflare-worker
run: pnpm install
- name: 🧪 Run Cloudflare Workers Compatibility Test (dry-run)
working-directory: internal-packages/sdk-compat-tests/src/fixtures/cloudflare-worker
run: npx wrangler deploy --dry-run --outdir dist
+8 -5
View File
@@ -12,19 +12,20 @@ jobs:
steps:
- name: ⬇️ Checkout repo
uses: actions/checkout@v4
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
fetch-depth: 0
persist-credentials: false
- name: ⎔ Setup pnpm
uses: pnpm/action-setup@v4
uses: pnpm/action-setup@fc06bc1257f339d1d5d8b3a19a8cae5388b55320 # v5.0.0
with:
version: 8.15.5
version: 10.33.2
- name: ⎔ Setup node
uses: buildjet/setup-node@v4
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: 20.11.1
node-version: 20.20.0
cache: "pnpm"
- name: 📥 Download deps
@@ -35,6 +36,8 @@ jobs:
- name: 🔎 Type check
run: pnpm run typecheck
env:
NODE_OPTIONS: --max-old-space-size=8192
- name: 🔎 Check exports
run: pnpm run check-exports
+33 -15
View File
@@ -5,6 +5,11 @@ permissions:
on:
workflow_call:
secrets:
DOCKERHUB_USERNAME:
required: false
DOCKERHUB_TOKEN:
required: false
jobs:
unitTests:
@@ -46,25 +51,26 @@ jobs:
run: sudo systemctl restart docker
- name: ⬇️ Checkout repo
uses: actions/checkout@v4
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
fetch-depth: 0
persist-credentials: false
- name: ⎔ Setup pnpm
uses: pnpm/action-setup@v4
uses: pnpm/action-setup@fc06bc1257f339d1d5d8b3a19a8cae5388b55320 # v5.0.0
with:
version: 8.15.5
version: 10.33.2
- name: ⎔ Setup node
uses: buildjet/setup-node@v4
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: 20.11.1
node-version: 20.20.0
cache: "pnpm"
# ..to avoid rate limits when pulling images
- name: 🐳 Login to DockerHub
if: ${{ env.DOCKERHUB_USERNAME }}
uses: docker/login-action@v3
uses: docker/login-action@4907a6ddec9925e35a0a9e82d7399ccc52663121 # v4.1.0
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
@@ -72,6 +78,17 @@ jobs:
if: ${{ !env.DOCKERHUB_USERNAME }}
run: echo "DockerHub login skipped because secrets are not available."
- name: 🐳 Pre-pull testcontainer images
if: ${{ env.DOCKERHUB_USERNAME }}
run: |
echo "Pre-pulling Docker images with authenticated session..."
docker pull postgres:14
docker pull clickhouse/clickhouse-server:25.4-alpine
docker pull redis:7-alpine
docker pull testcontainers/ryuk:0.11.0
docker pull electricsql/electric:1.2.4
echo "Image pre-pull complete"
- name: 📥 Download deps
run: pnpm install --frozen-lockfile
@@ -79,7 +96,7 @@ jobs:
run: pnpm run generate
- name: 🧪 Run Internal Unit Tests
run: pnpm run test:internal --reporter=default --reporter=blob --shard=${{ matrix.shardIndex }}/${{ matrix.shardTotal }}
run: pnpm run test:internal --reporter=default --reporter=blob --shard=${{ matrix.shardIndex }}/${{ matrix.shardTotal }} --passWithNoTests
- name: Gather all reports
if: ${{ !cancelled() }}
@@ -90,7 +107,7 @@ jobs:
- name: Upload blob reports to GitHub Actions Artifacts
if: ${{ !cancelled() }}
uses: actions/upload-artifact@v4
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: internal-blob-report-${{ matrix.shardIndex }}
path: .vitest-reports/*
@@ -104,27 +121,28 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: ⬇️ Checkout repo
uses: actions/checkout@v4
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
fetch-depth: 0
persist-credentials: false
- name: ⎔ Setup pnpm
uses: pnpm/action-setup@v4
uses: pnpm/action-setup@fc06bc1257f339d1d5d8b3a19a8cae5388b55320 # v5.0.0
with:
version: 8.15.5
version: 10.33.2
- name: ⎔ Setup node
uses: buildjet/setup-node@v4
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: 20.11.1
node-version: 20.20.0
# no cache enabled, we're not installing deps
- name: Download blob reports from GitHub Actions Artifacts
uses: actions/download-artifact@v4
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
path: .vitest-reports
pattern: internal-blob-report-*
merge-multiple: true
- name: Merge reports
run: pnpm dlx vitest@3.1.4 run --merge-reports --pass-with-no-tests
run: pnpm dlx vitest@4.1.7 run --merge-reports --pass-with-no-tests
+33 -15
View File
@@ -5,6 +5,11 @@ permissions:
on:
workflow_call:
secrets:
DOCKERHUB_USERNAME:
required: false
DOCKERHUB_TOKEN:
required: false
jobs:
unitTests:
@@ -46,25 +51,26 @@ jobs:
run: sudo systemctl restart docker
- name: ⬇️ Checkout repo
uses: actions/checkout@v4
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
fetch-depth: 0
persist-credentials: false
- name: ⎔ Setup pnpm
uses: pnpm/action-setup@v4
uses: pnpm/action-setup@fc06bc1257f339d1d5d8b3a19a8cae5388b55320 # v5.0.0
with:
version: 8.15.5
version: 10.33.2
- name: ⎔ Setup node
uses: buildjet/setup-node@v4
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: 20.11.1
node-version: 20.20.0
cache: "pnpm"
# ..to avoid rate limits when pulling images
- name: 🐳 Login to DockerHub
if: ${{ env.DOCKERHUB_USERNAME }}
uses: docker/login-action@v3
uses: docker/login-action@4907a6ddec9925e35a0a9e82d7399ccc52663121 # v4.1.0
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
@@ -72,6 +78,17 @@ jobs:
if: ${{ !env.DOCKERHUB_USERNAME }}
run: echo "DockerHub login skipped because secrets are not available."
- name: 🐳 Pre-pull testcontainer images
if: ${{ env.DOCKERHUB_USERNAME }}
run: |
echo "Pre-pulling Docker images with authenticated session..."
docker pull postgres:14
docker pull clickhouse/clickhouse-server:25.4-alpine
docker pull redis:7-alpine
docker pull testcontainers/ryuk:0.11.0
docker pull electricsql/electric:1.2.4
echo "Image pre-pull complete"
- name: 📥 Download deps
run: pnpm install --frozen-lockfile
@@ -79,7 +96,7 @@ jobs:
run: pnpm run generate
- name: 🧪 Run Package Unit Tests
run: pnpm run test:packages --reporter=default --reporter=blob --shard=${{ matrix.shardIndex }}/${{ matrix.shardTotal }}
run: pnpm run test:packages --reporter=default --reporter=blob --shard=${{ matrix.shardIndex }}/${{ matrix.shardTotal }} --passWithNoTests
- name: Gather all reports
if: ${{ !cancelled() }}
@@ -90,7 +107,7 @@ jobs:
- name: Upload blob reports to GitHub Actions Artifacts
if: ${{ !cancelled() }}
uses: actions/upload-artifact@v4
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: packages-blob-report-${{ matrix.shardIndex }}
path: .vitest-reports/*
@@ -104,27 +121,28 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: ⬇️ Checkout repo
uses: actions/checkout@v4
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
fetch-depth: 0
persist-credentials: false
- name: ⎔ Setup pnpm
uses: pnpm/action-setup@v4
uses: pnpm/action-setup@fc06bc1257f339d1d5d8b3a19a8cae5388b55320 # v5.0.0
with:
version: 8.15.5
version: 10.33.2
- name: ⎔ Setup node
uses: buildjet/setup-node@v4
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: 20.11.1
node-version: 20.20.0
# no cache enabled, we're not installing deps
- name: Download blob reports from GitHub Actions Artifacts
uses: actions/download-artifact@v4
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
path: .vitest-reports
pattern: packages-blob-report-*
merge-multiple: true
- name: Merge reports
run: pnpm dlx vitest@3.1.4 run --merge-reports --pass-with-no-tests
run: pnpm dlx vitest@4.1.7 run --merge-reports --pass-with-no-tests
+33 -15
View File
@@ -5,6 +5,11 @@ permissions:
on:
workflow_call:
secrets:
DOCKERHUB_USERNAME:
required: false
DOCKERHUB_TOKEN:
required: false
jobs:
unitTests:
@@ -46,25 +51,26 @@ jobs:
run: sudo systemctl restart docker
- name: ⬇️ Checkout repo
uses: actions/checkout@v4
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
fetch-depth: 0
persist-credentials: false
- name: ⎔ Setup pnpm
uses: pnpm/action-setup@v4
uses: pnpm/action-setup@fc06bc1257f339d1d5d8b3a19a8cae5388b55320 # v5.0.0
with:
version: 8.15.5
version: 10.33.2
- name: ⎔ Setup node
uses: buildjet/setup-node@v4
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: 20.11.1
node-version: 20.20.0
cache: "pnpm"
# ..to avoid rate limits when pulling images
- name: 🐳 Login to DockerHub
if: ${{ env.DOCKERHUB_USERNAME }}
uses: docker/login-action@v3
uses: docker/login-action@4907a6ddec9925e35a0a9e82d7399ccc52663121 # v4.1.0
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
@@ -72,6 +78,17 @@ jobs:
if: ${{ !env.DOCKERHUB_USERNAME }}
run: echo "DockerHub login skipped because secrets are not available."
- name: 🐳 Pre-pull testcontainer images
if: ${{ env.DOCKERHUB_USERNAME }}
run: |
echo "Pre-pulling Docker images with authenticated session..."
docker pull postgres:14
docker pull clickhouse/clickhouse-server:25.4-alpine
docker pull redis:7-alpine
docker pull testcontainers/ryuk:0.11.0
docker pull electricsql/electric:1.2.4
echo "Image pre-pull complete"
- name: 📥 Download deps
run: pnpm install --frozen-lockfile
@@ -79,7 +96,7 @@ jobs:
run: pnpm run generate
- name: 🧪 Run Webapp Unit Tests
run: pnpm run test:webapp --reporter=default --reporter=blob --shard=${{ matrix.shardIndex }}/${{ matrix.shardTotal }}
run: pnpm run test:webapp --reporter=default --reporter=blob --shard=${{ matrix.shardIndex }}/${{ matrix.shardTotal }} --passWithNoTests
env:
DATABASE_URL: postgresql://postgres:postgres@localhost:5432/postgres
DIRECT_URL: postgresql://postgres:postgres@localhost:5432/postgres
@@ -98,7 +115,7 @@ jobs:
- name: Upload blob reports to GitHub Actions Artifacts
if: ${{ !cancelled() }}
uses: actions/upload-artifact@v4
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: webapp-blob-report-${{ matrix.shardIndex }}
path: .vitest-reports/*
@@ -112,27 +129,28 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: ⬇️ Checkout repo
uses: actions/checkout@v4
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
fetch-depth: 0
persist-credentials: false
- name: ⎔ Setup pnpm
uses: pnpm/action-setup@v4
uses: pnpm/action-setup@fc06bc1257f339d1d5d8b3a19a8cae5388b55320 # v5.0.0
with:
version: 8.15.5
version: 10.33.2
- name: ⎔ Setup node
uses: buildjet/setup-node@v4
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: 20.11.1
node-version: 20.20.0
# no cache enabled, we're not installing deps
- name: Download blob reports from GitHub Actions Artifacts
uses: actions/download-artifact@v4
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
path: .vitest-reports
pattern: webapp-blob-report-*
merge-multiple: true
- name: Merge reports
run: pnpm dlx vitest@3.1.4 run --merge-reports --pass-with-no-tests
run: pnpm dlx vitest@4.1.7 run --merge-reports --pass-with-no-tests
+19 -3
View File
@@ -5,14 +5,30 @@ permissions:
on:
workflow_call:
secrets:
DOCKERHUB_USERNAME:
required: false
DOCKERHUB_TOKEN:
required: false
jobs:
webapp:
uses: ./.github/workflows/unit-tests-webapp.yml
secrets: inherit
secrets:
DOCKERHUB_USERNAME: ${{ secrets.DOCKERHUB_USERNAME }}
DOCKERHUB_TOKEN: ${{ secrets.DOCKERHUB_TOKEN }}
e2e-webapp:
uses: ./.github/workflows/e2e-webapp.yml
secrets:
DOCKERHUB_USERNAME: ${{ secrets.DOCKERHUB_USERNAME }}
DOCKERHUB_TOKEN: ${{ secrets.DOCKERHUB_TOKEN }}
packages:
uses: ./.github/workflows/unit-tests-packages.yml
secrets: inherit
secrets:
DOCKERHUB_USERNAME: ${{ secrets.DOCKERHUB_USERNAME }}
DOCKERHUB_TOKEN: ${{ secrets.DOCKERHUB_TOKEN }}
internal:
uses: ./.github/workflows/unit-tests-internal.yml
secrets: inherit
secrets:
DOCKERHUB_USERNAME: ${{ secrets.DOCKERHUB_USERNAME }}
DOCKERHUB_TOKEN: ${{ secrets.DOCKERHUB_TOKEN }}
+50
View File
@@ -0,0 +1,50 @@
name: Vouch - Check PR
on:
pull_request_target: # zizmor: ignore[dangerous-triggers] needed to comment/close fork PRs; safe because we never check out PR HEAD ref so no fork-controlled code runs
types: [opened, reopened]
permissions: {}
jobs:
check-vouch:
runs-on: ubuntu-latest
permissions:
contents: read
pull-requests: write # auto-close unvouched PRs
issues: read
steps:
- uses: mitchellh/vouch/action/check-pr@c6d80ead49839655b61b422700b7a3bc9d0804a9 # v1.4.2
with:
pr-number: ${{ github.event.pull_request.number }}
auto-close: true
require-vouch: true
env:
GH_TOKEN: ${{ github.token }}
require-draft:
needs: check-vouch
permissions:
pull-requests: write # close non-draft PRs with a comment
if: >
github.event.pull_request.draft == false &&
github.event.pull_request.author_association != 'MEMBER' &&
github.event.pull_request.author_association != 'OWNER' &&
github.event.pull_request.author_association != 'COLLABORATOR' &&
github.event.pull_request.user.login != 'devin-ai-integration[bot]' &&
github.event.pull_request.user.login != 'dependabot[bot]' &&
github.event.pull_request.user.login != 'github-actions[bot]'
runs-on: ubuntu-latest
steps:
- name: Close non-draft PR
env:
GH_TOKEN: ${{ github.token }}
run: |
STATE=$(gh pr view ${{ github.event.pull_request.number }} --repo ${{ github.repository }} --json state -q '.state')
if [ "$STATE" != "OPEN" ]; then
echo "PR is already closed, skipping."
exit 0
fi
gh pr close ${{ github.event.pull_request.number }} \
--repo ${{ github.repository }} \
--comment "Thanks for your contribution! We require all external PRs to be opened in **draft** status first so you can address CodeRabbit review comments and ensure CI passes before requesting a review. Please re-open this PR as a draft. See [CONTRIBUTING.md](https://github.com/${{ github.repository }}/blob/main/CONTRIBUTING.md#pr-workflow) for details."
@@ -0,0 +1,24 @@
name: Vouch - Manage by Issue
on:
issue_comment:
types: [created]
permissions:
contents: write
issues: write
jobs:
manage:
runs-on: ubuntu-latest
if: >-
contains(github.event.comment.body, 'vouch') ||
contains(github.event.comment.body, 'denounce') ||
contains(github.event.comment.body, 'unvouch')
steps:
- uses: mitchellh/vouch/action/manage-by-issue@c6d80ead49839655b61b422700b7a3bc9d0804a9 # v1.4.2
with:
comment-id: ${{ github.event.comment.id }}
issue-id: ${{ github.event.issue.number }}
env:
GH_TOKEN: ${{ github.token }}
+51
View File
@@ -0,0 +1,51 @@
name: Workflow Checks
on:
push:
branches: [main]
paths:
- '.github/workflows/**'
- '.github/actions/**'
- '.github/zizmor.yml'
pull_request:
paths:
- '.github/workflows/**'
- '.github/actions/**'
- '.github/zizmor.yml'
permissions: {}
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
actionlint:
name: Actionlint
runs-on: ubuntu-latest
permissions:
contents: read
steps:
- name: Checkout repository
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
persist-credentials: false
- name: Run actionlint
uses: docker://rhysd/actionlint:1.7.12@sha256:b1934ee5f1c509618f2508e6eb47ee0d3520686341fec936f3b79331f9315667
zizmor:
name: Zizmor
runs-on: ubuntu-latest
permissions:
security-events: write # Upload SARIF to GitHub Security tab
contents: read # Read workflow files for analysis
actions: read # Read workflow run metadata
steps:
- name: Checkout repository
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
persist-credentials: false
- name: Run zizmor
uses: zizmorcore/zizmor-action@b1d7e1fb5de872772f31590499237e7cce841e8e # v0.5.3
+5
View File
@@ -0,0 +1,5 @@
rules:
unpinned-uses:
config:
policies:
'*': hash-pin
+15 -7
View File
@@ -15,6 +15,9 @@ out/
dist
packages/**/dist
# vendored bundles (generated during build)
packages/**/src/**/vendor
# Tailwind
apps/**/styles/tailwind.css
packages/**/styles/tailwind.css
@@ -29,12 +32,10 @@ yarn-debug.log*
yarn-error.log*
# local env files
.env.docker
.env
.env.*
.docker/*.env
.env.local
.env.development.local
.env.test.local
.env.production.local
!.env.example
# turbo
.turbo
@@ -63,5 +64,12 @@ apps/**/public/build
/packages/core/src/package.json
/packages/trigger-sdk/src/package.json
/packages/python/src/package.json
.claude
.mcp.log
**/.claude/settings.local.json
.claude/architecture/
.claude/docs-plans/
.claude/review-guides/
.claude/scheduled_tasks.lock
.mcp.log
.mcp.json
.cursor/debug.log
ailogger-output.log
-5
View File
@@ -1,5 +0,0 @@
link-workspace-packages=false
public-hoist-pattern[]=*prisma*
prefer-workspace-packages=true
update-notifier=false
side-effects-cache=false
+1 -1
View File
@@ -1 +1 @@
v20.11.1
v20.20.0
View File
+81
View File
@@ -0,0 +1,81 @@
# Server Changes
This directory tracks changes to server-only components (webapp, supervisor, coordinator, etc.) that are not captured by changesets. Changesets only track published npm packages — server changes would otherwise go undocumented.
## When to add a file
**Server-only PRs**: If your PR only changes `apps/webapp/`, `apps/supervisor/`, `apps/coordinator/`, or other server components (and does NOT change anything in `packages/`), add a `.server-changes/` file.
**Mixed PRs** (both packages and server): Just add a changeset as usual. No `.server-changes/` file needed — the changeset covers it.
**Package-only PRs**: Just add a changeset as usual.
## File format
Create a markdown file with a descriptive name:
```
.server-changes/fix-batch-queue-stalls.md
```
With this format:
```markdown
---
area: webapp
type: fix
---
Speed up batch queue processing by removing stalls and fixing retry race
```
### Fields
- **area** (required): `webapp` | `supervisor` | `coordinator` | `kubernetes-provider` | `docker-provider`
- **type** (required): `feature` | `fix` | `improvement` | `breaking`
### Description
The body text (below the frontmatter) is a one-line description of the change. Keep it concise — it will appear in release notes.
## Lifecycle
1. Engineer adds a `.server-changes/` file in their PR
2. Files accumulate on `main` as PRs merge
3. The changeset release PR includes these in its summary
4. After the release merges, CI cleans up the consumed files
## Examples
**New feature:**
```markdown
---
area: webapp
type: feature
---
TRQL query language and the Query page
```
**Bug fix:**
```markdown
---
area: webapp
type: fix
---
Fix schedule limit counting for orgs with custom limits
```
**Improvement:**
```markdown
---
area: webapp
type: improvement
---
Use the replica for API auth queries to reduce primary load
```
@@ -0,0 +1,16 @@
---
area: webapp
type: fix
---
Fix the dashboard AI assistant (`dashboard-assistant` chat.agent) silently not
responding to messages. The session is started via `chat.createStartSessionAction`,
which triggers the first run with `trigger: "preload"`, so every chat boots
preloaded. The agent only created its `AiChat`/`AiChatSession` rows in
`onChatStart`, which early-returns on preloaded runs — so the rows were never
created and `onTurnStart`'s `aiChat.update(...)` threw before `run()` streamed.
Adds an `onPreload` hook that creates the rows (with `onChatStart` kept as the
non-preloaded fallback), and declares `tools` on the agent config (function form)
read back from the `run()` payload so the SDK re-applies each tool's
`toModelOutput` when re-converting history on later turns.
+6
View File
@@ -0,0 +1,6 @@
---
area: webapp
type: fix
---
Upgrade posthog-node to v5 (drops its axios dependency) and remove the now-stale axios override so axios resolves to patched 1.16.1
+6
View File
@@ -0,0 +1,6 @@
---
area: webapp
type: feature
---
Added `/engine/v1/dev/disconnect` endpoint to auto-cancel runs when the CLI disconnects. Maximum of 500 runs can be cancelled. Uses the bulk action system when there are more than 25 runs to cancel.
+3
View File
@@ -0,0 +1,3 @@
vouch:
- github: edosrecki
- github: GautamBytes
+9
View File
@@ -31,6 +31,15 @@
"cwd": "${workspaceFolder}/apps/webapp",
"sourceMaps": true
},
{
"type": "node-terminal",
"request": "launch",
"name": "Debug opened test file",
"command": "pnpm run test -- ./${relativeFile}",
"envFile": "${workspaceFolder}/.env",
"cwd": "${workspaceFolder}",
"sourceMaps": true
},
{
"type": "chrome",
"request": "launch",
+1 -1
View File
@@ -7,5 +7,5 @@
"packages/cli-v3/e2e": true
},
"vitest.disableWorkspaceWarning": true,
"typescript.experimental.useTsgo": false
"chat.agent.maxRequests": 10000
}

Some files were not shown because too many files have changed in this diff Show More