triggerdotdev--trigger.dev

github-featured/triggerdotdev--trigger.dev

Author	SHA1	Message	Date
github-actions[bot]	d825427cbc	chore: release v4.4.5 (#3406 ) ## Summary 8 new features, 18 improvements, 11 bug fixes. ## Breaking changes - Add server-side deprecation gate for deploys from v3 CLI versions (gated by `DEPRECATE_V3_CLI_DEPLOYS_ENABLED`). v4 CLI deploys are unaffected. ([#3415](https://github.com/triggerdotdev/trigger.dev/pull/3415)) ## Improvements - Add `--no-browser` flag to `init` and `login` to skip auto-opening the browser during authentication. Also error loudly when `init` is run without `--yes` under non-TTY stdin (previously default-and-exited silently, leaving the project half-initialized). Both commands now show an `Examples` section in `--help`. ([#3483](https://github.com/triggerdotdev/trigger.dev/pull/3483)) - Add `isReplay` boolean to the run context (`ctx.run.isReplay`), derived from the existing `replayedFromTaskRunFriendlyId` database field. Defaults to `false` for backwards compatibility. ([#3454](https://github.com/triggerdotdev/trigger.dev/pull/3454)) - Redact the `resolveWaitpoint` runtime log so it only emits `id` and `type` instead of the full completed waitpoint. Previously the log printed the entire waitpoint (including `output`) to stdout in production runs, which could leak sensitive payloads. The value returned by `wait.forToken()` is unchanged. ([#3490](https://github.com/triggerdotdev/trigger.dev/pull/3490)) - Add `SessionId` friendly ID generator and schemas for the new durable Session primitive. Exported from `@trigger.dev/core/v3/isomorphic` alongside `RunId`, `BatchId`, etc. Ships the `CreateSessionStreamWaitpoint` request/response schemas alongside the main Session CRUD. ([#3417](https://github.com/triggerdotdev/trigger.dev/pull/3417)) - Truncate large error stacks and messages to prevent OOM crashes. Stack traces are capped at 50 frames (keeping top 5 + bottom 45 with an omission notice), individual stack lines at 1024 chars, and error messages at 1000 chars. Applied in parseError, sanitizeError, and OTel span recording. ([#3405](https://github.com/triggerdotdev/trigger.dev/pull/3405)) ## Server changes These changes affect the self-hosted Docker image and Trigger.dev Cloud: - Add a "Back office" tab to `/admin` and a per-organization detail page at `/admin/back-office/orgs/:orgId`. The first action available on that page is editing the org's API rate limit: admins can save a `tokenBucket` override (refill rate, interval, max tokens) and see a plain-English preview of the resulting sustained rate and burst allowance. Writes are audit-logged via the server logger. ([#3434](https://github.com/triggerdotdev/trigger.dev/pull/3434)) - Optional `DEPLOY_REGISTRY_ECR_DEFAULT_REPOSITORY_POLICY` env var to apply a default repository policy when the webapp creates new ECR repos ([#3467](https://github.com/triggerdotdev/trigger.dev/pull/3467)) - Ship the Errors page to all users, with a polish + bug-fix pass: pinned "No channel" item in the Slack alert channel picker, viewer-timezone alert timestamps via Slack's `<!date^>` token, Activity sparkline peak tooltip, centered loading spinner and bug-icon empty state on the error detail page, ellipsis on the Configure alerts trigger. ([#3477](https://github.com/triggerdotdev/trigger.dev/pull/3477)) - Configure the set of machine presets to build boot snapshots for at deploy time via `COMPUTE_TEMPLATE_MACHINE_PRESETS` (CSV of preset names, default `small-1x`). Use `COMPUTE_TEMPLATE_MACHINE_PRESETS_REQUIRED` (CSV, default = full PRESETS list) to scope which preset failures fail a required-mode deploy. Optional preset failures are logged and don't block the deploy. ([#3492](https://github.com/triggerdotdev/trigger.dev/pull/3492)) - Regenerating a RuntimeEnvironment API key no longer invalidates the previous key immediately. The old key is recorded in a new `RevokedApiKey` table with a 24 hour grace window, and `findEnvironmentByApiKey` falls back to it when the submitted key doesn't match any live environment. The grace window can be ended early (or extended) by updating `expiresAt` on the row. ([#3420](https://github.com/triggerdotdev/trigger.dev/pull/3420)) - Add the `Session` primitive — a durable, task-bound, bidirectional I/O channel that outlives a single run and acts as the run manager for `chat.agent`. Ships the Postgres `Session` + `SessionRun` tables, ClickHouse `sessions_v1` + replication service, the `sessions` JWT scope, and the public CRUD + realtime routes (`/api/v1/sessions`, `/realtime/v1/sessions/:session/:io`) including `end-and-continue` for server-orchestrated run handoffs and session-stream waitpoints. ([#3417](https://github.com/triggerdotdev/trigger.dev/pull/3417)) - Add `KUBERNETES_POD_DNS_NDOTS_OVERRIDE_ENABLED` flag (off by default) that overrides the cluster default and sets `dnsConfig.options.ndots` on runner pods (defaulting to 2, configurable via `KUBERNETES_POD_DNS_NDOTS`). Kubernetes defaults pods to `ndots: 5`, so any name with fewer than 5 dots — including typical external domains like `api.example.com` — is first walked through every entry in the cluster search list (`<ns>.svc.cluster.local`, `svc.cluster.local`, `cluster.local`) before being tried as-is, turning one resolution into 4+ CoreDNS queries (×2 with A+AAAA). Using a lower `ndots` value reduces DNS query amplification in the `cluster.local` zone. Note: before enabling, make sure no code path relies on search-list expansion for names with dots ≥ the configured value — those names will hit their as-is form first and could resolve externally before falling back to the cluster search path. ([#3441](https://github.com/triggerdotdev/trigger.dev/pull/3441)) - Vercel integration option to disable auto promotions ([#3376](https://github.com/triggerdotdev/trigger.dev/pull/3376)) - Make it clear in the admin that feature flags are global and should rarely be changed. ([#3408](https://github.com/triggerdotdev/trigger.dev/pull/3408)) - Admin worker groups API: add GET loader and expose more fields on POST. ([#3390](https://github.com/triggerdotdev/trigger.dev/pull/3390)) - Add 60s fresh / 60s stale SWR cache to `getEntitlement` in `platform.v3.server.ts`. Eliminates a synchronous billing-service HTTP round trip on every trigger. Reuses the existing `platformCache` (LRU memory + Redis) pattern already used for `limits` and `usage`. Cache key is `${orgId}`. Errors return a permissive `{ hasAccess: true }` fallback (existing behavior) and are also cached to prevent thundering-herd on billing outages. ([#3388](https://github.com/triggerdotdev/trigger.dev/pull/3388)) - Show a `MicroVM` badge next to the region name on the regions page. ([#3407](https://github.com/triggerdotdev/trigger.dev/pull/3407)) - Increase default maximum project count per organization from 10 to 25 ([#3409](https://github.com/triggerdotdev/trigger.dev/pull/3409)) - Merge execution snapshot creation into the dequeue taskRun.update transaction, reducing 2 DB commits to 1 per dequeue operation ([#3395](https://github.com/triggerdotdev/trigger.dev/pull/3395)) - Add per-worker Node.js heap metrics to the OTel meter — `nodejs.memory.heap.used`, `nodejs.memory.heap.total`, `nodejs.memory.heap.limit`, `nodejs.memory.external`, `nodejs.memory.array_buffers`, `nodejs.memory.rss`. Host-metrics only publishes RSS, which overstates V8 heap by the external + native footprint; these give direct heap visibility per cluster worker so `NODE_MAX_OLD_SPACE_SIZE` can be sized against observed heap peaks rather than RSS. ([#3437](https://github.com/triggerdotdev/trigger.dev/pull/3437)) - Tag Prisma spans with `db.datasource: "writer" \| "replica"` so monitors and trace queries can distinguish the writer pool from the replica pool. Applies to all `prisma:engine:` spans (including `prisma:engine:connection` used by the connection-pool monitors) and the outer `prisma:client:operation` span. ([#3422](https://github.com/triggerdotdev/trigger.dev/pull/3422)) - Clarify the cross-region intent in the Terraform and AI-prompt helpers on the Add Private Connection page. Both already default `supported_regions` to `["us-east-1", "eu-central-1"]`; added an inline comment / parenthetical so the user understands why both regions are listed (Trigger.dev runs in both, so the service must be consumable from either). ([#3465](https://github.com/triggerdotdev/trigger.dev/pull/3465)) - Add `RUN_ENGINE_READ_REPLICA_SNAPSHOTS_SINCE_ENABLED` flag (default off) to route the Prisma reads inside `RunEngine.getSnapshotsSince` through the read-only replica client. Offloads the snapshot polling queries (fired by every running task runner) from the primary. When disabled, behavior is unchanged. ([#3423](https://github.com/triggerdotdev/trigger.dev/pull/3423)) - Stop creating TaskRunTag records and _TaskRunToTaskRunTag join table entries during task triggering. The denormalized runTags string array on TaskRun already stores tag names, making the M2M relation redundant write overhead. ([#3369](https://github.com/triggerdotdev/trigger.dev/pull/3369)) - Stop writing per-tick state (`lastScheduledTimestamp`, `nextScheduledTimestamp`, `lastRunTriggeredAt`) on `TaskSchedule` and `TaskScheduleInstance`. The schedule engine now carries the previous fire time forward via the worker queue payload, eliminating ~270K dead-tuple-driven autovacuums per year on these hot tables and the associated `IO:XactSync` mini-spikes on the writer. Customer-facing `payload.lastTimestamp` semantics are unchanged. ([#3476](https://github.com/triggerdotdev/trigger.dev/pull/3476)) - Replace the expensive DISTINCT query for task filter dropdowns with a dedicated TaskIdentifier registry table backed by Redis. Environments migrate automatically on their next deploy, with a transparent fallback to the legacy query for unmigrated environments. Also fixes duplicate dropdown entries when a task changes trigger source, and adds active/archived grouping for removed tasks. Moves BackgroundWorkerTask reads in the trigger hot path to the read replica. ([#3368](https://github.com/triggerdotdev/trigger.dev/pull/3368)) - Public Access Tokens (PATs) minted before an API key rotation now keep working during the 24h grace window. `validatePublicJwtKey` falls back to any non-expired `RevokedApiKey` rows for the signing environment when the primary signature check against the env's current `apiKey` fails. The fallback query only runs on the failure path, so the hot success path is unchanged. ([#3464](https://github.com/triggerdotdev/trigger.dev/pull/3464)) - Batch items that hit the environment queue size limit now fast-fail without retries and without creating pre-failed TaskRuns. ([#3352](https://github.com/triggerdotdev/trigger.dev/pull/3352)) - Show the cancel button in the runs list for runs in `DEQUEUED` status. `DEQUEUED` was missing from `NON_FINAL_RUN_STATUSES` so the list hid the button even though the single run page allowed it. ([#3421](https://github.com/triggerdotdev/trigger.dev/pull/3421)) - Reduce 5xx feedback loops on hot debounce keys by quantizing `delayUntil`, adding an unlocked fast-path skip, and gracefully handling redlock contention in `handleDebounce` so the SDK no longer retries into a herd. ([#3453](https://github.com/triggerdotdev/trigger.dev/pull/3453)) - Fix RSS memory leak in the realtime proxy routes. `/realtime/v1/runs`, `/realtime/v1/runs/:id`, and `/realtime/v1/batches/:id` called `fetch()` into Electric with no abort signal, so when a client disconnected mid long-poll, undici kept the upstream socket open and buffered response chunks that would never be consumed — retained only in RSS, invisible to V8 heap tooling. Thread `getRequestAbortSignal()` through `RealtimeClient.streamRun/streamRuns/streamBatch` to `longPollingFetch` and cancel the upstream body in the error path. Isolated reproducer showed ~44 KB retained per leaked request; signal propagation releases it cleanly. ([#3442](https://github.com/triggerdotdev/trigger.dev/pull/3442)) - Fix memory leak where every aborted SSE connection pinned the full request/response graph on Node 20, caused by `AbortSignal.any()` in `sse.ts` retaining its source signals indefinitely (see nodejs/node#54614, nodejs/node#55351). Also clear the `setTimeout(abort)` timer in `entry.server.tsx` so successful HTML renders don't pin the React tree for 30s per request. ([#3430](https://github.com/triggerdotdev/trigger.dev/pull/3430)) - Preserve filters on the queues page when submitting modal actions. ([#3471](https://github.com/triggerdotdev/trigger.dev/pull/3471)) - Fix Redis connection leak in realtime streams and broken abort signal propagation. Redis connections: Non-blocking methods (ingestData, appendPart, getLastChunkIndex) now share a single Redis connection instead of creating one per request. streamResponse still uses dedicated connections (required for XREAD BLOCK) but now tears them down immediately via disconnect() instead of graceful quit(), with a 15s inactivity fallback. Abort signal*: request.signal is broken in Remix/Express due to a Node.js undici GC bug (nodejs/node#55428) that severs the signal chain when Remix clones the Request internally. Added getRequestAbortSignal() wired to Express res.on("close") via httpAsyncStorage, which fires reliably on client disconnect. All SSE/streaming routes updated to use it. ([#3399](https://github.com/triggerdotdev/trigger.dev/pull/3399)) - Prevent dashboard crash (React error #31) when span accessory item text is not a string. Filters out malformed accessory items in SpanCodePathAccessory instead of passing objects to React as children. ([#3400](https://github.com/triggerdotdev/trigger.dev/pull/3400)) - Upgrade Remix packages from 2.1.0 to 2.17.4 to address security vulnerabilities in React Router ([#3372](https://github.com/triggerdotdev/trigger.dev/pull/3372)) - Fix Vercel integration settings page (remove redundant section toggles) and improve the Vercel onboarding flow so the modal closes after connecting a GitHub repo and the marketplace `next` URL is preserved across the GitHub app install redirect. ([#3424](https://github.com/triggerdotdev/trigger.dev/pull/3424)) <details> <summary>Raw changeset output</summary> # Releases ## @trigger.dev/build@4.4.5 ### Patch Changes - Updated dependencies: - `@trigger.dev/core@4.4.5` ## trigger.dev@4.4.5 ### Patch Changes - Add `--no-browser` flag to `init` and `login` to skip auto-opening the browser during authentication. Also error loudly when `init` is run without `--yes` under non-TTY stdin (previously default-and-exited silently, leaving the project half-initialized). Both commands now show an `Examples` section in `--help`. ([#3483](https://github.com/triggerdotdev/trigger.dev/pull/3483)) - Updated dependencies: - `@trigger.dev/core@4.4.5` - `@trigger.dev/build@4.4.5` - `@trigger.dev/schema-to-json@4.4.5` ## @trigger.dev/core@4.4.5 ### Patch Changes - Add `isReplay` boolean to the run context (`ctx.run.isReplay`), derived from the existing `replayedFromTaskRunFriendlyId` database field. Defaults to `false` for backwards compatibility. ([#3454](https://github.com/triggerdotdev/trigger.dev/pull/3454)) - Redact the `resolveWaitpoint` runtime log so it only emits `id` and `type` instead of the full completed waitpoint. Previously the log printed the entire waitpoint (including `output`) to stdout in production runs, which could leak sensitive payloads. The value returned by `wait.forToken()` is unchanged. ([#3490](https://github.com/triggerdotdev/trigger.dev/pull/3490)) - Add `SessionId` friendly ID generator and schemas for the new durable Session primitive. Exported from `@trigger.dev/core/v3/isomorphic` alongside `RunId`, `BatchId`, etc. Ships the `CreateSessionStreamWaitpoint` request/response schemas alongside the main Session CRUD. ([#3417](https://github.com/triggerdotdev/trigger.dev/pull/3417)) - Truncate large error stacks and messages to prevent OOM crashes. Stack traces are capped at 50 frames (keeping top 5 + bottom 45 with an omission notice), individual stack lines at 1024 chars, and error messages at 1000 chars. Applied in parseError, sanitizeError, and OTel span recording. ([#3405](https://github.com/triggerdotdev/trigger.dev/pull/3405)) ## @trigger.dev/python@4.4.5 ### Patch Changes - Updated dependencies: - `@trigger.dev/core@4.4.5` - `@trigger.dev/build@4.4.5` - `@trigger.dev/sdk@4.4.5` ## @trigger.dev/react-hooks@4.4.5 ### Patch Changes - Updated dependencies: - `@trigger.dev/core@4.4.5` ## @trigger.dev/redis-worker@4.4.5 ### Patch Changes - Updated dependencies: - `@trigger.dev/core@4.4.5` ## @trigger.dev/rsc@4.4.5 ### Patch Changes - Updated dependencies: - `@trigger.dev/core@4.4.5` ## @trigger.dev/schema-to-json@4.4.5 ### Patch Changes - Updated dependencies: - `@trigger.dev/core@4.4.5` ## @trigger.dev/sdk@4.4.5 ### Patch Changes - Updated dependencies: - `@trigger.dev/core@4.4.5` </details> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>	2026-05-01 15:13:11 +01:00
Matt Aitken	7c7d785552	Don't log waitpoint output when resolving (#3490 ) Redact the `resolveWaitpoint` runtime log so it only emits `id` and `type` instead of the full completed waitpoint. Previously the log printed the entire waitpoint (including `output`) to stdout in production runs, which could leak sensitive payloads. The value returned by `wait.forToken()` is unchanged.	2026-05-01 12:37:57 +01:00
nicktrn	e2b9e0f9f5	feat(cli-v3): add --no-browser flag and examples to init/login --help (#3483 ) Closes the most common friction point hit while setting up a fresh project from an agent harness: the CLI auto-opens the user's default browser during auth and there is no supported way to skip it (the existing `isLinuxServer()` path only triggers when `xdg-open` is missing entirely). `--no-browser` on `login` and `init` prints the URL and waits to be visited from any browser. The flag threads through the embedded `login()` call inside `init`. While here: - `init` now errors loudly when stdin is non-TTY without `--yes` instead of default-and-exiting silently at the first prompt (which left the project half-initialized: deps installed, no config or example file). - Both commands gain an `Examples` block in `--help` rendered between the description and the arguments/options list, so `--help \| head` surfaces the common invocations. Other commands also call `login()` embedded and would benefit from `--no-browser` too, but kept this PR scoped to the cases the friction log called out.	2026-05-01 09:39:22 +01:00
Eric Allam	c69e939c34	feat: Sessions - bidirectional durable agent streams (#3417 ) > ⚠️ Not released yet. This PR is the server-side foundation only. The SDK changes that customers will actually use (`chat.agent` migration, `chat.createStartSessionAction`, `useTriggerChatTransport` updates) live on a separate branch and ship together in an upcoming `@trigger.dev/sdk` prerelease. Until that prerelease is published, this surface is reachable only via direct HTTP. ## What this gives Trigger.dev users A new first-class primitive, Session, for durable, task-bound, bidirectional I/O that outlives any single run. Sessions are the run manager for `chat.agent` going forward, and they unblock anything else that needs "one identifier, many runs over time" with a stable channel pair the client can write to and subscribe to. ### Use cases unblocked - Chat agents that persist across many runs. One session per chat (keyed on your own `chatId` via `externalId`), turns 1..N attach to the same Session, the UI subscribes once and keeps receiving output as new runs take over. - Approval loops and long-running tasks with user feedback. The task waits on `.in`, the client writes to `.in`, the server enforces no-writes-after-close. - Workflow progress streams that live past the run. Subscribe to `.out` after the task finishes to replay history. - Resume-next-day flows. A session is a durable row, not a transient stream. Send a message a day later and the server triggers a fresh run on the same session. ### How it works (Session-as-run-manager) A Session row is task-bound (`taskIdentifier` + `triggerConfig` are required) and owns its current run via `currentRunId` + `currentRunVersion` for optimistic claim. Three trigger paths: 1. Session create — `POST /api/v1/sessions` creates the row and triggers the first run synchronously. 2. Append-time probe — `POST /realtime/v1/sessions/:session/in/append` checks if the current run is alive; if it has terminated (idle exit, crash, etc.), the server triggers a new run before processing the append. 3. End-and-continue handoff — `POST /api/v1/sessions/:session/end-and-continue`, called by the running agent, triggers a fresh run and atomically swaps `currentRunId`. Used by `chat.requestUpgrade()` for version handoffs. Every triggered run is recorded in the `SessionRun` audit table with a reason (`initial`, `continuation`, `upgrade`, `manual`). ## Public API surface ### Control plane - `POST /api/v1/sessions` — create. Idempotent on `(env, externalId)`. Triggers the first run, returns the session and a session-scoped public access token. Returns 409 if the upserted row is already closed. - `GET /api/v1/sessions/:session` — retrieve by friendlyId (`session_abc...`) or by your own externalId (server disambiguates by prefix). - `GET /api/v1/sessions` — list with filters (`type`, `tag`, `taskIdentifier`, `externalId`, derived `status` ACTIVE/CLOSED/EXPIRED, created-at range) and cursor pagination. Backed by ClickHouse. - `PATCH /api/v1/sessions/:session` — update tags / metadata / externalId. - `POST /api/v1/sessions/:session/close` — terminate. Idempotent, hard-blocks new server-brokered writes. - `POST /api/v1/sessions/:session/end-and-continue` — agent-only handoff to a fresh run. ### Realtime - `PUT /realtime/v1/sessions/:session/:io` — initialize a channel. Returns S2 credentials in headers so high-throughput clients can write direct to S2. - `GET /realtime/v1/sessions/:session/:io` — SSE subscribe. Supports Last-Event-ID resume and an opt-in `X-Peek-Settled: 1` header that fast-closes the stream when the upstream is already settled (`trigger:turn-complete`), eliminating long-poll wait on reconnect-on-reload paths. - `POST /realtime/v1/sessions/:session/:io/append` — server-side appends. - `POST /api/v1/runs/:runFriendlyId/session-streams/wait` — runs wait on a session stream as a waitpoint, with a race-check to avoid suspending if data already landed. ### Auth scopes `sessions` is a new resource type. `read:sessions:{id}`, `write:sessions:{id}`, `admin:sessions:{id}` flow through the existing JWT validator. Session-scoped public access tokens minted by the server replace browser-held trigger-task tokens for chat-style flows — the browser never sees a run identifier or a run-scoped token in steady state. ## What's coming after this PR - SDK + chat.agent migration: separate branch, separate PR, ships in the next `@trigger.dev/sdk` prerelease alongside this server deploy. Customers using the prerelease `chat.agent` will follow the [upgrade guide](https://github.com/triggerdotdev/trigger.dev/blob/docs/tri-7532-ai-sdk-chat-transport-and-chat-task-system/docs/ai-chat/upgrade-guide.mdx). - Dashboard surfaces: dedicated agent list, agent playground, agent view on the run dashboard. Tracking separately. ## Implementation notes - Postgres `Session` table: scalar scoping columns (`projectId`, `runtimeEnvironmentId`, `environmentType`, `organizationId`) without FKs, matching the January TaskRun FK-removal decision. Point-lookup indexes only — list queries go to ClickHouse. Terminal markers (`closedAt`, `expiresAt`) are write-once. - ClickHouse `sessions_v1`: ReplacingMergeTree, partitioned by month, ordered by `(org_id, project_id, environment_id, created_at, session_id)`. Tags indexed via `tokenbf_v1` skip index. - `SessionsReplicationService`: mirrors `RunsReplicationService` exactly — leader-locked logical replication consumer, `ConcurrentFlushScheduler`, retry with exponential backoff + jitter, identical metric shape. Dedicated slot + publication so the two consume independently. - S2 keys: `sessions/{addressingKey}/{out\|in}`. The existing `runs/{runId}/{streamId}` key format for run-scoped streams is untouched. - Optimistic claim: `ensureRunForSession` triggers a run upfront (cheap to cancel if it loses the race), then attempts an `updateMany` keyed on `currentRunVersion`. Loser cancels its triggered run and reuses the winner's. No DB lock held across the trigger. ### What did NOT change Run-scoped `streams.pipe` / `streams.input` and the existing `/realtime/v1/streams/{runId}/...` routes are unchanged. Sessions are net-new — not a reshaping of the current streams API. ## Deploy notes - Set `SESSION_REPLICATION_CLICKHOUSE_URL` and `SESSION_REPLICATION_ENABLED=1` to enable the replication consumer. - The `Session` table needs `REPLICA IDENTITY FULL` set on the prod source DB before the publication is created (same one-time DDL we did for `TaskRun`). Required for delete events to carry full column values. - Cross-form authorization on the `GET /api/v1/sessions/:session` loader (a JWT minted for either form authorizes both URL forms). Action routes are URL-form-specific, matching how the SDK mints PATs. ## Verification - Webapp typecheck clean (10/10). - `apps/webapp/test/sessionsReplicationService.test.ts` — round-trip tests for insert/update/delete through Postgres logical replication into ClickHouse via testcontainers. - Live end-to-end against local dev: create + retrieve (both forms) + update + close, `.out.initialize` + `.out.append` x2 + `.in.send` + `.out.subscribe` over SSE, list with all filter combinations + pagination, `end-and-continue` swap, `X-Peek-Settled` fast-close (verified in browser via reconnect-on-reload and via curl). Replicated row lands in ClickHouse within ~1s. - Multi-round Devin + CodeRabbit review feedback addressed (read-after-write paths use `prisma` writer, info-leak on auth-routes masked as 403, peek-settled discriminator parsing fix, etc.). ## Test plan - [ ] `pnpm run typecheck --filter webapp` - [ ] `pnpm run test --filter webapp ./test/sessionsReplicationService.test.ts --run` - [ ] Start the webapp with `SESSION_REPLICATION_CLICKHOUSE_URL` and `SESSION_REPLICATION_ENABLED=1`. Confirm the slot and publication auto-create on boot. - [ ] `POST /api/v1/sessions` and verify the row replicates to `trigger_dev.sessions_v1` within a couple of seconds. - [ ] `POST /api/v1/sessions/:id/close`, then confirm `POST /realtime/v1/sessions/:id/out/append` returns 400. - [ ] Reuse a closed session's `externalId` on `POST /api/v1/sessions` and confirm 409. - [ ] `GET /realtime/v1/sessions/:id/out` with `X-Peek-Settled: 1` after a turn completes and confirm `X-Session-Settled: true` response header + immediate close.	2026-04-28 12:35:55 +01:00
devin-ai-integration[bot]	4b28080ed4	feat: add `isReplay` to run context (#3454 ) ## Summary Adds `isReplay` boolean to the run context (`ctx.run.isReplay`), following the same pattern as the existing `isTest`. The value is derived from the existing `replayedFromTaskRunFriendlyId` database field, so no schema migration is needed. ## ✅ Checklist - [x] I have followed every step in the [contributing guide](https://github.com/triggerdotdev/trigger.dev/blob/main/CONTRIBUTING.md) - [x] The PR title follows the convention. - [x] I ran and tested the code works --- ## Testing - Verified `@trigger.dev/core` builds successfully - Verified `webapp` typechecks successfully - All new fields use `default(false)` for backwards compatibility --- ## Changelog - Added `isReplay` to `TaskRun` and `V3TaskRun` schemas in `common.ts` - Added `RUN_IS_REPLAY` semantic attribute and wired it in `taskContext` - Propagated `isReplay` through the dequeue system, run attempt system, and all execution context construction paths (V1 + V2) - Added `isReplay` to `DequeuedMessage` and `TaskRunExecutionLazyAttemptPayload` schemas - Added patch changeset for `@trigger.dev/core` - Updated docs: added `isReplay` to context reference, added "Detecting replays" section to replaying page --- 💯 Link to Devin session: https://app.devin.ai/sessions/1d6f1b3cc39a4623b72d05bf00f2d70c --------- Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: nick <55853254+nicktrn@users.noreply.github.com>	2026-04-28 11:57:44 +02:00
Eric Allam	69acdc2b32	fix(core): truncate large error stacks and messages to prevent OOM (#3405 ) ## Summary Large error stacks and messages can OOM the worker process when serialized into OTel spans or `TaskRunError` objects. This was reported when throwing an error with a massive `.stack` property from a chat agent hook. This adds frame-based stack truncation (similar to Sentry's approach) plus message length limits, applied consistently across all error serialization paths. ### What changed `packages/core/src/v3/errors.ts` - `truncateStack()` — parses `error.stack` into message lines + frame lines, caps at 50 frames (keep top 5 closest to throw + bottom 45 entry points, with "... N frames omitted ..." in between). Individual lines capped at 1024 chars. - `truncateMessage()` — caps error messages at 1000 chars - Applied in `parseError()` and `sanitizeError()` `packages/core/src/v3/otel/utils.ts` - `sanitizeSpanError()` now uses `truncateStack` and `truncateMessage` from `errors.ts` instead of duplicating truncation logic - Non-Error values (strings, JSON) capped at 5000 chars `packages/core/src/v3/tracer.ts` - `startActiveSpan` catch block now delegates to `recordSpanException()` instead of calling `span.recordException()` directly ### Limits \| What \| Limit \| Rationale \| \|------\|-------\|-----------\| \| Stack frames \| 50 \| Matches Sentry's `STACKTRACE_FRAME_LIMIT` \| \| Top frames kept \| 5 \| Closest to throw site \| \| Bottom frames kept \| 45 \| Entry points / framework frames \| \| Per-line length \| 1024 \| Matches Sentry, prevents regex DoS \| \| Message length \| 1000 \| Bounded but generous \| \| Generic string (non-Error) \| 5000 \| Fallback for JSON/string errors in spans \| ## Test plan - [x] 17 unit tests in `packages/core/test/errors.test.ts` - [x] E2E: threw a 300-frame / 5000-char-message error in the ai-chat reference app, verified truncated stack and message in span via `get_span_details` - [x] Verified the run survived the error (no OOM, continued waiting for next message)	2026-04-17 13:25:01 +01:00
github-actions[bot]	5ea36e08f2	chore: release v4.4.4 (#3228 ) 🚀 Publish Trigger.dev Docker / typecheck (push) Failing after 0s Details 🚀 Publish Trigger.dev Docker / publish-webapp (push) Has been skipped Details 🚀 Publish Trigger.dev Docker / publish-worker (push) Has been skipped Details 🚀 Publish Trigger.dev Docker / publish-worker-v4 (push) Has been skipped Details 🚀 Publish Trigger.dev Docker / units (push) Failing after 0s Details ## Summary 12 new features, 59 improvements, 17 bug fixes. ## Highlights - Add support for setting TTL (time-to-live) defaults at the task level and globally in trigger.config.ts, with per-trigger overrides still taking precedence ([#3196](https://github.com/triggerdotdev/trigger.dev/pull/3196)) - Large run outputs can use the new API which allows switching object storage providers. ([#3275](https://github.com/triggerdotdev/trigger.dev/pull/3275)) ## Improvements - Add platform notifications support to the CLI. The `trigger dev` and `trigger login` commands now fetch and display platform notifications (info, warn, error, success) from the server. Includes discovery-based filtering to conditionally show notifications based on project file patterns, color markup rendering for styled terminal output, and a non-blocking display flow with a spinner fallback for slow fetches. Use `--skip-platform-notifications` flag with `trigger dev` to disable the notification check. ([#3254](https://github.com/triggerdotdev/trigger.dev/pull/3254)) - Add `get_span_details` MCP tool for inspecting individual spans within a run trace. ([#3255](https://github.com/triggerdotdev/trigger.dev/pull/3255)) - New `get_span_details` tool returns full span attributes, timing, events, and AI enrichment (model, tokens, cost, speed) - Span IDs now shown in `get_run_details` trace output for easy discovery - New API endpoint `GET /api/v1/runs/:runId/spans/:spanId` - New `retrieveSpan()` method on the API client - `get_query_schema` — discover available TRQL tables and columns - `query` — execute TRQL queries against your data - `list_dashboards` — list built-in dashboards and their widgets - `run_dashboard_query` — execute a single dashboard widget query - `whoami` — show current profile, user, and API URL - `list_profiles` — list all configured CLI profiles - `switch_profile` — switch active profile for the MCP session - `start_dev_server` — start `trigger dev` in the background and stream output - `stop_dev_server` — stop the running dev server - `dev_server_status` — check dev server status and view recent logs - `GET /api/v1/query/schema` — query table schema discovery - `GET /api/v1/query/dashboards` — list built-in dashboards - `--readonly` flag hides write tools (`deploy`, `trigger_task`, `cancel_run`) so the AI cannot make changes - `read:query` JWT scope for query endpoint authorization - `get_run_details` trace output is now paginated with cursor support - MCP tool annotations (`readOnlyHint`, `destructiveHint`) for all tools - `get_query_schema` now requires a table name and returns only one table's schema (was returning all tables) - `get_current_worker` no longer inlines payload schemas; use new `get_task_schema` tool instead - Query results formatted as text tables instead of JSON (~50% fewer tokens) - `cancel_run`, `list_deploys`, `list_preview_branches` formatted as text instead of raw JSON - Schema and dashboard API responses cached to avoid redundant fetches - Adapted the CLI API client to propagate the trigger source via http headers. ([#3241](https://github.com/triggerdotdev/trigger.dev/pull/3241)) - Propagate run tags to span attributes so they can be extracted server-side for LLM cost attribution metadata. ([#3213](https://github.com/triggerdotdev/trigger.dev/pull/3213)) - New `get_span_details` tool returns full span attributes, timing, events, and AI enrichment (model, tokens, cost, speed) - Span IDs now shown in `get_run_details` trace output for easy discovery - New API endpoint `GET /api/v1/runs/:runId/spans/:spanId` - New `retrieveSpan()` method on the API client - `get_query_schema` — discover available TRQL tables and columns - `query` — execute TRQL queries against your data - `list_dashboards` — list built-in dashboards and their widgets - `run_dashboard_query` — execute a single dashboard widget query - `whoami` — show current profile, user, and API URL - `list_profiles` — list all configured CLI profiles - `switch_profile` — switch active profile for the MCP session - `start_dev_server` — start `trigger dev` in the background and stream output - `stop_dev_server` — stop the running dev server - `dev_server_status` — check dev server status and view recent logs - `GET /api/v1/query/schema` — query table schema discovery - `GET /api/v1/query/dashboards` — list built-in dashboards - `--readonly` flag hides write tools (`deploy`, `trigger_task`, `cancel_run`) so the AI cannot make changes - `read:query` JWT scope for query endpoint authorization - `get_run_details` trace output is now paginated with cursor support - MCP tool annotations (`readOnlyHint`, `destructiveHint`) for all tools - `get_query_schema` now requires a table name and returns only one table's schema (was returning all tables) - `get_current_worker` no longer inlines payload schemas; use new `get_task_schema` tool instead - Query results formatted as text tables instead of JSON (~50% fewer tokens) - `cancel_run`, `list_deploys`, `list_preview_branches` formatted as text instead of raw JSON - Schema and dashboard API responses cached to avoid redundant fetches - Add optional `hasPrivateLink` field to the dequeue message organization object for private networking support ([#3264](https://github.com/triggerdotdev/trigger.dev/pull/3264)) - Define and manage AI prompts with `prompts.define()`. Create typesafe prompt templates with variables, resolve them at runtime, and manage versions and overrides from the dashboard without redeploying. ([#3244](https://github.com/triggerdotdev/trigger.dev/pull/3244)) ## Bug fixes - Fix dev CLI leaking build directories on rebuild, causing disk space accumulation. Deprecated workers are now pruned (capped at 2 retained) when no active runs reference them. The watchdog process also cleans up `.trigger/tmp/` when the dev CLI is killed ungracefully (e.g. SIGKILL from pnpm). ([#3224](https://github.com/triggerdotdev/trigger.dev/pull/3224)) - Fix `--load` flag being silently ignored on local/self-hosted builds. ([#3114](https://github.com/triggerdotdev/trigger.dev/pull/3114)) - Fixed `search_docs` tool failing due to renamed upstream Mintlify tool (`SearchTriggerDev` → `search_trigger_dev`) - Fixed `list_deploys` failing when deployments have null `runtime`/`runtimeVersion` fields (#3139) - Fixed `list_preview_branches` crashing due to incorrect response shape access - Fixed `metrics` table column documented as `value` instead of `metric_value` in query docs - Fixed dev CLI leaking build directories on rebuild — deprecated workers now clean up their build dirs when their last run completes - Fixed `search_docs` tool failing due to renamed upstream Mintlify tool (`SearchTriggerDev` → `search_trigger_dev`) - Fixed `list_deploys` failing when deployments have null `runtime`/`runtimeVersion` fields (#3139) - Fixed `list_preview_branches` crashing due to incorrect response shape access - Fixed `metrics` table column documented as `value` instead of `metric_value` in query docs - Fixed dev CLI leaking build directories on rebuild — deprecated workers now clean up their build dirs when their last run completes ## Server changes These changes affect the self-hosted Docker image and Trigger.dev Cloud: - Add admin UI for viewing and editing feature flags (org-level overrides and global defaults). ([#3291](https://github.com/triggerdotdev/trigger.dev/pull/3291)) - AI prompt management dashboard and enhanced span inspectors. Prompt management: - Prompts list page with version status, model, override indicators, and 24h usage sparklines - Prompt detail page with template viewer, variable preview, version history timeline, and override editor - Create, edit, and remove overrides to change prompt content or model without redeploying - Promote any code-deployed version to current - Generations tab with infinite scroll, live polling, and inline span inspector - Per-prompt metrics: total generations, avg tokens, avg cost, latency, with version-level breakdowns AI span inspectors: - Custom inspectors for `ai.generateText`, `ai.streamText`, `ai.generateObject`, `ai.streamObject` parent spans - `ai.toolCall` inspector showing tool name, call ID, and input arguments - `ai.embed` inspector showing model, provider, and input text - Prompt tab on AI spans linking to prompt version with template and input variables - Compact timestamp and duration header on all AI span inspectors AI metrics dashboard: - Operations, Providers, and Prompts filters on the AI Metrics dashboard - Cost by prompt widget - "AI" section in the sidebar with Prompts and AI Metrics links Other improvements: - Resizable panel sizes now persist across page refreshes - Fixed `<div>` inside `<p>` DOM nesting warnings in span titles and chat messages ([#3244](https://github.com/triggerdotdev/trigger.dev/pull/3244)) - Add allowRollbacks query param to the promote deployment API to enable version downgrades ([#3214](https://github.com/triggerdotdev/trigger.dev/pull/3214)) - Pre-warm compute templates on deploy for orgs with compute access. Required for projects using a compute region, background-only for others. ([#3114](https://github.com/triggerdotdev/trigger.dev/pull/3114)) - Add automatic LLM cost calculation for spans with GenAI semantic conventions. When a span arrives with `gen_ai.response.model` and token usage data, costs are calculated from an in-memory pricing registry backed by Postgres and dual-written to both span attributes (`trigger.llm.`) and a new `llm_metrics_v1` ClickHouse table that captures usage, cost, performance (TTFC, tokens/sec), and behavioral (finish reason, operation type) metrics. ([#3213](https://github.com/triggerdotdev/trigger.dev/pull/3213)) - Add API endpoint `GET /api/v1/runs/:runId/spans/:spanId` that returns detailed span information including properties, events, AI enrichment (model, tokens, cost), and triggered child runs. ([#3255](https://github.com/triggerdotdev/trigger.dev/pull/3255)) - Multi-provider object storage with protocol-based routing for zero-downtime migration ([#3275](https://github.com/triggerdotdev/trigger.dev/pull/3275)) - Add IAM role-based auth support for object stores (no access keys required). ([#3275](https://github.com/triggerdotdev/trigger.dev/pull/3275)) - Add platform notifications to inform users about new features, changelogs, and platform events directly in the dashboard. ([#3254](https://github.com/triggerdotdev/trigger.dev/pull/3254)) - Add private networking support via AWS PrivateLink. Includes BillingClient methods for managing private connections, org settings UI pages for connection management, and supervisor changes to apply `privatelink` pod labels for CiliumNetworkPolicy matching. ([#3264](https://github.com/triggerdotdev/trigger.dev/pull/3264)) - Reduce run start latency by skipping the intermediate queue when concurrency is available. This optimization is rolled out per-region and enabled automatically for development environments. ([#3299](https://github.com/triggerdotdev/trigger.dev/pull/3299)) - Extended the search filter on the environment variables page to match on environment type (production, staging, development, preview) and branch name, not just variable name and value. ([#3302](https://github.com/triggerdotdev/trigger.dev/pull/3302)) - Set `application_name` on Prisma connections from SERVICE_NAME so DB load can be attributed by service ([#3348](https://github.com/triggerdotdev/trigger.dev/pull/3348)) - Fix transient R2/object store upload failures during batchTrigger() item streaming. - Added p-retry (3 attempts, 500ms–2s exponential backoff) around `uploadPacketToObjectStore` in `BatchPayloadProcessor.process()` so transient network errors self-heal server-side rather than aborting the entire batch stream. - Removed `x-should-retry: false` from the 500 response on the batch items route so the SDK's existing 5xx retry path can recover if server-side retries are exhausted. Item deduplication by index makes full-stream retries safe. ([#3331](https://github.com/triggerdotdev/trigger.dev/pull/3331)) - Concurrency-keyed queues now use a single master queue entry per base queue instead of one entry per key. Prevents high-CK-count tenants from consuming the entire parentQueueLimit window and starving other tenants on the same shard. ([#3219](https://github.com/triggerdotdev/trigger.dev/pull/3219)) - Reduce lock contention when processing large `batchTriggerAndWait` batches. Previously, each batch item acquired a Redis lock on the parent run to insert a `TaskRunWaitpoint` row, causing `LockAcquisitionTimeoutError` with high concurrency (880 errors/24h in prod). Since `blockRunWithCreatedBatch` already transitions the parent to `EXECUTING_WITH_WAITPOINTS` before items are processed, the per-item lock is unnecessary. The new `blockRunWithWaitpointLockless` method performs only the idempotent CTE insert without acquiring the lock. ([#3232](https://github.com/triggerdotdev/trigger.dev/pull/3232)) - Strip `secure` query parameter from QUERY_CLICKHOUSE_URL before passing to ClickHouse client. This was already done for the main and logs ClickHouse clients but was missing for the query client, causing a startup crash with `Error: Unknown URL parameters: secure`. ([#3204](https://github.com/triggerdotdev/trigger.dev/pull/3204)) - Fix `OrganizationsPresenter.#getEnvironment` matching the wrong development environment on teams with multiple members. All dev environments share the slug `"dev"`, so the previous `find` by slug alone could return another member's environment. Now filters DEVELOPMENT environments by `orgMember.userId` to ensure the logged-in user's dev environment is selected. ([#3273](https://github.com/triggerdotdev/trigger.dev/pull/3273)) <details> <summary>Raw changeset output</summary> # Releases ## @trigger.dev/build@4.4.4 ### Patch Changes - Updated dependencies: - `@trigger.dev/core@4.4.4` ## trigger.dev@4.4.4 ### Patch Changes - Add platform notifications support to the CLI. The `trigger dev` and `trigger login` commands now fetch and display platform notifications (info, warn, error, success) from the server. Includes discovery-based filtering to conditionally show notifications based on project file patterns, color markup rendering for styled terminal output, and a non-blocking display flow with a spinner fallback for slow fetches. Use `--skip-platform-notifications` flag with `trigger dev` to disable the notification check. ([#3254](https://github.com/triggerdotdev/trigger.dev/pull/3254)) - Fix dev CLI leaking build directories on rebuild, causing disk space accumulation. Deprecated workers are now pruned (capped at 2 retained) when no active runs reference them. The watchdog process also cleans up `.trigger/tmp/` when the dev CLI is killed ungracefully (e.g. SIGKILL from pnpm). ([#3224](https://github.com/triggerdotdev/trigger.dev/pull/3224)) - Fix `--load` flag being silently ignored on local/self-hosted builds. ([#3114](https://github.com/triggerdotdev/trigger.dev/pull/3114)) - Add `get_span_details` MCP tool for inspecting individual spans within a run trace. ([#3255](https://github.com/triggerdotdev/trigger.dev/pull/3255)) - New `get_span_details` tool returns full span attributes, timing, events, and AI enrichment (model, tokens, cost, speed) - Span IDs now shown in `get_run_details` trace output for easy discovery - New API endpoint `GET /api/v1/runs/:runId/spans/:spanId` - New `retrieveSpan()` method on the API client - MCP server improvements: new tools, bug fixes, and new flags. ([#3224](https://github.com/triggerdotdev/trigger.dev/pull/3224)) New tools:* - `get_query_schema` — discover available TRQL tables and columns - `query` — execute TRQL queries against your data - `list_dashboards` — list built-in dashboards and their widgets - `run_dashboard_query` — execute a single dashboard widget query - `whoami` — show current profile, user, and API URL - `list_profiles` — list all configured CLI profiles - `switch_profile` — switch active profile for the MCP session - `start_dev_server` — start `trigger dev` in the background and stream output - `stop_dev_server` — stop the running dev server - `dev_server_status` — check dev server status and view recent logs New API endpoints: - `GET /api/v1/query/schema` — query table schema discovery - `GET /api/v1/query/dashboards` — list built-in dashboards New features: - `--readonly` flag hides write tools (`deploy`, `trigger_task`, `cancel_run`) so the AI cannot make changes - `read:query` JWT scope for query endpoint authorization - `get_run_details` trace output is now paginated with cursor support - MCP tool annotations (`readOnlyHint`, `destructiveHint`) for all tools Bug fixes: - Fixed `search_docs` tool failing due to renamed upstream Mintlify tool (`SearchTriggerDev` → `search_trigger_dev`) - Fixed `list_deploys` failing when deployments have null `runtime`/`runtimeVersion` fields (#3139) - Fixed `list_preview_branches` crashing due to incorrect response shape access - Fixed `metrics` table column documented as `value` instead of `metric_value` in query docs - Fixed dev CLI leaking build directories on rebuild — deprecated workers now clean up their build dirs when their last run completes Context optimizations: - `get_query_schema` now requires a table name and returns only one table's schema (was returning all tables) - `get_current_worker` no longer inlines payload schemas; use new `get_task_schema` tool instead - Query results formatted as text tables instead of JSON (~50% fewer tokens) - `cancel_run`, `list_deploys`, `list_preview_branches` formatted as text instead of raw JSON - Schema and dashboard API responses cached to avoid redundant fetches - Add support for setting TTL (time-to-live) defaults at the task level and globally in trigger.config.ts, with per-trigger overrides still taking precedence ([#3196](https://github.com/triggerdotdev/trigger.dev/pull/3196)) - Adapted the CLI API client to propagate the trigger source via http headers. ([#3241](https://github.com/triggerdotdev/trigger.dev/pull/3241)) - Updated dependencies: - `@trigger.dev/core@4.4.4` - `@trigger.dev/build@4.4.4` - `@trigger.dev/schema-to-json@4.4.4` ## @trigger.dev/core@4.4.4 ### Patch Changes - Fix `list_deploys` MCP tool failing when deployments have null `runtime` or `runtimeVersion` fields. ([#3224](https://github.com/triggerdotdev/trigger.dev/pull/3224)) - Propagate run tags to span attributes so they can be extracted server-side for LLM cost attribution metadata. ([#3213](https://github.com/triggerdotdev/trigger.dev/pull/3213)) - Add `get_span_details` MCP tool for inspecting individual spans within a run trace. ([#3255](https://github.com/triggerdotdev/trigger.dev/pull/3255)) - New `get_span_details` tool returns full span attributes, timing, events, and AI enrichment (model, tokens, cost, speed) - Span IDs now shown in `get_run_details` trace output for easy discovery - New API endpoint `GET /api/v1/runs/:runId/spans/:spanId` - New `retrieveSpan()` method on the API client - MCP server improvements: new tools, bug fixes, and new flags. ([#3224](https://github.com/triggerdotdev/trigger.dev/pull/3224)) New tools: - `get_query_schema` — discover available TRQL tables and columns - `query` — execute TRQL queries against your data - `list_dashboards` — list built-in dashboards and their widgets - `run_dashboard_query` — execute a single dashboard widget query - `whoami` — show current profile, user, and API URL - `list_profiles` — list all configured CLI profiles - `switch_profile` — switch active profile for the MCP session - `start_dev_server` — start `trigger dev` in the background and stream output - `stop_dev_server` — stop the running dev server - `dev_server_status` — check dev server status and view recent logs New API endpoints: - `GET /api/v1/query/schema` — query table schema discovery - `GET /api/v1/query/dashboards` — list built-in dashboards New features: - `--readonly` flag hides write tools (`deploy`, `trigger_task`, `cancel_run`) so the AI cannot make changes - `read:query` JWT scope for query endpoint authorization - `get_run_details` trace output is now paginated with cursor support - MCP tool annotations (`readOnlyHint`, `destructiveHint`) for all tools Bug fixes: - Fixed `search_docs` tool failing due to renamed upstream Mintlify tool (`SearchTriggerDev` → `search_trigger_dev`) - Fixed `list_deploys` failing when deployments have null `runtime`/`runtimeVersion` fields (#3139) - Fixed `list_preview_branches` crashing due to incorrect response shape access - Fixed `metrics` table column documented as `value` instead of `metric_value` in query docs - Fixed dev CLI leaking build directories on rebuild — deprecated workers now clean up their build dirs when their last run completes Context optimizations: - `get_query_schema` now requires a table name and returns only one table's schema (was returning all tables) - `get_current_worker` no longer inlines payload schemas; use new `get_task_schema` tool instead - Query results formatted as text tables instead of JSON (~50% fewer tokens) - `cancel_run`, `list_deploys`, `list_preview_branches` formatted as text instead of raw JSON - Schema and dashboard API responses cached to avoid redundant fetches - Large run outputs can use the new API which allows switching object storage providers. ([#3275](https://github.com/triggerdotdev/trigger.dev/pull/3275)) - Add optional `hasPrivateLink` field to the dequeue message organization object for private networking support ([#3264](https://github.com/triggerdotdev/trigger.dev/pull/3264)) - Add support for setting TTL (time-to-live) defaults at the task level and globally in trigger.config.ts, with per-trigger overrides still taking precedence ([#3196](https://github.com/triggerdotdev/trigger.dev/pull/3196)) - Adapted the CLI API client to propagate the trigger source via http headers. ([#3241](https://github.com/triggerdotdev/trigger.dev/pull/3241)) ## @trigger.dev/python@4.4.4 ### Patch Changes - Updated dependencies: - `@trigger.dev/sdk@4.4.4` - `@trigger.dev/core@4.4.4` - `@trigger.dev/build@4.4.4` ## @trigger.dev/react-hooks@4.4.4 ### Patch Changes - Updated dependencies: - `@trigger.dev/core@4.4.4` ## @trigger.dev/redis-worker@4.4.4 ### Patch Changes - Adapted the CLI API client to propagate the trigger source via http headers. ([#3241](https://github.com/triggerdotdev/trigger.dev/pull/3241)) - Updated dependencies: - `@trigger.dev/core@4.4.4` ## @trigger.dev/rsc@4.4.4 ### Patch Changes - Updated dependencies: - `@trigger.dev/core@4.4.4` ## @trigger.dev/schema-to-json@4.4.4 ### Patch Changes - Updated dependencies: - `@trigger.dev/core@4.4.4` ## @trigger.dev/sdk@4.4.4 ### Patch Changes - Define and manage AI prompts with `prompts.define()`. Create typesafe prompt templates with variables, resolve them at runtime, and manage versions and overrides from the dashboard without redeploying. ([#3244](https://github.com/triggerdotdev/trigger.dev/pull/3244)) - Add support for setting TTL (time-to-live) defaults at the task level and globally in trigger.config.ts, with per-trigger overrides still taking precedence ([#3196](https://github.com/triggerdotdev/trigger.dev/pull/3196)) - Adapted the CLI API client to propagate the trigger source via http headers. ([#3241](https://github.com/triggerdotdev/trigger.dev/pull/3241)) - Updated dependencies: - `@trigger.dev/core@4.4.4` </details> --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>	2026-04-13 13:24:38 +01:00
Matt Aitken	68e88d0d71	Object Storage seamless migration (#3275 ) This allows seamless migration to different object storage. Existing runs that have offloaded payloads/outputs will continue to use the default object store (configured using `OBJECT_STORE_*` env vars). You can add additional stores by setting new env vars: - `OBJECT_STORE_DEFAULT_PROTOCOL` this determines where new run large payloads will get stored. - If you set that you need to set new env vars for that protocol. Example: ``` OBJECT_STORE_DEFAULT_PROTOCOL=“s3" OBJECT_STORE_S3_BASE_URL=https://s3.us-east-1.amazonaws.com OBJECT_STORE_S3_ACCESS_KEY_ID=<val> OBJECT_STORE_S3_SECRET_ACCESS_KEY=<val> OBJECT_STORE_S3_REGION=us-east-1 OBJECT_STORE_S3_SERVICE=s3 ``` --------- Co-authored-by: nicktrn <55853254+nicktrn@users.noreply.github.com>	2026-04-01 10:06:12 +01:00
nicktrn	0e63f8317e	feat: add ttl support at task and config levels (#3196 ) Add TTL (time-to-live) defaults at task-level and config-level, with precedence: per-trigger > task > config > dev default (10m). Docs PR: #3200 (merge after packages are released)	2026-03-30 23:25:07 +01:00
nicktrn	9cb3dcb07c	feat(supervisor): compute workload manager (#3114 ) Adds the `ComputeWorkloadManager` for routing task execution through the compute gateway, including full checkpoint/restore support, OTel trace integration, and template pre-warming. ## Changes Compute workload manager (`apps/supervisor/src/workloadManager/compute.ts`) - Routes instance create, snapshot, delete, and restore through the compute gateway API - Wide event logging on create with full timing and context - Configurable gateway timeout, auth token, image digest stripping Compute snapshot service (`apps/supervisor/src/services/computeSnapshotService.ts`) - Timer wheel for delayed snapshot dispatch (avoids wasted work on short-lived waitpoints) - Configurable dispatch concurrency limit (`COMPUTE_SNAPSHOT_DISPATCH_LIMIT`) - Snapshot-complete callback handler with suspend completion reporting - Trace context management and OTel span emission for snapshot operations OTel trace service (`apps/supervisor/src/services/otlpTraceService.ts`) - Fire-and-forget OTLP span emission for compute operations (provision, restore, snapshot) - BigInt nanosecond conversion preserving sub-ms precision for span ordering Template creation (`apps/webapp/app/v3/services/computeTemplateCreation.server.ts`) - Three-mode rollout: required (MICROVM projects), shadow (feature flag / percentage), skip - Integrated into deploy finalize flow Shared compute package (`internal-packages/compute/`) - Gateway client with namespace-based API (instances, templates, snapshots) - Zod schemas for all gateway request/response types Database - `COMPUTE` variant added to `TaskRunCheckpointType` enum - `WorkloadType` enum and column on `WorkerInstanceGroup` - `hasComputeAccess` feature flag Env / config - Compute gateway URL, auth token, timeout - Snapshot enable flag, delay, dispatch limit - Dedicated OTLP endpoint for compute spans (`COMPUTE_TRACE_OTLP_ENDPOINT`)	2026-03-29 22:03:59 +01:00
Oskar Otwinowski	efe24f9c2a	feat(private-link): Add private links UI (#3264 ) ⚒️ Publish Worker (v4) / build (supervisor) (push) Has been cancelled Details	2026-03-27 15:34:39 +01:00
Oskar Otwinowski	8244ac6f84	feat(Notifications): Platform notifications for CLI and Dashboard (#3254 ) For human reviewer: - Check if Redis connection + code makes sense - Check CLI methods (it's on a hotpath) - Check DB Migrations and new tables ## ✅ Checklist - [x] I have followed every step in the [contributing guide](https://github.com/triggerdotdev/trigger.dev/blob/main/CONTRIBUTING.md) - [x] The PR title follows the convention. - [x] I ran and tested the code works --- ## Testing Spawning new CLI / Dashboard notifications, check MVP, check if failures not produce any problems with CLI/Dashboard --- ## Changelog Added notifications mechanism for Dashboard and CLI --- ## Screenshots 💯	2026-03-26 14:19:17 +01:00
Eric Allam	c00dae006b	feat(mcp): add get_span_details tool (#3255 ) Returns the fully detailed span with attributes and AI enrichment data	2026-03-24 13:18:49 +00:00
Saadi Myftija	d4772b5f60	feat: run annotations (#3241 ) Adds an `annotations` JSONB column to task runs that captures where and how each run was triggered. This enables filtering and analyzing trigger origins without querying up the run tree. Also enables making scheduling decisions based on the trigger source, e.g., use separate affinities for scheduled runs. Each run records: - triggerSource: who initiated it (sdk, api, dashboard, cli, mcp, schedule) - triggerAction: what kind of action (trigger, replay, test) - rootTriggerSource: the trigger source of the root ancestor, propagated through the entire run tree - rootScheduleId: schedule id, in case the run tree was triggered from a schedule Currently the main motivation for annotations it to determine whether a run is part of a schedule-originated tree without traversing ancestors. ### A couple of design considerations - Decoupled source from method: triggerSource and triggerAction are separate fields to avoid combinatorial explosion (every new source × every new action) - Server-side first: all annotation values are primarily determined on the server, only a minor SDK change needed - Forward-compatible: annotation fields use `z.enum([...]).or(anyString)` so new values can be added without breaking validation; we currently don't need an explicit version field for annotations. Note: `metadata` would have been a more fitting name for the db column, as it is consistent with other tables where we store this type of information. It is already in use to store user metadata though, so we go with `annotations` instead.	2026-03-23 16:07:30 +01:00
Eric Allam	54d95ee4b9	feat: AI prompt management dashboard and enhanced span inspectors (#3244 ) - Full prompt management UI: list, detail, override, and version management for AI prompts defined with `prompts.define()` - Rich AI span inspectors for all AI SDK operations with token usage, messages, and prompt context - Real-time generation tracking with live polling and filtering ## Prompt management Define prompts in your code with `prompts.define()`, then manage versions and overrides from the dashboard without redeploying: ```typescript import { task, prompts } from "@trigger.dev/sdk"; import { generateText } from "ai"; import { openai } from "@ai-sdk/openai"; import { z } from "zod"; const supportPrompt = prompts.define({ id: "customer-support", model: "gpt-4o", variables: z.object({ customerName: z.string(), plan: z.string(), issue: z.string(), }), content: `You are a support agent for Acme SaaS. Customer: {{customerName}} ({{plan}} plan) Issue: {{issue}} Respond with empathy and precision.`, }); export const supportTask = task({ id: "handle-support", run: async (payload) => { const resolved = await supportPrompt.resolve({ customerName: payload.name, plan: payload.plan, issue: payload.issue, }); const result = await generateText({ model: openai(resolved.model ?? "gpt-4o"), system: resolved.text, prompt: payload.issue, ...resolved.toAISDKTelemetry(), }); return { response: result.text }; }, }); ``` The prompts list page shows each prompt with its current version, model, override status, and a usage sparkline over the last 24 hours. From the prompt detail page you can: - Create overrides to change the prompt template or model without redeploying. Overrides take priority over the deployed version when `prompt.resolve()` is called. - Promote any code-deployed version to be the current version - Browse generations across all versions with infinite scroll and live polling for new results - Filter by version, model, operation type, and provider - View metrics (total generations, avg tokens, avg cost, latency) broken down by version ## AI span inspectors Every AI SDK operation now gets a custom inspector in the run trace view: - `ai.generateText` / `ai.streamText` — Shows model, token usage, cost, the full message thread (system prompt, user message, assistant response), and linked prompt details - `ai.generateObject` / `ai.streamObject` — Same as above plus the JSON schema and structured output - `ai.toolCall` — Shows tool name, call ID, and input arguments - `ai.embed` — Shows model and the text being embedded For generation spans linked to a prompt, a "Prompt" tab shows the prompt metadata, the input variables passed to `resolve()`, and the template content from the prompt version. All AI span inspectors include a compact timestamp and duration header. ## Other improvements - Resizable panel sizes now persist across page refreshes (patched `@window-splitter/state` to fix snapshot restoration) - Run page panels also persist their sizes - Fixed `<div>` inside `<p>` DOM nesting warnings in span titles and chat messages - Added Operations and Providers filters to the AI metrics dashboard ## Screenshots <img width="3680" height="2392" alt="CleanShot 2026-03-21 at 10 14 17@2x" src="https://github.com/user-attachments/assets/f3e59989-a2fa-4990-a9d0-3cacda431868" /> <img width="3680" height="2392" alt="CleanShot 2026-03-21 at 10 15 37@2x" src="https://github.com/user-attachments/assets/2f2d02df-2d2b-44fb-ac6f-9153f6a6c387" /> <img width="3680" height="2392" alt="CleanShot 2026-03-21 at 10 15 54@2x" src="https://github.com/user-attachments/assets/baa161e0-ef91-4fa4-a55f-986b71cccdf0" />	2026-03-23 06:23:19 +00:00
Eric Allam	1cfc296c6b	feat(ai): LLM metrics tracking and AI span inspector (#3213 ) - Automatic LLM cost enrichment for AI SDK spans (streamText, generateText, generateObject) or any other spans that use semantic gen_ai attributes with support for 145+ models - New AI span inspector sidebar showing model, tokens, cost, messages, tool calls, and response text - LLM metrics dual-write to ClickHouse `llm_metrics_v1` table for analytics - LLM metrics built-in dashboard (unlinked at the moment) - Provider cost fallback — uses gateway/OpenRouter reported costs from `providerMetadata` when registry pricing is unavailable - Prefix-stripping for gateway/OpenRouter model names (e.g. `mistral/mistral-large-3` matches `mistral-large-3` pricing) - Admin dashboard for managing LLM model pricing (list, create, edit, delete, search, test pattern matching) - Missing models detection page — queries ClickHouse for unpriced models with sample spans and Claude Code-ready prompts for adding pricing - AI span seed script (`pnpm run db:seed:ai-spans`) with 51 spans across 12 provider systems for local dev testing - UI fixes: `completionTokens`/`promptTokens` aliases, `ai.response.object` display for generateObject, cache read/write token breakdown ## Screenshots: <img width="1030" height="104" alt="CleanShot 2026-03-17 at 16 48 54@2x" src="https://github.com/user-attachments/assets/bc8fccda-e48b-4d0c-bfb1-e620064e5979" /> <img width="1094" height="1512" alt="CleanShot 2026-03-17 at 16 49 23@2x" src="https://github.com/user-attachments/assets/c2424569-d07e-4d67-a436-e8250043a1ee" /> <img width="1074" height="1412" alt="CleanShot 2026-03-17 at 16 49 18@2x" src="https://github.com/user-attachments/assets/22342ac4-4769-45d1-a328-a24fb9a82a50" /> <img width="1012" height="2292" alt="CleanShot 2026-03-17 at 16 39 01@2x" src="https://github.com/user-attachments/assets/59e327d1-6652-4293-8be0-bb8326e5fbc5" /> <img width="3680" height="2392" alt="CleanShot 2026-03-15 at 08 29 38@2x" src="https://github.com/user-attachments/assets/1f77beb8-de67-495b-b890-bcdb8d7f1fe8" /> --------- Co-authored-by: James Ritchie <james@trigger.dev>	2026-03-17 18:26:43 +00:00
Eric Allam	dbbe9f77f9	feat(cli): Expand and improve the MCP server and dev CLI command (#3224 ) ## Summary Major expansion of the MCP server (14 → 25 tools), context efficiency optimizations, new API endpoints, and a fix for the dev CLI leaking build directories on disk. ### New MCP tools - Query & analytics: `get_query_schema`, `query`, `list_dashboards`, `run_dashboard_query` — query your data using TRQL directly from AI assistants - Profile management: `whoami`, `list_profiles`, `switch_profile` — see and switch CLI profiles per-project (persisted to `.trigger/mcp.json`) - Dev server control: `start_dev_server`, `stop_dev_server`, `dev_server_status` — start/stop `trigger dev` and stream build output - Task introspection: `get_task_schema` — get payload schema for a specific task (split out from `get_current_worker` to reduce context) ### New API endpoints - `GET /api/v1/query/schema` — discover TRQL tables and columns (server-driven, multi-table) - `GET /api/v1/query/dashboards` — list built-in dashboard widgets and their queries ### New features - `--readonly` flag — hides write tools (`deploy`, `trigger_task`, `cancel_run`) so agents can't make changes - `read:query` JWT scope — new authorization scope for query endpoints, with per-table granularity (`read:query:runs`, `read:query:llm_metrics`, etc.) - Paginated trace output — `get_run_details` now paginates trace events via cursor, caching the full trace in a temp file so subsequent pages don't re-fetch - MCP tool annotations — all tools now have `readOnlyHint`/`destructiveHint` annotations for clients that support them - Project-scoped profile persistence — `switch_profile` saves to `.trigger/mcp.json` (gitignored), automatically loaded on next MCP server start ### Context optimizations - `get_query_schema` requires a table name — returns one table's schema instead of all tables (60-80% fewer tokens) - `get_current_worker` no longer inlines payload schemas — use `get_task_schema` for specific tasks - Query results formatted as text tables instead of JSON (~50% fewer tokens for flat data) - `cancel_run`, `list_deploys`, `list_preview_branches` formatted as text instead of raw `JSON.stringify()` - Schema and dashboard API responses cached (1hr and 5min respectively) ### Bug fixes - Fixed `search_docs` failing due to renamed upstream Mintlify tool (`SearchTriggerDev` → `search_trigger_dev`) - Fixed `list_deploys` failing when deployments have null `runtime`/`runtimeVersion` fields (fixes #3139) - Fixed `list_preview_branches` crashing due to incorrect response shape access - Fixed `metrics` table column documented as `value` instead of `metric_value` in query docs - Fixed `/api/v1/query` not accepting JWT auth (added `allowJWT: true`) ### Dev CLI build directory fix The dev CLI was leaking `build-` directories in `.trigger/tmp/` on every rebuild, accumulating hundreds of MB over time (842MB observed). Three layers of protection added: 1. During session: deprecated workers are pruned (capped at 2 retained) when no active runs reference them, preventing unbounded accumulation 2. On SIGKILL/crash: the watchdog process now cleans up `.trigger/tmp/` when it detects the parent CLI was killed 3. On next startup*: existing `clearTmpDirs()` wipes any remaining orphans ## Test plan - [ ] `pnpm run mcp:smoke` — 17 automated smoke tests for all read-only MCP tools - [ ] `pnpm run mcp:test list` — verify 25 tools registered (21 in `--readonly` mode) - [ ] `pnpm run mcp:test --readonly list` — verify write tools hidden - [ ] Manual: start dev server, trigger task, rebuild multiple times, verify build dirs stay capped at 4 - [ ] Manual: SIGKILL the dev CLI, verify watchdog cleans up `.trigger/tmp/` - [ ] Verify new API endpoints return correct data: `GET /api/v1/query/schema`, `GET /api/v1/query/dashboards` 🤖 Generated with [Claude Code](https://claude.com/claude-code)	2026-03-17 11:38:23 +00:00
github-actions[bot]	c0b6309c72	chore: release v4.4.3 (#3182 ) 🚀 Publish Trigger.dev Docker / units (push) Failing after 4s Details 🚀 Publish Trigger.dev Docker / typecheck (push) Failing after 20s Details 🚀 Publish Trigger.dev Docker / publish-webapp (push) Has been skipped Details 🚀 Publish Trigger.dev Docker / publish-worker (push) Has been skipped Details 🚀 Publish Trigger.dev Docker / publish-worker-v4 (push) Has been skipped Details ## Summary 2 new features, 2 improvements. ## Improvements - Add syncSupabaseEnvVars to pull database connection strings and save them as trigger.dev environment variables ([#3152](https://github.com/triggerdotdev/trigger.dev/pull/3152)) - Auto-cancel in-flight dev runs when the CLI exits, using a detached watchdog process that survives pnpm SIGKILL ([#3191](https://github.com/triggerdotdev/trigger.dev/pull/3191)) ## Server changes These changes affect the self-hosted Docker image and Trigger.dev Cloud: - A new Errors page for viewing and tracking errors that cause runs to fail - Errors are grouped using error fingerprinting - View top errors for a time period, filter by task, or search the text - View occurrences over time - View all the runs for an error and bulk replay them ([#3172](https://github.com/triggerdotdev/trigger.dev/pull/3172)) - Add sidebar tabs (Options, AI, Schema) to the Test page for schemaTask payload generation and schema viewing. ([#3188](https://github.com/triggerdotdev/trigger.dev/pull/3188)) <details> <summary>Raw changeset output</summary> # Releases ## @trigger.dev/build@4.4.3 ### Patch Changes - Add syncSupabaseEnvVars to pull database connection strings and save them as trigger.dev environment variables ([#3152](https://github.com/triggerdotdev/trigger.dev/pull/3152)) - Updated dependencies: - `@trigger.dev/core@4.4.3` ## trigger.dev@4.4.3 ### Patch Changes - Auto-cancel in-flight dev runs when the CLI exits, using a detached watchdog process that survives pnpm SIGKILL ([#3191](https://github.com/triggerdotdev/trigger.dev/pull/3191)) - Updated dependencies: - `@trigger.dev/core@4.4.3` - `@trigger.dev/build@4.4.3` - `@trigger.dev/schema-to-json@4.4.3` ## @trigger.dev/core@4.4.3 ### Patch Changes - Auto-cancel in-flight dev runs when the CLI exits, using a detached watchdog process that survives pnpm SIGKILL ([#3191](https://github.com/triggerdotdev/trigger.dev/pull/3191)) ## @trigger.dev/python@4.4.3 ### Patch Changes - Updated dependencies: - `@trigger.dev/core@4.4.3` - `@trigger.dev/build@4.4.3` - `@trigger.dev/sdk@4.4.3` ## @trigger.dev/react-hooks@4.4.3 ### Patch Changes - Updated dependencies: - `@trigger.dev/core@4.4.3` ## @trigger.dev/redis-worker@4.4.3 ### Patch Changes - Updated dependencies: - `@trigger.dev/core@4.4.3` ## @trigger.dev/rsc@4.4.3 ### Patch Changes - Updated dependencies: - `@trigger.dev/core@4.4.3` ## @trigger.dev/schema-to-json@4.4.3 ### Patch Changes - Updated dependencies: - `@trigger.dev/core@4.4.3` ## @trigger.dev/sdk@4.4.3 ### Patch Changes - Updated dependencies: - `@trigger.dev/core@4.4.3` </details> --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>	2026-03-10 10:17:24 +00:00
Eric Allam	436f20efc6	feat(cli): auto-cancel dev runs on CLI exit via detached watchdog (#3191 ) When the dev CLI exits (e.g. ctrl+c via pnpm), runs that were mid-execution previously stayed stuck in EXECUTING status for up to 5 minutes until the heartbeat timeout fired. Now they are cancelled within seconds. The dev CLI spawns a lightweight detached watchdog process at startup. The watchdog monitors the CLI process ID and, when it detects the CLI has exited, calls a new POST /engine/v1/dev/disconnect endpoint to cancel all in-flight runs immediately (skipping PENDING_CANCEL since the worker is known to be dead). Watchdog design: - Fully detached (detached: true, stdio: ignore, unref()) so it survives even when pnpm sends SIGKILL to the process tree - Active run IDs maintained via atomic file write (.trigger/active-runs.json) - Single-instance guarantee via PID file (.trigger/watchdog.pid) - Safety timeout: exits after 24 hours to prevent zombie processes - On clean shutdown, the watchdog is killed (no disconnect needed) Disconnect endpoint: - Rate-limited: 5 calls/min per environment - Capped at 500 runs per call - Small counts (<= 25): cancelled inline with pMap concurrency 10 - Large counts: delegated to the bulk action system - Uses finalizeRun: true to skip PENDING_CANCEL and go straight to FINISHED Run engine change: - cancelRun() now respects finalizeRun when the run is in EXECUTING status, skipping the PENDING_CANCEL waiting state and going directly to FINISHED	2026-03-09 12:14:43 +00:00
Oskar Otwinowski	e49ccc1226	feat(buildExtensions): syncSupabaseEnvVars build extension (#3152 ) with docs	2026-03-05 11:04:26 +01:00
github-actions[bot]	e36b75785e	chore: release v4.4.2 (#3127 ) 🚀 Publish Trigger.dev Docker / units (push) Failing after 4s Details 🚀 Publish Trigger.dev Docker / typecheck (push) Failing after 20s Details 🚀 Publish Trigger.dev Docker / publish-webapp (push) Has been skipped Details 🚀 Publish Trigger.dev Docker / publish-worker (push) Has been skipped Details 🚀 Publish Trigger.dev Docker / publish-worker-v4 (push) Has been skipped Details # trigger.dev v4.4.2 ## Summary 2 new features, 2 improvements, 8 bug fixes. ## Improvements - Add input streams for bidirectional communication with running tasks. Define typed input streams with `streams.input<T>({ id })`, then consume inside tasks via `.wait()` (suspends the process), `.once()` (waits for next message), or `.on()` (subscribes to a continuous stream). Send data from backends with `.send(runId, data)` or from frontends with the new `useInputStreamSend` React hook. ([#3146](https://github.com/triggerdotdev/trigger.dev/pull/3146)) - Add PAYLOAD_TOO_LARGE error to handle graceful recovery of sending batch trigger items with payloads that exceed the maximum payload size ([#3137](https://github.com/triggerdotdev/trigger.dev/pull/3137)) ## Bug fixes - Fix slow batch queue processing by removing spurious cooloff on concurrency blocks and fixing a race condition where retry attempt counts were not atomically updated during message re-queue. ([#3079](https://github.com/triggerdotdev/trigger.dev/pull/3079)) - fix(sdk): batch triggerAndWait variants now return correct run.taskIdentifier instead of unknown ([#3080](https://github.com/triggerdotdev/trigger.dev/pull/3080)) ## Server changes These changes affect the self-hosted Docker image and Trigger.dev Cloud: - Two-level tenant dispatch architecture for batch queue processing. Replaces the single master queue with a two-level index: a dispatch index (tenant → shard) and per-tenant queue indexes (tenant → queues). This enables O(1) tenant selection and fair scheduling across tenants regardless of queue count. Improves batch queue processing performance. ([#3133](https://github.com/triggerdotdev/trigger.dev/pull/3133)) - Add input streams with API routes for sending data to running tasks, SSE reading, and waitpoint creation. Includes Redis cache for fast `.send()` to `.wait()` bridging, dashboard span support for input stream operations, and s2-lite support with configurable S2 endpoint, access token skipping, and S2-Basin headers for self-hosted deployments. Adds s2-lite to Docker Compose for local development. ([#3146](https://github.com/triggerdotdev/trigger.dev/pull/3146)) - Speed up batch queue processing by disabling cooloff and increasing the batch queue processing concurrency limits on the cloud: - Pro plan: increase to 50 from 10. - Hobby plan: increase to 10 from 5. - Free plan: increase to 5 from 1. ([#3079](https://github.com/triggerdotdev/trigger.dev/pull/3079)) - Move batch queue global rate limiter from FairQueue claim phase to BatchQueue worker queue consumer for accurate per-item rate limiting. Add worker queue depth cap to prevent unbounded growth that could cause visibility timeouts. ([#3166](https://github.com/triggerdotdev/trigger.dev/pull/3166)) - Fix a race condition in the waitpoint system where a run could be blocked by a completed waitpoint but never be resumed because of a PostgreSQL MVCC issue. This was most likely to occur when creating a waitpoint via `wait.forToken()` at the same moment as completing the token with `wait.completeToken()`. Other types of waitpoints (timed, child runs) were not affected. ([#3075](https://github.com/triggerdotdev/trigger.dev/pull/3075)) - Fix metrics dashboard chart series colors going out of sync and widgets not reloading stale data when scrolled back into view ([#3126](https://github.com/triggerdotdev/trigger.dev/pull/3126)) - Gracefully handle oversized batch items instead of aborting the stream. When an NDJSON batch item exceeds the maximum size, the parser now emits an error marker instead of throwing, allowing the batch to seal normally. The oversized item becomes a pre-failed run with `PAYLOAD_TOO_LARGE` error code, while other items in the batch process successfully. This prevents `batchTriggerAndWait` from seeing connection errors and retrying with exponential backoff. Also fixes the NDJSON parser not consuming the remainder of an oversized line split across multiple chunks, which caused "Invalid JSON" errors on subsequent lines. ([#3137](https://github.com/triggerdotdev/trigger.dev/pull/3137)) - Require the user is an admin during an impersonation session. Previously only the impersonation cookie was checked; now the real user's admin flag is verified on every request. If admin has been revoked, the session falls back to the real user's ID. ([#3078](https://github.com/triggerdotdev/trigger.dev/pull/3078)) <details> <summary>Raw changeset output</summary> # Releases ## @trigger.dev/build@4.4.2 ### Patch Changes - Updated dependencies: - `@trigger.dev/core@4.4.2` ## trigger.dev@4.4.2 ### Patch Changes - Updated dependencies: - `@trigger.dev/build@4.4.2` - `@trigger.dev/core@4.4.2` - `@trigger.dev/schema-to-json@4.4.2` ## @trigger.dev/python@4.4.2 ### Patch Changes - Updated dependencies: - `@trigger.dev/sdk@4.4.2` - `@trigger.dev/build@4.4.2` - `@trigger.dev/core@4.4.2` ## @trigger.dev/react-hooks@4.4.2 ### Patch Changes - Add input streams for bidirectional communication with running tasks. Define typed input streams with `streams.input<T>({ id })`, then consume inside tasks via `.wait()` (suspends the process), `.once()` (waits for next message), or `.on()` (subscribes to a continuous stream). Send data from backends with `.send(runId, data)` or from frontends with the new `useInputStreamSend` React hook. ([#3146](https://github.com/triggerdotdev/trigger.dev/pull/3146)) Upgrade S2 SDK from 0.17 to 0.22 with support for custom endpoints (s2-lite) via the new `endpoints` configuration, `AppendRecord.string()` API, and `maxInflightBytes` session option. - Updated dependencies: - `@trigger.dev/core@4.4.2` ## @trigger.dev/redis-worker@4.4.2 ### Patch Changes - Fix slow batch queue processing by removing spurious cooloff on concurrency blocks and fixing a race condition where retry attempt counts were not atomically updated during message re-queue. ([#3079](https://github.com/triggerdotdev/trigger.dev/pull/3079)) - Updated dependencies: - `@trigger.dev/core@4.4.2` ## @trigger.dev/rsc@4.4.2 ### Patch Changes - Updated dependencies: - `@trigger.dev/core@4.4.2` ## @trigger.dev/schema-to-json@4.4.2 ### Patch Changes - Updated dependencies: - `@trigger.dev/core@4.4.2` ## @trigger.dev/sdk@4.4.2 ### Patch Changes - Add input streams for bidirectional communication with running tasks. Define typed input streams with `streams.input<T>({ id })`, then consume inside tasks via `.wait()` (suspends the process), `.once()` (waits for next message), or `.on()` (subscribes to a continuous stream). Send data from backends with `.send(runId, data)` or from frontends with the new `useInputStreamSend` React hook. ([#3146](https://github.com/triggerdotdev/trigger.dev/pull/3146)) Upgrade S2 SDK from 0.17 to 0.22 with support for custom endpoints (s2-lite) via the new `endpoints` configuration, `AppendRecord.string()` API, and `maxInflightBytes` session option. - fix(sdk): batch triggerAndWait variants now return correct run.taskIdentifier instead of unknown ([#3080](https://github.com/triggerdotdev/trigger.dev/pull/3080)) - Add PAYLOAD_TOO_LARGE error to handle graceful recovery of sending batch trigger items with payloads that exceed the maximum payload size ([#3137](https://github.com/triggerdotdev/trigger.dev/pull/3137)) - Updated dependencies: - `@trigger.dev/core@4.4.2` ## @trigger.dev/core@4.4.2 </details> --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>	2026-03-04 09:47:24 +00:00
Eric Allam	540e1c86a4	feat: Input Streams - Bidirectional task communication (#3146 ) Input streams enable sending typed data to executing tasks from external callers — backends, frontends, or other tasks. This unlocks interactive use cases like approval UIs, cancel buttons, chat interfaces, and human-in-the-loop AI workflows where the task needs to receive data while running. Three consumption patterns inside a task: * `.wait()` — Suspend the task until data arrives (process freed, most efficient) * `.once()` — Wait for the next message (process stays alive) * `.on()` — Subscribe to a continuous stream of messages One send pattern from outside: * `.send(runId, data)` — Send typed data to a specific run's input stream ## User-facing API ### Define a typed input stream ```ts import { streams, task } from "@trigger.dev/sdk"; const approval = streams.input<{ approved: boolean; reviewer: string }>({ id: "approval" }); ``` ### Consume inside a task ```ts export const myTask = task({ id: "my-task", run: async () => { // Pattern 1: Suspend until data arrives (most efficient — frees the process) const result = await approval.wait({ timeout: "5m" }); // Pattern 2: Wait for next message (process stays alive) const data = await approval.once().unwrap(); // Pattern 3: Subscribe to multiple messages approval.on((data) => { /* handle each message */ }); }, }); ``` ### Send from outside ```ts // From a backend (using secret API key) await approval.send(runId, { approved: true, reviewer: "alice" }); // From a frontend (using public JWT token from trigger response) const { send } = useInputStreamSend("approval", runId, { accessToken }); send({ approved: true, reviewer: "alice" }); ``` --------- Co-authored-by: Claude <noreply@anthropic.com>	2026-03-02 16:49:54 +00:00
Eric Allam	8003923598	feat(server): Gracefully handle oversized batch items instead of aborting the stream (#3137 ) Gracefully handle oversized batch items instead of aborting the stream. When an NDJSON batch item exceeds the maximum size, the parser now emits an error marker instead of throwing, allowing the batch to seal normally. The oversized item becomes a pre-failed run with `PAYLOAD_TOO_LARGE` error code, while other items in the batch process successfully. This prevents `batchTriggerAndWait` from seeing connection errors and retrying with exponential backoff. Also fixes the NDJSON parser not consuming the remainder of an oversized line split across multiple chunks, which caused "Invalid JSON" errors on subsequent lines.	2026-02-27 10:11:42 +00:00
Eric Allam	bed3789c31	fix(batch-queue): speed up batch queue processing by disabling cooloff and fixing retry race (#3079 ) Fix slow fair queue processing by removing spurious cooloff on concurrency blocks and fixing a race condition where retry attempt counts were not atomically updated during message re-queue. Removed cooloff entirely from the batch queue	2026-02-25 17:33:01 +00:00
Eric Allam	f37bdaac84	fix(sdk): batch triggerAndWait variants now return correct run.taskIdentifier instead of unknown (#3080 ) Fixes #2942	2026-02-25 15:37:04 +00:00
github-actions[bot]	98bf706437	chore: release v4.4.1 (#3100 ) 🚀 Publish Trigger.dev Docker / units (push) Failing after 0s Details 🚀 Publish Trigger.dev Docker / typecheck (push) Failing after 6s Details 🚀 Publish Trigger.dev Docker / publish-webapp (push) Has been skipped Details 🚀 Publish Trigger.dev Docker / publish-worker (push) Has been skipped Details 🚀 Publish Trigger.dev Docker / publish-worker-v4 (push) Has been skipped Details This PR was opened by the [Changesets release](https://github.com/changesets/action) GitHub action. When you're ready to do a release, you can merge this and publish to npm yourself or [setup this action to publish automatically](https://github.com/changesets/action#with-publishing). If you're not ready to do a release yet, that's fine, whenever you add more changesets to main, this PR will be updated. # Releases ## @trigger.dev/build@4.4.1 ### Patch Changes - Updated dependencies: - `@trigger.dev/core@4.4.1` ## trigger.dev@4.4.1 ### Patch Changes - Add OTEL metrics pipeline for task workers. Workers collect process CPU/memory, Node.js runtime metrics (event loop utilization, event loop delay, heap usage), and user-defined custom metrics via `otel.metrics.getMeter()`. Metrics are exported to ClickHouse with 10-second aggregation buckets and 1m/5m rollups, and are queryable through the dashboard query engine with typed attribute columns, `prettyFormat()` for human-readable values, and AI query support. ([#3061](https://github.com/triggerdotdev/trigger.dev/pull/3061)) - Updated dependencies: - `@trigger.dev/build@4.4.1` - `@trigger.dev/core@4.4.1` - `@trigger.dev/schema-to-json@4.4.1` ## @trigger.dev/python@4.4.1 ### Patch Changes - Updated dependencies: - `@trigger.dev/sdk@4.4.1` - `@trigger.dev/build@4.4.1` - `@trigger.dev/core@4.4.1` ## @trigger.dev/react-hooks@4.4.1 ### Patch Changes - Updated dependencies: - `@trigger.dev/core@4.4.1` ## @trigger.dev/redis-worker@4.4.1 ### Patch Changes - Updated dependencies: - `@trigger.dev/core@4.4.1` ## @trigger.dev/rsc@4.4.1 ### Patch Changes - Updated dependencies: - `@trigger.dev/core@4.4.1` ## @trigger.dev/schema-to-json@4.4.1 ### Patch Changes - Updated dependencies: - `@trigger.dev/core@4.4.1` ## @trigger.dev/sdk@4.4.1 ### Patch Changes - Add OTEL metrics pipeline for task workers. Workers collect process CPU/memory, Node.js runtime metrics (event loop utilization, event loop delay, heap usage), and user-defined custom metrics via `otel.metrics.getMeter()`. Metrics are exported to ClickHouse with 10-second aggregation buckets and 1m/5m rollups, and are queryable through the dashboard query engine with typed attribute columns, `prettyFormat()` for human-readable values, and AI query support. ([#3061](https://github.com/triggerdotdev/trigger.dev/pull/3061)) - Updated dependencies: - `@trigger.dev/core@4.4.1` ## @trigger.dev/core@4.4.1 --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>	2026-02-20 16:40:06 +00:00
Eric Allam	469b039090	feat: OTEL metrics pipeline for task workers (#3061 ) - Adds an end-to-end OTEL metrics pipeline: task workers collect and export metrics via OpenTelemetry, the webapp ingests them into ClickHouse, and they're queryable through the existing dashboard query engine - Workers emit process CPU/memory metrics (via `@opentelemetry/host-metrics`) and Node.js runtime metrics (event loop utilization, event loop delay, heap usage) - Users can create custom metrics in their tasks via `otel.metrics.getMeter()` from `@trigger.dev/sdk` - Metrics are automatically tagged with run context (run ID, task slug, machine, worker version) so they can be sliced per-run, per-task, or per-machine - The TSQL query engine gains metrics table support with typed attribute columns, `prettyFormat()` for human-readable values, and per-schema time bucket thresholds - Includes reference tasks (`references/hello-world/src/trigger/metrics.ts`) demonstrating CPU-intensive, memory-ramp, bursty workload, and custom metrics patterns ## What changed ### Metrics collection (packages/core, packages/cli-v3) - Metrics export pipeline — `TracingSDK` now sets up a `MeterProvider` with a `PeriodicExportingMetricReader` that chains through `TaskContextMetricExporter` (adds run context attributes) and `BufferingMetricExporter` (batches exports to reduce overhead) - Host metrics — Enabled `@opentelemetry/host-metrics` for process CPU, memory, and system-level metrics - Node.js runtime metrics — New `nodejsRuntimeMetrics.ts` module using `performance.eventLoopUtilization()`, `monitorEventLoopDelay()`, and `process.memoryUsage()` to emit 6 observable gauges - File system and diskio metrics - Custom metrics — Exposed `otel.metrics` from `@trigger.dev/sdk` so users can create counters, histograms, and gauges in their tasks - Machine ID — Stable per-worker machine identifier for grouping metrics - Dev worker — Drops `system.` metrics to reduce noise, keeps sending metrics between runs in warm workers ### Metrics ingestion (apps/webapp) - OTEL endpoint* — `otel.v1.metrics.ts` accepts OTEL metric export requests (JSON and protobuf), converts to ClickHouse rows - ClickHouse schema — `017_create_metrics_v1.sql` with 10-second aggregation buckets, JSON attributes column, 60-day TTLs ### Query engine (internal-packages/tsql, apps/webapp) - Metrics query schema — Typed columns for metric attributes (`task_identifier`, `run_id`, `machine_name`, `worker_version`, etc.) extracted from the JSON attributes column - `prettyFormat()` — TSQL function that annotates columns with format hints (`bytes`, `percent`, `durationSeconds`) for frontend rendering without changing the underlying data - Per-schema time buckets — Different tables can define their own time bucket thresholds (metrics uses tighter intervals than runs) - AI query integration — The AI query service knows about the metrics table and can generate metric queries - Chart improvements — Better formatting for byte values, percentages, and durations in charts and tables ### Reference project - `references/hello-world/src/trigger/metrics.ts` — 6 example tasks: `cpu-intensive`, `memory-ramp`, `bursty-workload`, `sustained-workload`, `concurrent-load`, `custom-metrics` ## Test plan - [ ] Build all packages and webapp - [ ] Start dev worker with hello-world reference project - [ ] Run `cpu-intensive`, `memory-ramp`, and `custom-metrics` tasks - [ ] Verify metrics in ClickHouse: `SELECT DISTINCT metric_name FROM metrics_v1` - [ ] Query via dashboard AI: "show me CPU utilization over time" - [ ] Verify `prettyFormat` renders correctly in chart tooltips and table cells - [ ] Confirm dev worker drops `system.` metrics but keeps `process.` and `nodejs.*`	2026-02-20 13:16:34 +00:00
github-actions[bot]	84a00b6eb6	chore: release v4.4.0 (#2941 ) 🚀 Publish Trigger.dev Docker / typecheck (push) Failing after 0s Details 🚀 Publish Trigger.dev Docker / publish-webapp (push) Has been skipped Details 🚀 Publish Trigger.dev Docker / publish-worker (push) Has been skipped Details 🚀 Publish Trigger.dev Docker / publish-worker-v4 (push) Has been skipped Details 🚀 Publish Trigger.dev Docker / units (push) Failing after 3s Details This PR was opened by the [Changesets release](https://github.com/changesets/action) GitHub action. When you're ready to do a release, you can merge this and publish to npm yourself or [setup this action to publish automatically](https://github.com/changesets/action#with-publishing). If you're not ready to do a release yet, that's fine, whenever you add more changesets to main, this PR will be updated. # Releases ## @trigger.dev/sdk@4.4.0 ### Minor Changes - Added `query.execute()` which lets you query your Trigger.dev data using TRQL (Trigger Query Language) and returns results as typed JSON rows or CSV. It supports configurable scope (environment, project, or organization), time filtering via `period` or `from`/`to` ranges, and a `format` option for JSON or CSV output. ([#3060](https://github.com/triggerdotdev/trigger.dev/pull/3060)) ```typescript import { query } from "@trigger.dev/sdk"; import type { QueryTable } from "@trigger.dev/sdk"; // Basic untyped query const result = await query.execute("SELECT run_id, status FROM runs LIMIT 10"); // Type-safe query using QueryTable to pick specific columns const typedResult = await query.execute<QueryTable<"runs", "run_id" \| "status" \| "triggered_at">>( "SELECT run_id, status, triggered_at FROM runs LIMIT 10" ); typedResult.results.forEach((row) => { console.log(row.run_id, row.status); // Fully typed }); // Aggregation query with inline types const stats = await query.execute<{ status: string; count: number }>( "SELECT status, COUNT(*) as count FROM runs GROUP BY status", { scope: "project", period: "30d" } ); // CSV export const csv = await query.execute("SELECT run_id, status FROM runs", { format: "csv", period: "7d", }); console.log(csv.results); // Raw CSV string ``` ### Patch Changes - Add `maxDelay` option to debounce feature. This allows setting a maximum time limit for how long a debounced run can be delayed, ensuring execution happens within a specified window even with continuous triggers. ([#2984](https://github.com/triggerdotdev/trigger.dev/pull/2984)) ```typescript await myTask.trigger(payload, { debounce: { key: "my-key", delay: "5s", maxDelay: "30m", // Execute within 30 minutes regardless of continuous triggers }, }); ``` - Aligned the SDK's `getRunIdForOptions` logic with the Core package to handle semantic targets (`root`, `parent`) in root tasks. ([#2874](https://github.com/triggerdotdev/trigger.dev/pull/2874)) - Export `AnyOnStartAttemptHookFunction` type to allow defining `onStartAttempt` hooks for individual tasks. ([#2966](https://github.com/triggerdotdev/trigger.dev/pull/2966)) - Fixed a minor issue in the deployment command on distinguishing between local builds for the cloud vs local builds for self-hosting setups. ([#3070](https://github.com/triggerdotdev/trigger.dev/pull/3070)) - Updated dependencies: - `@trigger.dev/core@4.4.0` ## @trigger.dev/build@4.4.0 ### Patch Changes - Updated dependencies: - `@trigger.dev/core@4.4.0` ## trigger.dev@4.4.0 ### Patch Changes - Fix runner getting stuck indefinitely when `execute()` is called on a dead child process. ([#2978](https://github.com/triggerdotdev/trigger.dev/pull/2978)) - Add optional `timeoutInSeconds` parameter to the `wait_for_run_to_complete` MCP tool. Defaults to 60 seconds. If the run doesn't complete within the timeout, the current state of the run is returned instead of waiting indefinitely. ([#3035](https://github.com/triggerdotdev/trigger.dev/pull/3035)) - Fixed a minor issue in the deployment command on distinguishing between local builds for the cloud vs local builds for self-hosting setups. ([#3070](https://github.com/triggerdotdev/trigger.dev/pull/3070)) - Updated dependencies: - `@trigger.dev/core@4.4.0` - `@trigger.dev/build@4.4.0` - `@trigger.dev/schema-to-json@4.4.0` ## @trigger.dev/core@4.4.0 ### Patch Changes - Add `maxDelay` option to debounce feature. This allows setting a maximum time limit for how long a debounced run can be delayed, ensuring execution happens within a specified window even with continuous triggers. ([#2984](https://github.com/triggerdotdev/trigger.dev/pull/2984)) ```typescript await myTask.trigger(payload, { debounce: { key: "my-key", delay: "5s", maxDelay: "30m", // Execute within 30 minutes regardless of continuous triggers }, }); ``` - Fixed a minor issue in the deployment command on distinguishing between local builds for the cloud vs local builds for self-hosting setups. ([#3070](https://github.com/triggerdotdev/trigger.dev/pull/3070)) - fix: vendor superjson to fix ESM/CJS compatibility ([#2949](https://github.com/triggerdotdev/trigger.dev/pull/2949)) Bundle superjson during build to avoid `ERR_REQUIRE_ESM` errors on Node.js versions that don't support `require(ESM)` by default (< 22.12.0) and AWS Lambda which intentionally disables it. - Add Vercel integration support to API schemas: `commitSHA` and `integrationDeployments` on deployment responses, and `source` field for environment variable imports. ([#2994](https://github.com/triggerdotdev/trigger.dev/pull/2994)) ## @trigger.dev/python@4.4.0 ### Patch Changes - Updated dependencies: - `@trigger.dev/core@4.4.0` - `@trigger.dev/sdk@4.4.0` - `@trigger.dev/build@4.4.0` ## @trigger.dev/react-hooks@4.4.0 ### Patch Changes - Fix `onComplete` callback firing prematurely when the realtime stream disconnects before the run finishes. ([#2929](https://github.com/triggerdotdev/trigger.dev/pull/2929)) - Updated dependencies: - `@trigger.dev/core@4.4.0` ## @trigger.dev/redis-worker@4.4.0 ### Patch Changes - Updated dependencies: - `@trigger.dev/core@4.4.0` ## @trigger.dev/rsc@4.4.0 ### Patch Changes - Updated dependencies: - `@trigger.dev/core@4.4.0` ## @trigger.dev/schema-to-json@4.4.0 ### Patch Changes - Updated dependencies: - `@trigger.dev/core@4.4.0` --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>	2026-02-19 13:38:10 +00:00
Saadi Myftija	7af789bc3e	fix(deployments): external build token generation issue (#3070 ) Fixes an issue introduced in #3024. The behavior for local builds in older CLI versions relies on `externalBuildData` to be defined to distinguish from the self-hosting local build path, even though it doesn't actually use the token.	2026-02-16 19:05:40 +01:00
Matt Aitken	d4cd34094e	Query API and SDK (#3060 ) Summary - Add API endpoint to run TRQL queries - Implement SDK function for executing queries ## SDK Added `query.execute()` which lets you query your Trigger.dev data using TRQL (Trigger Query Language) and returns results as typed JSON rows or CSV. It supports configurable scope (environment, project, or organization), time filtering via `period` or `from`/`to` ranges, and a `format` option for JSON or CSV output. ```typescript import { query } from "@trigger.dev/sdk"; import type { QueryTable } from "@trigger.dev/sdk"; // Basic untyped query const result = await query.execute("SELECT run_id, status FROM runs LIMIT 10"); // Type-safe query using QueryTable to pick specific columns const typedResult = await query.execute<QueryTable<"runs", "run_id" \| "status" \| "triggered_at">>( "SELECT run_id, status, triggered_at FROM runs LIMIT 10" ); typedResult.results.forEach(row => { console.log(row.run_id, row.status); // Fully typed }); // Aggregation query with inline types const stats = await query.execute<{ status: string; count: number }>( "SELECT status, COUNT(*) as count FROM runs GROUP BY status", { scope: "project", period: "30d" } ); // CSV export const csv = await query.execute( "SELECT run_id, status FROM runs", { format: "csv", period: "7d" } ); console.log(csv.results); // Raw CSV string ```	2026-02-16 10:53:41 +00:00
Eric Allam	062bcaece8	feat(mcp): add timeout parameter to wait_for_run_to_complete tool (#3035 ) ## Summary - Adds an optional `timeoutInSeconds` parameter (default 60s) to the `wait_for_run_to_complete` MCP tool - If the run doesn't complete within the timeout, returns the current run state instead of blocking indefinitely - Uses `AbortSignal.timeout()` combined with the existing MCP signal Fixes #3032	2026-02-12 16:14:25 +00:00
Oskar Otwinowski	9b21f8d322	feat(webapp): Vercel integration (#2994 ) Vercel integration Desc + Vid coming soon For human reviewer: - check the db schema - check if posthog user attribution call is correct (telemetry.server.ts & `referralSource`) <!-- devin-review-badge-begin --> --- <a href="https://app.devin.ai/review/triggerdotdev/trigger.dev/pull/2994" target="_blank"> <picture> <source media="(prefers-color-scheme: dark)" srcset="https://static.devin.ai/assets/gh-open-in-devin-review-dark.svg?v=1"> <img src="https://static.devin.ai/assets/gh-open-in-devin-review-light.svg?v=1" alt="Open with Devin"> </picture> </a> <!-- devin-review-badge-end -->	2026-02-10 10:37:09 +01:00
Eric Allam	b72cacc671	feat(debounce): add maxDelay option to limit total debounce time (#2984 )	2026-02-02 20:15:06 +00:00
nicktrn	279102c17c	fix(cli): reject execute() immediately when child process is dead (#2978 ) ## Summary - When a child process crashes and a retry (`RETRY_IMMEDIATELY`) is attempted on the same `TaskRunProcess`, `execute()` hangs forever because the IPC send is silently skipped and the attempt promise can never resolve - This caused runner pods to stay up indefinitely with no heartbeats or polls - Fix: reject the attempt promise immediately when the child is not connected, so the controller can proceed to warm start or exit ## Test plan - [x] Added `taskRunProcess.test.ts` — verifies `execute()` rejects promptly instead of hanging when the child process is dead - [x] Deploy and verify no more stuck runner pods accumulate over time	2026-01-30 16:44:46 +00:00
Matt Aitken	bc7ce78103	fix(sdk): export AnyOnStartAttemptHookFunction type (#2966 ) Export AnyOnStartAttemptHookFunction type to allow defining onStartAttempt hooks for individual tasks. https://claude.ai/code/session_018jgSVcFtKVyv65ktGNQFFq <!-- devin-review-badge-begin --> --- <a href="https://app.devin.ai/review/triggerdotdev/trigger.dev/pull/2966"> <picture> <source media="(prefers-color-scheme: dark)" srcset="https://static.devin.ai/assets/gh-open-in-devin-review-dark.svg?v=1"> <img src="https://static.devin.ai/assets/gh-open-in-devin-review-light.svg?v=1" alt="Open with Devin"> </picture> </a> <!-- devin-review-badge-end --> Co-authored-by: Claude <noreply@anthropic.com>	2026-01-30 09:17:37 +00:00
Eric Allam	3925f8cc49	fix(core): vendor superjson to fix ESM/CJS compatibility (#2949 ) Bundle superjson and its dependency (copy-anything) during build to avoid ERR_REQUIRE_ESM errors on Node.js versions that don't support require(ESM) by default (< 22.12.0) and AWS Lambda which intentionally disables it. - Add scripts/bundle-superjson.mjs to bundle superjson with esbuild - Update build script to bundle vendor files before tshy compilation - Move superjson from dependencies to devDependencies - Update imports to use vendored bundles Fixes #2937 <!-- devin-review-badge-begin --> --- <a href="https://app.devin.ai/review/triggerdotdev/trigger.dev/pull/2949"> <picture> <source media="(prefers-color-scheme: dark)" srcset="https://static.devin.ai/assets/gh-open-in-devin-review-dark.svg?v=1"> <img src="https://static.devin.ai/assets/gh-open-in-devin-review-light.svg?v=1" alt="Open with Devin"> </picture> </a> <!-- devin-review-badge-end --> --------- Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com> Co-authored-by: Eric Allam <ericallam@users.noreply.github.com>	2026-01-30 09:15:02 +00:00
nicktrn	70c8d6d14b	fix(react-hooks): prevent onComplete from firing prematurely when stream disconnects (#2929 ) ## Summary Fixes #2856 - The `onComplete` callback in `useRealtimeRun` was firing prematurely ## Root Cause The callback was triggered when the long-poll stream ended, regardless of whether the run had actually completed. Reverse proxies often close idle connections, causing the stream to end prematurely. In this case it was caused by fetch abort due to React strict mode. ## Fix Changed the condition from checking if `run` exists to checking if `run?.finishedAt` exists, ensuring `onComplete` only fires when the run has reached a terminal state. --------- Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com> Co-authored-by: nicktrn <nicktrn@users.noreply.github.com>	2026-01-27 16:21:56 +00:00
bharath kumar	b143027d95	Fix/sdk stream root fallback (#2874 )	2026-01-25 06:53:26 +00:00
github-actions[bot]	700fe91bec	chore: release v4.3.3 (#2893 ) 🚀 Publish Trigger.dev Docker / typecheck (push) Failing after 0s Details 🚀 Publish Trigger.dev Docker / publish-webapp (push) Has been skipped Details 🚀 Publish Trigger.dev Docker / publish-worker (push) Has been skipped Details 🚀 Publish Trigger.dev Docker / publish-worker-v4 (push) Has been skipped Details 🚀 Publish Trigger.dev Docker / units (push) Failing after 0s Details This PR was opened by the [Changesets release](https://github.com/changesets/action) GitHub action. When you're ready to do a release, you can merge this and publish to npm yourself or [setup this action to publish automatically](https://github.com/changesets/action#with-publishing). If you're not ready to do a release yet, that's fine, whenever you add more changesets to main, this PR will be updated. # Releases ## @trigger.dev/build@4.3.3 ### Patch Changes - Updated dependencies: - `@trigger.dev/core@4.3.3` ## trigger.dev@4.3.3 ### Patch Changes - Updated dependencies: - `@trigger.dev/core@4.3.3` - `@trigger.dev/build@4.3.3` - `@trigger.dev/schema-to-json@4.3.3` ## @trigger.dev/core@4.3.3 ### Patch Changes - Add support for AI SDK v6 (Vercel AI SDK) ([#2919](https://github.com/triggerdotdev/trigger.dev/pull/2919)) - Updated peer dependency to allow `ai@^6.0.0` alongside v4 and v5 - Updated internal code to handle async validation from AI SDK v6's Schema type - Expose user-provided idempotency key and scope in task context. `ctx.run.idempotencyKey` now returns the original key passed to `idempotencyKeys.create()` instead of the hash, and `ctx.run.idempotencyKeyScope` shows the scope ("run", "attempt", or "global"). ([#2903](https://github.com/triggerdotdev/trigger.dev/pull/2903)) - Fix batch trigger failing with "ReadableStream is locked" error when network failures occur mid-stream. Added safe stream cancellation that gracefully handles locked streams during retry attempts. ([#2917](https://github.com/triggerdotdev/trigger.dev/pull/2917)) - Add a maxDepth to flatten/unflattenAttributes to prevent possible issues ([#2890](https://github.com/triggerdotdev/trigger.dev/pull/2890)) ## @trigger.dev/python@4.3.3 ### Patch Changes - Updated dependencies: - `@trigger.dev/sdk@4.3.3` - `@trigger.dev/core@4.3.3` - `@trigger.dev/build@4.3.3` ## @trigger.dev/react-hooks@4.3.3 ### Patch Changes - Updated dependencies: - `@trigger.dev/core@4.3.3` ## @trigger.dev/redis-worker@4.3.3 ### Patch Changes - Updated dependencies: - `@trigger.dev/core@4.3.3` ## @trigger.dev/rsc@4.3.3 ### Patch Changes - Updated dependencies: - `@trigger.dev/core@4.3.3` ## @trigger.dev/schema-to-json@4.3.3 ### Patch Changes - Updated dependencies: - `@trigger.dev/core@4.3.3` ## @trigger.dev/sdk@4.3.3 ### Patch Changes - Add support for AI SDK v6 (Vercel AI SDK) ([#2919](https://github.com/triggerdotdev/trigger.dev/pull/2919)) - Updated peer dependency to allow `ai@^6.0.0` alongside v4 and v5 - Updated internal code to handle async validation from AI SDK v6's Schema type - Expose user-provided idempotency key and scope in task context. `ctx.run.idempotencyKey` now returns the original key passed to `idempotencyKeys.create()` instead of the hash, and `ctx.run.idempotencyKeyScope` shows the scope ("run", "attempt", or "global"). ([#2903](https://github.com/triggerdotdev/trigger.dev/pull/2903)) - Updated dependencies: - `@trigger.dev/core@4.3.3` --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>	2026-01-23 17:36:11 +00:00
Eric Allam	23ec5ff8fa	feat(sdk): add support for AI SDK v6 (#2919 ) ## Summary - Add support for Vercel AI SDK v6 as a peer dependency - Update internal code to handle async validation from AI SDK v6's Schema type Closes #2918 Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com> Co-authored-by: Eric Allam <ericallam@users.noreply.github.com>	2026-01-20 22:31:41 +00:00
Eric Allam	31e4753122	fix(sdk): handle locked ReadableStream when retrying batch trigger (#2917 ) When fetch crashes mid-stream during batch item upload (e.g., connection reset, timeout), the request stream may remain locked by fetch's internal reader. Attempting to cancel a locked stream throws 'Invalid state: ReadableStream is locked', causing the batch operation to fail. Added safeStreamCancel() helper that gracefully handles locked streams by catching and ignoring the locked error. The stream will be cleaned up by garbage collection when fetch eventually releases the reader. Fixes customer issue where batchTrigger failed with ReadableStream locked error during network instability.	2026-01-20 15:55:23 +00:00
Eric Allam	36168b3eb6	feat(sdk): expose user-provided idempotency key and scope in task context (#2903 ) ## Summary - Store the original user-provided idempotency key and scope alongside the hash - Expose `ctx.run.idempotencyKey` as the user-provided key (not the hash) - Add `ctx.run.idempotencyKeyScope` to show the scope ("run", "attempt", or "global") <img width="539" height="450" alt="CleanShot 2026-01-19 at 11 40 46" src="https://github.com/user-attachments/assets/b6f42991-697e-4314-a164-aef77b8fd25c" /> ## Problem Idempotency keys were hashed (SHA-256) before storage, making debugging difficult since users couldn't see the value they originally set or search for runs by idempotency key. ## Solution Attach metadata to the `String` object returned by `idempotencyKeys.create()` using a Symbol, extract it in the SDK before the API call, and store it in the database alongside the hash. ```typescript const key = await idempotencyKeys.create("my-key", { scope: "global" }); await childTask.triggerAndWait(payload, { idempotencyKey: key }); // In child task: ctx.run.idempotencyKey // "my-key" (previously showed the hash) ctx.run.idempotencyKeyScope // "global" ``` Test plan - Trigger task with idempotencyKeys.create() using different scopes (run, attempt, global) - Verify ctx.run.idempotencyKey returns user-provided key - Verify ctx.run.idempotencyKeyScope returns correct scope - Verify PostgreSQL stores idempotencyKeyOptions JSON - Verify ClickHouse receives idempotency_key_user and idempotency_key_scope via replication --------- Co-authored-by: James Ritchie <james@trigger.dev>	2026-01-20 11:23:07 +00:00
Eric Allam	936bddf198	fix: upgrade Node.js to 20.20.0 to address async_hooks DoS vulnerability (#2890 ) ## Summary - Upgrades Node.js from 20.19.0 to 20.20.0 (and 22.12.0 to 22.22.0 for supervisor) to address the async_hooks stack overflow DoS vulnerability - Adds `maxDepth` parameter (default 128) to `flattenAttributes` and `unflattenAttributes` to prevent stack overflow on maliciously deep nested structures ## Details The vulnerability (patched in Node.js 20.20.0, 22.22.0, 24.13.0, 25.3.0) causes unrecoverable crashes (exit code 7) when stack overflow occurs during async_hooks callbacks. Since the webapp uses `AsyncLocalStorage`, it was theoretically vulnerable. ### Changes Node.js version updates: - `docker/Dockerfile`: 20.11.1 → 20.20.0 - `apps/supervisor/Containerfile`: 22-alpine → 22.22.0-alpine - `.nvmrc`: 20.19.0 → 20.20.0 - `apps/supervisor/.nvmrc`: 22.12.0 → 22.22.0 - `references/prisma-7/.nvmrc`: 20.19.0 → 20.20.0 - All GitHub workflows: 20.19.0 → 20.20.0 Defense in depth: - Added `maxDepth` parameter to `flattenAttributes()` and `unflattenAttributes()` in `packages/core` to prevent stack overflow on deeply nested user input ## Test plan - [x] All existing `flattenAttributes` tests pass (50 tests) - [x] New tests for depth limiting added - [x] Verify Docker builds work with new base images	2026-01-15 10:47:44 +00:00
github-actions[bot]	f73138fe6c	chore: release v4.3.2 (#2811 ) 🚀 Publish Trigger.dev Docker / typecheck (push) Failing after 4s Details 🚀 Publish Trigger.dev Docker / units (push) Failing after 0s Details 🚀 Publish Trigger.dev Docker / publish-webapp (push) Has been skipped Details 🚀 Publish Trigger.dev Docker / publish-worker (push) Has been skipped Details 🚀 Publish Trigger.dev Docker / publish-worker-v4 (push) Has been skipped Details This PR was opened by the [Changesets release](https://github.com/changesets/action) GitHub action. When you're ready to do a release, you can merge this and publish to npm yourself or [setup this action to publish automatically](https://github.com/changesets/action#with-publishing). If you're not ready to do a release yet, that's fine, whenever you add more changesets to main, this PR will be updated. # Releases ## @trigger.dev/build@4.3.2 ### Patch Changes - Updated dependencies: - `@trigger.dev/core@4.3.2` ## trigger.dev@4.3.2 ### Patch Changes - fix(cli): update command should preserve existing package.json order ([#2810](https://github.com/triggerdotdev/trigger.dev/pull/2810)) - Updated dependencies: - `@trigger.dev/build@4.3.2` - `@trigger.dev/core@4.3.2` - `@trigger.dev/schema-to-json@4.3.2` ## @trigger.dev/python@4.3.2 ### Patch Changes - Updated dependencies: - `@trigger.dev/sdk@4.3.2` - `@trigger.dev/build@4.3.2` - `@trigger.dev/core@4.3.2` ## @trigger.dev/react-hooks@4.3.2 ### Patch Changes - Updated dependencies: - `@trigger.dev/core@4.3.2` ## @trigger.dev/redis-worker@4.3.2 ### Patch Changes - Updated dependencies: - `@trigger.dev/core@4.3.2` ## @trigger.dev/rsc@4.3.2 ### Patch Changes - Updated dependencies: - `@trigger.dev/core@4.3.2` ## @trigger.dev/schema-to-json@4.3.2 ### Patch Changes - Updated dependencies: - `@trigger.dev/core@4.3.2` ## @trigger.dev/sdk@4.3.2 ### Patch Changes - Improve batch trigger error messages, especially when rate limited ([#2837](https://github.com/triggerdotdev/trigger.dev/pull/2837)) - Updated dependencies: - `@trigger.dev/core@4.3.2` ## @trigger.dev/core@4.3.2 --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>	2026-01-08 11:26:59 +00:00
Eric Allam	8ba7526d51	fix(batch): rate limiting by token bucket no longer incorrectly goes negative (#2837 ) Also improves the BatchTriggerError when a result of getting rate limited.	2026-01-07 14:02:19 +00:00
nicktrn	7a54a843e2	fix(cli): update command should preserve existing package.json order (#2810 ) This fixes a regression introduced in #2778 - stable sort is required for deterministic builds, but we can safely preserve order for the user package.json during package updates	2025-12-23 12:52:06 +00:00
github-actions[bot]	caa40ce925	chore: release v4.3.1 (#2788 ) 🚀 Publish Trigger.dev Docker / typecheck (push) Failing after 0s Details 🚀 Publish Trigger.dev Docker / publish-webapp (push) Has been skipped Details 🚀 Publish Trigger.dev Docker / publish-worker (push) Has been skipped Details 🚀 Publish Trigger.dev Docker / publish-worker-v4 (push) Has been skipped Details 🚀 Publish Trigger.dev Docker / units (push) Failing after 4s Details This PR was opened by the [Changesets release](https://github.com/changesets/action) GitHub action. When you're ready to do a release, you can merge this and publish to npm yourself or [setup this action to publish automatically](https://github.com/changesets/action#with-publishing). If you're not ready to do a release yet, that's fine, whenever you add more changesets to main, this PR will be updated. # Releases ## @trigger.dev/build@4.3.1 ### Patch Changes - Updated dependencies: - `@trigger.dev/core@4.3.1` ## trigger.dev@4.3.1 ### Patch Changes - Updated dependencies: - `@trigger.dev/core@4.3.1` - `@trigger.dev/build@4.3.1` - `@trigger.dev/schema-to-json@4.3.1` ## @trigger.dev/core@4.3.1 ### Patch Changes - Added support for idempotency reset ([#2777](https://github.com/triggerdotdev/trigger.dev/pull/2777)) ## @trigger.dev/python@4.3.1 ### Patch Changes - Updated dependencies: - `@trigger.dev/sdk@4.3.1` - `@trigger.dev/core@4.3.1` - `@trigger.dev/build@4.3.1` ## @trigger.dev/react-hooks@4.3.1 ### Patch Changes - Updated dependencies: - `@trigger.dev/core@4.3.1` ## @trigger.dev/redis-worker@4.3.1 ### Patch Changes - Updated dependencies: - `@trigger.dev/core@4.3.1` ## @trigger.dev/rsc@4.3.1 ### Patch Changes - Updated dependencies: - `@trigger.dev/core@4.3.1` ## @trigger.dev/schema-to-json@4.3.1 ### Patch Changes - Updated dependencies: - `@trigger.dev/core@4.3.1` ## @trigger.dev/sdk@4.3.1 ### Patch Changes - feat: Support for new batch trigger system ([#2779](https://github.com/triggerdotdev/trigger.dev/pull/2779)) - feat(sdk): Support debouncing runs when triggering with new debounce options ([#2794](https://github.com/triggerdotdev/trigger.dev/pull/2794)) - Added support for idempotency reset ([#2777](https://github.com/triggerdotdev/trigger.dev/pull/2777)) - Updated dependencies: - `@trigger.dev/core@4.3.1` --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>	2025-12-22 15:29:18 +00:00
Mihai Popescu	7574c69c2d	feat(webapp): Add support for resetting idempotency keys (#2777 ) Add support for resetting idempotency keys both from ui and sdk ## ✅ Checklist - [x] I have followed every step in the [contributing guide](https://github.com/triggerdotdev/trigger.dev/blob/main/CONTRIBUTING.md) - [x] The PR title follows the convention. - [x] I ran and tested the code works --- ## Testing - Created a new run with a idempotency idempotencyKey. - Started a new run with the same task and got redirected to the first run. - Deleted the key from the UI on the run details - Started a new run with the same task and it created a new one - Did the above steps using the SDK --- ## Changelog - Add new action route for resetting idempotency keys via UI - Add reset button in Idempotency section of run detail view - Added API and SDK for resetting imdepotency - Updated docs page for this feature --- ## Screenshots _[Screenshots]_ <img width="438" height="363" alt="Screenshot 2025-12-11 at 11 56 37" src="https://github.com/user-attachments/assets/30b8ef5e-8aac-4d04-b57a-9bf30d085dcb" />	2025-12-19 11:55:04 +00:00
Eric Allam	3875bb292a	feat(engine): run debounce system (#2794 ) Adds support for debounced task runs - when triggering a task with a debounce key, subsequent triggers with the same key will reschedule the existing delayed run instead of creating new runs. This continues until no new triggers occur within the delay window. ## Usage ```typescript await myTask.trigger({ userId: "123" }, { debounce: { key: "user-123-update", delay: "5s", mode: "leading", // default } }); ``` - key: Scoped to the task identifier - delay: How long to wait before executing (supports duration strings like `"5s"`, `"1m"`) - mode: Either `"leading"` or `"trailing"`. Leading debounce will use the payload and options from the first run created with the debounce key. Trailing will use payload and options from the last run. ### "trailing" mode overrides When using `mode: "trailing"` with debounce, the following options are updated from the last trigger: - `payload` - The task input data - `metadata` - Run metadata - `tags` - Run tags (replaces existing tags) - `maxAttempts` - Maximum retry attempts - `maxDuration` - Maximum compute time - `machine` - Machine preset (cpu/memory) ## Behavior - First run wins: The first trigger creates the run, subsequent triggers push its execution time later - Idempotency keys take precedence: If both are specified, idempotency is checked first - Max duration: Configurable via `DEBOUNCE_MAX_DURATION_MS` env var (default: 10 minutes) Works with `triggerAndWait` - parent runs correctly block on the debounced run.	2025-12-18 16:04:43 +00:00
Eric Allam	a999d9ea3f	feat(engine): Batch trigger reloaded (#2779 ) New batch trigger system with larger payloads, streaming ingestion, larger batch sizes, and a fair processing system. This PR introduces a new `FairQueue` abstraction inspired by our own `RunQueue` that enables multi-tenant fair queueing with concurrency limits. The new `BatchQueue` is built on top of the `FairQueue`, and handles processing Batch triggers in a fair manner with per-environment concurrency limits defined per-org. Additionally, there is a global concurrency limit to prevent the BatchQueue system from creating too many runs too quickly, which can cause downstream issues. For this new BatchQueue system we have a completely new batch trigger creation and ingestion system. Previously this was a single endpoint with a single JSON body that defined details about the batch as well as all the items in the batch. We're introducing a two-phase batch trigger ingestion system. In the first phase, the BatchTaskRun record is created (and possibly rate limited). The second phase is another endpoint that accepts an NDJSON body with each line being a single item/run with payload and options. At ingestion time all items are added to a queue, in order, and then processed by the BatchQueue system. ## New batch trigger rate limits This PR implements a new batch trigger specific rate limit, configured on the `Organization.batchRateLimitConfig` column, and defaults using these environment variables: - `BATCH_RATE_LIMIT_REFILL_RATE` defaults to 10 - `BATCH_RATE_LIMIT_REFILL_INTERVAL` the duration interval, defaults to `"10s"` - `BATCH_RATE_LIMIT_MAX` defaults to 1200 This rate limiter is scoped to the environment ID and controls how many runs can be submitted via batch triggers per interval. The SDK handles the retrying side. ## Batch queue concurrency limits The new column `Organization.batchQueueConcurrencyConfig` now defines an org specific `processingConcurrency` value, with a backup of the env var `BATCH_CONCURRENCY_LIMIT_DEFAULT` which defaults to 10. This controls how many batch queue items are processed concurrently per environment. There is also a global rate limit for the batch queue set via the `BATCH_QUEUE_GLOBAL_RATE_LIMIT` which defaults to being disabled. If set, the entire batch queue system won't process more than `BATCH_QUEUE_GLOBAL_RATE_LIMIT` items per second. This allows controlling the maximum number of runs created per second via batch triggers. ## Batch trigger settings - `STREAMING_BATCH_MAX_ITEMS` controls the maximum number of items in a single batch - `STREAMING_BATCH_ITEM_MAXIMUM_SIZE` controls the maximum size of each item in a batch - `BATCH_CONCURRENCY_DEFAULT_CONCURRENCY` controls the default environment concurrency - `BATCH_QUEUE_DRR_QUANTUM` how many credits each environment gets each round for the DRR scheduler - `BATCH_QUEUE_MAX_DEFICIT` the maximum deficit for the DRR scheduler - `BATCH_QUEUE_CONSUMER_COUNT` how many queue consumers to run - `BATCH_QUEUE_CONSUMER_INTERVAL_MS` how frequently they poll for items in the queue ### Configuration Recommendations by Use Case High-throughput priority (fairness acceptable at 0.98+): ```env BATCH_QUEUE_DRR_QUANTUM=25 BATCH_QUEUE_MAX_DEFICIT=100 BATCH_QUEUE_CONSUMER_COUNT=10 BATCH_QUEUE_CONSUMER_INTERVAL_MS=50 BATCH_CONCURRENCY_DEFAULT_CONCURRENCY=25 ``` Strict fairness priority (throughput can be lower): ```env BATCH_QUEUE_DRR_QUANTUM=5 BATCH_QUEUE_MAX_DEFICIT=25 BATCH_QUEUE_CONSUMER_COUNT=3 BATCH_QUEUE_CONSUMER_INTERVAL_MS=100 BATCH_CONCURRENCY_DEFAULT_CONCURRENCY=5 ```	2025-12-16 14:32:49 +00:00

1 2 3 4 5 ...

1060 Commits