Example cURL call using an admin user PAT (replace with a real one):
```sh
curl -X PUT https://cloud.trigger.dev/admin/api/v1/environments/<environmentId>/burst-factor \
-H "Authorization: Bearer tr_pat_1234" \
-H "Content-Type: application/json" \
-d '{"burstFactor": 1.5}'
```
Adds a `MicroVM` badge next to the region name on the regions page. Uses
the existing `small` badge variant for visual consistency with the
`Default` badge already on this page.
## Summary
Large error stacks and messages can OOM the worker process when
serialized into OTel spans or `TaskRunError` objects. This was reported
when throwing an error with a massive `.stack` property from a chat
agent hook.
This adds frame-based stack truncation (similar to Sentry's approach)
plus message length limits, applied consistently across all error
serialization paths.
### What changed
**`packages/core/src/v3/errors.ts`**
- `truncateStack()` — parses `error.stack` into message lines + frame
lines, caps at 50 frames (keep top 5 closest to throw + bottom 45 entry
points, with "... N frames omitted ..." in between). Individual lines
capped at 1024 chars.
- `truncateMessage()` — caps error messages at 1000 chars
- Applied in `parseError()` and `sanitizeError()`
**`packages/core/src/v3/otel/utils.ts`**
- `sanitizeSpanError()` now uses `truncateStack` and `truncateMessage`
from `errors.ts` instead of duplicating truncation logic
- Non-Error values (strings, JSON) capped at 5000 chars
**`packages/core/src/v3/tracer.ts`**
- `startActiveSpan` catch block now delegates to `recordSpanException()`
instead of calling `span.recordException()` directly
### Limits
| What | Limit | Rationale |
|------|-------|-----------|
| Stack frames | 50 | Matches Sentry's `STACKTRACE_FRAME_LIMIT` |
| Top frames kept | 5 | Closest to throw site |
| Bottom frames kept | 45 | Entry points / framework frames |
| Per-line length | 1024 | Matches Sentry, prevents regex DoS |
| Message length | 1000 | Bounded but generous |
| Generic string (non-Error) | 5000 | Fallback for JSON/string errors in
spans |
## Test plan
- [x] 17 unit tests in `packages/core/test/errors.test.ts`
- [x] E2E: threw a 300-frame / 5000-char-message error in the ai-chat
reference app, verified truncated stack and message in span via
`get_span_details`
- [x] Verified the run survived the error (no OOM, continued waiting for
next message)
Two changes to cut error volume from logs that represent handled
conditions, not real errors (combined ~1600/hr in prod):
1. api.v1.waitpoints.tokens.$waitpointFriendlyId.complete.ts
The route throws `json(..., { status: 404 })` when a waitpoint
isn't found, but the generic catch block caught that Response,
logged it as an error (with an empty {} body because Error fields
are non-enumerable), and rethrew as a 500 — so clients saw a 500
instead of the intended 404, and every stale-waitpoint request
produced a Sentry event.
Fix: re-throw Response objects unchanged so the correct status
propagates and we don't log user 404s as errors. Also serialize
remaining Error instances explicitly (name/message/stack) so the
logs are actionable when we do hit a real error.
2. v3/marqs/sharedQueueConsumer.server.ts:603
"Task run has invalid status for execution. Going to ack" — the
message itself says we're handling it gracefully. Benign race
between dequeue and completion/cancellation. Demote to warn.
## Summary
Nests the `TaskRunExecutionSnapshot` creation inside the
`taskRun.update()` Prisma call in the dequeue flow, reducing **2 DB
commits → 1** per dequeue operation. This is the highest-volume of the
five unmerged flows identified in TRI-8450 (~9,200 commits/sec on the
engine service).
**Pattern**: Follows the same nested-write approach already used in the
completion path (`runAttemptSystem.ts:735`) and trigger path
(`engine/index.ts:674`).
**Changes**:
- `dequeueSystem.ts`: Moved snapshot creation into `executionSnapshots:
{ create: {...} }` within the existing `taskRun.update()`. Pre-generates
the snapshot ID via `generateInternalId()` (plain cuid, matching what
Prisma's `@default(cuid())` produces) so the event emission, heartbeat
enqueue, and return value can all be constructed from data already in
scope — **no extra DB read needed** after the merged write.
`SnapshotId.toFriendlyId()` is used only for the return value's
`friendlyId` field, matching the original `createExecutionSnapshot`
behavior.
- `executionSnapshotSystem.ts`: Added public
`enqueueHeartbeatIfNeeded()` method that exposes the heartbeat
scheduling logic (previously only available internally via
`createExecutionSnapshot`). This is needed because `PENDING_EXECUTING`
requires a heartbeat, unlike the `FINISHED` status in the completion
reference pattern. This method is reusable by future merge targets
(retry-immediate, checkpoint, cancel, requeue).
**Net DB change per dequeue**: eliminates 1 write transaction (the
separate `TaskRunExecutionSnapshot.create`). No extra reads added — the
snapshot ID is pre-generated and the `executionSnapshotCreated` event
payload is constructed inline from values already available in the
closure.
## Review & Testing Checklist for Human
- [ ] **Verify manually-constructed event payload matches DB state**:
The `executionSnapshotCreated` event is now built inline (not read back
from DB). Confirm the field values (`runStatus: "PENDING"`,
`attemptNumber`, `checkpointId`, `workerId`, `runnerId`,
`completedWaitpointIds`) match what Prisma actually writes. A mismatch
here would be silent — event consumers would get stale/wrong data.
- [ ] **Verify `attemptNumber` source is equivalent**: Old code used
`lockedTaskRun.attemptNumber` (post-update result). New code uses
`result.run.attemptNumber` (pre-update). The `taskRun.update()` data
payload does NOT include `attemptNumber`, so they should be identical —
but confirm this assumption holds for all dequeue scenarios (e.g.
retried runs).
- [ ] **Verify `isValid` defaults to `true` in schema**: The old
`createExecutionSnapshot` explicitly set `isValid: error ? false :
true`. The nested create omits `isValid` (no error in the dequeue happy
path). Confirm the Prisma schema default for
`TaskRunExecutionSnapshot.isValid` is `true`.
- [ ] **Verify `runStatus: "PENDING"` hardcoding matches the mapping**:
The old code passed `lockedTaskRun.status` ("DEQUEUED") to
`createExecutionSnapshot`, which mapped it to "PENDING" via `run.status
=== "DEQUEUED" ? "PENDING" : run.status`. The new code hardcodes
`"PENDING"` directly. This is correct but brittle if `status` ever
changes from "DEQUEUED" to something else upstream.
- [ ] **Spot-check `completedWaitpoints` connect + order logic**: The
nested create replicates the connect/order logic from
`createExecutionSnapshot` (lines 387-393). Verify the
`snapshot.completedWaitpoints` type provides `id` and `index` fields
compatible with this usage.
- [ ] **Verify `checkpoint` in return value**: The return now uses
`snapshot.checkpoint` (from the *previous* snapshot) instead of reading
the newly-created snapshot's checkpoint relation. Since `checkpointId`
is passed through unchanged, they should be identical — but worth a
sanity check.
**Recommended test plan**: deploy to staging, run the
`sample_pg_activity.py` sampler for a 5-minute window, and verify the
COMMIT count drop on the engine service + proportional `IO:XactSync`
reduction.
### Notes
- This only covers the **dequeue** flow (flow #1 from TRI-8450). The
remaining four flows (retry-immediate, checkpoint, requeue, cancel) are
separate follow-ups.
- The new `enqueueHeartbeatIfNeeded` method is deliberately designed for
reuse by those follow-up PRs.
- CI note: the `priority.test.ts` failure in shard 7 is a flaky ordering
assertion unrelated to this change (it compares `friendlyId` values in
dequeue order). The `audit` check is also pre-existing/unrelated.
Link to Devin session:
https://app.devin.ai/sessions/034fe0e7224f49278a2de260203e1377
Requested by: @ericallam
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Eric Allam <eallam@icloud.com>
Wraps getEntitlement in platform.v3.server.ts with the existing
platformCache (LRU memory + Redis) under a new `entitlement` namespace.
Eliminates a synchronous billing-service HTTP round trip on every
trigger.
Cache config: 60s fresh / 60s stale SWR. Cache key is the
organization id. Errors are caught inside the loader and return the
existing permissive { hasAccess: true } fallback, which is also
cached to prevent thundering-herd on billing outages.
Trade-off: plan upgrade/downgrade is now visible after up to ~120s
worst-case (60s fresh + 60s stale revalidation). Acceptable since
the existing limits and usage namespaces use 5min/10min, and the
defensive hasAccess: true fallback already exists.
Replace the expensive DISTINCT query for task filter dropdowns with a
dedicated TaskIdentifier registry table backed by Redis. Environments
migrate automatically on their next deploy, with a transparent fallback
to the legacy query for unmigrated environments. Also fixes duplicate
dropdown entries when a task changes trigger source, and adds
active/archived grouping for removed tasks. Moves BackgroundWorkerTask
reads in the trigger hot path to the read replica.
Pool Redis connections for non-blocking ops (ingestData, appendPart,
getLastChunkIndex)
using a shared singleton instead of new Redis() per request. Use
redis.disconnect()
for immediate teardown in streamResponse cleanup. Add 15s inactivity
timeout fallback.
Fix broken request.signal in Remix/Express by wiring Express
res.on('close') to an
AbortController via httpAsyncStorage. All SSE/streaming routes now use
getRequestAbortSignal() which fires reliably on client disconnect,
bypassing the
Node.js undici GC bug (nodejs/node#55428) that severs the signal chain.
Extends the admin worker groups endpoint with a GET loader and more
fields on POST (type, hidden, workloadType, cloudProvider, location,
staticIPs, enableFastPath), and pulls the PAT + admin check that was
inlined or locally duplicated across every admin.api route into a shared
helper in personalAccessToken.server.ts. The generic
authenticateAdminRequest returns a discriminated result;
requireAdminApiRequest is the thin Remix loader/action wrapper that
throws. The neverthrow-style route (platform-notifications.ts) now
composes the generic helper instead of duplicating the check. Verified
locally against GET (listing) and POST (new fields, invalid enum,
minimal backwards-compat).
## Summary
Upgrades all `@remix-run/*` packages in `apps/webapp` from **2.1.0 →
2.17.4** to address security vulnerabilities. Recreation of #2951 on a
fresh checkout of `main`.
**Updated packages (`apps/webapp/package.json`):**
- `@remix-run/express`, `@remix-run/node`, `@remix-run/react`,
`@remix-run/serve`, `@remix-run/server-runtime`: 2.1.0 → 2.17.4
- `@remix-run/router`: ^1.15.3 → ^1.23.2
- `@remix-run/dev`, `@remix-run/eslint-config`, `@remix-run/testing`:
2.1.0 → 2.17.4
**Root `package.json` overrides:**
- `@remix-run/dev@2.17.4>tar-fs`: 2.1.3 → 2.1.4
- `testcontainers@10.28.0>tar-fs`: 3.0.9 → 3.1.1
**Documentation:** Updated Remix version references in `CLAUDE.md`,
`apps/webapp/CLAUDE.md`, and `.cursor/rules/webapp.mdc`.
**Server changes:** Added `.server-changes/upgrade-remix-security.md`
for release tracking per `CONTRIBUTING.md`.
No application code changes — only `package.json` files, documentation,
a server-changes entry, and the regenerated `pnpm-lock.yaml`.
### Updates since last revision
Addressed all 3 Devin Review findings:
1. **Missing `.server-changes/` file** — added
`.server-changes/upgrade-remix-security.md` (commit ce22a0bd4)
2. **Sentry Remix patch (`@sentry/remix@9.46.0`)** — verified the patch
at `patches/@sentry__remix@9.46.0.patch` applies cleanly against 2.17.4.
The patch modifies Sentry's own `RemixInstrumentation` wrapper (removing
`request.clone()` and form data attributes), not Remix internals. The
underlying Remix APIs it hooks into (`callRouteAction`,
`callRouteLoader`) are stable across 2.1→2.17.
3. **`remix-typedjson@0.3.1` compatibility** — peer deps declare
`@remix-run/react: ^1.16.0 || ^2.0`, covering 2.17.4. Confirmed working
at runtime across all 22 tested pages that use it (root.tsx, hooks,
route loaders).
### Verification performed during this session
- **Runtime:** Express+Remix integration, magic link login, client-side
routing, MetaFunction rendering
- **Operational:** hello-world task triggered via API, runs list, run
detail, tasks page
- **Comprehensive UI:** 22 pages, 11 filter types, environment/project
switchers, interactive elements
- **Docker:** Production Dockerfile (`docker/webapp/Dockerfile`) builds
successfully
- **Changelog audit:** All 16 minor versions reviewed — every breaking
change is behind opt-in future flags the webapp doesn't enable
## Review & Testing Checklist for Human
- [ ] **Verify auth flows in staging** — `remix-auth`,
`remix-auth-email-link`, and `remix-auth-github` declare peer deps on
`@remix-run/server-runtime@^1.x`, which is now 2.17.4. Login (magic link
+ OAuth) should be tested in a staging environment since local dev
testing may not exercise all auth code paths.
- [ ] **Verify tar-fs override versions** resolve the targeted security
advisories (2.1.4 and 3.1.1)
- [ ] **Review new transitive dependencies** added by the upgrade:
`turbo-stream@2.4.1`, `undici@6.25.0`, `valibot@1.3.1`, `ws@7.5.10`
Recommended test plan: deploy to staging and exercise core webapp flows
— login (email magic link + GitHub OAuth), dashboard navigation, task
triggering/viewing, and API endpoints — to catch runtime regressions not
covered by local testing.
### Notes
- Peer dependency warnings for `remix-auth-*` packages (expecting
`@remix-run/server-runtime@^1.x`) were present in the original PR #2951
as well and appear to be pre-existing
- The lockfile diff is large (~1200 lines) but mechanical — driven by
the Remix version bump cascading through transitive dependencies
- CI failures (`audit`, `units/internal/1-of-8`) are unrelated: `audit`
is a `claude-code-action` bot permissions issue; the internal test
failure is a ClickHouse testcontainers `Failed to connect to Reaper`
flake
Link to Devin session:
https://app.devin.ai/sessions/d9fa9953b9bf40e5a8d12b8f5ba5b86b
Requested by: @ericallam
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Eric Allam <eallam@icloud.com>
Adds missing list deployments API page, fixes defaultMachine → machine
in config docs, and clarifies browser CORS usage for wait token
completion with corrected warning placement
The TaskRun.runTags string array already stores tag names, making the
TaskRunTag M2M relation redundant write overhead. Remove createTags
calls, connect: tags, and join table writes from both V1 and V2 trigger
paths. Simplify the add-tags API to just push to runTags directly.
## Summary
12 new features, 59 improvements, 17 bug fixes.
## Highlights
- Add support for setting TTL (time-to-live) defaults at the task level
and globally in trigger.config.ts, with per-trigger overrides still
taking precedence
([#3196](https://github.com/triggerdotdev/trigger.dev/pull/3196))
- Large run outputs can use the new API which allows switching object
storage providers.
([#3275](https://github.com/triggerdotdev/trigger.dev/pull/3275))
## Improvements
- Add platform notifications support to the CLI. The `trigger dev` and
`trigger login` commands now fetch and display platform notifications
(info, warn, error, success) from the server. Includes discovery-based
filtering to conditionally show notifications based on project file
patterns, color markup rendering for styled terminal output, and a
non-blocking display flow with a spinner fallback for slow fetches. Use
`--skip-platform-notifications` flag with `trigger dev` to disable the
notification check.
([#3254](https://github.com/triggerdotdev/trigger.dev/pull/3254))
- Add `get_span_details` MCP tool for inspecting individual spans within
a run trace.
([#3255](https://github.com/triggerdotdev/trigger.dev/pull/3255))
- New `get_span_details` tool returns full span attributes, timing,
events, and AI enrichment (model, tokens, cost, speed)
- Span IDs now shown in `get_run_details` trace output for easy
discovery
- New API endpoint `GET /api/v1/runs/:runId/spans/:spanId`
- New `retrieveSpan()` method on the API client
- `get_query_schema` — discover available TRQL tables and columns
- `query` — execute TRQL queries against your data
- `list_dashboards` — list built-in dashboards and their widgets
- `run_dashboard_query` — execute a single dashboard widget query
- `whoami` — show current profile, user, and API URL
- `list_profiles` — list all configured CLI profiles
- `switch_profile` — switch active profile for the MCP session
- `start_dev_server` — start `trigger dev` in the background and stream
output
- `stop_dev_server` — stop the running dev server
- `dev_server_status` — check dev server status and view recent logs
- `GET /api/v1/query/schema` — query table schema discovery
- `GET /api/v1/query/dashboards` — list built-in dashboards
- `--readonly` flag hides write tools (`deploy`, `trigger_task`,
`cancel_run`) so the AI cannot make changes
- `read:query` JWT scope for query endpoint authorization
- `get_run_details` trace output is now paginated with cursor support
- MCP tool annotations (`readOnlyHint`, `destructiveHint`) for all tools
- `get_query_schema` now requires a table name and returns only one
table's schema (was returning all tables)
- `get_current_worker` no longer inlines payload schemas; use new
`get_task_schema` tool instead
- Query results formatted as text tables instead of JSON (~50% fewer
tokens)
- `cancel_run`, `list_deploys`, `list_preview_branches` formatted as
text instead of raw JSON
- Schema and dashboard API responses cached to avoid redundant fetches
- Adapted the CLI API client to propagate the trigger source via http
headers.
([#3241](https://github.com/triggerdotdev/trigger.dev/pull/3241))
- Propagate run tags to span attributes so they can be extracted
server-side for LLM cost attribution metadata.
([#3213](https://github.com/triggerdotdev/trigger.dev/pull/3213))
- New `get_span_details` tool returns full span attributes, timing,
events, and AI enrichment (model, tokens, cost, speed)
- Span IDs now shown in `get_run_details` trace output for easy
discovery
- New API endpoint `GET /api/v1/runs/:runId/spans/:spanId`
- New `retrieveSpan()` method on the API client
- `get_query_schema` — discover available TRQL tables and columns
- `query` — execute TRQL queries against your data
- `list_dashboards` — list built-in dashboards and their widgets
- `run_dashboard_query` — execute a single dashboard widget query
- `whoami` — show current profile, user, and API URL
- `list_profiles` — list all configured CLI profiles
- `switch_profile` — switch active profile for the MCP session
- `start_dev_server` — start `trigger dev` in the background and stream
output
- `stop_dev_server` — stop the running dev server
- `dev_server_status` — check dev server status and view recent logs
- `GET /api/v1/query/schema` — query table schema discovery
- `GET /api/v1/query/dashboards` — list built-in dashboards
- `--readonly` flag hides write tools (`deploy`, `trigger_task`,
`cancel_run`) so the AI cannot make changes
- `read:query` JWT scope for query endpoint authorization
- `get_run_details` trace output is now paginated with cursor support
- MCP tool annotations (`readOnlyHint`, `destructiveHint`) for all tools
- `get_query_schema` now requires a table name and returns only one
table's schema (was returning all tables)
- `get_current_worker` no longer inlines payload schemas; use new
`get_task_schema` tool instead
- Query results formatted as text tables instead of JSON (~50% fewer
tokens)
- `cancel_run`, `list_deploys`, `list_preview_branches` formatted as
text instead of raw JSON
- Schema and dashboard API responses cached to avoid redundant fetches
- Add optional `hasPrivateLink` field to the dequeue message
organization object for private networking support
([#3264](https://github.com/triggerdotdev/trigger.dev/pull/3264))
- Define and manage AI prompts with `prompts.define()`. Create typesafe
prompt templates with variables, resolve them at runtime, and manage
versions and overrides from the dashboard without redeploying.
([#3244](https://github.com/triggerdotdev/trigger.dev/pull/3244))
## Bug fixes
- Fix dev CLI leaking build directories on rebuild, causing disk space
accumulation. Deprecated workers are now pruned (capped at 2 retained)
when no active runs reference them. The watchdog process also cleans up
`.trigger/tmp/` when the dev CLI is killed ungracefully (e.g. SIGKILL
from pnpm).
([#3224](https://github.com/triggerdotdev/trigger.dev/pull/3224))
- Fix `--load` flag being silently ignored on local/self-hosted builds.
([#3114](https://github.com/triggerdotdev/trigger.dev/pull/3114))
- Fixed `search_docs` tool failing due to renamed upstream Mintlify tool
(`SearchTriggerDev` → `search_trigger_dev`)
- Fixed `list_deploys` failing when deployments have null
`runtime`/`runtimeVersion` fields (#3139)
- Fixed `list_preview_branches` crashing due to incorrect response shape
access
- Fixed `metrics` table column documented as `value` instead of
`metric_value` in query docs
- Fixed dev CLI leaking build directories on rebuild — deprecated
workers now clean up their build dirs when their last run completes
- Fixed `search_docs` tool failing due to renamed upstream Mintlify tool
(`SearchTriggerDev` → `search_trigger_dev`)
- Fixed `list_deploys` failing when deployments have null
`runtime`/`runtimeVersion` fields (#3139)
- Fixed `list_preview_branches` crashing due to incorrect response shape
access
- Fixed `metrics` table column documented as `value` instead of
`metric_value` in query docs
- Fixed dev CLI leaking build directories on rebuild — deprecated
workers now clean up their build dirs when their last run completes
## Server changes
These changes affect the self-hosted Docker image and Trigger.dev Cloud:
- Add admin UI for viewing and editing feature flags (org-level
overrides and global defaults).
([#3291](https://github.com/triggerdotdev/trigger.dev/pull/3291))
- AI prompt management dashboard and enhanced span inspectors.
**Prompt management:**
- Prompts list page with version status, model, override indicators, and
24h usage sparklines
- Prompt detail page with template viewer, variable preview, version
history timeline, and override editor
- Create, edit, and remove overrides to change prompt content or model
without redeploying
- Promote any code-deployed version to current
- Generations tab with infinite scroll, live polling, and inline span
inspector
- Per-prompt metrics: total generations, avg tokens, avg cost, latency,
with version-level breakdowns
**AI span inspectors:**
- Custom inspectors for `ai.generateText`, `ai.streamText`,
`ai.generateObject`, `ai.streamObject` parent spans
- `ai.toolCall` inspector showing tool name, call ID, and input
arguments
- `ai.embed` inspector showing model, provider, and input text
- Prompt tab on AI spans linking to prompt version with template and
input variables
- Compact timestamp and duration header on all AI span inspectors
**AI metrics dashboard:**
- Operations, Providers, and Prompts filters on the AI Metrics dashboard
- Cost by prompt widget
- "AI" section in the sidebar with Prompts and AI Metrics links
**Other improvements:**
- Resizable panel sizes now persist across page refreshes
- Fixed `<div>` inside `<p>` DOM nesting warnings in span titles and
chat messages
([#3244](https://github.com/triggerdotdev/trigger.dev/pull/3244))
- Add allowRollbacks query param to the promote deployment API to enable
version downgrades
([#3214](https://github.com/triggerdotdev/trigger.dev/pull/3214))
- Pre-warm compute templates on deploy for orgs with compute access.
Required for projects using a compute region, background-only for
others.
([#3114](https://github.com/triggerdotdev/trigger.dev/pull/3114))
- Add automatic LLM cost calculation for spans with GenAI semantic
conventions. When a span arrives with `gen_ai.response.model` and token
usage data, costs are calculated from an in-memory pricing registry
backed by Postgres and dual-written to both span attributes
(`trigger.llm.*`) and a new `llm_metrics_v1` ClickHouse table that
captures usage, cost, performance (TTFC, tokens/sec), and behavioral
(finish reason, operation type) metrics.
([#3213](https://github.com/triggerdotdev/trigger.dev/pull/3213))
- Add API endpoint `GET /api/v1/runs/:runId/spans/:spanId` that returns
detailed span information including properties, events, AI enrichment
(model, tokens, cost), and triggered child runs.
([#3255](https://github.com/triggerdotdev/trigger.dev/pull/3255))
- Multi-provider object storage with protocol-based routing for
zero-downtime migration
([#3275](https://github.com/triggerdotdev/trigger.dev/pull/3275))
- Add IAM role-based auth support for object stores (no access keys
required).
([#3275](https://github.com/triggerdotdev/trigger.dev/pull/3275))
- Add platform notifications to inform users about new features,
changelogs, and platform events directly in the dashboard.
([#3254](https://github.com/triggerdotdev/trigger.dev/pull/3254))
- Add private networking support via AWS PrivateLink. Includes
BillingClient methods for managing private connections, org settings UI
pages for connection management, and supervisor changes to apply
`privatelink` pod labels for CiliumNetworkPolicy matching.
([#3264](https://github.com/triggerdotdev/trigger.dev/pull/3264))
- Reduce run start latency by skipping the intermediate queue when
concurrency is available. This optimization is rolled out per-region and
enabled automatically for development environments.
([#3299](https://github.com/triggerdotdev/trigger.dev/pull/3299))
- Extended the search filter on the environment variables page to match
on environment type (production, staging, development, preview) and
branch name, not just variable name and value.
([#3302](https://github.com/triggerdotdev/trigger.dev/pull/3302))
- Set `application_name` on Prisma connections from SERVICE_NAME so DB
load can be attributed by service
([#3348](https://github.com/triggerdotdev/trigger.dev/pull/3348))
- Fix transient R2/object store upload failures during batchTrigger()
item streaming.
- Added p-retry (3 attempts, 500ms–2s exponential backoff) around
`uploadPacketToObjectStore` in `BatchPayloadProcessor.process()` so
transient network errors self-heal server-side rather than aborting the
entire batch stream.
- Removed `x-should-retry: false` from the 500 response on the batch
items route so the SDK's existing 5xx retry path can recover if
server-side retries are exhausted. Item deduplication by index makes
full-stream retries safe.
([#3331](https://github.com/triggerdotdev/trigger.dev/pull/3331))
- Concurrency-keyed queues now use a single master queue entry per base
queue instead of one entry per key. Prevents high-CK-count tenants from
consuming the entire parentQueueLimit window and starving other tenants
on the same shard.
([#3219](https://github.com/triggerdotdev/trigger.dev/pull/3219))
- Reduce lock contention when processing large `batchTriggerAndWait`
batches. Previously, each batch item acquired a Redis lock on the parent
run to insert a `TaskRunWaitpoint` row, causing
`LockAcquisitionTimeoutError` with high concurrency (880 errors/24h in
prod). Since `blockRunWithCreatedBatch` already transitions the parent
to `EXECUTING_WITH_WAITPOINTS` before items are processed, the per-item
lock is unnecessary. The new `blockRunWithWaitpointLockless` method
performs only the idempotent CTE insert without acquiring the lock.
([#3232](https://github.com/triggerdotdev/trigger.dev/pull/3232))
- Strip `secure` query parameter from QUERY_CLICKHOUSE_URL before
passing to ClickHouse client. This was already done for the main and
logs ClickHouse clients but was missing for the query client, causing a
startup crash with `Error: Unknown URL parameters: secure`.
([#3204](https://github.com/triggerdotdev/trigger.dev/pull/3204))
- Fix `OrganizationsPresenter.#getEnvironment` matching the wrong
development environment on teams with multiple members. All dev
environments share the slug `"dev"`, so the previous `find` by slug
alone could return another member's environment. Now filters DEVELOPMENT
environments by `orgMember.userId` to ensure the logged-in user's dev
environment is selected.
([#3273](https://github.com/triggerdotdev/trigger.dev/pull/3273))
<details>
<summary>Raw changeset output</summary>
# Releases
## @trigger.dev/build@4.4.4
### Patch Changes
- Updated dependencies:
- `@trigger.dev/core@4.4.4`
## trigger.dev@4.4.4
### Patch Changes
- Add platform notifications support to the CLI. The `trigger dev` and
`trigger login` commands now fetch and display platform notifications
(info, warn, error, success) from the server. Includes discovery-based
filtering to conditionally show notifications based on project file
patterns, color markup rendering for styled terminal output, and a
non-blocking display flow with a spinner fallback for slow fetches. Use
`--skip-platform-notifications` flag with `trigger dev` to disable the
notification check.
([#3254](https://github.com/triggerdotdev/trigger.dev/pull/3254))
- Fix dev CLI leaking build directories on rebuild, causing disk space
accumulation. Deprecated workers are now pruned (capped at 2 retained)
when no active runs reference them. The watchdog process also cleans up
`.trigger/tmp/` when the dev CLI is killed ungracefully (e.g. SIGKILL
from pnpm).
([#3224](https://github.com/triggerdotdev/trigger.dev/pull/3224))
- Fix `--load` flag being silently ignored on local/self-hosted builds.
([#3114](https://github.com/triggerdotdev/trigger.dev/pull/3114))
- Add `get_span_details` MCP tool for inspecting individual spans within
a run trace.
([#3255](https://github.com/triggerdotdev/trigger.dev/pull/3255))
- New `get_span_details` tool returns full span attributes, timing,
events, and AI enrichment (model, tokens, cost, speed)
- Span IDs now shown in `get_run_details` trace output for easy
discovery
- New API endpoint `GET /api/v1/runs/:runId/spans/:spanId`
- New `retrieveSpan()` method on the API client
- MCP server improvements: new tools, bug fixes, and new flags.
([#3224](https://github.com/triggerdotdev/trigger.dev/pull/3224))
**New tools:**
- `get_query_schema` — discover available TRQL tables and columns
- `query` — execute TRQL queries against your data
- `list_dashboards` — list built-in dashboards and their widgets
- `run_dashboard_query` — execute a single dashboard widget query
- `whoami` — show current profile, user, and API URL
- `list_profiles` — list all configured CLI profiles
- `switch_profile` — switch active profile for the MCP session
- `start_dev_server` — start `trigger dev` in the background and stream
output
- `stop_dev_server` — stop the running dev server
- `dev_server_status` — check dev server status and view recent logs
**New API endpoints:**
- `GET /api/v1/query/schema` — query table schema discovery
- `GET /api/v1/query/dashboards` — list built-in dashboards
**New features:**
- `--readonly` flag hides write tools (`deploy`, `trigger_task`,
`cancel_run`) so the AI cannot make changes
- `read:query` JWT scope for query endpoint authorization
- `get_run_details` trace output is now paginated with cursor support
- MCP tool annotations (`readOnlyHint`, `destructiveHint`) for all tools
**Bug fixes:**
- Fixed `search_docs` tool failing due to renamed upstream Mintlify tool
(`SearchTriggerDev` → `search_trigger_dev`)
- Fixed `list_deploys` failing when deployments have null
`runtime`/`runtimeVersion` fields (#3139)
- Fixed `list_preview_branches` crashing due to incorrect response shape
access
- Fixed `metrics` table column documented as `value` instead of
`metric_value` in query docs
- Fixed dev CLI leaking build directories on rebuild — deprecated
workers now clean up their build dirs when their last run completes
**Context optimizations:**
- `get_query_schema` now requires a table name and returns only one
table's schema (was returning all tables)
- `get_current_worker` no longer inlines payload schemas; use new
`get_task_schema` tool instead
- Query results formatted as text tables instead of JSON (~50% fewer
tokens)
- `cancel_run`, `list_deploys`, `list_preview_branches` formatted as
text instead of raw JSON
- Schema and dashboard API responses cached to avoid redundant fetches
- Add support for setting TTL (time-to-live) defaults at the task level
and globally in trigger.config.ts, with per-trigger overrides still
taking precedence
([#3196](https://github.com/triggerdotdev/trigger.dev/pull/3196))
- Adapted the CLI API client to propagate the trigger source via http
headers.
([#3241](https://github.com/triggerdotdev/trigger.dev/pull/3241))
- Updated dependencies:
- `@trigger.dev/core@4.4.4`
- `@trigger.dev/build@4.4.4`
- `@trigger.dev/schema-to-json@4.4.4`
## @trigger.dev/core@4.4.4
### Patch Changes
- Fix `list_deploys` MCP tool failing when deployments have null
`runtime` or `runtimeVersion` fields.
([#3224](https://github.com/triggerdotdev/trigger.dev/pull/3224))
- Propagate run tags to span attributes so they can be extracted
server-side for LLM cost attribution metadata.
([#3213](https://github.com/triggerdotdev/trigger.dev/pull/3213))
- Add `get_span_details` MCP tool for inspecting individual spans within
a run trace.
([#3255](https://github.com/triggerdotdev/trigger.dev/pull/3255))
- New `get_span_details` tool returns full span attributes, timing,
events, and AI enrichment (model, tokens, cost, speed)
- Span IDs now shown in `get_run_details` trace output for easy
discovery
- New API endpoint `GET /api/v1/runs/:runId/spans/:spanId`
- New `retrieveSpan()` method on the API client
- MCP server improvements: new tools, bug fixes, and new flags.
([#3224](https://github.com/triggerdotdev/trigger.dev/pull/3224))
**New tools:**
- `get_query_schema` — discover available TRQL tables and columns
- `query` — execute TRQL queries against your data
- `list_dashboards` — list built-in dashboards and their widgets
- `run_dashboard_query` — execute a single dashboard widget query
- `whoami` — show current profile, user, and API URL
- `list_profiles` — list all configured CLI profiles
- `switch_profile` — switch active profile for the MCP session
- `start_dev_server` — start `trigger dev` in the background and stream
output
- `stop_dev_server` — stop the running dev server
- `dev_server_status` — check dev server status and view recent logs
**New API endpoints:**
- `GET /api/v1/query/schema` — query table schema discovery
- `GET /api/v1/query/dashboards` — list built-in dashboards
**New features:**
- `--readonly` flag hides write tools (`deploy`, `trigger_task`,
`cancel_run`) so the AI cannot make changes
- `read:query` JWT scope for query endpoint authorization
- `get_run_details` trace output is now paginated with cursor support
- MCP tool annotations (`readOnlyHint`, `destructiveHint`) for all tools
**Bug fixes:**
- Fixed `search_docs` tool failing due to renamed upstream Mintlify tool
(`SearchTriggerDev` → `search_trigger_dev`)
- Fixed `list_deploys` failing when deployments have null
`runtime`/`runtimeVersion` fields (#3139)
- Fixed `list_preview_branches` crashing due to incorrect response shape
access
- Fixed `metrics` table column documented as `value` instead of
`metric_value` in query docs
- Fixed dev CLI leaking build directories on rebuild — deprecated
workers now clean up their build dirs when their last run completes
**Context optimizations:**
- `get_query_schema` now requires a table name and returns only one
table's schema (was returning all tables)
- `get_current_worker` no longer inlines payload schemas; use new
`get_task_schema` tool instead
- Query results formatted as text tables instead of JSON (~50% fewer
tokens)
- `cancel_run`, `list_deploys`, `list_preview_branches` formatted as
text instead of raw JSON
- Schema and dashboard API responses cached to avoid redundant fetches
- Large run outputs can use the new API which allows switching object
storage providers.
([#3275](https://github.com/triggerdotdev/trigger.dev/pull/3275))
- Add optional `hasPrivateLink` field to the dequeue message
organization object for private networking support
([#3264](https://github.com/triggerdotdev/trigger.dev/pull/3264))
- Add support for setting TTL (time-to-live) defaults at the task level
and globally in trigger.config.ts, with per-trigger overrides still
taking precedence
([#3196](https://github.com/triggerdotdev/trigger.dev/pull/3196))
- Adapted the CLI API client to propagate the trigger source via http
headers.
([#3241](https://github.com/triggerdotdev/trigger.dev/pull/3241))
## @trigger.dev/python@4.4.4
### Patch Changes
- Updated dependencies:
- `@trigger.dev/sdk@4.4.4`
- `@trigger.dev/core@4.4.4`
- `@trigger.dev/build@4.4.4`
## @trigger.dev/react-hooks@4.4.4
### Patch Changes
- Updated dependencies:
- `@trigger.dev/core@4.4.4`
## @trigger.dev/redis-worker@4.4.4
### Patch Changes
- Adapted the CLI API client to propagate the trigger source via http
headers.
([#3241](https://github.com/triggerdotdev/trigger.dev/pull/3241))
- Updated dependencies:
- `@trigger.dev/core@4.4.4`
## @trigger.dev/rsc@4.4.4
### Patch Changes
- Updated dependencies:
- `@trigger.dev/core@4.4.4`
## @trigger.dev/schema-to-json@4.4.4
### Patch Changes
- Updated dependencies:
- `@trigger.dev/core@4.4.4`
## @trigger.dev/sdk@4.4.4
### Patch Changes
- Define and manage AI prompts with `prompts.define()`. Create typesafe
prompt templates with variables, resolve them at runtime, and manage
versions and overrides from the dashboard without redeploying.
([#3244](https://github.com/triggerdotdev/trigger.dev/pull/3244))
- Add support for setting TTL (time-to-live) defaults at the task level
and globally in trigger.config.ts, with per-trigger overrides still
taking precedence
([#3196](https://github.com/triggerdotdev/trigger.dev/pull/3196))
- Adapted the CLI API client to propagate the trigger source via http
headers.
([#3241](https://github.com/triggerdotdev/trigger.dev/pull/3241))
- Updated dependencies:
- `@trigger.dev/core@4.4.4`
</details>
---------
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Adds region-level gating so MICROVM regions are only visible and usable
by orgs with the `hasComputeAccess` feature flag. Admins and explicit
allowlist behavior unchanged.
- New shared helper (`regionAccess.server.ts`) with
`resolveComputeAccess`, `defaultVisibilityFilter`, and
`isComputeRegionAccessible`
- `RegionsPresenter` filters out MICROVM regions for non-compute orgs
- `SetDefaultRegionService` blocks setting a MICROVM region as default
without compute access
- `WorkerGroupService` blocks triggering runs in MICROVM regions without
compute access
- `computeTemplateCreation` refactored to use shared
`resolveComputeAccess`
- Updated snapshot callback schema
Sets `application_name` on the Prisma writer and replica connection
strings using the existing `SERVICE_NAME` env var, so DB load can be
attributed by service.
A single "fetch failed" from the object store was aborting the entire
batch stream with no retry. Added p-retry (3 attempts, 500ms-2s backoff)
around ploadPacketToObjectStore so transient network errors self-heal
server-side instead of propagating to the SDK.
### Text wrapping fix
- Fixes message text not wrapping on the run inspector if there were no
spaces in the text
- Fixes inspector title truncation
- Adds a copy text button for the Message property
<img width="468" height="740" alt="CleanShot 2026-04-04 at 10 19 02@2x"
src="https://github.com/user-attachments/assets/71e42bf3-d103-44a2-b3b4-937c0b60a4bc"
/>
This is a small improvement mainly with the UI Skills file:
- Animate open and close the Resizable panels
- Uses the built in animation hooks from react-window-splitter
- Includes a global variable for the animation easing and timing for
consistency
https://github.com/user-attachments/assets/50ed0019-ed12-4e08-b95c-7c6d1fe5bac0
## Summary
- Drop all 8 foreign key constraints on TaskRun. The run listing path is
now fully ClickHouse-backed so we no longer need Postgres to enforce
referential integrity on this table. The FK constraints add write
overhead on every insert/update with no remaining benefit. Prisma
queries are unaffected.
- Remove PostgresRunsRepository and its associated feature flag
(runsListRepository), which was the last remaining code path querying
TaskRun directly for list/count operations.
- Drop three indexes that were only useful for the Postgres run list
path and have no remaining query consumers:
- TaskRun_runtimeEnvironmentId_id_idx — was the cursor pagination index
for PostgresRunsRepository; superseded by the (runtimeEnvironmentId,
createdAt DESC) composite index
- TaskRun_scheduleId_idx — redundant with the (scheduleId, createdAt
DESC) composite index; no direct Postgres queries filter by scheduleId
alone
- TaskRun_rootTaskRunId_idx — no queries filter TaskRun by rootTaskRunId
as a WHERE clause anywhere in the codebase
All index drops use CONCURRENTLY IF EXISTS to avoid table locks in
production.
## Test plan
- pnpm run db:migrate:deploy applies all migrations cleanly
- pnpm run typecheck --filter webapp passes
- Run list pages load correctly in the dashboard (ClickHouse path)
- Scheduled task runs still trigger and appear correctly
This allows seamless migration to different object storage.
Existing runs that have offloaded payloads/outputs will continue to use
the default object store (configured using `OBJECT_STORE_*` env vars).
You can add additional stores by setting new env vars:
- `OBJECT_STORE_DEFAULT_PROTOCOL` this determines where new run large
payloads will get stored.
- If you set that you need to set new env vars for that protocol.
Example:
```
OBJECT_STORE_DEFAULT_PROTOCOL=“s3"
OBJECT_STORE_S3_BASE_URL=https://s3.us-east-1.amazonaws.com
OBJECT_STORE_S3_ACCESS_KEY_ID=<val>
OBJECT_STORE_S3_SECRET_ACCESS_KEY=<val>
OBJECT_STORE_S3_REGION=us-east-1
OBJECT_STORE_S3_SERVICE=s3
```
---------
Co-authored-by: nicktrn <55853254+nicktrn@users.noreply.github.com>
Adds a migration reference for users moving from n8n to Trigger.dev.
Includes a concept map, four common patterns covering the
migration-specific gaps, and a full customer onboarding example. The
onboarding workflow highlights the 3-day wait pattern, an area where
n8n's execution model has known reliability issues at production scale
that Trigger.dev handles natively
- Added versions filtering on the Errors list and page
- Added errors stacked bars to the graph on the individual error page
---------
Co-authored-by: James Ritchie <james@trigger.dev>
The @internal/compute package had its main/types pointing to
./src/index.ts with no build step. This works in dev (tsc resolves .ts
at compile time) but fails at runtime in Docker because Node.js can't
load .ts files directly.
Added tsconfig.build.json and build/clean/dev scripts matching the
pattern used by schedule-engine and other internal packages. Exports now
point to dist/.
Temporary workaround that enables filtering by environment in the
envvars page, without changing any UI.
---------
Co-authored-by: Claude <noreply@anthropic.com>
## Summary
Currently, every triggered run follows a two-step path through Redis:
1. **Enqueue** — A Lua script atomically adds the message to a queue
sorted set (ordered by priority-adjusted timestamp)
2. **Dequeue** — A debounced `processQueueForWorkerQueue` job fires
~500ms later, checks concurrency limits, removes the message from the
sorted set, and pushes it to a worker queue (Redis list) where workers
pick it up via `BLPOP`
This means every run pays at least ~500ms of latency between being
triggered and being available for a worker to execute, even when the
queue is empty and concurrency is wide open.
### What changed
The enqueue Lua scripts now atomically decide whether to **skip the
queue sorted set entirely** and push directly to the worker queue. This
happens inside the same Lua script that handles normal enqueue, so the
decision is atomic with respect to concurrency bookkeeping.
A run takes the **fast path** when all of these are true:
- **Fast path is enabled** for this worker queue (gated per
`WorkerInstanceGroup`)
- **No available messages** in the queue (`ZRANGEBYSCORE` finds nothing
with score ≤ now) — this respects priority ordering and allows fast path
even when the queue has future-scored messages (e.g. nacked retries with
delay)
- **Environment concurrency** has capacity
- **Queue concurrency** has capacity (including per-concurrency-key
limits for CK queues)
When the fast path is taken:
- The message is stored and pushed directly to the worker queue
(`RPUSH`)
- Concurrency slots are claimed (`SADD` to the same sets used by the
normal dequeue path)
- The `processQueueForWorkerQueue` job is **not scheduled** (no work to
do)
- TTL sorted set is skipped (the `expireRun` worker job handles TTL
independently)
When any condition fails, the existing slow path runs unchanged.
### Rollout gating
- **Development environments**: Fast path is always enabled
- **Production environments**: Gated by a new `enableFastPath` boolean
on `WorkerInstanceGroup` (defaults to `false`), allowing
region-by-region rollout
### Rolling deploy safety
Each process registers its own Lua scripts via `defineCommand`
(identified by SHA hash). Old and new processes never share scripts. The
Redis data structures are fully compatible in both directions — ack,
nack, and release operations work identically regardless of which path a
message took.
## Test plan
- [x] Fast path taken when queue is empty and concurrency available
- [x] Slow path when `enableFastPath` is false
- [x] Slow path when queue has available messages (respects priority
ordering)
- [x] Fast path when queue only has future-scored messages
- [x] Slow path when env concurrency is full
- [x] Fast-path message can be acknowledged correctly
- [x] Fast-path message can be nacked and re-enqueued to the queue
sorted set
- [x] Run all existing run-queue tests (ack, nack, CK, concurrency
sweeper, dequeue) to verify no regressions
- [x] Typecheck passes for run-engine and webapp
Add TTL (time-to-live) defaults at task-level and config-level, with
precedence: per-trigger > task > config > dev default (10m).
Docs PR: #3200 (merge after packages are released)
- Rebuild llm_pricing_tiers and llm_prices in syncLlmCatalog for
source=default
- Add vitest config, sync regression tests, and pin vitest 3.1.4
- Update pnpm-lock.yaml for the new devDependency
Adds a dialog to the admin orgs page for viewing and editing per-org
feature flag overrides. Flags are introspected from the catalog so the
UI stays in sync with available flags automatically. Also adds a new tab
for global flags.
Refactors featureFlags.server.ts to split catalog definition (shared)
from server-only runtime (flags(), makeSetMultipleFlags). The shared
module exports flag metadata and validation so both the UI and API
routes can use it without pulling in server dependencies.
Adds support for taint tolerations for scheduled runs. Useful for
selectively tolerating taints on dedicated node pools.
The new `KUBERNETES_SCHEDULED_RUN_TOLERATIONS` env variable accepts a
comma-separated list in the format key=value:effect (or key:effect for
the Exists operator).
Drive-by: renames all `KUBERNETES_SCHEDULE_*` affinity env vars to
KUBERNETES_SCHEDULED_RUN_* for clarity — this feature isn't used in
production yet or published in a tagged image; the name change is fine.
For human reviewer:
- Check if Redis connection + code makes sense
- Check CLI methods (it's on a hotpath)
- Check DB Migrations and new tables
## ✅ Checklist
- [x] I have followed every step in the [contributing
guide](https://github.com/triggerdotdev/trigger.dev/blob/main/CONTRIBUTING.md)
- [x] The PR title follows the convention.
- [x] I ran and tested the code works
---
## Testing
Spawning new CLI / Dashboard notifications, check MVP, check if failures
not produce any problems with CLI/Dashboard
---
## Changelog
Added notifications mechanism for Dashboard and CLI
---
## Screenshots
💯
Adds three new top-level columns to the ClickHouse task_runs_v2 table
primarily for analytics:
- `trigger_source` / `root_trigger_source` - extracted from the existing
TaskRun.annotations JSON during WAL
replication
- `is_warm_start` - new nullable boolean on TaskRun in Postgres, set in
the existing taskRun.update() at attempt
start (no additional write). null until the first attempt starts.
Run region is already available via the existing `worker_queue` column
in ClickHouse.
Scheduled runs create predictable hourly spikes that compete with
on-demand runs for node capacity. Runs triggered "on-demand" via the
SDK, API, or dashboard, are more sensitive to cold start latency since
users are typically
waiting on the result. When a burst of scheduled runs lands at the top
of the hour, it can saturate the shared pool resources causing
contention, affecting cold starts across the board.
The idea in this change is to absorb these periodic spikes in a
dedicated pool without affecting the cold starts of on-demand runs.
Scheduled runs are inherently less sensitive to cold starts.
### Changes in this PR
Follows up on run annotations (#3241), which made trigger origin
available on every run in the tree. This PR exposes
annotations at dequeue time to the supervisor. This enables scheduling
decisions based on trigger source.
The affinities are soft preferences at schedule time, so runs fall back
gracefully if the target pool is out out of capacity.
Queue limit ServiceValidationErrors were being logged at error level.
These are
expected validation rejections, not bugs.
- Add logLevel property to ServiceValidationError (webapp + run-engine)
- Set logLevel: warn on all queue limit throws
- Schedule engine: detect queue limit failures and log as warn
- Redis-worker: respect logLevel on thrown errors
Adds an `annotations` JSONB column to task runs that captures where and
how each run was triggered.
This enables filtering and analyzing trigger origins without querying up
the run tree. Also enables making scheduling decisions based on the
trigger source, e.g., use separate affinities for scheduled runs.
Each run records:
- **triggerSource**: who initiated it (sdk, api, dashboard, cli, mcp,
schedule)
- **triggerAction**: what kind of action (trigger, replay, test)
- **rootTriggerSource**: the trigger source of the root ancestor,
propagated through the entire run
tree
- **rootScheduleId**: schedule id, in case the run tree was triggered
from a schedule
Currently the main motivation for annotations it to determine whether a
run is part of a schedule-originated tree without traversing ancestors.
### A couple of design considerations
- **Decoupled source from method**: triggerSource and triggerAction are
separate fields to avoid
combinatorial explosion (every new source × every new action)
- **Server-side first**: all annotation values are primarily determined
on the server, only a minor SDK change needed
- **Forward-compatible**: annotation fields use
`z.enum([...]).or(anyString)` so new values can be
added without breaking validation; we currently don't need an explicit
version field for annotations.
Note: `metadata` would have been a more fitting name for the db column,
as it is consistent with other tables where we store this type of
information. It is already in use to store user metadata though, so we
go with `annotations` instead.
- Full prompt management UI: list, detail, override, and version
management for AI prompts defined with `prompts.define()`
- Rich AI span inspectors for all AI SDK operations with token usage,
messages, and prompt context
- Real-time generation tracking with live polling and filtering
## Prompt management
Define prompts in your code with `prompts.define()`, then manage
versions and overrides from the dashboard without redeploying:
```typescript
import { task, prompts } from "@trigger.dev/sdk";
import { generateText } from "ai";
import { openai } from "@ai-sdk/openai";
import { z } from "zod";
const supportPrompt = prompts.define({
id: "customer-support",
model: "gpt-4o",
variables: z.object({
customerName: z.string(),
plan: z.string(),
issue: z.string(),
}),
content: `You are a support agent for Acme SaaS.
Customer: {{customerName}} ({{plan}} plan)
Issue: {{issue}}
Respond with empathy and precision.`,
});
export const supportTask = task({
id: "handle-support",
run: async (payload) => {
const resolved = await supportPrompt.resolve({
customerName: payload.name,
plan: payload.plan,
issue: payload.issue,
});
const result = await generateText({
model: openai(resolved.model ?? "gpt-4o"),
system: resolved.text,
prompt: payload.issue,
...resolved.toAISDKTelemetry(),
});
return { response: result.text };
},
});
```
The prompts list page shows each prompt with its current version, model,
override status, and a usage sparkline over the last 24 hours.
From the prompt detail page you can:
- **Create overrides** to change the prompt template or model without
redeploying. Overrides take priority over the deployed version when
`prompt.resolve()` is called.
- **Promote** any code-deployed version to be the current version
- **Browse generations** across all versions with infinite scroll and
live polling for new results
- **Filter** by version, model, operation type, and provider
- **View metrics** (total generations, avg tokens, avg cost, latency)
broken down by version
## AI span inspectors
Every AI SDK operation now gets a custom inspector in the run trace
view:
- **`ai.generateText` / `ai.streamText`** — Shows model, token usage,
cost, the full message thread (system prompt, user message, assistant
response), and linked prompt details
- **`ai.generateObject` / `ai.streamObject`** — Same as above plus the
JSON schema and structured output
- **`ai.toolCall`** — Shows tool name, call ID, and input arguments
- **`ai.embed`** — Shows model and the text being embedded
For generation spans linked to a prompt, a "Prompt" tab shows the prompt
metadata, the input variables passed to `resolve()`, and the template
content from the prompt version.
All AI span inspectors include a compact timestamp and duration header.
## Other improvements
- Resizable panel sizes now persist across page refreshes (patched
`@window-splitter/state` to fix snapshot restoration)
- Run page panels also persist their sizes
- Fixed `<div>` inside `<p>` DOM nesting warnings in span titles and
chat messages
- Added Operations and Providers filters to the AI metrics dashboard
## Screenshots
<img width="3680" height="2392" alt="CleanShot 2026-03-21 at 10 14
17@2x"
src="https://github.com/user-attachments/assets/f3e59989-a2fa-4990-a9d0-3cacda431868"
/>
<img width="3680" height="2392" alt="CleanShot 2026-03-21 at 10 15
37@2x"
src="https://github.com/user-attachments/assets/2f2d02df-2d2b-44fb-ac6f-9153f6a6c387"
/>
<img width="3680" height="2392" alt="CleanShot 2026-03-21 at 10 15
54@2x"
src="https://github.com/user-attachments/assets/baa161e0-ef91-4fa4-a55f-986b71cccdf0"
/>
- Automatic LLM cost enrichment for AI SDK spans (streamText,
generateText, generateObject) or any other spans that use semantic
gen_ai attributes with support for 145+ models
- New AI span inspector sidebar showing model, tokens, cost, messages,
tool calls, and response text
- LLM metrics dual-write to ClickHouse `llm_metrics_v1` table for
analytics
- LLM metrics built-in dashboard (unlinked at the moment)
- Provider cost fallback — uses gateway/OpenRouter reported costs from
`providerMetadata` when registry pricing is unavailable
- Prefix-stripping for gateway/OpenRouter model names (e.g.
`mistral/mistral-large-3` matches `mistral-large-3` pricing)
- Admin dashboard for managing LLM model pricing (list, create, edit,
delete, search, test pattern matching)
- Missing models detection page — queries ClickHouse for unpriced models
with sample spans and Claude Code-ready prompts for adding pricing
- AI span seed script (`pnpm run db:seed:ai-spans`) with 51 spans across
12 provider systems for local dev testing
- UI fixes: `completionTokens`/`promptTokens` aliases,
`ai.response.object` display for generateObject, cache read/write token
breakdown
## Screenshots:
<img width="1030" height="104" alt="CleanShot 2026-03-17 at 16 48 54@2x"
src="https://github.com/user-attachments/assets/bc8fccda-e48b-4d0c-bfb1-e620064e5979"
/>
<img width="1094" height="1512" alt="CleanShot 2026-03-17 at 16 49
23@2x"
src="https://github.com/user-attachments/assets/c2424569-d07e-4d67-a436-e8250043a1ee"
/>
<img width="1074" height="1412" alt="CleanShot 2026-03-17 at 16 49
18@2x"
src="https://github.com/user-attachments/assets/22342ac4-4769-45d1-a328-a24fb9a82a50"
/>
<img width="1012" height="2292" alt="CleanShot 2026-03-17 at 16 39
01@2x"
src="https://github.com/user-attachments/assets/59e327d1-6652-4293-8be0-bb8326e5fbc5"
/>
<img width="3680" height="2392" alt="CleanShot 2026-03-15 at 08 29
38@2x"
src="https://github.com/user-attachments/assets/1f77beb8-de67-495b-b890-bcdb8d7f1fe8"
/>
---------
Co-authored-by: James Ritchie <james@trigger.dev>
When processing batchTriggerAndWait items, each batch item was acquiring
a
Redis lock on the parent run to insert a TaskRunWaitpoint row. With high
concurrency (processingConcurrency=50), this caused
LockAcquisitionTimeoutError
(880 errors/24h in prod), orphaned runs, and stuck parent runs.
Since blockRunWithCreatedBatch already transitions the parent to
EXECUTING_WITH_WAITPOINTS before items are processed, the per-item lock
is
unnecessary. The new blockRunWithWaitpointLockless method performs only
the
idempotent CTE insert and timeout scheduling without acquiring the lock.
## Summary
Major expansion of the MCP server (14 → 25 tools), context efficiency
optimizations, new API endpoints, and a fix for the dev CLI leaking
build directories on disk.
### New MCP tools
- **Query & analytics**: `get_query_schema`, `query`, `list_dashboards`,
`run_dashboard_query` — query your data using TRQL directly from AI
assistants
- **Profile management**: `whoami`, `list_profiles`, `switch_profile` —
see and switch CLI profiles per-project (persisted to
`.trigger/mcp.json`)
- **Dev server control**: `start_dev_server`, `stop_dev_server`,
`dev_server_status` — start/stop `trigger dev` and stream build output
- **Task introspection**: `get_task_schema` — get payload schema for a
specific task (split out from `get_current_worker` to reduce context)
### New API endpoints
- `GET /api/v1/query/schema` — discover TRQL tables and columns
(server-driven, multi-table)
- `GET /api/v1/query/dashboards` — list built-in dashboard widgets and
their queries
### New features
- **`--readonly` flag** — hides write tools (`deploy`, `trigger_task`,
`cancel_run`) so agents can't make changes
- **`read:query` JWT scope** — new authorization scope for query
endpoints, with per-table granularity (`read:query:runs`,
`read:query:llm_metrics`, etc.)
- **Paginated trace output** — `get_run_details` now paginates trace
events via cursor, caching the full trace in a temp file so subsequent
pages don't re-fetch
- **MCP tool annotations** — all tools now have
`readOnlyHint`/`destructiveHint` annotations for clients that support
them
- **Project-scoped profile persistence** — `switch_profile` saves to
`.trigger/mcp.json` (gitignored), automatically loaded on next MCP
server start
### Context optimizations
- `get_query_schema` requires a table name — returns one table's schema
instead of all tables (60-80% fewer tokens)
- `get_current_worker` no longer inlines payload schemas — use
`get_task_schema` for specific tasks
- Query results formatted as text tables instead of JSON (~50% fewer
tokens for flat data)
- `cancel_run`, `list_deploys`, `list_preview_branches` formatted as
text instead of raw `JSON.stringify()`
- Schema and dashboard API responses cached (1hr and 5min respectively)
### Bug fixes
- Fixed `search_docs` failing due to renamed upstream Mintlify tool
(`SearchTriggerDev` → `search_trigger_dev`)
- Fixed `list_deploys` failing when deployments have null
`runtime`/`runtimeVersion` fields (fixes#3139)
- Fixed `list_preview_branches` crashing due to incorrect response shape
access
- Fixed `metrics` table column documented as `value` instead of
`metric_value` in query docs
- Fixed `/api/v1/query` not accepting JWT auth (added `allowJWT: true`)
### Dev CLI build directory fix
The dev CLI was leaking `build-*` directories in `.trigger/tmp/` on
every rebuild, accumulating hundreds of MB over time (842MB observed).
Three layers of protection added:
1. **During session**: deprecated workers are pruned (capped at 2
retained) when no active runs reference them, preventing unbounded
accumulation
2. **On SIGKILL/crash**: the watchdog process now cleans up
`.trigger/tmp/` when it detects the parent CLI was killed
3. **On next startup**: existing `clearTmpDirs()` wipes any remaining
orphans
## Test plan
- [ ] `pnpm run mcp:smoke` — 17 automated smoke tests for all read-only
MCP tools
- [ ] `pnpm run mcp:test list` — verify 25 tools registered (21 in
`--readonly` mode)
- [ ] `pnpm run mcp:test --readonly list` — verify write tools hidden
- [ ] Manual: start dev server, trigger task, rebuild multiple times,
verify build dirs stay capped at 4
- [ ] Manual: SIGKILL the dev CLI, verify watchdog cleans up
`.trigger/tmp/`
- [ ] Verify new API endpoints return correct data: `GET
/api/v1/query/schema`, `GET /api/v1/query/dashboards`
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Queues with concurrency keys now appear as a single entry in the master
queue instead of one entry per key. This prevents high-CK-count tenants
from consuming the entire `parentQueueLimit` window and starving other
tenants on the same shard.
A new per-queue **CK index** (sorted set) tracks active concurrency key
sub-queues. The master queue gets one `:ck:*` wildcard entry per base
queue. Dequeuing from that entry round-robins across sub-queues,
maintaining per-CK concurrency tracking and fairness.
All existing operations (enqueue, dequeue, ack, nack, DLQ, TTL expiry)
are CK-index-aware and keep the index consistent. Old-format entries
drain naturally during rollout — no migration step needed, single
deploy.
## Adds 2 self serve features
### 1. self serve preview branches
- Copies the patterns of the self serve concurrency
- Self serve only available on Pro plan (otherwise you are linked to the
billing plans page)
- Global self serve branches limit: 180 (+20 for the Pro plan). It can
be overridden per Org
- You need to archive branches before reducing the number of extra
branches you're paying for
- Branches are removed immediately but remain billed until the end of
the billing cycle like extra concurrency
### 2. self serve team members
- Copies the patterns of the self serve concurrency
- Self serve only available on Pro plan (otherwise you are linked to the
billing plans page)
- Global self serve members is unlimited but can be limited with the
same env var quota and overridden per org if needed
- You need to remove team members before reducing the number of members
you pay for
- Team members are removed immediately but remain billed until the end
of the billing cycle like extra concurrency
Deprecates the syncVercelEnvVars build extension and adds warnings in
both the Vercel integration docs and the extension's own page to prevent
conflicts with the native env var sync
## Summary
2 new features, 2 improvements.
## Improvements
- Add syncSupabaseEnvVars to pull database connection strings and save
them as trigger.dev environment variables
([#3152](https://github.com/triggerdotdev/trigger.dev/pull/3152))
- Auto-cancel in-flight dev runs when the CLI exits, using a detached
watchdog process that survives pnpm SIGKILL
([#3191](https://github.com/triggerdotdev/trigger.dev/pull/3191))
## Server changes
These changes affect the self-hosted Docker image and Trigger.dev Cloud:
- A new Errors page for viewing and tracking errors that cause runs to
fail
- Errors are grouped using error fingerprinting
- View top errors for a time period, filter by task, or search the text
- View occurrences over time
- View all the runs for an error and bulk replay them
([#3172](https://github.com/triggerdotdev/trigger.dev/pull/3172))
- Add sidebar tabs (Options, AI, Schema) to the Test page for schemaTask
payload generation and schema viewing.
([#3188](https://github.com/triggerdotdev/trigger.dev/pull/3188))
<details>
<summary>Raw changeset output</summary>
# Releases
## @trigger.dev/build@4.4.3
### Patch Changes
- Add syncSupabaseEnvVars to pull database connection strings and save
them as trigger.dev environment variables
([#3152](https://github.com/triggerdotdev/trigger.dev/pull/3152))
- Updated dependencies:
- `@trigger.dev/core@4.4.3`
## trigger.dev@4.4.3
### Patch Changes
- Auto-cancel in-flight dev runs when the CLI exits, using a detached
watchdog process that survives pnpm SIGKILL
([#3191](https://github.com/triggerdotdev/trigger.dev/pull/3191))
- Updated dependencies:
- `@trigger.dev/core@4.4.3`
- `@trigger.dev/build@4.4.3`
- `@trigger.dev/schema-to-json@4.4.3`
## @trigger.dev/core@4.4.3
### Patch Changes
- Auto-cancel in-flight dev runs when the CLI exits, using a detached
watchdog process that survives pnpm SIGKILL
([#3191](https://github.com/triggerdotdev/trigger.dev/pull/3191))
## @trigger.dev/python@4.4.3
### Patch Changes
- Updated dependencies:
- `@trigger.dev/core@4.4.3`
- `@trigger.dev/build@4.4.3`
- `@trigger.dev/sdk@4.4.3`
## @trigger.dev/react-hooks@4.4.3
### Patch Changes
- Updated dependencies:
- `@trigger.dev/core@4.4.3`
## @trigger.dev/redis-worker@4.4.3
### Patch Changes
- Updated dependencies:
- `@trigger.dev/core@4.4.3`
## @trigger.dev/rsc@4.4.3
### Patch Changes
- Updated dependencies:
- `@trigger.dev/core@4.4.3`
## @trigger.dev/schema-to-json@4.4.3
### Patch Changes
- Updated dependencies:
- `@trigger.dev/core@4.4.3`
## @trigger.dev/sdk@4.4.3
### Patch Changes
- Updated dependencies:
- `@trigger.dev/core@4.4.3`
</details>
---------
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
When the dev CLI exits (e.g. ctrl+c via pnpm), runs that were
mid-execution
previously stayed stuck in EXECUTING status for up to 5 minutes until
the
heartbeat timeout fired. Now they are cancelled within seconds.
The dev CLI spawns a lightweight detached watchdog process at startup.
The
watchdog monitors the CLI process ID and, when it detects the CLI has
exited,
calls a new POST /engine/v1/dev/disconnect endpoint to cancel all
in-flight
runs immediately (skipping PENDING_CANCEL since the worker is known to
be dead).
Watchdog design:
- Fully detached (detached: true, stdio: ignore, unref()) so it survives
even when pnpm sends SIGKILL to the process tree
- Active run IDs maintained via atomic file write
(.trigger/active-runs.json)
- Single-instance guarantee via PID file (.trigger/watchdog.pid)
- Safety timeout: exits after 24 hours to prevent zombie processes
- On clean shutdown, the watchdog is killed (no disconnect needed)
Disconnect endpoint:
- Rate-limited: 5 calls/min per environment
- Capped at 500 runs per call
- Small counts (<= 25): cancelled inline with pMap concurrency 10
- Large counts: delegated to the bulk action system
- Uses finalizeRun: true to skip PENDING_CANCEL and go straight to
FINISHED
Run engine change:
- cancelRun() now respects finalizeRun when the run is in EXECUTING
status,
skipping the PENDING_CANCEL waiting state and going directly to FINISHED
### Fixes and improvements to the onboarding questions:
**This change is worth double checking @matt-aitken**
- Update to the Button.tsx file: it now takes `isLoading` that shows a
spinner in the middle of the button (replacing the button text and any
icons) and sets it to `disabled`. It does this nicely by keeping the
button width the same so there's no layout shift.
**Other fixes**
- Fixes an issue where if you type a custom option in the "What
technologies do you use" question, it doesn't check the list to see if
it matches. Now it checks the box if you've typed an option from that
list.
- When we randomize the list of onboarding question options, we now
store the position they appeared in the list
<img width="2191" height="1023" alt="CleanShot 2026-03-06 at 13 36 53"
src="https://github.com/user-attachments/assets/4eba0d1a-1528-49a3-be5b-6bde89030193"
/>
<img width="411" height="1069" alt="CleanShot 2026-03-06 at 13 37 28"
src="https://github.com/user-attachments/assets/e5f7bb9c-c894-41cc-9ca6-96b43fcf6005"
/>
Add a tabbed sidebar to the Test page for standard tasks, reusing the
ClientTabs pattern from the Query page.
- Options tab: existing sidebar content (machine, version, queue, etc.)
- AI tab: AI-powered payload generation with streaming, supports JSON
Schema, inferred schema from recent runs, and task source code lookup
via tool calling for tasks without schemas
- Schema tab: displays payload JSON Schema (from schemaTask), inferred
schema (from recent runs via @jsonhero/schema-infer), or empty state
with schemaTask docs and example code
Data layer changes:
- Surface payloadSchema and inferredPayloadSchema from TestTaskPresenter
- Add payloadSchema and fileId to WorkerDeploymentWithWorkerTasks type
- Decompress zlib-deflated source files for AI context
A top-level Errors page that aggregates errors from failed runs with
occurrences metrics.
https://github.com/user-attachments/assets/8f0ef55e-90dd-4faa-9051-59f4665181e4
Errors are “fingerprinted” so similar errors are grouped together (e.g.
has an ID in the error message).
You can view an individual error to view a timeline of when it fired,
the runs, and bulk replay them.
# trigger.dev v4.4.2
## Summary
2 new features, 2 improvements, 8 bug fixes.
## Improvements
- Add input streams for bidirectional communication with running tasks.
Define typed input streams with `streams.input<T>({ id })`, then consume
inside tasks via `.wait()` (suspends the process), `.once()` (waits for
next message), or `.on()` (subscribes to a continuous stream). Send data
from backends with `.send(runId, data)` or from frontends with the new
`useInputStreamSend` React hook.
([#3146](https://github.com/triggerdotdev/trigger.dev/pull/3146))
- Add PAYLOAD_TOO_LARGE error to handle graceful recovery of sending
batch trigger items with payloads that exceed the maximum payload size
([#3137](https://github.com/triggerdotdev/trigger.dev/pull/3137))
## Bug fixes
- Fix slow batch queue processing by removing spurious cooloff on
concurrency blocks and fixing a race condition where retry attempt
counts were not atomically updated during message re-queue.
([#3079](https://github.com/triggerdotdev/trigger.dev/pull/3079))
- fix(sdk): batch triggerAndWait variants now return correct
run.taskIdentifier instead of unknown
([#3080](https://github.com/triggerdotdev/trigger.dev/pull/3080))
## Server changes
These changes affect the self-hosted Docker image and Trigger.dev Cloud:
- Two-level tenant dispatch architecture for batch queue processing.
Replaces the
single master queue with a two-level index: a dispatch index (tenant →
shard)
and per-tenant queue indexes (tenant → queues). This enables O(1) tenant
selection and fair scheduling across tenants regardless of queue count.
Improves batch queue processing performance.
([#3133](https://github.com/triggerdotdev/trigger.dev/pull/3133))
- Add input streams with API routes for sending data to running tasks,
SSE reading, and waitpoint creation. Includes Redis cache for fast
`.send()` to `.wait()` bridging, dashboard span support for input stream
operations, and s2-lite support with configurable S2 endpoint, access
token skipping, and S2-Basin headers for self-hosted deployments. Adds
s2-lite to Docker Compose for local development.
([#3146](https://github.com/triggerdotdev/trigger.dev/pull/3146))
- Speed up batch queue processing by disabling cooloff and increasing
the batch queue processing concurrency limits on the cloud:
- Pro plan: increase to 50 from 10.
- Hobby plan: increase to 10 from 5.
- Free plan: increase to 5 from 1.
([#3079](https://github.com/triggerdotdev/trigger.dev/pull/3079))
- Move batch queue global rate limiter from FairQueue claim phase to
BatchQueue worker queue consumer for accurate per-item rate limiting.
Add worker queue depth cap to prevent unbounded growth that could cause
visibility timeouts.
([#3166](https://github.com/triggerdotdev/trigger.dev/pull/3166))
- Fix a race condition in the waitpoint system where a run could be
blocked by a completed waitpoint but never be resumed because of a
PostgreSQL MVCC issue. This was most likely to occur when creating a
waitpoint via `wait.forToken()` at the same moment as completing the
token with `wait.completeToken()`. Other types of waitpoints (timed,
child runs) were not affected.
([#3075](https://github.com/triggerdotdev/trigger.dev/pull/3075))
- Fix metrics dashboard chart series colors going out of sync and
widgets not reloading stale data when scrolled back into view
([#3126](https://github.com/triggerdotdev/trigger.dev/pull/3126))
- Gracefully handle oversized batch items instead of aborting the
stream.
When an NDJSON batch item exceeds the maximum size, the parser now emits
an error marker instead of throwing, allowing the batch to seal
normally. The oversized item becomes a pre-failed run with
`PAYLOAD_TOO_LARGE` error code, while other items in the batch process
successfully. This prevents `batchTriggerAndWait` from seeing connection
errors and retrying with exponential backoff.
Also fixes the NDJSON parser not consuming the remainder of an oversized
line split across multiple chunks, which caused "Invalid JSON" errors on
subsequent lines.
([#3137](https://github.com/triggerdotdev/trigger.dev/pull/3137))
- Require the user is an admin during an impersonation session.
Previously only the impersonation cookie was checked; now the real
user's admin flag is verified on every request. If admin has been
revoked, the session falls back to the real user's ID.
([#3078](https://github.com/triggerdotdev/trigger.dev/pull/3078))
<details>
<summary>Raw changeset output</summary>
# Releases
## @trigger.dev/build@4.4.2
### Patch Changes
- Updated dependencies:
- `@trigger.dev/core@4.4.2`
## trigger.dev@4.4.2
### Patch Changes
- Updated dependencies:
- `@trigger.dev/build@4.4.2`
- `@trigger.dev/core@4.4.2`
- `@trigger.dev/schema-to-json@4.4.2`
## @trigger.dev/python@4.4.2
### Patch Changes
- Updated dependencies:
- `@trigger.dev/sdk@4.4.2`
- `@trigger.dev/build@4.4.2`
- `@trigger.dev/core@4.4.2`
## @trigger.dev/react-hooks@4.4.2
### Patch Changes
- Add input streams for bidirectional communication with running tasks.
Define typed input streams with `streams.input<T>({ id })`, then consume
inside tasks via `.wait()` (suspends the process), `.once()` (waits for
next message), or `.on()` (subscribes to a continuous stream). Send data
from backends with `.send(runId, data)` or from frontends with the new
`useInputStreamSend` React hook.
([#3146](https://github.com/triggerdotdev/trigger.dev/pull/3146))
Upgrade S2 SDK from 0.17 to 0.22 with support for custom endpoints
(s2-lite) via the new `endpoints` configuration, `AppendRecord.string()`
API, and `maxInflightBytes` session option.
- Updated dependencies:
- `@trigger.dev/core@4.4.2`
## @trigger.dev/redis-worker@4.4.2
### Patch Changes
- Fix slow batch queue processing by removing spurious cooloff on
concurrency blocks and fixing a race condition where retry attempt
counts were not atomically updated during message re-queue.
([#3079](https://github.com/triggerdotdev/trigger.dev/pull/3079))
- Updated dependencies:
- `@trigger.dev/core@4.4.2`
## @trigger.dev/rsc@4.4.2
### Patch Changes
- Updated dependencies:
- `@trigger.dev/core@4.4.2`
## @trigger.dev/schema-to-json@4.4.2
### Patch Changes
- Updated dependencies:
- `@trigger.dev/core@4.4.2`
## @trigger.dev/sdk@4.4.2
### Patch Changes
- Add input streams for bidirectional communication with running tasks.
Define typed input streams with `streams.input<T>({ id })`, then consume
inside tasks via `.wait()` (suspends the process), `.once()` (waits for
next message), or `.on()` (subscribes to a continuous stream). Send data
from backends with `.send(runId, data)` or from frontends with the new
`useInputStreamSend` React hook.
([#3146](https://github.com/triggerdotdev/trigger.dev/pull/3146))
Upgrade S2 SDK from 0.17 to 0.22 with support for custom endpoints
(s2-lite) via the new `endpoints` configuration, `AppendRecord.string()`
API, and `maxInflightBytes` session option.
- fix(sdk): batch triggerAndWait variants now return correct
run.taskIdentifier instead of unknown
([#3080](https://github.com/triggerdotdev/trigger.dev/pull/3080))
- Add PAYLOAD_TOO_LARGE error to handle graceful recovery of sending
batch trigger items with payloads that exceed the maximum payload size
([#3137](https://github.com/triggerdotdev/trigger.dev/pull/3137))
- Updated dependencies:
- `@trigger.dev/core@4.4.2`
## @trigger.dev/core@4.4.2
</details>
---------
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
The global rate limiter was being applied at the FairQueue claim phase,
consuming 1 token per queue-claim-attempt rather than per item
processed.
With many small queues (each batch is its own queue), consumers burned
through tokens on empty or single-item queues, causing aggressive
throttling well below the intended items/sec limit.
Changes:
- Move rate limiter from FairQueue claim phase to BatchQueue worker
queue
consumer loop (before blockingPop), so each token = 1 item processed
- Replace the FairQueue rate limiter with a worker queue depth cap to
prevent unbounded growth that could cause visibility timeouts
- Add BATCH_QUEUE_WORKER_QUEUE_MAX_DEPTH env var (optional, disabled by
default)
Input streams enable sending typed data to executing tasks from external
callers — backends, frontends, or other tasks. This unlocks interactive
use cases like approval UIs, cancel buttons, chat interfaces, and
human-in-the-loop AI workflows where the task needs to receive data
while running.
Three consumption patterns inside a task:
* `.wait()` — Suspend the task until data arrives (process freed, most
efficient)
* `.once()` — Wait for the next message (process stays alive)
* `.on()` — Subscribe to a continuous stream of messages
One send pattern from outside:
* `.send(runId, data)` — Send typed data to a specific run's input
stream
## User-facing API
### Define a typed input stream
```ts
import { streams, task } from "@trigger.dev/sdk";
const approval = streams.input<{ approved: boolean; reviewer: string }>({ id: "approval" });
```
### Consume inside a task
```ts
export const myTask = task({
id: "my-task",
run: async () => {
// Pattern 1: Suspend until data arrives (most efficient — frees the process)
const result = await approval.wait({ timeout: "5m" });
// Pattern 2: Wait for next message (process stays alive)
const data = await approval.once().unwrap();
// Pattern 3: Subscribe to multiple messages
approval.on((data) => { /* handle each message */ });
},
});
```
### Send from outside
```ts
// From a backend (using secret API key)
await approval.send(runId, { approved: true, reviewer: "alice" });
// From a frontend (using public JWT token from trigger response)
const { send } = useInputStreamSend("approval", runId, { accessToken });
send({ approved: true, reviewer: "alice" });
```
---------
Co-authored-by: Claude <noreply@anthropic.com>
- New User onboarding questions added and stored in a new
`onboardingData` col
- Keeps the same Org creation screen and stores the data in the same
format in same DB column
- New Org onboarding questions addded and stored in a new
`onboardingData` col
https://github.com/user-attachments/assets/244e4bae-f74d-4ed4-a545-92c9b927e98b
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Pins vouch actions to `c6d80ead49839655b61b422700b7a3bc9d0804a9`
(v1.4.2) in favor of security practices. We were previously using the
`@main` tag.
Also removes the checkout steps as they're not needed in these
workflows.
There's a bug in react-window-splitter on Firefox. When trying to expand
the inspector panel in the query editor, it checks if the main panel has
space but gets an object instead of a number for the auto-sized
query-main panel. This causes the expand calculation to fail and it
snaps it back to collapsed.
I've removed this behavior for now as it's not an important feature.
Gracefully handle oversized batch items instead of aborting the stream.
When an NDJSON batch item exceeds the maximum size, the parser now emits
an error marker instead of throwing, allowing the batch to seal
normally. The oversized item becomes a pre-failed run with
`PAYLOAD_TOO_LARGE` error code, while other items in the batch process
successfully. This prevents `batchTriggerAndWait` from seeing connection
errors and retrying with exponential backoff.
Also fixes the NDJSON parser not consuming the remainder of an oversized
line split across multiple chunks, which caused "Invalid JSON" errors on
subsequent lines.
Replace flat master queue index with two-level tenant dispatch to fix
noisy neighbor problem. When a tenant has many queues at capacity, the
scheduler now iterates tenants (Level 1) not queues, then fetches
per-tenant queues (Level 2) only for eligible tenants.
Single-deploy migration: new enqueues write to dispatch indexes only,
consumer drains old master queue alongside new dispatch path until
empty.
Clarifies in the Query docs that run metadata is not available on the
Query page and that the output column is JSON, so dot notation (e.g.
output.externalId) should be used for selecting and filtering. Adds an
example that filters by an output field in WHERE
Split the CTE in blockRunWithWaitpoint so the pending waitpoint check
is a separate SQL statement. In READ COMMITTED isolation, each statement
gets its own snapshot, so a separate SELECT sees the latest committed
state from concurrent completeWaitpoint calls.
Previously, the CTE did INSERT + pending check in one statement (one
snapshot). If completeWaitpoint committed between the CTE start and
the SELECT, the SELECT would still see PENDING due to the stale
snapshot. Neither side would enqueue continueRunIfUnblocked, leaving
the run stuck forever.
Closes #
## ✅ Checklist
- [ ] I have followed every step in the [contributing
guide](https://github.com/triggerdotdev/trigger.dev/blob/main/CONTRIBUTING.md)
- [ ] The PR title follows the convention.
- [ ] I ran and tested the code works
---
## Testing
Verified the pricing definition displays the correct tier amount on the
plan selection page.
---
## Changelog
Fixed incorrect pricing tier for additional realtime connections from
$10/month per 100 to $10/month per 1000.
---
## Screenshots
N/A
💯https://claude.ai/code/session_015QrZZJHPWta3QCBnhX2Pff
Co-authored-by: Claude <noreply@anthropic.com>
Fix slow fair queue processing by removing spurious cooloff on
concurrency blocks and fixing a race condition where retry attempt
counts were not atomically updated during message re-queue.
Removed cooloff entirely from the batch queue
- Fix for series color assignment being out of sync with the graph
(ensures added series colors match their graph representation)
- Reload widgets when returning to screen if props changed (prevents
stale widgets after filtering and scrolling)
- Add .server-changes/ convention for tracking server-only changes
- Create scripts/enhance-release-pr.mjs to deduplicate and categorize
changeset PR body
- Create scripts/generate-github-release.mjs to format unified GitHub
release body
- Change release.yml to create one unified GitHub release instead of
per-package releases
- Add update-release job to patch Docker image link after images are
pushed to GHCR
- Update changesets-pr.yml to trigger on .server-changes, enhance PR
body, and clean up consumed files
- Document server changes in CLAUDE.md, CONTRIBUTING.md, CHANGESETS.md,
and RELEASE.md
What changed
- Fixed some functions like dateAdd, toString, ifNotFinite
- Removed all functions that accept lambdas as they're not supported
(yet)
- Added tests for all TRQL functions that use ClickHouse
Adds OpenAPI specs and sidebar pages for four previously undocumented
public endpoints: retrieve
run result, per-task batch trigger, retrieve batch, and retrieve batch
results.
This PR implements a new run TTL system and queue size limits to prevent
unbounded queue growth which should help prevent situations where queues
enter a "death spiral" where the queue will never be able to catch up.
The main/correct way to battle this situation is to enforce a maximum
TTL on all runs (e.g. up to 14 days) where runs that have been queued
for that maximum TTL will get auto-expired, making room for newer runs
to execute. This required creating a new TTL system that can handle
higher workloads and is now deeply integrated into the RunQueue. When
runs are enqueued with a TTL, they are added to their normal queue as
well as to the TTL queue. When runs are dequeued, they are removed from
both their normal queue and the TTL queue. If runs are dequeued by the
TTL system, they are removed from their normal queue. Both these
dequeues happen automatically so there is no race condition.
The TTL expiration system is also made reliable by expiring runs via a
Redis worker, which is enqueued to atomically inside the TTL dequeue lua
script.
### Optional associated waitpoints
Additionally, this PR implements an optimization where runs that aren't
triggered with a dependent parent run will no longer create an
associated waitpoint. Associated waitpoints are then lazily created if a
dependent run wants to wait for the child run post-facto (via debounce
or idempotency), which is a rare situation but is possible. This means
fewer waitpoint creations but also fewer waitpoint completions for runs
with no dependencies.
### Environment Queue Limits
Prevents any single queue growing too large by enforcing queue size
limits at trigger time.
- Queue size checks happen at trigger time - runs are rejected if queue
would exceed limit
- Dashboard UI shows queue limits on both the Queues page and a new
Limits page
- In-memory caching for queue size checks to reduce Redis load
### Batch trigger fixes
Currently when a batch item cannot be created for whatever reason (e.g.
queue limits) the run will never get created, which means a stalled run
if using `batchTriggerAndWait`. We've updated the system to handle this
differently: now when a batch item cannot be triggered and converted
into a run, we will eventually (after retrying 8 times up to 30s) we
will create a "pre-failed" run with the error details, correctly
resolving the batchTriggerAndWait.
Without doing an expensive query we can’t tell if it’s definitely a v3
projects – like getting run counts.
So let’s just assume if the project hasn’t been upgraded to v4 (by
running dev/deploy CLI with v4) AND the project is older than the v4
release then it’s v3.
Small fixes and improvements to the logs page:
- Clicking the Run ID didn't open inspector
- Swapped the "open link in tab" icon with Runs icon
- Prevent tooltip hovering on Level info
<img width="350" height="206" alt="CleanShot 2026-02-20 at 10 00 37@2x"
src="https://github.com/user-attachments/assets/3e82f24a-c0a1-4c01-a8e9-9e06a8af982a"
/>
This PR was opened by the [Changesets
release](https://github.com/changesets/action) GitHub action. When
you're ready to do a release, you can merge this and publish to npm
yourself or [setup this action to publish
automatically](https://github.com/changesets/action#with-publishing). If
you're not ready to do a release yet, that's fine, whenever you add more
changesets to main, this PR will be updated.
# Releases
## @trigger.dev/build@4.4.1
### Patch Changes
- Updated dependencies:
- `@trigger.dev/core@4.4.1`
## trigger.dev@4.4.1
### Patch Changes
- Add OTEL metrics pipeline for task workers. Workers collect process
CPU/memory, Node.js runtime metrics (event loop utilization, event loop
delay, heap usage), and user-defined custom metrics via
`otel.metrics.getMeter()`. Metrics are exported to ClickHouse with
10-second aggregation buckets and 1m/5m rollups, and are queryable
through the dashboard query engine with typed attribute columns,
`prettyFormat()` for human-readable values, and AI query support.
([#3061](https://github.com/triggerdotdev/trigger.dev/pull/3061))
- Updated dependencies:
- `@trigger.dev/build@4.4.1`
- `@trigger.dev/core@4.4.1`
- `@trigger.dev/schema-to-json@4.4.1`
## @trigger.dev/python@4.4.1
### Patch Changes
- Updated dependencies:
- `@trigger.dev/sdk@4.4.1`
- `@trigger.dev/build@4.4.1`
- `@trigger.dev/core@4.4.1`
## @trigger.dev/react-hooks@4.4.1
### Patch Changes
- Updated dependencies:
- `@trigger.dev/core@4.4.1`
## @trigger.dev/redis-worker@4.4.1
### Patch Changes
- Updated dependencies:
- `@trigger.dev/core@4.4.1`
## @trigger.dev/rsc@4.4.1
### Patch Changes
- Updated dependencies:
- `@trigger.dev/core@4.4.1`
## @trigger.dev/schema-to-json@4.4.1
### Patch Changes
- Updated dependencies:
- `@trigger.dev/core@4.4.1`
## @trigger.dev/sdk@4.4.1
### Patch Changes
- Add OTEL metrics pipeline for task workers. Workers collect process
CPU/memory, Node.js runtime metrics (event loop utilization, event loop
delay, heap usage), and user-defined custom metrics via
`otel.metrics.getMeter()`. Metrics are exported to ClickHouse with
10-second aggregation buckets and 1m/5m rollups, and are queryable
through the dashboard query engine with typed attribute columns,
`prettyFormat()` for human-readable values, and AI query support.
([#3061](https://github.com/triggerdotdev/trigger.dev/pull/3061))
- Updated dependencies:
- `@trigger.dev/core@4.4.1`
## @trigger.dev/core@4.4.1
---------
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Adds a direct Vercel Marketplace link, documents configuring build
options via the project config page, and adds a warning and workaround
for projects using a Vercel Root Directory
- Adds an end-to-end OTEL metrics pipeline: task workers collect and
export metrics via OpenTelemetry, the webapp ingests them into
ClickHouse, and they're queryable through the existing dashboard query
engine
- Workers emit process CPU/memory metrics (via
`@opentelemetry/host-metrics`) and Node.js runtime metrics (event loop
utilization, event loop delay, heap usage)
- Users can create custom metrics in their tasks via
`otel.metrics.getMeter()` from `@trigger.dev/sdk`
- Metrics are automatically tagged with run context (run ID, task slug,
machine, worker version) so they can be sliced per-run, per-task, or
per-machine
- The TSQL query engine gains metrics table support with typed attribute
columns, `prettyFormat()` for human-readable values, and per-schema time
bucket thresholds
- Includes reference tasks
(`references/hello-world/src/trigger/metrics.ts`) demonstrating
CPU-intensive, memory-ramp, bursty workload, and custom metrics patterns
## What changed
### Metrics collection (packages/core, packages/cli-v3)
- **Metrics export pipeline** — `TracingSDK` now sets up a
`MeterProvider` with a `PeriodicExportingMetricReader` that chains
through `TaskContextMetricExporter` (adds run context attributes) and
`BufferingMetricExporter` (batches exports to reduce overhead)
- **Host metrics** — Enabled `@opentelemetry/host-metrics` for process
CPU, memory, and system-level metrics
- **Node.js runtime metrics** — New `nodejsRuntimeMetrics.ts` module
using `performance.eventLoopUtilization()`, `monitorEventLoopDelay()`,
and `process.memoryUsage()` to emit 6 observable gauges
- File system and diskio metrics
- **Custom metrics** — Exposed `otel.metrics` from `@trigger.dev/sdk` so
users can create counters, histograms, and gauges in their tasks
- **Machine ID** — Stable per-worker machine identifier for grouping
metrics
- **Dev worker** — Drops `system.*` metrics to reduce noise, keeps
sending metrics between runs in warm workers
### Metrics ingestion (apps/webapp)
- **OTEL endpoint** — `otel.v1.metrics.ts` accepts OTEL metric export
requests (JSON and protobuf), converts to ClickHouse rows
- **ClickHouse schema** — `017_create_metrics_v1.sql` with 10-second
aggregation buckets, JSON attributes column, 60-day TTLs
### Query engine (internal-packages/tsql, apps/webapp)
- **Metrics query schema** — Typed columns for metric attributes
(`task_identifier`, `run_id`, `machine_name`, `worker_version`, etc.)
extracted from the JSON attributes column
- **`prettyFormat()`** — TSQL function that annotates columns with
format hints (`bytes`, `percent`, `durationSeconds`) for frontend
rendering without changing the underlying data
- **Per-schema time buckets** — Different tables can define their own
time bucket thresholds (metrics uses tighter intervals than runs)
- **AI query integration** — The AI query service knows about the
metrics table and can generate metric queries
- **Chart improvements** — Better formatting for byte values,
percentages, and durations in charts and tables
### Reference project
- **`references/hello-world/src/trigger/metrics.ts`** — 6 example tasks:
`cpu-intensive`, `memory-ramp`, `bursty-workload`, `sustained-workload`,
`concurrent-load`, `custom-metrics`
## Test plan
- [ ] Build all packages and webapp
- [ ] Start dev worker with hello-world reference project
- [ ] Run `cpu-intensive`, `memory-ramp`, and `custom-metrics` tasks
- [ ] Verify metrics in ClickHouse: `SELECT DISTINCT metric_name FROM
metrics_v1`
- [ ] Query via dashboard AI: "show me CPU utilization over time"
- [ ] Verify `prettyFormat` renders correctly in chart tooltips and
table cells
- [ ] Confirm dev worker drops `system.*` metrics but keeps `process.*`
and `nodejs.*`
This PR was opened by the [Changesets
release](https://github.com/changesets/action) GitHub action. When
you're ready to do a release, you can merge this and publish to npm
yourself or [setup this action to publish
automatically](https://github.com/changesets/action#with-publishing). If
you're not ready to do a release yet, that's fine, whenever you add more
changesets to main, this PR will be updated.
# Releases
## @trigger.dev/sdk@4.4.0
### Minor Changes
- Added `query.execute()` which lets you query your Trigger.dev data
using TRQL (Trigger Query Language) and returns results as typed JSON
rows or CSV. It supports configurable scope (environment, project, or
organization), time filtering via `period` or `from`/`to` ranges, and a
`format` option for JSON or CSV output.
([#3060](https://github.com/triggerdotdev/trigger.dev/pull/3060))
```typescript
import { query } from "@trigger.dev/sdk";
import type { QueryTable } from "@trigger.dev/sdk";
// Basic untyped query
const result = await query.execute("SELECT run_id, status FROM runs
LIMIT 10");
// Type-safe query using QueryTable to pick specific columns
const typedResult = await query.execute<QueryTable<"runs", "run_id" |
"status" | "triggered_at">>(
"SELECT run_id, status, triggered_at FROM runs LIMIT 10"
);
typedResult.results.forEach((row) => {
console.log(row.run_id, row.status); // Fully typed
});
// Aggregation query with inline types
const stats = await query.execute<{ status: string; count: number }>(
"SELECT status, COUNT(*) as count FROM runs GROUP BY status",
{ scope: "project", period: "30d" }
);
// CSV export
const csv = await query.execute("SELECT run_id, status FROM runs", {
format: "csv",
period: "7d",
});
console.log(csv.results); // Raw CSV string
```
### Patch Changes
- Add `maxDelay` option to debounce feature. This allows setting a
maximum time limit for how long a debounced run can be delayed, ensuring
execution happens within a specified window even with continuous
triggers.
([#2984](https://github.com/triggerdotdev/trigger.dev/pull/2984))
```typescript
await myTask.trigger(payload, {
debounce: {
key: "my-key",
delay: "5s",
maxDelay: "30m", // Execute within 30 minutes regardless of continuous
triggers
},
});
```
- Aligned the SDK's `getRunIdForOptions` logic with the Core package to
handle semantic targets (`root`, `parent`) in root tasks.
([#2874](https://github.com/triggerdotdev/trigger.dev/pull/2874))
- Export `AnyOnStartAttemptHookFunction` type to allow defining
`onStartAttempt` hooks for individual tasks.
([#2966](https://github.com/triggerdotdev/trigger.dev/pull/2966))
- Fixed a minor issue in the deployment command on distinguishing
between local builds for the cloud vs local builds for self-hosting
setups.
([#3070](https://github.com/triggerdotdev/trigger.dev/pull/3070))
- Updated dependencies:
- `@trigger.dev/core@4.4.0`
## @trigger.dev/build@4.4.0
### Patch Changes
- Updated dependencies:
- `@trigger.dev/core@4.4.0`
## trigger.dev@4.4.0
### Patch Changes
- Fix runner getting stuck indefinitely when `execute()` is called on a
dead child process.
([#2978](https://github.com/triggerdotdev/trigger.dev/pull/2978))
- Add optional `timeoutInSeconds` parameter to the
`wait_for_run_to_complete` MCP tool. Defaults to 60 seconds. If the run
doesn't complete within the timeout, the current state of the run is
returned instead of waiting indefinitely.
([#3035](https://github.com/triggerdotdev/trigger.dev/pull/3035))
- Fixed a minor issue in the deployment command on distinguishing
between local builds for the cloud vs local builds for self-hosting
setups.
([#3070](https://github.com/triggerdotdev/trigger.dev/pull/3070))
- Updated dependencies:
- `@trigger.dev/core@4.4.0`
- `@trigger.dev/build@4.4.0`
- `@trigger.dev/schema-to-json@4.4.0`
## @trigger.dev/core@4.4.0
### Patch Changes
- Add `maxDelay` option to debounce feature. This allows setting a
maximum time limit for how long a debounced run can be delayed, ensuring
execution happens within a specified window even with continuous
triggers.
([#2984](https://github.com/triggerdotdev/trigger.dev/pull/2984))
```typescript
await myTask.trigger(payload, {
debounce: {
key: "my-key",
delay: "5s",
maxDelay: "30m", // Execute within 30 minutes regardless of continuous
triggers
},
});
```
- Fixed a minor issue in the deployment command on distinguishing
between local builds for the cloud vs local builds for self-hosting
setups.
([#3070](https://github.com/triggerdotdev/trigger.dev/pull/3070))
- fix: vendor superjson to fix ESM/CJS compatibility
([#2949](https://github.com/triggerdotdev/trigger.dev/pull/2949))
Bundle superjson during build to avoid `ERR_REQUIRE_ESM` errors on
Node.js versions that don't support `require(ESM)` by default (<
22.12.0) and AWS Lambda which intentionally disables it.
- Add Vercel integration support to API schemas: `commitSHA` and
`integrationDeployments` on deployment responses, and `source` field for
environment variable imports.
([#2994](https://github.com/triggerdotdev/trigger.dev/pull/2994))
## @trigger.dev/python@4.4.0
### Patch Changes
- Updated dependencies:
- `@trigger.dev/core@4.4.0`
- `@trigger.dev/sdk@4.4.0`
- `@trigger.dev/build@4.4.0`
## @trigger.dev/react-hooks@4.4.0
### Patch Changes
- Fix `onComplete` callback firing prematurely when the realtime stream
disconnects before the run finishes.
([#2929](https://github.com/triggerdotdev/trigger.dev/pull/2929))
- Updated dependencies:
- `@trigger.dev/core@4.4.0`
## @trigger.dev/redis-worker@4.4.0
### Patch Changes
- Updated dependencies:
- `@trigger.dev/core@4.4.0`
## @trigger.dev/rsc@4.4.0
### Patch Changes
- Updated dependencies:
- `@trigger.dev/core@4.4.0`
## @trigger.dev/schema-to-json@4.4.0
### Patch Changes
- Updated dependencies:
- `@trigger.dev/core@4.4.0`
---------
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Closes #<issue>
## ✅ Checklist
- [ ] I have followed every step in the [contributing
guide](https://github.com/triggerdotdev/trigger.dev/blob/main/CONTRIBUTING.md)
- [ ] The PR title follows the convention.
- [ ] I ran and tested the code works
---
## Testing
N/A - Documentation and OpenAPI schema updates only.
---
## Changelog
Added comprehensive Queue Management API support:
**OpenAPI Endpoints:**
- `GET /api/v1/queues` - List all queues with pagination support
- `GET /api/v1/queues/{queueParam}` - Retrieve a specific queue by ID,
task ID, or custom queue name
- `POST /api/v1/queues/{queueParam}/pause` - Pause or resume a queue
- `POST /api/v1/queues/{queueParam}/concurrency/override` - Override
queue concurrency limits
- `POST /api/v1/queues/{queueParam}/concurrency/reset` - Reset
concurrency limits to base values
**Schema Definitions:**
- `QueueObject` - Complete queue representation with concurrency details
- `ListQueuesResult` - Paginated queue listing response
**Documentation:**
- Updated `queue-concurrency.mdx` with SDK usage examples for queue
management
- Added 5 new management API documentation pages for each endpoint
- Updated `docs.json` navigation structure with new "Queues API" section
All endpoints support flexible queue identification (by ID, task ID, or
custom queue name) and include TypeScript code samples.
---
## Screenshots
N/A
💯https://claude.ai/code/session_01LyrXwxHCbejvi34fykifPP
Co-authored-by: Claude <noreply@anthropic.com>
Adds a region selector to the Test task page and Replay run dialog, so
users can override the region from the dashboard. Disabled with a
placeholder for dev environments.
Closes#3016
This will prevent internal logs to be added to the
task_events_search_table
Closes #<issue>
## ✅ Checklist
- [x] I have followed every step in the [contributing
guide](https://github.com/triggerdotdev/trigger.dev/blob/main/CONTRIBUTING.md)
- [x] The PR title follows the convention.
- [x] I ran and tested the code works
---
## Testing
Ran the migration, deleted the old invalid rows and ran new tasks.
The undesired logs are not added to the table.
---
## Changelog
Updated the MATERIALIZED VIEW to also filter for `trace_id != ''`
---------
Co-authored-by: Matt Aitken <matt@mattaitken.com>
Expand documentation for the Vercel integration with detailed usage,
installation, environment variable sync, atomic deployments, and
environment mapping. Replace the previous "coming soon" placeholder with
complete instructions and UI flow for connecting via the Trigger.dev
dashboard or the Vercel Marketplace. Explain required GitHub
integration,
how env vars sync in both directions, which vars are excluded, and how
to control sync behavior. Describe atomic deployments (default for
production), how they gate Vercel deployments to ensure task/app
consistency, and note related configuration changes. Add tips and notes
to guide setup and troubleshooting.
This provides users with actionable guidance to connect Vercel, map
environments, and keep app and tasks in sync without custom CI scripts.
Fixes an issue introduced in #3024.
The behavior for local builds in older CLI versions relies on
`externalBuildData` to be defined to distinguish from the self-hosting
local build path, even though it doesn't actually use the token.
Summary
- Add API endpoint to run TRQL queries
- Implement SDK function for executing queries
## SDK
Added `query.execute()` which lets you query your Trigger.dev data using
TRQL (Trigger Query Language) and returns results as typed JSON rows or
CSV. It supports configurable scope (environment, project, or
organization), time filtering via `period` or `from`/`to` ranges, and a
`format` option for JSON or CSV output.
```typescript
import { query } from "@trigger.dev/sdk";
import type { QueryTable } from "@trigger.dev/sdk";
// Basic untyped query
const result = await query.execute("SELECT run_id, status FROM runs LIMIT 10");
// Type-safe query using QueryTable to pick specific columns
const typedResult = await query.execute<QueryTable<"runs", "run_id" | "status" | "triggered_at">>(
"SELECT run_id, status, triggered_at FROM runs LIMIT 10"
);
typedResult.results.forEach(row => {
console.log(row.run_id, row.status); // Fully typed
});
// Aggregation query with inline types
const stats = await query.execute<{ status: string; count: number }>(
"SELECT status, COUNT(*) as count FROM runs GROUP BY status",
{ scope: "project", period: "30d" }
);
// CSV export
const csv = await query.execute(
"SELECT run_id, status FROM runs",
{ format: "csv", period: "7d" }
);
console.log(csv.results); // Raw CSV string
```
Documents the skipColumns option on useRealtimeRun and
useRealtimeRunsWithTag for status-only subscriptions (smaller payloads,
e.g. for progress/completion UI). Adds a troubleshooting section for the
“Failed to index deployment” source-map error when using the Bun
runtime, with a pnpm patch workaround and link to the GitHub issue
Extract `applyPeriod` callback from `applySelection` so preset period
buttons ("Created in the last X") apply immediately when clicked,
instead of only updating the selection state and requiring a separate
apply step.
Also validates `maxPeriodDays` on instant-apply so the upgrade prompt
still works correctly for plan-limited periods.
For now we’re going to always add FINAL to TRQL queries for data
correctness.
In the future we will implement an automated optimization where we use
`SELECT argMax(column, _version)` and `WHERE _is_deleted = 0`. But this
is a more complex change and needs more investigation of downsides.
A customer experienced a bug where their subscription downgraded to the
free plan unintentionally. This was due to a concurrency upgrade payment
attempt that failed a card check. We auto retry the payment across 2
weeks of attempts. When the final attempt failed, the whole subscription
downgraded.
Now we check if the payment is successful and if not, return an error
immediately so the subscription isn't modified until a successful
payment is made for an upgrade.
Summary
- Remove LIMIT from built-in dashboard queries
- Make concurrency configurable per project via environment variables
- Fix widget fallback period to Metrics default (1d) instead of 7d
- Handle concurrency at the project level
- Sort series for graphs so largest is displayed at the bottom (legend
shows largest at top)
- Use average aggregation for some built-in charts
- Improve aggregation handling for the legend
- Only render chart points when there is data; render dots on line
charts
- Truncate legend items and show tooltip on hover
- Better preserve chart configuration when the underlying query changes
## ✅ Checklist
- [X] I have followed every step in the [contributing
guide](https://github.com/triggerdotdev/trigger.dev/blob/main/CONTRIBUTING.md)
- [X] The PR title follows the convention.
- [X] I ran and tested the code works
---
## Testing
Tested the migration on test env and locally.
Tested query and merge performance.
Generated tasks and observed the ingested data and searches.
---
## Changelog
* New ClickHouse table & MV (task_events_search_v1): A search-optimized
materialized view that filters out debug events, partial spans, and
empty span events at ingestion time.
* ClickHouse client updates: New getLogsSearchListQueryBuilder and
taskEventsSearch accessor on the ClickHouse class.
* LogsListPresenter: Switches to the new search table, uses
triggered_timestamp for cursor pagination instead of unixTimestamp.
* Spans route: Also switches to the new search query builder.
* Seed spanSpammer task: Adds a 10s trace with events and metadata
operations for testing.
Summary
- Implemented metrics dashboards with a built-in dashboard and custom
dashboards
- Added a "Big number” display type
What changed
- New data format for metric layouts and saving/editing layouts
(editing, saving, cancel revert)
- QueryWidget usable on Query page and Metrics dashboards
- Time filtering, auto-reloading and timeBucket() auto-bin support
- Filters added to metrics; widget popover/improved history and blank
states
- Side menu:
- Metrics/Insights section with icons, colors, padding, collapsible
behavior and reordering of custom dashboards
- Move action logic into service for reuse and API querying; refactor
reordering for reuse
<!-- devin-review-badge-begin -->
---
<a
href="https://app.devin.ai/review/triggerdotdev/trigger.dev/pull/3019"
target="_blank">
<picture>
<source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-open-in-devin-review-dark.svg?v=1">
<img
src="https://static.devin.ai/assets/gh-open-in-devin-review-light.svg?v=1"
alt="Open with Devin">
</picture>
</a>
<!-- devin-review-badge-end -->
---------
Co-authored-by: James Ritchie <james@trigger.dev>
## Summary
- Adds an optional `timeoutInSeconds` parameter (default 60s) to the
`wait_for_run_to_complete` MCP tool
- If the run doesn't complete within the timeout, returns the current
run state instead of blocking indefinitely
- Uses `AbortSignal.timeout()` combined with the existing MCP signal
Fixes#3032
## ✅ Checklist
- [X] I have followed every step in the [contributing
guide](https://github.com/triggerdotdev/trigger.dev/blob/main/CONTRIBUTING.md)
- [X] The PR title follows the convention.
- [X] I ran and tested the code works
---
## Testing
Manually tested each implementation.
---
## Changelog
* Updated Logs Page with the new implementation in time filter component
* In TRQL editor users can now click on empty/blank spaces in the editor
and the cursor will appear
* Added CMD + / for line commenting in TRQL
* Activated proper undo/redo functionality in CodeMirror (TRQL editor)
* Added a check for new logs button, previously once the user got to the
end of the logs he could not check for newer logs
* Added showing MS in logs page Dates
* Removed LOG_INFO internal logs, they are available with Admin Debug
flag
* Added support for correct timezone render on server side.
* Increased CLICKHOUSE_LOGS_LIST_MAX_MEMORY_USAGE to 1GB
* Changed Previous run/ Next run to J/K, consistent with previous/next
page in Runs list
## Summary
- **Fix Docker publish automation**: The `v.docker.*` tags pushed by the
release workflow using `GITHUB_TOKEN` don't trigger the publish workflow
(GitHub Actions limitation to prevent infinite loops). Added a
`workflow_call` to `publish.yml` directly from the release job so Docker
images are built automatically after npm publish. Tags are still pushed
for reference.
- **Fix worker Containerfiles**: The coordinator, docker-provider, and
kubernetes-provider builds have been failing since the superjson
vendoring change in `@trigger.dev/core` (#2949). The Containerfiles now
run `bundle-vendor` before `build:bundle` to generate the vendor files
that esbuild needs.
### Context
- Docker images on GHCR have been stuck at v4.3.0 — v4.3.1, v4.3.2,
v4.3.3 tags existed on GitHub but never triggered publish runs
- The worker builds (publish-worker) have been failing on every push to
main since Jan 30
## Test plan
- [x] Verified kubernetes-provider Containerfile builds locally with the
fix
- [x] Manually dispatched publish workflow for v4.3.1 — all jobs
succeeded
<!-- devin-review-badge-begin -->
---
<a
href="https://app.devin.ai/review/triggerdotdev/trigger.dev/pull/3013"
target="_blank">
<picture>
<source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-open-in-devin-review-dark.svg?v=1">
<img
src="https://static.devin.ai/assets/gh-open-in-devin-review-light.svg?v=1"
alt="Open with Devin">
</picture>
</a>
<!-- devin-review-badge-end -->
This pull request overhauls the "Building with AI" documentation
section. It includes a comprehensive restructuring of the main
building-with-ai page with new setup guides and troubleshooting
sections, reorganizes the navigation hierarchy to elevate
mcp-agent-rules as a top-level page, and updates multiple documentation
pages to clarify the relationships between three AI tools: Skills, Agent
Rules, and MCP Server. Changes also include formatting improvements,
such as replacing italicized text with inline code formatting, and
consistent additions of explanatory Note blocks and CardGroup components
across related pages.
Display the deployment trigger source (CLI, CI/CD, Dashboard, GitHub
Integration) with appropriate icons on the deployment details page. The
triggeredVia field was already in the database but not displayed.
Co-authored-by: Claude <noreply@anthropic.com>
Adds optional pod affinity so pods from the same project prefer
scheduling on the same node. This can help improve image cache hit
rates; subsequent pods benefit from already-pulled image layers,
reducing startup time.
Complements the built-in ImageLocality scheduler plugin by helping
during burst scheduling scenarios. Pod affinity sees scheduled pods
immediately, while ImageLocality only sees images after they're fully
pulled.
Configuration:
- `KUBERNETES_PROJECT_AFFINITY_ENABLED` - Enable/disable (default:
false)
- `KUBERNETES_PROJECT_AFFINITY_WEIGHT` - Scheduler weight 1-100
(default: 50)
- `KUBERNETES_PROJECT_AFFINITY_TOPOLOGY_KEY` - Topology key (default:
kubernetes.io/hostname)
Uses soft (preferred) affinity so pods always schedule even if preferred
node is full.
<!-- devin-review-badge-begin -->
---
<a
href="https://app.devin.ai/review/triggerdotdev/trigger.dev/pull/2995">
<picture>
<source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-open-in-devin-review-dark.svg?v=1">
<img
src="https://static.devin.ai/assets/gh-open-in-devin-review-light.svg?v=1"
alt="Open with Devin">
</picture>
</a>
<!-- devin-review-badge-end -->
## Summary
- When a child process crashes and a retry (`RETRY_IMMEDIATELY`) is
attempted on the same `TaskRunProcess`, `execute()` hangs forever
because the IPC send is silently skipped and the attempt promise can
never resolve
- This caused runner pods to stay up indefinitely with no heartbeats or
polls
- Fix: reject the attempt promise immediately when the child is not
connected, so the controller can proceed to warm start or exit
## Test plan
- [x] Added `taskRunProcess.test.ts` — verifies `execute()` rejects
promptly instead of hanging when the child process is dead
- [x] Deploy and verify no more stuck runner pods accumulate over time
Closes#2798
When a run finished the logs UI could get stuck and so be pending and
never update again. If you did a hard reload it would be correct.
This happened because when we insert a log/span we ping Redis which
causes a reload of the UI. However there was a race condition – the
insert into ClickHouse can take a while so we were refreshing the UI too
early. Then never refreshing it again.
Changes
- Send refresh pings every 5s to keep run page logs live
- Throttle updates so the run UI is never updated more than once per
second
- Stop auto-reloading when a run has been completed for >= 30s
- Add type inference improvements for the throttle function
<!-- devin-review-badge-begin -->
---
<a
href="https://app.devin.ai/review/triggerdotdev/trigger.dev/pull/2971">
<picture>
<source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-open-in-devin-review-dark.svg?v=1">
<img
src="https://static.devin.ai/assets/gh-open-in-devin-review-light.svg?v=1"
alt="Open with Devin">
</picture>
</a>
<!-- devin-review-badge-end -->
## ✅ Checklist
- [ ] I have followed every step in the [contributing
guide](https://github.com/triggerdotdev/trigger.dev/blob/main/CONTRIBUTING.md)
- [ ] The PR title follows the convention.
- [ ] I ran and tested the code works
---
## Description
This PR standardizes the `@types/node` dependency across the entire
monorepo to version `20.14.14`. Previously, different packages were
using different versions (ranging from 12.20.55 to 22.13.9), which could
cause type conflicts and inconsistencies.
### Changes Made
1. **tsconfig.json** - Added `"node"` to the `types` array in
`apps/webapp/tsconfig.json` to ensure Node.js types are properly
recognized
2. **package.json overrides** - Added `@types/node` version override to
`20.14.14` in the root `package.json`
3. **pnpm-lock.yaml** - Updated lock file to reflect the standardized
version across all packages and their dependencies
4. **Fixture package.json** - Updated
`packages/cli-v3/e2e/fixtures/emit-decorator-metadata/package.json` to
use the standardized version
This ensures consistent type definitions across the monorepo and
prevents version mismatches that could lead to type errors or unexpected
behavior.
---
## Testing
- Verified that all package references to `@types/node` now point to
version `20.14.14`
- Confirmed that the lock file properly reflects the override across all
transitive dependencies
- Ensured TypeScript configuration includes Node.js types for proper
type checking
---
## Changelog
- Standardized `@types/node` to version `20.14.14` across all packages
in the monorepo
- Added `"node"` to TypeScript compiler types in webapp configuration
- Updated all package dependencies to use the consistent version through
pnpm overrides
💯https://claude.ai/code/session_018eqp2LvvErkFSN9oK5xBh1
<!-- devin-review-badge-begin -->
---
<a
href="https://app.devin.ai/review/triggerdotdev/trigger.dev/pull/2970">
<picture>
<source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-open-in-devin-review-dark.svg?v=1">
<img
src="https://static.devin.ai/assets/gh-open-in-devin-review-light.svg?v=1"
alt="Open with Devin">
</picture>
</a>
<!-- devin-review-badge-end -->
---------
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Eric Allam <ericallam@users.noreply.github.com>
Bundle superjson and its dependency (copy-anything) during build to
avoid
ERR_REQUIRE_ESM errors on Node.js versions that don't support
require(ESM)
by default (< 22.12.0) and AWS Lambda which intentionally disables it.
- Add scripts/bundle-superjson.mjs to bundle superjson with esbuild
- Update build script to bundle vendor files before tshy compilation
- Move superjson from dependencies to devDependencies
- Update imports to use vendored bundles
Fixes#2937
<!-- devin-review-badge-begin -->
---
<a
href="https://app.devin.ai/review/triggerdotdev/trigger.dev/pull/2949">
<picture>
<source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-open-in-devin-review-dark.svg?v=1">
<img
src="https://static.devin.ai/assets/gh-open-in-devin-review-light.svg?v=1"
alt="Open with Devin">
</picture>
</a>
<!-- devin-review-badge-end -->
---------
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Eric Allam <ericallam@users.noreply.github.com>
Summary
- Query: add time limits, performance improvements, and styling updates
Changes
- Add ClickHouse output_text and error_text columns with indexes
- Automatically use _text columns for JSON based on query pattern;
support JSON column data prefixes
- Add idempotency key and scope columns
- Add enforcedWhereClause for tenant and time restrictions, instead of
the old tenant stuff.
- Implement basic time filter limiting and set default time period based
on plan; show message when results are clipped
- UX: resizable code area (including vertical splits), collapsible
sidebar, fix table/chart vertical sizing, max height for chart legend in
fullscreen
- Styling and UI tweaks: improved chart legend styling, more chart
colours, thinner line chart stroke, pricing callout color, improved
layout for callouts
- Features: generate and save AI titles
<!-- devin-review-badge-begin -->
---
<a
href="https://app.devin.ai/review/triggerdotdev/trigger.dev/pull/2953">
<picture>
<source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-open-in-devin-review-dark.svg?v=1">
<img
src="https://static.devin.ai/assets/gh-open-in-devin-review-light.svg?v=1"
alt="Open with Devin">
</picture>
</a>
<!-- devin-review-badge-end -->
Closes #
## ✅ Checklist
- [ ] I have followed every step in the [contributing
guide](https://github.com/triggerdotdev/trigger.dev/blob/main/CONTRIBUTING.md)
- [ ] The PR title follows the convention.
- [ ] I ran and tested the code works
---
## Testing
Tested the MiddleTruncate component in the TasksDropdown by:
1. Verifying that long task names (e.g.,
"namespace:category:subcategory:task-name") are truncated in the middle
2. Confirming the full text appears in a tooltip on hover
3. Testing responsive behavior - truncation adjusts when the container
is resized
4. Verifying that short task names that fit within the container are
displayed in full without truncation
---
## Changelog
Added a new `MiddleTruncate` primitive component that intelligently
truncates text in the middle while preserving the beginning and end
portions. This is particularly useful for long hierarchical identifiers
like task slugs.
**Key features:**
- Truncates text in the middle with an ellipsis (…) when it exceeds
available width
- Shows full text in a tooltip on hover when truncated
- Responsive - recalculates truncation on container resize using
ResizeObserver
- Maintains minimum character visibility (4 chars minimum on each side
for readability)
- Integrated into TasksDropdown to handle long task names
**Changes:**
- Created new `MiddleTruncate.tsx` component with binary search
algorithm for optimal character distribution
- Updated TasksDropdown to use MiddleTruncate for task slug display
- Increased TasksDropdown popover width from 240px to 360px to provide
better space for truncated text
---
## Screenshots
💯https://github.com/user-attachments/assets/a7a2191a-2e36-437e-ab3f-517fe7620b93
<!-- devin-review-badge-begin -->
---
<a
href="https://app.devin.ai/review/triggerdotdev/trigger.dev/pull/2946">
<picture>
<source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-open-in-devin-review-dark.svg?v=1">
<img
src="https://static.devin.ai/assets/gh-open-in-devin-review-light.svg?v=1"
alt="Open with Devin">
</picture>
</a>
<!-- devin-review-badge-end -->
---------
Co-authored-by: Claude <noreply@anthropic.com>
## ✨ Changes
### UI & UX
- Normalized log level display across table and detail view
- Fixed table header scroll behavior and sidebar positioning
- Improved loading state with taller segment and disabled resizing
- Added "no more logs" message with count
- Enhanced keyboard shortcuts
### Filtering & Search
- Streamlined filters: RunId and Task only (removed run filters)
- Side panel closes when filters change
- Fixed logs from previous search remaining in table
- Fixed table scroll position when changing filters
### Backend
- Added performance indexes on message and attributes
(`014_add_task_runs_v2_search_indexes.sql`)
- Added DEBUG level logging by default
- Removed internal logs from display
- Fixed ServiceValidationError forwarding to frontend
- Removed v1 logs API support
## Summary
Fixes#2856 - The `onComplete` callback in `useRealtimeRun` was firing prematurely
## Root Cause
The callback was triggered when the long-poll stream ended, regardless
of whether the run had actually completed. Reverse proxies often close
idle connections, causing the stream to end prematurely. In this case it
was caused by fetch abort due to React strict mode.
## Fix
Changed the condition from checking if `run` exists to checking if
`run?.finishedAt` exists, ensuring `onComplete` only fires when the run
has reached a terminal state.
---------
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: nicktrn <nicktrn@users.noreply.github.com>
## Summary
Fixes a concurrency leak in the batch queue where visibility timeout
reclaims do not release concurrency slots.
**The bug:** When a message visibility timeout expires (60s),
`reclaimTimedOut` puts the message back in the queue but does NOT
release the concurrency slot. The messageId stays in the concurrency set
(`engine:batch:concurrency:tenant:{envId}`), counting against the tenant
limit even though the message is no longer in-flight.
This causes:
1. Tenant appears at capacity when checking `SCARD >= limit`
2. New messages get released back to queue instead of being processed
3. Messages stuck in infinite loop, master queue grows indefinitely
**The fix:**
- Modified `reclaimTimedOut` to capture message data (including
tenantId) BEFORE releasing from in-flight
- Returns `ReclaimedMessageInfo[]` with messageId, queueId, tenantId,
and metadata
- `#reclaimTimedOutMessages` now iterates over reclaimed messages and
calls `concurrencyManager.release()` for each
## Test plan
- [x] Added test: `should return reclaimed message info with tenantId
for concurrency release`
- [x] Added test: `should return empty array when no messages have timed
out`
- [x] Added test: `should reclaim multiple timed-out messages and return
all their info`
- [x] Updated `raceConditions.test.ts` for new return type
- [x] All tests passing
- [ ] Monitor production after deploy for concurrency leak recurrence
refs TRI-7049
<!-- devin-review-badge-begin -->
---
<a
href="https://app.devin.ai/review/triggerdotdev/trigger.dev/pull/2907">
<picture>
<source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-open-in-devin-review-dark.svg?v=1">
<img
src="https://static.devin.ai/assets/gh-open-in-devin-review-light.svg?v=1"
alt="Open with Devin">
</picture>
</a>
<!-- devin-review-badge-end -->
Adds documentation for creating additional worker groups via the admin
API endpoint, including how to make users admin (new vs existing users),
and clarifies that ADMIN_EMAILS only applies on signup.
Adds documentation notes clarifying that:
- delayed runs execute on the current deployment version
- guidance for Date objects in payloads
- static IP availability
- version locking behavior for delayed runs
This PR was opened by the [Changesets
release](https://github.com/changesets/action) GitHub action. When
you're ready to do a release, you can merge this and publish to npm
yourself or [setup this action to publish
automatically](https://github.com/changesets/action#with-publishing). If
you're not ready to do a release yet, that's fine, whenever you add more
changesets to main, this PR will be updated.
# Releases
## @trigger.dev/build@4.3.3
### Patch Changes
- Updated dependencies:
- `@trigger.dev/core@4.3.3`
## trigger.dev@4.3.3
### Patch Changes
- Updated dependencies:
- `@trigger.dev/core@4.3.3`
- `@trigger.dev/build@4.3.3`
- `@trigger.dev/schema-to-json@4.3.3`
## @trigger.dev/core@4.3.3
### Patch Changes
- Add support for AI SDK v6 (Vercel AI SDK)
([#2919](https://github.com/triggerdotdev/trigger.dev/pull/2919))
- Updated peer dependency to allow `ai@^6.0.0` alongside v4 and v5
- Updated internal code to handle async validation from AI SDK v6's
Schema type
- Expose user-provided idempotency key and scope in task context.
`ctx.run.idempotencyKey` now returns the original key passed to
`idempotencyKeys.create()` instead of the hash, and
`ctx.run.idempotencyKeyScope` shows the scope ("run", "attempt", or
"global").
([#2903](https://github.com/triggerdotdev/trigger.dev/pull/2903))
- Fix batch trigger failing with "ReadableStream is locked" error when
network failures occur mid-stream. Added safe stream cancellation that
gracefully handles locked streams during retry attempts.
([#2917](https://github.com/triggerdotdev/trigger.dev/pull/2917))
- Add a maxDepth to flatten/unflattenAttributes to prevent possible
issues ([#2890](https://github.com/triggerdotdev/trigger.dev/pull/2890))
## @trigger.dev/python@4.3.3
### Patch Changes
- Updated dependencies:
- `@trigger.dev/sdk@4.3.3`
- `@trigger.dev/core@4.3.3`
- `@trigger.dev/build@4.3.3`
## @trigger.dev/react-hooks@4.3.3
### Patch Changes
- Updated dependencies:
- `@trigger.dev/core@4.3.3`
## @trigger.dev/redis-worker@4.3.3
### Patch Changes
- Updated dependencies:
- `@trigger.dev/core@4.3.3`
## @trigger.dev/rsc@4.3.3
### Patch Changes
- Updated dependencies:
- `@trigger.dev/core@4.3.3`
## @trigger.dev/schema-to-json@4.3.3
### Patch Changes
- Updated dependencies:
- `@trigger.dev/core@4.3.3`
## @trigger.dev/sdk@4.3.3
### Patch Changes
- Add support for AI SDK v6 (Vercel AI SDK)
([#2919](https://github.com/triggerdotdev/trigger.dev/pull/2919))
- Updated peer dependency to allow `ai@^6.0.0` alongside v4 and v5
- Updated internal code to handle async validation from AI SDK v6's
Schema type
- Expose user-provided idempotency key and scope in task context.
`ctx.run.idempotencyKey` now returns the original key passed to
`idempotencyKeys.create()` instead of the hash, and
`ctx.run.idempotencyKeyScope` shows the scope ("run", "attempt", or
"global").
([#2903](https://github.com/triggerdotdev/trigger.dev/pull/2903))
- Updated dependencies:
- `@trigger.dev/core@4.3.3`
---------
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
There’s an edge case that means runs can end up in the
currentConcurrency set when they’re not in the correct state for
execution. This means they will be permanently stuck in queued.
Given an environmentId this will fix those runs.
This is a temporary fix while we permanently fix the issue.
---------
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Eric Allam <ericallam@users.noreply.github.com>
When we auth API keys we get the environment, project and org. This is a
very hot path so even though these queries are fast they contribute a
significant percentage of total load.
This moves them to use the read replica instead.
Moving usage updates into the run engine to prevent inefficient &
additional incremental updates to the TaskRun table. Read/Modify/Write
pattern is safe inside of the run engine because of the run lock. We can
also now cap the usageDurationMs value from overflowing and causing an
error.
## Why?
This is preventing at least one update per TaskRun and instead updating
these values piggybacking on other updates.
## Aurora PostgreSQL Reader Consistency Notes
### TL;DR
Aurora readers share the same storage as the writer, but maintain
separate in-memory page caches. This means:
- **Storage is always consistent** - writes are synchronously committed
to shared storage
- **Page cache can lag** - typically <100ms, but can cause stale reads
if data is cached
### How It Works
1. Writer commits to shared storage (synchronous 4/6 quorum)
2. Writer sends cache invalidation messages to readers (asynchronous)
3. If reader has data in cache → returns cached (potentially stale)
value
4. If reader has cache miss → fetches from shared storage (always
current)
### Monitoring
```sql
SELECT server_id,
CASE WHEN session_id = 'MASTER_SESSION_ID' THEN 'Writer' ELSE 'Reader' END AS role,
replica_lag_in_msec
FROM aurora_replica_status();
```
Move expensive findMany queries for PENDING_VERSION and
WAITING_FOR_DEPLOY
runs to read replicas to avoid blocking migrations on the primary
database.
Changes:
- Add readOnlyPrisma to SystemResources type
- Pass readOnlyPrisma to systems in RunEngine constructor
- Update pendingVersionSystem to use readOnlyPrisma for findMany
- Update executeTasksWaitingForDeploy to use _replica for findMany
## Summary
- Adds `IF NOT EXISTS` to the migration that adds
`idempotencyKeyOptions` column to prevent errors if the column already
exists
## Migration Checksum Fix
If you've already applied the previous version of this migration, you'll
need to update the checksum in your `_prisma_migrations` table to match
the new migration file.
**Previous checksum:**
`f8876e274e3f7735312275eb24a9c4b40f512ac12a286b2de3add47f66df5b27`
**New checksum:**
`0620a914ddbaf01279576274432e51c41f41502cd4c8de38621625380750e397`
### Fix instructions
Run this SQL command against your database:
```sql
UPDATE "_prisma_migrations"
SET checksum = '0620a914ddbaf01279576274432e51c41f41502cd4c8de38621625380750e397'
WHERE migration_name = '20260116154810_add_idempotency_key_options_to_task_run';
```
This updates the stored checksum to match the modified migration file,
allowing future migrations to proceed without checksum mismatch errors.
## Test plan
- [x] Verified migration applies cleanly on fresh database
- [ ] Verified checksum update works on database with previous migration
applied
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Adds support for skipping Postgres migrations on container startup via
the new `SKIP_POSTGRES_MIGRATIONS` environment variable.
Set `SKIP_POSTGRES_MIGRATIONS=1` to skip migrations, matching the
existing behavior of `SKIP_CLICKHOUSE_MIGRATIONS`.
What changed
- Upgraded recharts to 2.15.2
- Added multiple chart types and components: big number, line, stacked,
bar (including zoomable & reference line), big dataset bar, and usage
graph
- Implemented custom legend with animated values, tooltip showing x-axis
data, and hover/highlight behaviors for stacks and legend
- Added loading, no-data, and invalid chart states plus loading spinners
and improved loading animations/layout
- Storybook integration: initial charts setup, separate chart files,
alphabetized menu, chart state toggles, and story updates
- Interaction & UX improvements: zooming (drag/select), crosshair
pointer, show/select dates while zooming, prevent text selection on
drag, hide mouse wheel zoom, capped legend items, axis/legend styling
tweaks, better spacing, and min-height for charts
- Data & state handling: moved date data to route for unified zooming,
moved chartState to main Chart component, moved hard-coded/mock data out
of components, and set chart data when zooming to start/end dates
- Performance & animation: turned off/reduced chart animations, sped up
animated numbers, removed hover transitions for bars
- New UI primitives and layout: Card component, small card updates, SVG
icons, improved segmented control and popover variants, table
improvements (resizable columns, filtering, sorting, scrolling fixes)
- Various fixes and polish: tooltip style fixes, legend value updates,
hover/leave state resets, bar width fixes for small datasets,
type/import fixes, and numerous small style/typo tweaks
---------
Co-authored-by: James Ritchie <james@trigger.dev>
## Summary
- Add support for Vercel AI SDK v6 as a peer dependency
- Update internal code to handle async validation from AI SDK v6's
Schema type
Closes#2918
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Eric Allam <ericallam@users.noreply.github.com>
When fetch crashes mid-stream during batch item upload (e.g., connection
reset, timeout), the request stream may remain locked by fetch's
internal reader. Attempting to cancel a locked stream throws 'Invalid
state: ReadableStream is locked', causing the batch operation to fail.
Added safeStreamCancel() helper that gracefully handles locked streams
by catching and ignoring the locked error. The stream will be cleaned up
by garbage collection when fetch eventually releases the reader.
Fixes customer issue where batchTrigger failed with ReadableStream
locked error during network instability.
## Summary
- Store the original user-provided idempotency key and scope alongside
the hash
- Expose `ctx.run.idempotencyKey` as the user-provided key (not the
hash)
- Add `ctx.run.idempotencyKeyScope` to show the scope ("run", "attempt",
or "global")
<img width="539" height="450" alt="CleanShot 2026-01-19 at 11 40 46"
src="https://github.com/user-attachments/assets/b6f42991-697e-4314-a164-aef77b8fd25c"
/>
## Problem
Idempotency keys were hashed (SHA-256) before storage, making debugging
difficult since users couldn't see the value they originally set or
search for runs by idempotency key.
## Solution
Attach metadata to the `String` object returned by
`idempotencyKeys.create()` using a Symbol, extract it in the SDK before
the API call, and store it in the database alongside the hash.
```typescript
const key = await idempotencyKeys.create("my-key", { scope: "global" });
await childTask.triggerAndWait(payload, { idempotencyKey: key });
// In child task:
ctx.run.idempotencyKey // "my-key" (previously showed the hash)
ctx.run.idempotencyKeyScope // "global"
```
Test plan
- Trigger task with idempotencyKeys.create() using different scopes (run, attempt, global)
- Verify ctx.run.idempotencyKey returns user-provided key
- Verify ctx.run.idempotencyKeyScope returns correct scope
- Verify PostgreSQL stores idempotencyKeyOptions JSON
- Verify ClickHouse receives idempotency_key_user and idempotency_key_scope via replication
---------
Co-authored-by: James Ritchie <james@trigger.dev>
Added documentation for the new Limits page feature that allows users to
view their current limits, quotas, and rate limit usage in real-time
from the dashboard. The page displays rate limit token availability,
quota usage, and plan features for organizations.
## Files changed
- `docs/limits.mdx` - Added introductory paragraph about the new Limits
page in the dashboard
Generated from [feat(webapp): New limits
page](https://github.com/triggerdotdev/trigger.dev/pull/2885) @samejr
Co-authored-by: mintlify[bot] <109931778+mintlify[bot]@users.noreply.github.com>
Co-authored-by: James Ritchie <james@trigger.dev>
Adds support to configure CPU/memory request ratios per machine preset.
Falls back to the global request ratio configs if no specific override
is specified.
Runs across different machine presets have different usage patters, so
this enables use to manage the available capacity better.
Closes #<issue>
## ✅ Checklist
- [ ] I have followed every step in the [contributing
guide](https://github.com/triggerdotdev/trigger.dev/blob/main/CONTRIBUTING.md)
- [ ] The PR title follows the convention.
- [ ] I ran and tested the code works
---
## Testing
_[Describe the steps you took to test this change]_
---
## Changelog
_[Short description of what has changed]_
---
## Screenshots
_[Screenshots]_
💯
---------
Co-authored-by: Matt Aitken <matt@mattaitken.com>
Summary
- Add nullable projectId field to TaskScheduleInstance.
- Create an index for TaskScheduleInstance.environmentId (added only if
it doesn’t exist, handled concurrently).
- Ensure TaskScheduleInstance.projectId is set everywhere in the
codebase.
Backfilling projectIds, once this is live
```sql
UPDATE "TaskScheduleInstance" tsi
SET "projectId" = ts."projectId"
FROM "TaskSchedule" ts
WHERE tsi."taskScheduleId" = ts."id";
```
* Removed EVENT_REPOSITORY_CLICKHOUSE_ROLLOUT_PERCENT
* Added hasLogsPageAccess featureFlag for logs page
* Replaced attributes with attributes_text for logs to reduce memory
usage and improve query performance
* Added support for event_v1 for logs, now depending on the settings the
logs are fetched either from `task_events_v1` or `task_events_v2`
* Show an error in the interface in cast the repository store is
`postgres`
## Summary
- Upgrades Node.js from 20.19.0 to 20.20.0 (and 22.12.0 to 22.22.0 for
supervisor) to address the async_hooks stack overflow DoS vulnerability
- Adds `maxDepth` parameter (default 128) to `flattenAttributes` and
`unflattenAttributes` to prevent stack overflow on maliciously deep
nested structures
## Details
The vulnerability (patched in Node.js 20.20.0, 22.22.0, 24.13.0, 25.3.0)
causes unrecoverable crashes (exit code 7) when stack overflow occurs
during async_hooks callbacks. Since the webapp uses `AsyncLocalStorage`,
it was theoretically vulnerable.
### Changes
**Node.js version updates:**
- `docker/Dockerfile`: 20.11.1 → 20.20.0
- `apps/supervisor/Containerfile`: 22-alpine → 22.22.0-alpine
- `.nvmrc`: 20.19.0 → 20.20.0
- `apps/supervisor/.nvmrc`: 22.12.0 → 22.22.0
- `references/prisma-7/.nvmrc`: 20.19.0 → 20.20.0
- All GitHub workflows: 20.19.0 → 20.20.0
**Defense in depth:**
- Added `maxDepth` parameter to `flattenAttributes()` and
`unflattenAttributes()` in `packages/core` to prevent stack overflow on
deeply nested user input
## Test plan
- [x] All existing `flattenAttributes` tests pass (50 tests)
- [x] New tests for depth limiting added
- [x] Verify Docker builds work with new base images
Fixed documentation examples to use correct 'data' field instead of
'output' for the waitpoint token completion endpoint.
The API schema expects 'data' in the request body, but all code examples
(curl, Python, Ruby, Go) incorrectly showed 'output', causing waitpoints
to complete with empty/undefined output when users followed the docs.
Fixes#2872
Closes #<issue>
## ✅ Checklist
- [x] I have followed every step in the [contributing
guide](https://github.com/triggerdotdev/trigger.dev/blob/main/CONTRIBUTING.md)
- [x] The PR title follows the convention.
- [x] I ran and tested the code works
---
## Testing
_[Describe the steps you took to test this change]_
This is simple doc fix
---
## Changelog
_[Short description of what has changed]_
Documentation example for the specific api route had a wrong field,
fixed that to have correct field
---
## Screenshots
_[Screenshots]_
<img width="780" height="252" alt="Screenshot 2026-01-14 at 10 49 50 PM"
src="https://github.com/user-attachments/assets/50a03f2c-edfb-4bd4-bb72-a0fd79e77216"
/>
The above image shows the correct request format, but docs previously
had incorrect payload.
<img width="723" height="307" alt="Screenshot 2026-01-14 at 11 24 01 PM"
src="https://github.com/user-attachments/assets/b9096225-3f7c-4511-b8c2-e8144c896900"
/>
This is the exact wrong field in docs, that was fixed
https://trigger.dev/docs/wait-for-token#from-another-language💯
Co-authored-by: appdevelopers9a <appdeveloper@s9alabs.com>
It’s useful to know when they were modified for debugging and auditing.
For existing rows createdAt and updatedAt are set to now() during the
migration, to avoid a nullable column.
When users set `secrets.enabled=false` to use an external secret via
`secrets.existingSecret`, the environment variables `SESSION_SECRET`,
`MAGIC_LINK_SECRET`, `ENCRYPTION_KEY`, and `MANAGED_WORKER_SECRET` were
not being populated from the secret.
Fixes#2859
Also adds automatic helm prereleases for PRs
---------
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: nicktrn <nicktrn@users.noreply.github.com>
Change EVENT_REPOSITORY_DEFAULT_STORE default from "postgres" to
"clickhouse_v2" so new deployments favor the ClickHouse v2 event store
by default. This updates runtime behavior to use the newer store
implementation unless explicitly overridden.
Don’t allow aliased columns to be queried – it was actually safe but
confusing. We call `created_at` -> `triggered_at` but we still allowed
created_at which was confusing.
Now we have nice errors if you try select columns that aren’t
selectable.
Also removed a ClickHouse setting `allow_experimental_object_type` which
worked fine locally but stopped all queries working on ClickHouse Cloud
🤦♂️
The Cancel button was missing an onClick handler to close the modal
dialog. This caused confusing behavior where clicking Cancel would not
dismiss the dialog. Also added type="button" to prevent form submission
since the button is inside a form.
Co-authored-by: Claude <noreply@anthropic.com>
## Summary
Optimizes the runs replication service for better CPU efficiency and
throughput when inserting task runs into ClickHouse.
### Key Changes
- **Switch to compact array format** - Uses
`JSONCompactEachRowWithNames` instead of `JSONEachRow` for ClickHouse
inserts, reducing JSON serialization overhead
- **Type-safe tuple arrays** - Introduces `TaskRunInsertArray` and
`PayloadInsertArray` tuple types with compile-time column order
validation
- **Pre-sorted batch inserts** - Sorts inserts by primary key before
flushing for better ClickHouse insert performance
- **Programmatic index generation** - `TASK_RUN_INDEX` and
`PAYLOAD_INDEX` are generated from column arrays to prevent manual
synchronization errors
### Files Changed
- `runsReplicationService.server.ts` - Core optimization to use compact
array inserts
- `@internal/clickhouse` - Added `insertCompactRaw` method and tuple
types
- `taskRuns.ts` - Column definitions, index constants, and insert
functions
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Closes #<issue>
## ✅ Checklist
- [x] I have followed every step in the [contributing
guide](https://github.com/triggerdotdev/trigger.dev/blob/main/CONTRIBUTING.md)
- [x] The PR title follows the convention.
- [x] I ran and tested the code works
---
## Testing
- Verified log detail view displays correctly with message, metadata,
and attributes
- Tested search highlighting functionality in log messages (escapes
special regex characters)
- Confirmed tabs (Details/Run) switch properly with keyboard shortcuts
(d/r)
- Verified run information loads via async fetcher in Run tab
- Tested close button and Escape key for dismissing the panel
- Verified log details display correct information: level badges, kind
badges, timestamps, trace IDs, span IDs
- Confirmed links to parent spans and run pages work correctly
- Tested with various log levels (ERROR, WARN, INFO, DEBUG, TRACE) and
kinds (SPAN, SPAN_EVENT, LOG_*)
- Verified admin-only fields display correctly when user has admin
access
- Tested data loading states and error states (log not found, run not
found)
---
## Changelog
Created new Logs page.
The information shown is gathered from the spans from each run.
The feature supports all run filters with two new filters for level and
logs text search.
---
## Screenshots
<img width="2059" height="1196" alt="Logs page preview"
src="https://github.com/user-attachments/assets/70b667b4-98cc-4728-855a-2766dd5c1aa5"
/>
💯
---------
Co-authored-by: James Ritchie <james@trigger.dev>
Summary
- Improve query experience and safety across ClickHouse and TSQL.
Changes
- Display JSON columns when in non-pretty mode (no longer show [Object
Object]).
- Sanitize ClickHouse errors originating from TSQL.
- Remove tenant details from errors.
- Add AI-assisted error-fixing for queries.
- Improve code quality and readability.
- Provide autocomplete support for enum values.
- Enforce limits on ClickHouse queries (10s query limit).
- Add org-level and global concurrency limits.
- Warn and train AI to avoid SELECT *; when used, only return core
columns and show info.
- If AI suggests no time range, default to past 7 days.
- Format the default query for readability.
- Add an admin-only EXPLAIN button.
- Prevent impersonation queries from being saved to history.
**Background**
Runs with `large-1x` or `large-2x` machine presets are disproportionally
affected by scheduling delays during peak times. This is in part caused
by the fact that the worker pool is shared for all runs, meaning large
runs compete with smaller runs for available capacity. Because large
runs require significantly more CPU and memory, they are harder for the
scheduler to bin-pack onto existing nodes, often requiring a node with a
significant amount of free resources or waiting for a new node to spin
up entirely. This effect is amplified during peak times when nodes are
already densely packed with smaller workloads, leaving insufficient
contiguous resources for large runs. Also, large runs make up a small
percentage of the total runs.
**Changes**
This PR adds Kubernetes node affinity settings to separate large and
standard machine workloads across node pools.
- Controlled via `KUBERNETES_LARGE_MACHINE_POOL_LABEL` env var (disabled
when not set)
- Large machine presets (large-*) get a soft preference to schedule on
the large pool, with fallback to standard nodes
- Non-large machines are excluded from the large pool via required
anti-affinity
- This ensures the large machine pool is reserved for large workloads
while allowing large workloads to spill over to standard nodes if needed
UI/UX improvement to the date/time picker:
- You can now choose a custom duration
- Adds a new DateTimePicker.tsx component, using a new shadcn
Calendar.tsx component
- Clear UI separation between the 2 actions, choosing a duration or
choosing date range
- Adds new quick select options for picking a date range quickly
https://github.com/user-attachments/assets/6b59b49d-2a56-4354-ad72-d8426437e56e
---------
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Eric Allam <ericallam@users.noreply.github.com>
- Add CLAUDE.md providing Claude Code guidance and documenting the
Claude Code skill
- Add trigger-dev-tasks skill to assist writing Trigger.dev tasks
- Add SDK rules version 4.3.0 including batch trigger v2 and debouncing
features
TRQL (pronounced Treacle like the delicious British dark sweet syrup) is
the TRiggerQueryLanguage. It allows users to safely write queries on
their data. The queries are safely turned into ClickHouse queries which
are tenant-safe and not SQL injectable.
https://github.com/user-attachments/assets/bbfca473-b3fc-4150-8fe6-79e8840a2d29
This started out as a translation of HogQL by PostHog from Python to
TypeScript.
Features
- Tenant safe queries.
- Many underlying ClickHouse features including functions and
aggregations.
- Virtual columns, which are exposed to users as real columns but are
actually expressions.
- Transformations of data types and where clauses.
- Simple JSON path querying.
- Limits on execution time.
- Reporting of query statistics.
## Query page
There’s a new Query page (currently behind a feature flag) where you can
write TRQL queries and execute them against your environment, project or
organization.
Features
- Executing TRQL queries
- Syntax highlighting and errors
- Autocomplete
- AI generation/editing of queries
- Help and examples
- Table with auto-inferred data types from the table schema
- Table cell renderers for our special types like Run ids, environments,
machines, tasks, queues, etc.
- Copy/export as CSV/JSON
- Line and bar graphs with grouping and stacking
- History of queries
Replaces `redirectDocument` with `useFetcher` for editing environment
variables. This allows background form submission without full page
reload, which preserves:
- Scroll position in the env vars list
- "Reveal values" toggle state
- Search filter state
Fixes#2845
Generated with [Claude Code](https://claude.ai/code)
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Eric Allam <ericallam@users.noreply.github.com>
Updated the billing alerts documentation to reflect the new spike alerts
feature added in PR #2829. The documentation now explains both standard
alerts (75%, 90%, 100%, 200%, 500%) and spike alerts (10x, 20x, 50x,
100x) that help catch runaway usage from bugs or errors.
## Files changed
- `docs/how-to-reduce-your-spend.mdx` - Added section explaining the two
types of billing alerts
Generated from [Chore(webapp): Adds additional billing
alerts](https://github.com/triggerdotdev/trigger.dev/pull/2829) @samejr
Co-authored-by: mintlify[bot] <109931778+mintlify[bot]@users.noreply.github.com>
Add colored console warnings when the event loop is blocked and wire
a feature flag to enable/disable notifications- Introduce
notifyEventLoopBlocked() in eventLoopMonitor.server.ts to
log a colored warning with blocked and async type.
- Call notifyEventLoopBlocked() when an event-loop stall is detected.
- Add EVENT_LOOP_MONITOR_NOTIFY_ENABLED to env schema with a default of
"0" so notifications are off by default.
- Will notify when over the `EVENT_LOOP_MONITOR_THRESHOLD_MS` env var
This makes it easier to spot long event-loop stalls during development
or when notifications are explicitly enabled.
<img width="840" height="132" alt="CleanShot 2026-01-07 at 15 03 24@2x"
src="https://github.com/user-attachments/assets/be20fa6a-be2b-46a1-aa89-d0913ed8b5b3"
/>
This PR fixes some issues with the new BatchQueue by implementing the
full two-phase dequeue process in the FairQueue, and moving the
responsibility of consuming the worker queue to the BatchQueue and
independently enabling it via the `BATCH_QUEUE_WORKER_QUEUE_ENABLED` env
var. We've also introduced the `BATCH_QUEUE_SHARD_COUNT` env var to
control the count of master queue shards in the FairQueue. We can also
control how many queues are considered in each iteration of the master
queue consumer via the `BATCH_QUEUE_MASTER_QUEUE_LIMIT` env var.
This PR will also now skip trying to dequeue from tenants that are at
concurrency capacity, which should lead to fewer issues with low
concurrency tenants blocking higher concurrency tenants from processing.
The Override concurrency limit modal has 2 type="submit" buttons. The
first one in the DOM was firing when the "enter" key is hit which
canceled and reset the limit instead which is a bad UX.
### The fix
This fix adds a hidden button above in the DOM order which mirrors the
Update Override button. Having a double submit button is rare in our
modals so feels safe to add this to the specific modal that needs it.
### Alternative solution
Switching the order of the buttons in the main FormButton component,
then using `flex-row-reverse` to flip them back in CSS works, but it
reverses the tab order. Adding a `tabIndex` to fix that issue didn't
seem to work reliably.
### UI Improvements to the Concurrency page:
- Truncates long branch names and includes a tooltip
- The Tables have a new variant if you don't want the rows to highlight
on hover
- Small fix to pluralize some words in the purchase modal
- Fix to prevent tooltip buttons being `type=submit`
- Updates the /limits docs page to include purchasing more concurrency
- Adds a clear banner when you have a positive balance of unallocated
concurrency
https://github.com/user-attachments/assets/54d927c3-84e3-4d55-8f42-726098f4daf0
- Adds 4 additional alert thresholds to ensure customers are emailed if
they have runaway usage.
- Separated these into a new section called "Spike alerts" with a
tooltip so it's clear what they are.
- Tooltip message is: "Catch runaway usage from bugs or errors. We
recommend keeping these enabled as a safety net."
- A billing service PR now returns all orgs to populate the email list,
rather than oldest 5.
- Adds `defaultChecked` logic to honour existing orgs who have
configured alerts in the DB. New orgs get all alerts checked on by
default.
<img width="1316" height="1560" alt="CleanShot 2026-01-05 at 09 43
56@2x"
src="https://github.com/user-attachments/assets/ce749407-2b7f-4864-9c09-9333c5ac495a"
/>
Fixes#2835
There were still some flags in here we removed, deploying is a lot
simpler now for self-hosters.
Also updates the github actions guide.
---------
Co-authored-by: Claude <noreply@anthropic.com>
**Improvements to the run ID copy button and run navigation buttons for
consistency**
- Adds some x-padding and layout adjustment to the copy ID button.
<img width="664" height="114" alt="CleanShot 2025-12-19 at 16 05 21@2x"
src="https://github.com/user-attachments/assets/ebc8e0de-011b-419c-bdcc-eb4157553d1c"
/>
- New custom navigation icons that work better at tiny sizes
<img width="330" height="196" alt="CleanShot 2025-12-19 at 16 06 38@2x"
src="https://github.com/user-attachments/assets/bfd8d6b8-8a65-4eac-9ce1-d70acf0ad265"
/>
Some other small improvements/fixes:
- Fixes a browser html error where there was a <button> inside a
<button>
- Updates the shortcut description to match the tooltip text for
consistency
- Made the hover states more consistent
- The shortcut bar at the bottom snaps to the list sooner because there
are more items now
Closes #<issue>
## ✅ Checklist
- [ ] I have followed every step in the [contributing
guide](https://github.com/triggerdotdev/trigger.dev/blob/main/CONTRIBUTING.md)
- [ ] The PR title follows the convention.
- [ ] I ran and tested the code works
---
## Testing
_[Describe the steps you took to test this change]_
---
## Changelog
_[Short description of what has changed]_
---
## Screenshots
_[Screenshots]_
💯
This fixes a regression introduced in #2778 - stable sort is required
for deterministic builds, but we can safely preserve order for the user
package.json during package updates
This PR was opened by the [Changesets
release](https://github.com/changesets/action) GitHub action. When
you're ready to do a release, you can merge this and publish to npm
yourself or [setup this action to publish
automatically](https://github.com/changesets/action#with-publishing). If
you're not ready to do a release yet, that's fine, whenever you add more
changesets to main, this PR will be updated.
# Releases
## @trigger.dev/build@4.3.1
### Patch Changes
- Updated dependencies:
- `@trigger.dev/core@4.3.1`
## trigger.dev@4.3.1
### Patch Changes
- Updated dependencies:
- `@trigger.dev/core@4.3.1`
- `@trigger.dev/build@4.3.1`
- `@trigger.dev/schema-to-json@4.3.1`
## @trigger.dev/core@4.3.1
### Patch Changes
- Added support for idempotency reset
([#2777](https://github.com/triggerdotdev/trigger.dev/pull/2777))
## @trigger.dev/python@4.3.1
### Patch Changes
- Updated dependencies:
- `@trigger.dev/sdk@4.3.1`
- `@trigger.dev/core@4.3.1`
- `@trigger.dev/build@4.3.1`
## @trigger.dev/react-hooks@4.3.1
### Patch Changes
- Updated dependencies:
- `@trigger.dev/core@4.3.1`
## @trigger.dev/redis-worker@4.3.1
### Patch Changes
- Updated dependencies:
- `@trigger.dev/core@4.3.1`
## @trigger.dev/rsc@4.3.1
### Patch Changes
- Updated dependencies:
- `@trigger.dev/core@4.3.1`
## @trigger.dev/schema-to-json@4.3.1
### Patch Changes
- Updated dependencies:
- `@trigger.dev/core@4.3.1`
## @trigger.dev/sdk@4.3.1
### Patch Changes
- feat: Support for new batch trigger system
([#2779](https://github.com/triggerdotdev/trigger.dev/pull/2779))
- feat(sdk): Support debouncing runs when triggering with new debounce
options
([#2794](https://github.com/triggerdotdev/trigger.dev/pull/2794))
- Added support for idempotency reset
([#2777](https://github.com/triggerdotdev/trigger.dev/pull/2777))
- Updated dependencies:
- `@trigger.dev/core@4.3.1`
---------
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Add support for resetting idempotency keys both from ui and sdk
## ✅ Checklist
- [x] I have followed every step in the [contributing
guide](https://github.com/triggerdotdev/trigger.dev/blob/main/CONTRIBUTING.md)
- [x] The PR title follows the convention.
- [x] I ran and tested the code works
---
## Testing
- Created a new run with a idempotency idempotencyKey.
- Started a new run with the same task and got redirected to the first
run.
- Deleted the key from the UI on the run details
- Started a new run with the same task and it created a new one
- Did the above steps using the SDK
---
## Changelog
- Add new action route for resetting idempotency keys via UI
- Add reset button in Idempotency section of run detail view
- Added API and SDK for resetting imdepotency
- Updated docs page for this feature
---
## Screenshots
_[Screenshots]_
<img width="438" height="363" alt="Screenshot 2025-12-11 at 11 56 37"
src="https://github.com/user-attachments/assets/30b8ef5e-8aac-4d04-b57a-9bf30d085dcb"
/>
Adds support for **debounced task runs** - when triggering a task with a
debounce key, subsequent triggers with the same key will reschedule the
existing delayed run instead of creating new runs. This continues until
no new triggers occur within the delay window.
## Usage
```typescript
await myTask.trigger({ userId: "123" }, {
debounce: {
key: "user-123-update",
delay: "5s",
mode: "leading", // default
}
});
```
- **key**: Scoped to the task identifier
- **delay**: How long to wait before executing (supports duration
strings like `"5s"`, `"1m"`)
- **mode**: Either `"leading"` or `"trailing"`. Leading debounce will
use the payload and options from the first run created with the debounce
key. Trailing will use payload and options from the last run.
### "trailing" mode overrides
When using `mode: "trailing"` with debounce, the following options are
updated from the **last** trigger:
- **`payload`** - The task input data
- **`metadata`** - Run metadata
- **`tags`** - Run tags (replaces existing tags)
- **`maxAttempts`** - Maximum retry attempts
- **`maxDuration`** - Maximum compute time
- **`machine`** - Machine preset (cpu/memory)
## Behavior
- **First run wins**: The first trigger creates the run, subsequent
triggers push its execution time later
- **Idempotency keys take precedence**: If both are specified,
idempotency is checked first
- **Max duration**: Configurable via `DEBOUNCE_MAX_DURATION_MS` env var
(default: 10 minutes)
Works with `triggerAndWait` - parent runs correctly block on the
debounced run.
New batch trigger system with larger payloads, streaming ingestion,
larger batch sizes, and a fair processing system.
This PR introduces a new `FairQueue` abstraction inspired by our own
`RunQueue` that enables multi-tenant fair queueing with concurrency
limits. The new `BatchQueue` is built on top of the `FairQueue`, and
handles processing Batch triggers in a fair manner with per-environment
concurrency limits defined per-org. Additionally, there is a global
concurrency limit to prevent the BatchQueue system from creating too
many runs too quickly, which can cause downstream issues.
For this new BatchQueue system we have a completely new batch trigger
creation and ingestion system. Previously this was a single endpoint
with a single JSON body that defined details about the batch as well as
all the items in the batch.
We're introducing a two-phase batch trigger ingestion system. In the
first phase, the BatchTaskRun record is created (and possibly rate
limited). The second phase is another endpoint that accepts an NDJSON
body with each line being a single item/run with payload and options.
At ingestion time all items are added to a queue, in order, and then
processed by the BatchQueue system.
## New batch trigger rate limits
This PR implements a new batch trigger specific rate limit, configured
on the `Organization.batchRateLimitConfig` column, and defaults using
these environment variables:
- `BATCH_RATE_LIMIT_REFILL_RATE` defaults to 10
- `BATCH_RATE_LIMIT_REFILL_INTERVAL` the duration interval, defaults to
`"10s"`
- `BATCH_RATE_LIMIT_MAX` defaults to 1200
This rate limiter is scoped to the environment ID and controls how many
runs can be submitted via batch triggers per interval. The SDK handles
the retrying side.
## Batch queue concurrency limits
The new column `Organization.batchQueueConcurrencyConfig` now defines an
org specific `processingConcurrency` value, with a backup of the env var
`BATCH_CONCURRENCY_LIMIT_DEFAULT` which defaults to 10. This controls
how many batch queue items are processed concurrently per environment.
There is also a global rate limit for the batch queue set via the
`BATCH_QUEUE_GLOBAL_RATE_LIMIT` which defaults to being disabled. If
set, the entire batch queue system won't process more than
`BATCH_QUEUE_GLOBAL_RATE_LIMIT` items per second. This allows
controlling the maximum number of runs created per second via batch
triggers.
## Batch trigger settings
- `STREAMING_BATCH_MAX_ITEMS` controls the maximum number of items in a
single batch
- `STREAMING_BATCH_ITEM_MAXIMUM_SIZE` controls the maximum size of each
item in a batch
- `BATCH_CONCURRENCY_DEFAULT_CONCURRENCY` controls the default
environment concurrency
- `BATCH_QUEUE_DRR_QUANTUM` how many credits each environment gets each
round for the DRR scheduler
- `BATCH_QUEUE_MAX_DEFICIT` the maximum deficit for the DRR scheduler
- `BATCH_QUEUE_CONSUMER_COUNT` how many queue consumers to run
- `BATCH_QUEUE_CONSUMER_INTERVAL_MS` how frequently they poll for items
in the queue
### Configuration Recommendations by Use Case
**High-throughput priority (fairness acceptable at 0.98+):**
```env
BATCH_QUEUE_DRR_QUANTUM=25
BATCH_QUEUE_MAX_DEFICIT=100
BATCH_QUEUE_CONSUMER_COUNT=10
BATCH_QUEUE_CONSUMER_INTERVAL_MS=50
BATCH_CONCURRENCY_DEFAULT_CONCURRENCY=25
```
**Strict fairness priority (throughput can be lower):**
```env
BATCH_QUEUE_DRR_QUANTUM=5
BATCH_QUEUE_MAX_DEFICIT=25
BATCH_QUEUE_CONSUMER_COUNT=3
BATCH_QUEUE_CONSUMER_INTERVAL_MS=100
BATCH_CONCURRENCY_DEFAULT_CONCURRENCY=5
```
## Changelog
- Add disableAdjacentRows prop to TaskRunsTable component to control
table state encoding
- Pass rootOnlyDefault prop from loader to TaskRunsTable for proper
state management
- Disable adjacent run navigation in schedule, waitpoint, and other
inspector views
- Preserve adjacent run navigation on main runs list page with rootOnly
filter support
This PR was opened by the [Changesets
release](https://github.com/changesets/action) GitHub action. When
you're ready to do a release, you can merge this and publish to npm
yourself or [setup this action to publish
automatically](https://github.com/changesets/action#with-publishing). If
you're not ready to do a release yet, that's fine, whenever you add more
changesets to main, this PR will be updated.
# Releases
## trigger.dev@4.3.0
### Minor Changes
- feat(cli): deterministic image builds for deployments
([#2778](https://github.com/triggerdotdev/trigger.dev/pull/2778))
- feat(cli): enable zstd compression for deployment images
([#2773](https://github.com/triggerdotdev/trigger.dev/pull/2773))
### Patch Changes
- The new `triggeredVia` field is now populated in deployments via the
CLI. ([#2767](https://github.com/triggerdotdev/trigger.dev/pull/2767))
- fix(dev): stop max listeners exceeded warning messages when running
more than 10 runs concurrently
([#2771](https://github.com/triggerdotdev/trigger.dev/pull/2771))
- Upgrade @modelcontextprotocol/sdk to 1.24.3
([#2768](https://github.com/triggerdotdev/trigger.dev/pull/2768))
- Updated dependencies:
- `@trigger.dev/core@4.3.0`
- `@trigger.dev/build@4.3.0`
- `@trigger.dev/schema-to-json@4.3.0`
## @trigger.dev/core@4.3.0
### Minor Changes
- feat(cli): deterministic image builds for deployments
([#2778](https://github.com/triggerdotdev/trigger.dev/pull/2778))
### Patch Changes
- The new `triggeredVia` field is now populated in deployments via the
CLI. ([#2767](https://github.com/triggerdotdev/trigger.dev/pull/2767))
## @trigger.dev/build@4.3.0
### Patch Changes
- Updated dependencies:
- `@trigger.dev/core@4.3.0`
## @trigger.dev/python@4.3.0
### Patch Changes
- Updated dependencies:
- `@trigger.dev/core@4.3.0`
- `@trigger.dev/build@4.3.0`
- `@trigger.dev/sdk@4.3.0`
## @trigger.dev/react-hooks@4.3.0
### Patch Changes
- Updated dependencies:
- `@trigger.dev/core@4.3.0`
## @trigger.dev/redis-worker@4.3.0
### Patch Changes
- Updated dependencies:
- `@trigger.dev/core@4.3.0`
## @trigger.dev/rsc@4.3.0
### Patch Changes
- Updated dependencies:
- `@trigger.dev/core@4.3.0`
## @trigger.dev/schema-to-json@4.3.0
### Patch Changes
- Updated dependencies:
- `@trigger.dev/core@4.3.0`
## @trigger.dev/sdk@4.3.0
### Patch Changes
- Updated dependencies:
- `@trigger.dev/core@4.3.0`
---------
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
This PR makes our image builds deterministic and reproducible by
ensuring that identical source code always produces the same image
layers and image digest. This means that deployments where nothing has
changed will no longer invalidate the image cache in our worker cluster
nodes, thus avoid making the cold starts for runs worse.
**Context**
New deployments currently increase the cold start times for runs, as
they generate a new image which needs to be pulled in the worker cluster
where runs are executed. It happens also when the source code for the
deployment has not changed due to non-deterministic steps in our build
system. This addresses the latter issue by making builds reproducible.
**Main changes**
- Avoided baking `TRIGGER_DEPLOYMENT_ID` and
`TRIGGER_DEPLOYMENT_VERSION` in the image, we now pass these via the
supervisor instead.
- Used `json-stable-stringify` for consistent key ordering in the files
we generate for the build, e.g., `package.json`, `build.json`,
`index.json`.
- Removed `metafile.json` from the image contents as it is not actually
used in the container. This is only relevant for the `analyze` command.
- Added `SOURCE_DATE_EPOCH=0` and `rewrite-timestamp=true` to Docker
builds to normalize file timestamps.
- Removed some `timings` and `outputHashes` from build outputs and
manifests.
The builds are now reproducible for both native build server and Depot
paths. This should also lead to better image layer cache reuse in
general.
- Add replace prop to LinkButton to use history replacement for adjacent
run navigation
- Preserve span and tab params when navigating between adjacent runs
- Disable animations for completed spans in timeline to improve
performance
- Include spanId in runs list navigation for better context preservation
- Direct link to task test page when filtering by single task with no
runs
- Fix minor styling issue with run friendlyId display padding
## ✅ Checklist
- [x] I have followed every step in the [contributing
guide](https://github.com/triggerdotdev/trigger.dev/blob/main/CONTRIBUTING.md)
- [x] The PR title follows the convention.
- [x] I ran and tested the code works
---
## Testing
Set up the local github application, and tested its connection with
trigger.dev
Checked:
- Change is backwards compatible
- Actions/ux are uniform across the pages (empty Tasks, Deployments,
project settings)
- Connecting GH, Connecting Repo, disconnecting Repo, modifying settings
---
## Changelog
- Create new resource route for GitHub settings management with loader
and actions
- Add GitHubSettingsPresenter to fetch connected repos and installations
- Implement GitHubSettingsPanel component for reusable GitHub
configuration UI
- Refactor project settings page to use shared GitHubSettingsPanel
component
- Integrate GitHub connection flow into empty state onboarding for Tasks
and Deployments
- Add support for GitHub repo connection, disconnection, and branch
tracking settings
- Include redirect URL support for seamless navigation after GitHub
actions
- Remove duplicate GitHub connection code from project settings route
---
## Screenshots
https://github.com/user-attachments/assets/8fc24699-640b-4f9e-afd8-b26edc945218🐐
---------
Co-authored-by: James Ritchie <james@trigger.dev>
This will speed up ice cold starts (*) for two reasons:
- better compression ratio
- faster decompression
This is a minor release because zstd compression will now be enabled by
default for all deployments.
(*) ice cold starts happen when deploy images are not cached on the
worker node yet. These cold start durations are highly dependent on
image size and as it turns out, also the type of compression used.
Fixed the issue where Cmd+Left Arrow was being intercepted by the
TreeView component on task runs screen.
Solution:
Added a check in the getTreeProps keyboard handler to detect when
metaKey (Cmd on macOS) is pressed with Left Arrow. When detected, the
handler returns early without preventing the default browser behavior,
allowing Chrome's native back navigation to work.
Closes #<issue>
## ✅ Checklist
- [ x] I have followed every step in the [contributing
guide](https://github.com/triggerdotdev/trigger.dev/blob/main/CONTRIBUTING.md)
- [ x] The PR title follows the convention.
- [ x] I ran and tested the code works
---
## Testing
Went to the task details page and confirmed that Cmd + Left Arrow will
navigate back
---
## Changelog
Modified TreeView.tsx to check for e.metaKey before handling Left Arrow
key events
When Cmd+Left is pressed, the event is no longer prevented, allowing
browser default behavior
---
Co-authored-by: Mihai Popescu <mihaipopescu@Mihais-MacBook-Pro.local>
Deployments are affected by general API rate limits, this is just a
quick fix by whitelisting the deployment related endpoints. In a follow
up PR we'll add a separate rate limiter for this group of endpoints.
## ✅ Checklist
- [x] I have followed every step in the [contributing
guide](https://github.com/triggerdotdev/trigger.dev/blob/main/CONTRIBUTING.md)
- [x] The PR title follows the convention.
- [x] I ran and tested the code works
---
## Testing
Manual testing of the task run pages
---
## Changelog
- Add previous/next run navigation buttons to run detail page header
- Support [ and ] keyboard shortcuts to jump between adjacent runs
- Preserve runs table state (filters, pagination) when navigating
- Preload adjacent page runs at boundaries for seamless navigation
- Add actions prop to PageTitle component
- Document shortcut in keyboard shortcuts panel
- Store current filter state from runs table as `tableState` search
param when navigating to individual run pages
- Restore filters when navigating back from run detail view to runs list
- Update `v3RunPath` and `v3RunSpanPath` helpers to accept optional
searchParams
- Use `useOptimisticLocation` to capture current search params in
TaskRunsTable
- Parse `tableState` param in run detail route and pass filters to back
button
- This improves UX by remembering filter selections (task, status, date
range, etc.) when users click into a run and then navigate back to the
runs list
- Add new text-below variant that shows "Click to copy" tooltip on hover
and "Copied" on click. Also add controlled open/onOpenChange props to
SimpleTooltip for managing tooltip visibility.
---
## Screenshots
https://github.com/user-attachments/assets/5067bbe0-1bcd-4e75-80a7-f56dabd5ed69
Changes in this PR:
- Arbitrary refs are now allowed when triggering the release workflow
manually (ref must be on the main branch).
- Release summary is now displayed in the GH job output; makes for a
nicer experience when approving the release workflow.
<!-- CURSOR_SUMMARY -->
> [!NOTE]
> Updates `BatchTriggerV3Service` to send `batch.id` (not `friendlyId`)
as `batchId` to `TriggerTaskService.call`.
>
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
68bf8df4a29bdcac5bfb2806bc417541f69ffd6e. This will update automatically
on new commits. Configure
[here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->
<!-- CURSOR_SUMMARY -->
> [!NOTE]
> Centralizes SIGTERM handling in `DevSupervisor` and removes per-run
SIGTERM listeners in `DevRunController` to avoid
MaxListenersExceededWarning under high concurrency.
>
> - **Dev runtime**:
> - **SIGTERM handling**: Add centralized handler in
`packages/cli-v3/src/dev/devSupervisor.ts` to gracefully stop all run
controllers; unregisters on `shutdown()`.
> - **Cleanup**: Remove per-controller `SIGTERM` listener and handler
from `packages/cli-v3/src/entryPoints/dev-run-controller.ts` to reduce
event listeners and warnings.
> - **Changeset**: Add patch note in
`.changeset/fuzzy-ghosts-admire.md`.
>
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
5ad2f5341829cebf6fd37a3c616a2db5e4ad936a. This will update automatically
on new commits. Configure
[here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->
This PR applies a small change to the deployments table to keep track
of:
- where the deployment was triggered from
- build server metadata, if the build server was involved
We recently upgraded the ECR sdk version. Our ECR repo exists check
relies on the type of the error thrown and the new ECR sdk version seems
to have broken that behavior. This PR adds a workaround to the issue.
The manual trigger is currently enabled only for the prerelease job.
This PR adds it for the normal release flow too, as it is useful to
retrigger failed release workflows.
This PR was opened by the [Changesets
release](https://github.com/changesets/action) GitHub action. When
you're ready to do a release, you can merge this and publish to npm
yourself or [setup this action to publish
automatically](https://github.com/changesets/action#with-publishing). If
you're not ready to do a release yet, that's fine, whenever you add more
changesets to main, this PR will be updated.
# Releases
## trigger.dev@4.2.0
### Minor Changes
- feat(cli): upgrade bun deployments to v1.3.3
([#2756](https://github.com/triggerdotdev/trigger.dev/pull/2756))
### Patch Changes
- fix(otel): exported logs and spans will now have matching trace IDs
([#2724](https://github.com/triggerdotdev/trigger.dev/pull/2724))
- The `--force-local-build` flag is now renamed to just `--local-build`
([#2702](https://github.com/triggerdotdev/trigger.dev/pull/2702))
- fix(cli): header will always print the correct profile
([#2728](https://github.com/triggerdotdev/trigger.dev/pull/2728))
- feat: add ability to set custom resource properties through
trigger.config.ts or via the OTEL_RESOURCE_ATTRIBUTES env var
([#2704](https://github.com/triggerdotdev/trigger.dev/pull/2704))
- feat(cli): implements content-addressable store for the dev CLI build
outputs, reducing disk usage
([#2725](https://github.com/triggerdotdev/trigger.dev/pull/2725))
- Added support for native build server builds in the deploy command
(`--native-build-server`)
([#2702](https://github.com/triggerdotdev/trigger.dev/pull/2702))
- Updated dependencies:
- `@trigger.dev/build@4.2.0`
- `@trigger.dev/core@4.2.0`
- `@trigger.dev/schema-to-json@4.2.0`
## @trigger.dev/build@4.2.0
### Patch Changes
- syncVercelEnvVars to skip API and read env vars directly from
env.process for Vercel build environments. New syncNeonEnvVars build
extension for syncing environment variablesfrom Neon database projects
to Trigger.dev. The extension automatically detects branches and builds
appropriate PostgreSQL connection strings for non-production, non-dev
environments (staging, preview).
([#2729](https://github.com/triggerdotdev/trigger.dev/pull/2729))
- Updated dependencies:
- `@trigger.dev/core@4.2.0`
## @trigger.dev/core@4.2.0
### Patch Changes
- fix: prevent ERR_IPC_CHANNEL_CLOSED errors from causing an unhandled
exception on TaskRunProcess
([#2743](https://github.com/triggerdotdev/trigger.dev/pull/2743))
- Added support for native build server builds in the deploy command
(`--native-build-server`)
([#2702](https://github.com/triggerdotdev/trigger.dev/pull/2702))
## @trigger.dev/python@4.2.0
### Patch Changes
- Updated dependencies:
- `@trigger.dev/build@4.2.0`
- `@trigger.dev/sdk@4.2.0`
- `@trigger.dev/core@4.2.0`
## @trigger.dev/react-hooks@4.2.0
### Patch Changes
- fix: prevent infinite useEffect when passing an array of tags to
useRealtimeRunsWithTag
([#2705](https://github.com/triggerdotdev/trigger.dev/pull/2705))
- Updated dependencies:
- `@trigger.dev/core@4.2.0`
## @trigger.dev/redis-worker@4.2.0
### Patch Changes
- Updated dependencies:
- `@trigger.dev/core@4.2.0`
## @trigger.dev/rsc@4.2.0
### Patch Changes
- Updated dependencies:
- `@trigger.dev/core@4.2.0`
## @trigger.dev/schema-to-json@4.2.0
### Patch Changes
- Updated dependencies:
- `@trigger.dev/core@4.2.0`
## @trigger.dev/sdk@4.2.0
### Patch Changes
- fix(sdk): Re-export schemaTask types to prevent the TypeScript error
TS2742: The inferred type of 'task' cannot be named without a reference
to '@trigger.dev/core/v3'. This is likely not portable.
([#2735](https://github.com/triggerdotdev/trigger.dev/pull/2735))
- feat: add ability to set custom resource properties through
trigger.config.ts or via the OTEL_RESOURCE_ATTRIBUTES env var
([#2704](https://github.com/triggerdotdev/trigger.dev/pull/2704))
- Updated dependencies:
- `@trigger.dev/core@4.2.0`
---------
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
**Background**
Currently the changeset PR creation and the publishing is handled by the
same workflow. This is not ideal:
- The build steps are executed on every run of the pipeline, even though
they're only needed for the publish case.
- The PR creation workflow does not need permissions to publish to npm,
only the release path needs them.
- Adding an approval step is painful as we'd need to also approve each
changeset PR creation workflow run.
**Changes in this PR**
- Separated the changeset PR creation into its own workflow and minimum
permission set.
- Added a GH environment with an approval step for the package
publishing workflow (also for prereleases).
- New publish workflow runs will not cancel in-progress runs; helps
avoid partial failures in publishing.
These changes also enable hardening the npm OIDC setup by tying it to a
GH environment that requires approval.
Vercel's NeonDB integration renders database connection environment
variables at runtime, which means Trigger.dev cannot directly sync these
values during the build process. This change adds support for fetching
branch-specific NeonDB connection strings via the Neon API.
feat(build): Add syncNeonEnvVars extension and improve Vercel env var
syncing
Add a new `syncNeonEnvVars` build extension for syncing environment
variables
from Neon database projects to Trigger.dev. The extension automatically
detects
branches and builds appropriate PostgreSQL connection strings for
non-production
environments (staging, dev, preview).
Features of `syncNeonEnvVars`:
- Fetches branch-specific database credentials from Neon API
- Generates all standard Postgres connection strings (DATABASE_URL,
POSTGRES_URL,
POSTGRES_PRISMA_URL, etc.) with both pooled and unpooled variants
- Supports custom database name, role name, and env var prefix options
- Skips automatically in Vercel environments (Neon's Vercel integration
handles this)
- Skips for production environments (designed for preview/staging/dev
branches)
Improvements to `syncVercelEnvVars`:
- When running in a Vercel build environment (detected via VERCEL env
var),
values are now read from process.env instead of the Vercel API response
- This ensures the build uses the actual runtime values Vercel provides
- Removed embedded Neon-specific logic (now handled by separate
extension)
- Simplified and cleaned up the extension code
Documentation updates for both extensions with usage examples and
configuration
options.
Closes#2714
## ✅ Checklist
- [x] I have followed every step in the [contributing
guide](https://github.com/triggerdotdev/trigger.dev/blob/main/CONTRIBUTING.md)
- [x] The PR title follows the convention.
- [x] I ran and tested the code works
---
## Testing
Set up Vercel + Trigger.dev envs, used Vercel's chat-bot-ai template.
When viewing runs filtered to a single task, the "Create a test run" and
"Run a test" buttons now navigate directly to the task-specific test
page instead of the generic test page.
This improves UX by pre-populating the test form with the filtered task,
saving users from having to manually select it again.
## ✅ Checklist
- [x] I have followed every step in the [contributing
guide](https://github.com/triggerdotdev/trigger.dev/blob/main/CONTRIBUTING.md)
- [x] The PR title follows the convention.
- [x] I ran and tested the code works
## Screenshots
<img width="1706" height="1392" alt="image"
src="https://github.com/user-attachments/assets/d8b5d445-73b5-426c-83a4-90ac2a95b955"
/>
Add validation for logical replication publication configuration. Helps
diagnose an issue where runs are no longer replicated to clickhouse
because of a configuration issue with the replication publication.
## Problem
The `LogicalReplicationClient` only checked if a publication existed,
not if it was correctly configured. This caused a silent failure where:
- Replication would start successfully
- Transaction boundaries (begin/commit) were received
- **But no actual data changes were replicated**
This happened when a publication existed but:
1. Had no tables associated with it
2. Was missing required actions (e.g., `delete`)
## Solution
Added `#validatePublicationConfiguration()` method that validates:
- ✅ Publication includes the expected table
- ✅ Publication has all required actions configured
When validation fails, error messages include the exact SQL command to
fix the issue:
**Missing table:**
```
Publication 'task_runs_to_clickhouse_v1_publication' exists but has NO TABLES configured.
Expected table: "public.TaskRun".
Run: ALTER PUBLICATION task_runs_to_clickhouse_v1_publication ADD TABLE "TaskRun";
```
**Missing actions:**
```
Publication 'task_runs_to_clickhouse_v1_publication' is missing required actions.
Expected: [insert, update, delete], Current: [insert, update], Missing: [delete].
Run: ALTER PUBLICATION task_runs_to_clickhouse_v1_publication SET (publish = 'insert, update, delete');
```
This prevents silent data loss and makes debugging configuration issues
much easier.
Adds a manual trigger to the `release.yml` workflow for publishing
prerelease versions. Needs to be in the same workflow file due to a NPM
limitation on how OIDC claims are checked.
Currently there is a validation step on the ref for the prerelease: it
must be merged to the main branch. We can revisit this in the future in
case we find it too limiting.
To avoid doing a migration for the ECR repo cache settings, we lazily do
it on the next deployment for that project. Failures to update the repo
settings are just logged and will not cause the deployment to fail.
Fixes this type of error when exporting a `schemaTask` in a monorepo:
```
error TS2742: The inferred type of 'helloWorldSchema' cannot be named without a reference to '@trigger.dev/core/v3'. This is likely not portable.
```
This PR adds support for CLI deployments using the native build server.
**Background**
The deployment command currently does the following:
- bundles the code
- submits the build context to our external build provider and waits for
the build
- triggers deployment state transitions using the platform API
Upstream build provider outages cause issue with deployments,
potentially blocking deployments entirely. We recently introduced the
`--force-local-build` flag as a fallback to enable deployment without a
dependency on the upstream build provider, though it requires users to
have docker in their systems. This PR continues that work by providing a
remote build path which uses our own build server and does not rely on
the external provider.
**Changes in this PR**
Introduced the new `--native-build-server` flag, which does the
following:
- scans all files relevant for the Trigger deployment and evaluates
ignore rules
- packages it up in an archive and uploads it as a deployment artifact
- queues the deployment and triggers the build
- streams logs from the build server
This no longer relies on external build services. Also deployment state
transitions happen on the server-side, giving us more flexibility to
evolve the flow and schemas of related deployment API endpoints. In
general it gives us better control of the whole build and deployment
process. This path will eventually become the default.
The `--detach` flag is also new, allowing to trigger deployments without
waiting for the result.
The deployment artifacts are uploaded via pre-signed URLs to avoid
unnecessary load on the platform. The new `/artifacts` endpoint
generates the pre-signed URLs; size limits are enforced on s3. This
endpoint is deliberately generic, we could extend it in the future to
upload other artifacts client-side in a similar way, e.g., large payload
packets.
This is a step which we currently need to do manually and it's rather
painful. The lockfile update is necessary due to cross references in our
packages.
Added it as a separate job instead of a step to start from fresh
workspace, as the state that the `changeset` step leaves the workdir is
not explicitly clear to the reader.
Add a new `syncNeonEnvVars` build extension for syncing environment variables
from Neon database projects to Trigger.dev. The extension automatically detects
branches and builds appropriate PostgreSQL connection strings for non-production
environments (staging, dev, preview).
Features of `syncNeonEnvVars`:
- Fetches branch-specific database credentials from Neon API
- Generates all standard Postgres connection strings (DATABASE_URL, POSTGRES_URL,
POSTGRES_PRISMA_URL, etc.) with both pooled and unpooled variants
- Supports custom database name, role name, and env var prefix options
- Skips automatically in Vercel environments (Neon's Vercel integration handles this)
- Skips for production environments (designed for preview/staging/dev branches)
Improvements to `syncVercelEnvVars`:
- When running in a Vercel build environment (detected via VERCEL env var),
values are now read from process.env instead of the Vercel API response
- This ensures the build uses the actual runtime values Vercel provides
- Removed embedded Neon-specific logic (now handled by separate extension)
- Simplified and cleaned up the extension code
Documentation updates for both extensions with usage examples and configuration
options.
* Add the release version to changeset PRs
* Add missing id-token permission, needed for oidc
* Remove a couple of unnecesary steps
* Reference the `changeset-release/main` branch explicitly
Vercel's NeonDB integration renders database connection environment
variables at runtime, which means Trigger.dev cannot directly sync
these values during the build process. This change adds support for
fetching branch-specific NeonDB connection strings via the Neon API.
Changes:
- Discover NEON_PROJECT_ID from incoming Vercel environment variables
- Call NeonDB API to search for branches matching the git branch name
- Filter branches to find exact matches with Vercel environment prefix
(e.g., "preview/branch-name") to avoid false positives from partial
string matches
- Retrieve branch endpoints and select the write endpoint (or first
available)
- Build connection strings (DATABASE_URL, POSTGRES_URL, etc.) using
the branch endpoint host while preserving user/password credentials
Safety measures for non-production environments:
- Filter out all Neon-related env vars (DATABASE_URL, PGHOST, etc.)
before calling the Neon API to prevent accidental use of production
database credentials
- Only add branch-specific database env vars if a matching Neon branch
is found and the API call succeeds
- If neonDbAccessToken is not provided or the API fails, non-production
environments will not receive any database connection env vars
Usage:
Users must provide a NEON_ACCESS_TOKEN (via options or env var) to
enable automatic branch resolution for preview deployments. Production
environments continue to use Vercel's standard env var sync without
modification.
schedule spans can sometimes show as generic spans when using the
task_events_v2 table because of the inserted_at filter. Increasing the
buffer for the start time does the trick and doesn’t cause any perf
Issues (and is in general just more robust)
When using custom OTLP exporters via `telemetry.exporters` and
This occurred when tasks were triggered **without** a parent trace
context (e.g., via API or dashboard). In this scenario: - Spans were
correctly rewritten to use the generated `externalTraceId` - Logs kept
their original internal trace ID due to a bug in the early return logic
### Root Cause
In `ExternalLogRecordExporterWrapper.transformLogRecord()`, the early
return condition incorrectly included `!this.externalTraceContext`:
```typescript
if (!logRecord.spanContext || !this.externalTraceId ||
!this.externalTraceContext) { return logRecord; // Bug: Returns early
when externalTraceContext is undefined }
// This fallback logic was never reached:
const externalTraceId = this.externalTraceContext
? this.externalTraceContext.traceId
: this.externalTraceId;
```
### Fix
1. **Reordered logic in `transformLogRecord()`**: Move the
1. `externalTraceId` calculation before the early return, and check the
1. culated value instead of `this.externalTraceContext`:
```typescript
const externalTraceId = this.externalTraceContext
? this.externalTraceContext.traceId
: this.externalTraceId;
if (!logRecord.spanContext || !externalTraceId) {
return logRecord;
}
```
2. **Clarified `_isExternallySampled` logic**: Updated both
2. `ExternalSpanExporterWrapper` and `ExternalLogRecordExporterWrapper`
2. explicitly handle the case where there's no external trace context
2. a generated `externalTraceId` exists:
```typescript
this._isExternallySampled = externalTraceContext
? isTraceFlagSampled(externalTraceContext.traceFlags)
: !!externalTraceId;
```
### Impact
Logs and spans from the same task run will now have matching trace IDs
when exported to external observability tools, enabling proper trace correlation regardless of whether the task was triggered with or without a parent trace context.
`telemetry.logExporters` in `trigger.config.ts`, logs and spans were
exported with **different trace IDs**, breaking trace correlation in
external observability tools like Datadog.
* stop deleting the first dev version files on the first change, prevents system failures
* prevent dev runs getting stuck in dequeued status by deleting workers
* add changeset
This PR fixes the `x-trigger-branch` header support for targeting specific preview branches when managing environment variables. The header was documented but not actually being extracted or used in the environment variable API routes. Additionally, the query logic in `authenticatedEnvironmentForAuthentication` was fundamentally broken—it searched for environments with both `slug: "preview"` (parent environment property) AND a specific `branchName` (child environment property), which no environment could satisfy simultaneously. The fix extracts the branch name using `branchNameFromRequest()` and correctly queries for child branch environments using `type: "PREVIEW"` and the specific `branchName`. This ensures that environment variable operations (create, update, get, list) properly target individual preview branches instead of affecting all preview environments.
* Don't use the organization max concurrency anymore
* Early draft of the concurrency page
* WIP adding a new stepper input component
* Move stepper to be alphabetical
* When max value is reached, disabled the + button
* Show placeholder if you delete all numbers
* Make all the html input values available to the component
* Adds size variants
* Move stepper into its own component
* Work on showing the extra concurrency
* The purchase form styling and functionality (minus actually purchasing)
* New style for outline input fields
* Concurrency purchasing working
* Purchasing concurrency and quota emails working
* Improvements to the modal
* Show cost breakdown in the modal
* Fix for allocated concurrency including DEV
* Improved types
* Allocating concurrency is working
* Live updates total env concurrency
* Implemented reset
* Fix for concurrency allocation editing across multiple projects
* Tabular numbers
* Added an error from allocating concurrency
* Fixes for allocating concurrency where it didn't calculate correctly
* "Increase limit" link to concurrency page
* Indent environments
* Added Preview limit when updating concurrency for an org
* Show error when changing plan fails
* Added maximumProjectCount column to Org
* Limit project count and display a rich error toast (with title and button now)
* Added title and button to toasts. Use it for new project error
* @trigger.dev/platform 1.0.20
* Allow submitting zero concurrency so you can downgrade back to nothing
* Use the server as the truth for omitted environments
* Updated the pricing panels
---------
Co-authored-by: James Ritchie <james@trigger.dev>
* prisma extension fixes WIP
* More prisma stuff
* more prisma stuff
* remove changelog
* upgrade github workflows to use node 20.19 because installing prisma@7 breaks with lower versions
* Don't use generate for the prisma reference projects
* make sure it works if no mode is passed in
* Update aws sdk ecr client to the latest version
* Exlude the cache tag from the immutability enforcement
* Attach a policy to ECR repos to expire untagged images
* Fix filterType
2025-11-17 12:34:54 +01:00
1496 changed files with 264462 additions and 32671 deletions
Truncate large error stacks and messages to prevent OOM crashes. Stack traces are capped at 50 frames (keeping top 5 + bottom 45 with an omission notice), individual stack lines at 1024 chars, and error messages at 1000 chars. Applied in parseError, sanitizeError, and OTel span recording.
- When adding indexes to **existing tables**, use `CREATE INDEX CONCURRENTLY IF NOT EXISTS` to avoid table locks. These must be in their own separate migration file (one index per file).
- Indexes on **newly created tables** (same migration as `CREATE TABLE`) do not need CONCURRENTLY.
- When indexing a **new column on an existing table**, split into two migrations: first `ADD COLUMN IF NOT EXISTS`, then `CREATE INDEX CONCURRENTLY IF NOT EXISTS` in a separate file.
- After generating a migration with Prisma, remove extraneous lines for: `_BackgroundWorkerToBackgroundWorkerFile`, `_BackgroundWorkerToTaskQueue`, `_TaskRunToTaskRunTag`, `_WaitpointRunConnections`, `_completedWaitpoints`, `SecretStore_key_idx`, and unrelated TaskRun indexes.
- Never drop columns or tables without explicit approval.
- New code should target `RunEngineVersion.V2` only.
The `v3/` directory name is misleading - most code here is actively used by the current V2 engine. Only the specific files below are legacy V1-only code.
description: Use when adding, modifying, or debugging OTel span timeline events in the trace view. Covers event structure, ClickHouse storage constraints, rendering in SpanTimeline component, admin visibility, and the step-by-step process for adding new events.
The trace view's right panel shows a timeline of events for the selected span. These are OTel span events rendered by `app/utils/timelineSpanEvents.ts` and the `SpanTimeline` component.
## How They Work
1.**Span events** in OTel are attached to a parent span. In ClickHouse, they're stored as separate rows with `kind: "SPAN_EVENT"` sharing the parent span's `span_id`. The `#mergeRecordsIntoSpanDetail` method reassembles them into the span's `events` array at query time.
2. The timeline only renders events whose `name` starts with `trigger.dev/` - all others are silently filtered out.
3. The **display name** comes from `properties.event` (not the span event name), mapped through `getFriendlyNameForEvent()`.
4. Events are shown on the **span they belong to** - events on one span don't appear in another span's timeline.
## ClickHouse Storage Constraint
When events are written to ClickHouse, `spanEventsToTaskEventV1Input()` filters out events whose `start_time` is not greater than the parent span's `startTime`. Events at or before the span start are silently dropped. This means span events must have timestamps strictly after the span's own `startTimeUnixNano`.
## Timeline Rendering (SpanTimeline component)
The `SpanTimeline` component in `app/components/run/RunTimeline.tsx` renders:
1.**Events** (thin 1px line with hollow dots) - all events from `createTimelineSpanEventsFromSpanEvents()`
2.**"Started"** marker (thick cap) - at the span's `startTime`
3.**Duration bar** (thick 7px line) - from "Started" to "Finished"
4.**"Finished"** marker (thick cap) - at `startTime + duration`
The thin line before "Started" only appears when there are events with timestamps between the span start and the first child span. For the Attempt span this works well (Dequeued -> Pod scheduled -> Launched -> etc. all happen before execution starts). Events all get `lineVariant: "light"` (thin) while the execution bar gets `variant: "normal"` (thick).
## Trace View Sort Order
Sibling spans (same parent) are sorted by `start_time ASC` from the ClickHouse query. The `createTreeFromFlatItems` function preserves this order. Event timestamps don't affect sort order - only the span's own `start_time`.
## Event Structure
```typescript
// OTel span event format
{
name:"trigger.dev/run",// Must start with "trigger.dev/" to render
timeUnixNano:"1711200000000000000",
attributes:[
{key:"event",value:{stringValue:"dequeue"}},// The actual event type
{key:"duration",value:{intValue: 150}},// Optional: duration in ms
]
}
```
## Admin-Only Events
`getAdminOnlyForEvent()` controls visibility. Events default to **admin-only** (`true`).
| Event | Admin-only | Friendly name |
|-------|-----------|---------------|
| `dequeue` | No | Dequeued |
| `fork` | No | Launched |
| `import` | No (if no fork event) | Importing task file |
description: Use this skill when writing, designing, or optimizing Trigger.dev background tasks and workflows. This includes creating reliable async tasks, implementing AI workflows, setting up scheduled jobs, structuring complex task hierarchies with subtasks, configuring build extensions for tools like ffmpeg or Puppeteer/Playwright, and handling task schemas with Zod validation.
You are an expert Trigger.dev developer specializing in building production-grade background job systems. Tasks deployed to Trigger.dev run in Node.js 21+ and use the `@trigger.dev/sdk` package.
## Critical Rules
1.**Always use `@trigger.dev/sdk`** - Never use `@trigger.dev/sdk/v3` or deprecated `client.defineJob` pattern
2.**Never use `node-fetch`** - Use the built-in `fetch` function
3.**Export all tasks** - Every task must be exported, including subtasks
4.**Never wrap wait/trigger calls in Promise.all** - `triggerAndWait`, `batchTriggerAndWait`, and `wait.*` calls cannot be wrapped in `Promise.all` or `Promise.allSettled`
mode:"trailing",// "leading" (default) or "trailing"
},
});
```
## Debouncing
Consolidate multiple triggers into a single execution:
```ts
// Rapid triggers with same key = single execution
awaitmyTask.trigger({userId:"123"},{
debounce:{
key:"user-123-update",
delay:"5s",
},
});
// Trailing mode: use payload from LAST trigger
awaitmyTask.trigger({data:"latest"},{
debounce:{
key:"my-key",
delay:"10s",
mode:"trailing",
},
});
```
Use cases: user activity updates, webhook deduplication, search indexing, notification batching.
## Batch Triggering
Up to 1,000 items per batch, 3MB per payload:
```ts
constresults=awaitmyTask.batchTriggerAndWait([
{payload:{userId:"1"}},
{payload:{userId:"2"}},
]);
for(constresultofresults){
if(result.ok)console.log(result.output);
}
```
## Machine Presets
| Preset | vCPU | Memory |
|-------------|------|--------|
| micro | 0.25 | 0.25GB |
| small-1x | 0.5 | 0.5GB |
| small-2x | 1 | 1GB |
| medium-1x | 1 | 2GB |
| medium-2x | 2 | 4GB |
| large-1x | 4 | 8GB |
| large-2x | 8 | 16GB |
## Design Principles
1.**Break complex workflows into subtasks** that can be independently retried and made idempotent
2.**Don't over-complicate** - Sometimes `Promise.allSettled` inside a single task is better than many subtasks (each task has dedicated process and is charged by millisecond)
3.**Always configure retries** - Set appropriate `maxAttempts` based on the operation
4.**Use idempotency keys** - Especially for payment/critical operations
5.**Group related subtasks** - Keep subtasks only used by one parent in the same file, don't export them
6.**Use logger** - Log at key execution points with `logger.info()`, `logger.error()`, etc.
## Reference Documentation
For detailed documentation on specific topics, read these files:
description: Guidelines for creating OpenTelemetry metrics to avoid cardinality issues
globs:
- "**/*.ts"
---
# OpenTelemetry Metrics Guidelines
When creating or editing OTEL metrics (counters, histograms, gauges), always ensure metric attributes have **low cardinality**.
## What is Cardinality?
Cardinality refers to the number of unique values an attribute can have. Each unique combination of attribute values creates a new time series, which consumes memory and storage in your metrics backend.
The main trigger.dev webapp, which powers it's API and dashboard and makes up the docker image that is produced as an OSS image, is a Remix 2.1.0 app that uses an express server, written in TypeScript. The following subsystems are either included in the webapp or are used by the webapp in another part of the monorepo:
The main trigger.dev webapp, which powers it's API and dashboard and makes up the docker image that is produced as an OSS image, is a Remix 2.17.4 app that uses an express server, written in TypeScript. The following subsystems are either included in the webapp or are used by the webapp in another part of the monorepo:
- `@trigger.dev/database` exports a Prisma 6.14.0 client that is used extensively in the webapp to access a PostgreSQL instance. The schema file is [schema.prisma](mdc:internal-packages/database/prisma/schema.prisma)
- `@trigger.dev/core` is a published package and is used to share code between the `@trigger.dev/sdk` and the webapp. It includes functionality but also a load of Zod schemas for data validation. When importing from `@trigger.dev/core` in the webapp, we never import the root `@trigger.dev/core` path, instead we favor one of the subpath exports that you can find in [package.json](mdc:packages/core/package.json)
gh pr close ${{ github.event.pull_request.number }} \
--repo ${{ github.repository }} \
--comment "Thanks for your contribution! We require all external PRs to be opened in **draft** status first so you can address CodeRabbit review comments and ensure CI passes before requesting a review. Please re-open this PR as a draft. See [CONTRIBUTING.md](https://github.com/${{ github.repository }}/blob/main/CONTRIBUTING.md#pr-workflow) for details."
This directory tracks changes to server-only components (webapp, supervisor, coordinator, etc.) that are not captured by changesets. Changesets only track published npm packages — server changes would otherwise go undocumented.
## When to add a file
**Server-only PRs**: If your PR only changes `apps/webapp/`, `apps/supervisor/`, `apps/coordinator/`, or other server components (and does NOT change anything in `packages/`), add a `.server-changes/` file.
**Mixed PRs** (both packages and server): Just add a changeset as usual. No `.server-changes/` file needed — the changeset covers it.
**Package-only PRs**: Just add a changeset as usual.
## File format
Create a markdown file with a descriptive name:
```
.server-changes/fix-batch-queue-stalls.md
```
With this format:
```markdown
---
area: webapp
type: fix
---
Speed up batch queue processing by removing stalls and fixing retry race
Added `/engine/v1/dev/disconnect` endpoint to auto-cancel runs when the CLI disconnects. Maximum of 500 runs can be cancelled. Uses the bulk action system when there are more than 25 runs to cancel.
Add 60s fresh / 60s stale SWR cache to `getEntitlement` in `platform.v3.server.ts`. Eliminates a synchronous billing-service HTTP round trip on every trigger. Reuses the existing `platformCache` (LRU memory + Redis) pattern already used for `limits` and `usage`. Cache key is `${orgId}`. Errors return a permissive `{ hasAccess: true }` fallback (existing behavior) and are also cached to prevent thundering-herd on billing outages.
Fix Redis connection leak in realtime streams and broken abort signal propagation.
**Redis connections**: Non-blocking methods (ingestData, appendPart, getLastChunkIndex) now share a single Redis connection instead of creating one per request. streamResponse still uses dedicated connections (required for XREAD BLOCK) but now tears them down immediately via disconnect() instead of graceful quit(), with a 15s inactivity fallback.
**Abort signal**: request.signal is broken in Remix/Express due to a Node.js undici GC bug (nodejs/node#55428) that severs the signal chain when Remix clones the Request internally. Added getRequestAbortSignal() wired to Express res.on("close") via httpAsyncStorage, which fires reliably on client disconnect. All SSE/streaming routes updated to use it.
Prevent dashboard crash (React error #31) when span accessory item text is not a string. Filters out malformed accessory items in SpanCodePathAccessory instead of passing objects to React as children.
Stop creating TaskRunTag records and _TaskRunToTaskRunTag join table entries during task triggering. The denormalized runTags string array on TaskRun already stores tag names, making the M2M relation redundant write overhead.
Replace the expensive DISTINCT query for task filter dropdowns with a dedicated TaskIdentifier registry table backed by Redis. Environments migrate automatically on their next deploy, with a transparent fallback to the legacy query for unmigrated environments. Also fixes duplicate dropdown entries when a task changes trigger source, and adds active/archived grouping for removed tasks. Moves BackgroundWorkerTask reads in the trigger hot path to the read replica.
@@ -13,7 +13,7 @@ This repository is a pnpm monorepo managed with Turbo. It contains multiple apps
See `ai/references/repo.md` for a more complete explanation of the workspaces.
## Development setup
1. Install dependencies with `pnpm i` (pnpm `8.15.5` and Node.js `20.11.1` are required).
1. Install dependencies with `pnpm i` (pnpm `10.23.0` and Node.js `20.20.0` are required).
2. Copy `.env.example` to `.env` and generate a random 16 byte hex string for `ENCRYPTION_KEY` (`openssl rand -hex 16`). Update other secrets if needed.
Trigger.dev uses [changesets](https://github.com/changesets/changesets) to manage updated our packages and releasing them to npm.
Trigger.dev uses [changesets](https://github.com/changesets/changesets) to manage package versions and releasing them to npm. For server-only changes, we use a lightweight `.server-changes/` convention.
## Adding a changeset
## Adding a changeset (package changes)
To add a changeset, use `pnpm run changeset:add` and follow the instructions [here](https://github.com/changesets/changesets/blob/main/docs/adding-a-changeset.md). Please only ever select one of our public packages when adding a changeset.
## Release instructions (local only)
## Adding a server change (server-only changes)
Based on the instructions [here](https://github.com/changesets/changesets/blob/main/docs/intro-to-using-changesets.md)
If your PR only changes server components (`apps/webapp/`, `apps/supervisor/`, etc.) and does NOT change any published packages, add a `.server-changes/` file instead of a changeset:
For **mixed PRs** (both packages and server): just add a changeset. No `.server-changes/` file needed.
See `.server-changes/README.md` for full documentation.
## When to add which
| PR changes | What to add |
|---|---|
| Only packages (`packages/`) | Changeset (`pnpm run changeset:add`) |
| Only server (`apps/`) | `.server-changes/` file |
| Both packages and server | Just the changeset |
## Release instructions (CI)
Please follow the best-practice of adding changesets in the same commit as the code making the change with `pnpm run changeset:add`, as it will allow our release.yml CI workflow to function properly:
- Anytime new changesets are added in a commit in the `main` branch, the [release.yml](./.github/workflows/release.yml) workflow will run and will automatically create/update a PR with a fresh run of `pnpm run changeset:version`.
-When the version PR is merged into `main`, the release.yml workflow will automatically run `pnpm run changeset:release` to build and release packages to npm.
- Anytime new changesets are added in a commit in the `main` branch, the [changesets-pr.yml](./.github/workflows/changesets-pr.yml) workflow will run and will automatically create/update a PR with a fresh run of `pnpm run changeset:version`.
-The release PR body is automatically enhanced with a clean, deduplicated summary that includes both package changes and `.server-changes/` entries.
- Consumed `.server-changes/` files are removed on the `changeset-release/main` branch — the same way changesets deletes `.changeset/*.md` files. When the release PR merges, they're gone from main.
- When the version PR is merged into `main`, the [release.yml](./.github/workflows/release.yml) workflow will automatically build, release packages to npm, and create a single unified GitHub release.
This file provides guidance to Claude Code when working with this repository. Subdirectory CLAUDE.md files provide deeper context when you navigate into specific areas.
## Build and Development Commands
This is a pnpm 10.23.0 monorepo using Turborepo. Run commands from root with `pnpm run`.
```bash
pnpm run docker # Start Docker services (PostgreSQL, Redis, Electric)
pnpm run db:migrate # Run database migrations
pnpm run db:seed # Seed the database (required for reference projects)
# Build packages (required before running)
pnpm run build --filter webapp && pnpm run build --filter trigger.dev && pnpm run build --filter @trigger.dev/sdk
pnpm run dev --filter webapp # Run webapp (http://localhost:3030)
pnpm run dev --filter trigger.dev --filter "@trigger.dev/*"# Watch CLI and packages
```
### Verifying Changes
The verification command depends on where the change lives:
- **Apps and internal packages** (`apps/*`, `internal-packages/*`): Use `typecheck`. **Never use `build`** for these — building proves almost nothing about correctness.
- **Public packages** (`packages/*`): Use `build`.
```bash
# Apps and internal packages — use typecheck
pnpm run typecheck --filter webapp # ~1-2 minutes
pnpm run typecheck --filter @internal/run-engine
# Public packages — use build
pnpm run build --filter @trigger.dev/sdk
pnpm run build --filter @trigger.dev/core
```
Only run typecheck/build after major changes (new files, significant refactors, schema changes). For small edits, trust the types and let CI catch issues.
## Testing
We use vitest exclusively. **Never mock anything** - use testcontainers instead.
```bash
pnpm run test --filter webapp # All tests for a package
cd internal-packages/run-engine
pnpm run test ./src/engine/tests/ttl.test.ts --run # Single test file
pnpm run build --filter @internal/run-engine # May need to build deps first
```
Test files go next to source files (e.g., `MyService.ts` -> `MyService.test.ts`).
redisTest("should use redis",async({redisOptions})=>{
/* ... */
});
postgresTest("should use postgres",async({prisma})=>{
/* ... */
});
containerTest("should use both",async({prisma,redisOptions})=>{
/* ... */
});
```
## Changesets and Server Changes
When modifying any public package (`packages/*` or `integrations/*`), add a changeset:
```bash
pnpm run changeset:add
```
- Default to **patch** for bug fixes and minor changes
- Confirm with maintainers before selecting **minor** (new features)
- **Never** select major without explicit approval
When modifying only server components (`apps/webapp/`, `apps/supervisor/`, etc.) with no package changes, add a `.server-changes/` file instead. See `.server-changes/README.md` for format and documentation.
## Dependency Pinning
Zod is pinned to a single version across the entire monorepo (currently `3.25.76`). When adding zod to a new or existing package, use the **exact same version** as the rest of the repo - never a different version or a range. Mismatched zod versions cause runtime type incompatibilities (e.g., schemas from one package can't be used as body validators in another).
## Architecture Overview
### Request Flow
User API call -> Webapp routes -> Services -> RunEngine -> Redis Queue -> Supervisor -> Container execution -> Results back through RunEngine -> ClickHouse (analytics) + PostgreSQL (state)
- **internal-packages/zodworker**: Graphile-worker wrapper (DEPRECATED - use redis-worker)
### Legacy V1 Engine Code
The `apps/webapp/app/v3/` directory name is misleading - most code there is actively used by V2. Only specific files are V1-only legacy (MarQS queue, triggerTaskV1, cancelTaskRunV1, etc.). See `apps/webapp/CLAUDE.md` for the exact list. When you encounter V1/V2 branching in services, only modify V2 code paths. All new work uses Run Engine 2.0 (`@internal/run-engine`) and redis-worker.
### Documentation
Docs live in `docs/` as a Mintlify site (MDX format). See `docs/CLAUDE.md` for conventions.
### Reference Projects
The `references/` directory contains test workspaces for testing SDK and platform features. Use `references/hello-world` to manually test changes before submitting PRs.
## Docker Image Guidelines
When updating Docker image references:
- **Always use multiplatform/index digests**, not architecture-specific digests
- Architecture-specific digests cause CI failures on different build environments
- Use the digest from the main Docker Hub page, not from a specific OS/ARCH variant
## Writing Trigger.dev Tasks
Always import from `@trigger.dev/sdk`. Never use `@trigger.dev/sdk/v3` or deprecated `client.defineJob`.
```typescript
import{task}from"@trigger.dev/sdk";
exportconstmyTask=task({
id:"my-task",
run: async(payload:{message: string})=>{
// Task logic
},
});
```
### SDK Documentation Rules
The `rules/` directory contains versioned SDK documentation distributed via the SDK installer. Current version: `rules/manifest.json`. Do NOT update `rules/` or `.claude/skills/trigger-dev-tasks/` unless explicitly asked - these are maintained in separate dedicated passes.
The preferred way to query for crumbs is to use `pnpm exec agentcrumbs query --app trigger` with the `--limit` option and cursor pagination, and clear existing crumbs before reproducing a bug via `pnpm exec agentcrumbs clear --app trigger`.
Thank you for taking the time to contribute to Trigger.dev. Your involvement is not just welcomed, but we encourage it! 🚀
Please take some time to read this guide to understand contributing best practices for Trigger.dev.
Please take some time to read this guide to understand contributing best practices for Trigger.dev. Note that we use [vouch](https://github.com/mitchellh/vouch) to manage contributor trust, so you'll need to be vouched before opening a PR.
Thank you for helping us make Trigger.dev even better! 🤩
> **Important:** We only accept PRs that address a single issue. Please do not submit PRs containing multiple unrelated fixes or features. If you have multiple contributions, open a separate PR for each one.
## Getting vouched (required before opening a PR)
We use [vouch](https://github.com/mitchellh/vouch) to manage contributor trust. **PRs from unvouched users are automatically closed.**
Before you open your first pull request, you need to be vouched by a maintainer. Here's how:
1. Open a [Vouch Request](https://github.com/triggerdotdev/trigger.dev/issues/new?template=vouch-request.yml) issue.
2. Tell us what you'd like to work on and share any relevant background.
3. A maintainer will review your request and vouch for you by commenting on the issue.
4. Once vouched, your PRs will be accepted normally.
If you're unsure whether you're already vouched, go ahead and open a PR — the check will tell you.
## Developing
The development branch is `main`. This is the branch that all pull
@@ -14,8 +29,8 @@ branch are tagged into a release periodically.
### Prerequisites
- [Node.js](https://nodejs.org/en) version 20.11.1
- [pnpm package manager](https://pnpm.io/installation) version 8.15.5
- [Node.js](https://nodejs.org/en) version 20.20.0
- [pnpm package manager](https://pnpm.io/installation) version 10.23.0
@@ -34,9 +49,9 @@ branch are tagged into a release periodically.
```
cd trigger.dev
```
3. Ensure you are on the correct version of Node.js (20.11.1). If you are using `nvm`, there is an `.nvmrc` file that will automatically select the correct version of Node.js when you navigate to the repository.
3. Ensure you are on the correct version of Node.js (20.20.0). If you are using `nvm`, there is an `.nvmrc` file that will automatically select the correct version of Node.js when you navigate to the repository.
4. Run `corepack enable` to use the correct version of pnpm (`8.15.5`) as specified in the root `package.json` file.
4. Run `corepack enable` to use the correct version of pnpm (`10.23.0`) as specified in the root `package.json` file.
5. Install the required packages using pnpm.
```
@@ -92,7 +107,7 @@ We use the `<root>/references/hello-world` subdirectory as a staging ground for
First, make sure you are running the webapp according to the instructions above. Then:
1. Visit http://localhost:3030 in your browser and create a new V3 project called "hello-world".
1. Visit http://localhost:3030 in your browser and create a new project called "hello-world".
2. In Postgres go to the "Projects" table and for the project you create change the `externalRef` to `proj_rrkpdguyagvsoktglnod`.
@@ -127,7 +142,7 @@ pnpm exec trigger deploy --profile local
### Running
The following steps should be followed any time you start working on a new feature you want to test in v3:
The following steps should be followed any time you start working on a new feature you want to test:
1. Make sure the webapp is running on localhost:3030
@@ -227,9 +242,23 @@ See the [Job Catalog](./references/job-catalog/README.md) file for more.
**If you get errors, be sure to fix them before committing.**
- Be sure to [check the "Allow edits from maintainers" option](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/allowing-changes-to-a-pull-request-branch-created-from-a-fork) while creating you PR.
- If your PR refers to or fixes an issue, be sure to add `refs #XXX` or `fixes #XXX` to the PR description. Replacing `XXX` with the respective issue number. See more about [Linking a pull request to an issue
> **Note:** We may close PRs if we decide that the cost of integrating the change outweighs the benefits. To improve the chances of your PR getting accepted, follow the guidelines below.
### PR workflow
1. **Always open your PR in draft status first.** Do not mark it as "Ready for Review" until the steps below are complete.
2. **Address all CodeRabbit code review comments.** Our CI runs an automated code review via CodeRabbit. Go through each comment and either fix the issue or resolve it with a comment explaining why no change is needed.
3. **Wait for all CI checks to pass.** Do not mark the PR as "Ready for Review" until every check is green.
4. **Then mark the PR as "Ready for Review"** so a maintainer can take a look.
### Cost/benefit analysis for risky changes
If your change touches core infrastructure, modifies widely-used code paths, or could introduce regressions, consider doing a brief cost/benefit analysis and including it in the PR description. Explain what the benefit is to users and why the risk is worth it. This goes a long way toward helping maintainers evaluate your contribution.
### General guidelines
- Be sure to [check the "Allow edits from maintainers" option](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/allowing-changes-to-a-pull-request-branch-created-from-a-fork) while creating your PR.
- If your PR refers to or fixes an issue, be sure to add `refs #XXX` or `fixes #XXX` to the PR description. Replacing `XXX` with the respective issue number. See more about [Linking a pull request to an issue](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue).
- Be sure to fill the PR Template accordingly.
## Adding changesets
@@ -252,6 +281,39 @@ You will be prompted to select which packages to include in the changeset. Only
Most of the time the changes you'll make are likely to be categorized as patch releases. If you feel like there is the need for a minor or major release of the package based on the changes being made, add the changeset as such and it will be discussed during PR review.
## Adding server changes
Changesets only track published npm packages. If your PR only changes server components (`apps/webapp/`, `apps/supervisor/`, `apps/coordinator/`, etc.) with no package changes, add a `.server-changes/` file so the change appears in release notes.
1. PRs merge to `main` with changesets (for package changes) and/or `.server-changes/` files (for server-only changes).
2. The [changesets-pr.yml](./.github/workflows/changesets-pr.yml) workflow automatically creates/updates the `changeset-release/main` PR with version bumps and an enhanced summary of all changes. Consumed `.server-changes/` files are removed on the release branch (same approach changesets uses for `.changeset/` files — they're deleted on the branch, so merging the PR cleans them up).
3. When ready to release, merge the changeset release PR into `main`.
4. The [release.yml](./.github/workflows/release.yml) workflow automatically:
- Publishes all packages to npm
- Creates a single unified GitHub release (e.g., "trigger.dev v4.3.4")
- Tags and triggers Docker image builds
- After Docker images are pushed, updates the GitHub release with the exact GHCR tag link
### What engineers need to do
- **Package changes**: Add a changeset with `pnpm run changeset:add`
- **Server-only changes**: Add a `.server-changes/` file (see `.server-changes/README.md`)
- **Mixed PRs**: Just the changeset is enough
See `CHANGESETS.md` for full details on changesets and server changes.
### Legacy release (v3)
1. Merge in the changeset PR into main, making sure to cancel both the release and publish github actions from that merge.
2. Pull the changes locally into main
3. Run `pnpm i` which will update the pnpm lock file with the new versions
We use prisma migrations to manage the database schema. Please follow the following steps when editing the `internal-packages/database/prisma/schema.prisma` file:
Edit the `schema.prisma` file to add or modify the schema.
Create a new migration file but don't apply it yet:
```bash
cd internal-packages/database
pnpm run db:migrate:dev:create --name "add_new_column_to_table"
```
The migration file will be created in the `prisma/migrations` directory, but it will have a bunch of edits to the schema that are not needed and will need to be removed before we can apply the migration. Here's an example of what the migration file might look like:
Node.js app that manages task execution containers. Receives work from the platform, starts Docker/Kubernetes containers, monitors execution, and reports results.
## Key Directories
-`src/services/` - Core service logic
-`src/workloadManager/` - Container orchestration abstraction (Docker or Kubernetes)
-`src/workloadServer/` - HTTP server for workload communication (heartbeats, snapshots)
-`src/clients/` - Platform communication (webapp/coordinator)
-`src/env.ts` - Environment configuration
## Architecture
- **WorkloadManager**: Abstracts Docker vs Kubernetes execution
- **SupervisorSession**: Manages the dequeue loop with EWMA-based dynamic scaling
- **ResourceMonitor**: Tracks CPU/memory during execution
* Network mode to use for all runners. Supported standard values are: `bridge`, `host`, `none`, and `container:<name|id>`.
* Any other value is taken as a custom network's name to which all runners should connect to.
*
* Accepts a list of comma-separated values to attach to multiple networks. Additional networks are interpreted as network names and will be attached after container creation.
*
* **WARNING**: Specifying multiple networks will slightly increase startup times.
* Network mode to use for all runners. Supported standard values are: `bridge`, `host`, `none`, and `container:<name|id>`.
* Any other value is taken as a custom network's name to which all runners should connect to.
*
* Accepts a list of comma-separated values to attach to multiple networks. Additional networks are interpreted as network names and will be attached after container creation.
*
* **WARNING**: Specifying multiple networks will slightly increase startup times.
Remix 2.17.4 app serving as the main API, dashboard, and orchestration engine. Uses an Express server (`server.ts`).
## Verifying Changes
**Never run `pnpm run build --filter webapp` to verify changes.** Building proves almost nothing about correctness. The webapp is an app, not a public package — use typecheck from the repo root:
```bash
pnpm run typecheck --filter webapp # ~1-2 minutes
```
Only run typecheck after major changes (new files, significant refactors, schema changes). For small edits, trust the types and let CI catch issues.
Note: Public packages (`packages/*`) use `build` instead. See the root CLAUDE.md for details.
## Testing Dashboard Changes with Chrome DevTools MCP
Use the `chrome-devtools` MCP server to visually verify local dashboard changes. The webapp must be running (`pnpm run dev --filter webapp` from repo root).
2. mcp__chrome-devtools__click the "Continue with Email" link
3. mcp__chrome-devtools__fill the email field with "local@trigger.dev"
4. mcp__chrome-devtools__click "Send a magic link"
→ Auto-logs in and redirects to the dashboard (no email verification needed locally)
```
### Navigating and Verifying
- **take_snapshot**: Get an a11y tree of the page (text content, element UIDs for interaction). Prefer this over screenshots for understanding page structure.
- **take_screenshot**: Capture what the page looks like visually. Use to verify styling, layout, and visual changes.
- **navigate_page**: Go to specific URLs, e.g. `http://localhost:3030/orgs/references-bc08/projects/hello-world-SiWs/env/dev/runs`
- **click / fill**: Interact with elements using UIDs from `take_snapshot`.
- **evaluate_script**: Run JS in the browser console for debugging.
- **list_console_messages**: Check for console errors after navigating.
### Tips
- Snapshots can be very large on complex pages (200K+ chars). Use `take_screenshot` first to orient, then `take_snapshot` only when you need element UIDs to interact.
- The local seeded user email is `local@trigger.dev`.
**Never use `request.signal`** for detecting client disconnects. It is broken due to a Node.js bug ([nodejs/node#55428](https://github.com/nodejs/node/issues/55428)) where the AbortSignal chain is severed when Remix internally clones the Request object. Instead, use `getRequestAbortSignal()` from `app/services/httpAsyncStorage.server.ts`, which is wired directly to Express `res.on("close")` and fires reliably.
// In route handlers, SSE streams, or any server-side code:
constsignal=getRequestAbortSignal();
```
## Environment Variables
Access via `env` export from `app/env.server.ts`. **Never use `process.env` directly.**
For testable code, **never import env.server.ts** in test files. Pass configuration as options instead:
-`realtimeClient.server.ts` (testable service, takes config as constructor arg)
-`realtimeClientGlobal.server.ts` (creates singleton with env config)
## Run Engine 2.0
The webapp integrates `@internal/run-engine` via `app/v3/runEngine.server.ts`. This is the singleton engine instance. Services in `app/v3/services/` call engine methods for all run lifecycle operations (triggering, completing, cancelling, etc.).
The `engineVersion.server.ts` file determines V1 vs V2 for a given environment. New code should always target V2.
## Background Workers
Background job workers use `@trigger.dev/redis-worker`:
-`app/v3/commonWorker.server.ts`
-`app/v3/alertsWorker.server.ts`
-`app/v3/batchTriggerWorker.server.ts`
Do NOT add new jobs using zodworker/graphile-worker (legacy).
- Electric SQL: Powers real-time data sync for the dashboard
## Legacy V1 Code
The `app/v3/` directory name is misleading - most code is actively used by V2. Only these specific files are V1-only legacy:
-`app/v3/marqs/` (old MarQS queue system)
-`app/v3/legacyRunEngineWorker.server.ts`
-`app/v3/services/triggerTaskV1.server.ts`
-`app/v3/services/cancelTaskRunV1.server.ts`
-`app/v3/authenticatedSocketConnection.server.ts`
-`app/v3/sharedSocketConnection.ts`
Some services (e.g., `cancelTaskRun.server.ts`, `batchTriggerV3.server.ts`) branch on `RunEngineVersion` to support both V1 and V2. When editing these, only modify V2 code paths.
## Performance: Trigger Hot Path
The `triggerTask.server.ts` service is the **highest-throughput code path** in the system. Every API trigger call goes through it. Keep it fast:
- **Do NOT add database queries** to `triggerTask.server.ts` or `batchTriggerV3.server.ts`. Task defaults (TTL, etc.) are resolved via `backgroundWorkerTask.findFirst()` in the queue concern (`queues.server.ts`) - one query per request, in mutually exclusive branches depending on locked/non-locked path. Piggyback on the existing query instead of adding new ones.
- **Two-stage resolution pattern**: Task metadata is resolved in two stages by design:
1.**Trigger time** (`triggerTask.server.ts`): Only TTL is resolved from task defaults. Everything else uses whatever the caller provides.
2.**Dequeue time** (`dequeueSystem.ts`): Full `BackgroundWorkerTask` is loaded and retry config, machine config, maxDuration, etc. are resolved against task defaults.
- If you need to add a new task-level default, **add it to the existing `select` clause** in the `backgroundWorkerTask.findFirst()` query — do NOT add a second query. If the default doesn't need to be known at trigger time, resolve it at dequeue time instead.
- Batch triggers (`batchTriggerV3.server.ts`) follow the same pattern — keep batch paths equally fast.
## Prisma Query Patterns
- **Always use `findFirst` instead of `findUnique`.** Prisma's `findUnique` has an implicit DataLoader that batches concurrent calls into a single `IN` query. This batching cannot be disabled and has active bugs even in Prisma 6.x: uppercase UUIDs returning null (#25484, confirmed 6.4.1), composite key SQL correctness issues (#22202), and 5-10x worse performance than manual DataLoader (#6573, open since 2021). `findFirst` is never batched and avoids this entire class of issues.
## React Patterns
- Only use `useCallback`/`useMemo` for context provider values, expensive derived data that is a dependency elsewhere, or stable refs required by a dependency array. Don't wrap ordinary event handlers or trivial computations.
- Use named constants for sentinel/placeholder values (e.g. `const UNSET_VALUE = "__unset__"`) instead of raw string literals scattered across comparisons.
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.