1958 Commits

Author	SHA1	Message	Date
github-actions[bot]	6685cbd599	chore: release v4.5.11 (#4557) ... ## Summary 4 new features, 24 improvements, 10 bug fixes. ## Highlights - Allow `trigger deploy` to authenticate with an environment API key from `TRIGGER_ACCESS_TOKEN`. ([#4561](https://github.com/triggerdotdev/trigger.dev/pull/4561)) ## Improvements - Chat in the browser now reconnects when the connection drops mid-turn, instead of leaving the reply stuck as if it were still generating. Reports can be fetched as structured data with the `json` format, and the shortest report period is now one minute (`1m`, `30m`, `1h`, `7d`). The `mint-token` command's help is clearer too: a token minted without `--cap` is read-only, and `--ttl` shows the correct maximum lifetime of 7 days. ([#4418](https://github.com/triggerdotdev/trigger.dev/pull/4418)) - The dev environment onboarding now tracks real progress. After you run `init`, the setup checklist marks your project as initialized, and it updates live as your dev server connects and your tasks register. The blank state also adds a "Copy AI agent prompt" button that copies a ready-to-paste setup prompt (pre-filled with your project reference) for Claude Code, Cursor, or any coding agent. ([#4563](https://github.com/triggerdotdev/trigger.dev/pull/4563)) The `init` scaffold now imports from `@trigger.dev/sdk` instead of the deprecated `@trigger.dev/sdk/v3` subpath. - Deployed images now ship dependencies and bundled task code as separate layers. Repeat deploys with unchanged dependencies typically push and pull far less data, making deploys and worker image pulls faster. ([#4551](https://github.com/triggerdotdev/trigger.dev/pull/4551)) - The current-worker API now reports each task's queue, so you can see which tasks write to a given queue. ([#4525](https://github.com/triggerdotdev/trigger.dev/pull/4525)) - Watch-mode chat streams now survive quiet windows and page reloads, and a reply cut off by a lost connection shows an error instead of appearing finished. Aborting a resumed subscription only closes your local stream — call `stopGeneration(chatId)` or pass `stopOnAbort: true` to stop the run. Also fixed a race where quickly restarting a stream could break stop and reconnect, and stopping a chat now hands it back to your other tabs instead of leaving them read-only. ([#4516](https://github.com/triggerdotdev/trigger.dev/pull/4516)) ## Server changes These changes affect the self-hosted Docker image and Trigger.dev Cloud: - The dashboard agent now has a monthly message allowance and plan-based limits on watches. Queries stay read-only with clearer errors when busy, and messages with unusual characters no longer fail to send. ([#4516](https://github.com/triggerdotdev/trigger.dev/pull/4516)) - Meet the dashboard agent: a chat in every environment that answers questions about your runs, queues, errors and health with real data and links, replacing Ask AI everywhere it used to appear. Investigate a failed run, an error, a backed-up queue or a run that hasn't started to get a worked-through answer — what happened, why, and how to fix it, with every claim linked to the runs, errors and deploys behind it. It reads your data read-only, works on preview and dev branches with that branch's own data, and reads the same everywhere — dashboard, terminal, editor. A very long chat keeps working: the agent summarises the earlier part and carries on. **Watch…** on a run, queue, error or the health report tells you when things change: a run finishes, a queue clears or grows past a number you pick, an error comes back, an environment recovers. The answer arrives in the chat and, if you want, by email, Slack or webhook — and the agent can look into bad news on its own. A watch reaches you on any browser you sign in from, without opening the chat first. A sample of conversations is scored automatically so the agent keeps getting better; only the score and a one-line summary are kept, never your messages, data or code, and we can switch it off for your organization on request. Ask the agent instead of the Docs buttons in page headers — they stay there when the agent isn't available to you. Separately, a queue's wait times, peak depth, throughput and throttling can now be read from the API. ([#4418](https://github.com/triggerdotdev/trigger.dev/pull/4418)) - Add backend support for delaying cron schedules within a specified window with a minimum of 60 seconds. ([#4566](https://github.com/triggerdotdev/trigger.dev/pull/4566)) - Reduced recurring background database load from the billing-limit recovery check, so paused environments are reconciled with less overhead. ([#4590](https://github.com/triggerdotdev/trigger.dev/pull/4590)) - Validating a schedule when deploying or updating a schedule now does less work on projects with many preview branches, so those operations stay fast as branches accumulate. ([#4598](https://github.com/triggerdotdev/trigger.dev/pull/4598)) - Project pages now load faster for projects with a large number of preview branches, by no longer loading archived branch environments that aren't shown. ([#4595](https://github.com/triggerdotdev/trigger.dev/pull/4595)) - Database queries that filter on a list of values now reuse cached query plans more consistently, instead of forcing the database to re-plan whenever the list length changes. ([#4480](https://github.com/triggerdotdev/trigger.dev/pull/4480)) - Routine cleanup of old dashboard agent data now runs on its own schedule. ([#4599](https://github.com/triggerdotdev/trigger.dev/pull/4599)) - Database connection metrics are now reported for every configured database connection instead of only the primary one, and stay accurate regardless of connection type. ([#4541](https://github.com/triggerdotdev/trigger.dev/pull/4541)) - Deployment-related API endpoints now draw from their own generous rate limit budget, configurable via the `DEPLOYMENT_RATE_LIMIT_*` environment variables, so runtime API traffic no longer competes with deployments for the same per-environment budget. ([#4565](https://github.com/triggerdotdev/trigger.dev/pull/4565)) - Deleting or editing a secret environment variable is now fast and no longer slows down as a project accumulates variables. ([#4555](https://github.com/triggerdotdev/trigger.dev/pull/4555)) - Speed up personal access token lookups by indexing them on their owner ([#4588](https://github.com/triggerdotdev/trigger.dev/pull/4588)) - Switching project or organization in the sidebar now keeps you on the same page instead of sending you back to Tasks. Pages for a specific run, deploy or other single item open the matching list instead. ([#4585](https://github.com/triggerdotdev/trigger.dev/pull/4585)) - Reduced database load when loading the dashboard by removing an unused organization member count that was being calculated on every page navigation. ([#4587](https://github.com/triggerdotdev/trigger.dev/pull/4587)) - The environment variables page now loads a page at a time, keeping it fast for projects with a large number of variables. Search matches variable names across every page. ([#4597](https://github.com/triggerdotdev/trigger.dev/pull/4597)) - Groundwork for an alternative database connection driver, gated behind configuration and disabled by default, so there is no change to default behavior. ([#4539](https://github.com/triggerdotdev/trigger.dev/pull/4539)) - Deleting an alert channel is now fast and no longer slows down as a project builds up alert history. ([#4554](https://github.com/triggerdotdev/trigger.dev/pull/4554)) - Reduced internal overhead on the API under high load. ([#4532](https://github.com/triggerdotdev/trigger.dev/pull/4532)) - Out-of-date upgrade prompts no longer appear in the dashboard: the "V4" badges and the notices saying preview branches and the queues table need V4 have been removed. The side menu still warns you when a project is on v3, with updated wording and a link to the v4 upgrade guide. ([#4589](https://github.com/triggerdotdev/trigger.dev/pull/4589)) - Make background worker registration cheaper for projects with many scheduled tasks by scoping declarative schedule reconciliation to the current environment and dropping redundant schedule lookups. ([#4577](https://github.com/triggerdotdev/trigger.dev/pull/4577)) - Speed up setting and importing environment variables for projects with many variables. ([#4579](https://github.com/triggerdotdev/trigger.dev/pull/4579)) - Loading the deployments list is now faster, especially when filtering by deployment status on projects with many deployments. ([#4591](https://github.com/triggerdotdev/trigger.dev/pull/4591)) - Fixed the billing limits page timing out for organizations with many preview branches, especially while a spend limit was being enforced. The page now loads quickly, so you can raise or resolve your limit without delay. ([#4594](https://github.com/triggerdotdev/trigger.dev/pull/4594)) - Fix the Concurrency page showing the plan's default concurrency for the dev environment instead of the environment's actual limit. ([#4596](https://github.com/triggerdotdev/trigger.dev/pull/4596)) - Creating an organization sometimes left you back on the creation form even though the organization had already been created, so clicking Create again made a duplicate. Creating an organization now completes and takes you to your new organization. ([#4530](https://github.com/triggerdotdev/trigger.dev/pull/4530)) - Ensure creating a project completes instead of returning to its creation form after a navigation error. ([#4584](https://github.com/triggerdotdev/trigger.dev/pull/4584)) - Renaming a project now keeps you on the project settings page and tells you what happened, instead of silently moving you to the tasks page or clearing the form with no explanation. ([#4601](https://github.com/triggerdotdev/trigger.dev/pull/4601)) - Fixed support threads showing no account details for some customers, so the team can see your plan, organizations and projects when you get in touch. ([#4575](https://github.com/triggerdotdev/trigger.dev/pull/4575)) - In the light theme, the Format, Clear and Copy buttons on the query editor no longer blend into the query text behind them. ([#4592](https://github.com/triggerdotdev/trigger.dev/pull/4592)) - The health report now says start latency is "unknown" when there is no data for it, instead of showing a healthy-looking 0ms ([#4544](https://github.com/triggerdotdev/trigger.dev/pull/4544)) - Realtime streams written inside a chat session run now use the same backend as the session itself, and runs are no longer created against a backend that cannot serve them. ([#4564](https://github.com/triggerdotdev/trigger.dev/pull/4564)) - The grouped "watch updates" notification now shows the total number of results waiting, instead of only the most recent batch's count. ([#4525](https://github.com/triggerdotdev/trigger.dev/pull/4525)) <details> <summary>Raw changeset output</summary> # Releases ## @trigger.dev/build@4.5.11 ### Patch Changes - Updated dependencies: - `@trigger.dev/core@4.5.11` ## trigger.dev@4.5.11 ### Patch Changes - Chat in the browser now reconnects when the connection drops mid-turn, instead of leaving the reply stuck as if it were still generating. Reports can be fetched as structured data with the `json` format, and the shortest report period is now one minute (`1m`, `30m`, `1h`, `7d`). The `mint-token` command's help is clearer too: a token minted without `--cap` is read-only, and `--ttl` shows the correct maximum lifetime of 7 days. ([#4418](https://github.com/triggerdotdev/trigger.dev/pull/4418)) - Allow `trigger deploy` to authenticate with an environment API key from `TRIGGER_ACCESS_TOKEN`. ([#4561](https://github.com/triggerdotdev/trigger.dev/pull/4561)) - The dev environment onboarding now tracks real progress. After you run `init`, the setup checklist marks your project as initialized, and it updates live as your dev server connects and your tasks register. The blank state also adds a "Copy AI agent prompt" button that copies a ready-to-paste setup prompt (pre-filled with your project reference) for Claude Code, Cursor, or any coding agent. ([#4563](https://github.com/triggerdotdev/trigger.dev/pull/4563)) The `init` scaffold now imports from `@trigger.dev/sdk` instead of the deprecated `@trigger.dev/sdk/v3` subpath. - Deployed images now ship dependencies and bundled task code as separate layers. Repeat deploys with unchanged dependencies typically push and pull far less data, making deploys and worker image pulls faster. ([#4551](https://github.com/triggerdotdev/trigger.dev/pull/4551)) - Updated dependencies: - `@trigger.dev/core@4.5.11` - `@trigger.dev/build@4.5.11` - `@trigger.dev/schema-to-json@4.5.11` ## @trigger.dev/core@4.5.11 ### Patch Changes - Chat in the browser now reconnects when the connection drops mid-turn, instead of leaving the reply stuck as if it were still generating. Reports can be fetched as structured data with the `json` format, and the shortest report period is now one minute (`1m`, `30m`, `1h`, `7d`). The `mint-token` command's help is clearer too: a token minted without `--cap` is read-only, and `--ttl` shows the correct maximum lifetime of 7 days. ([#4418](https://github.com/triggerdotdev/trigger.dev/pull/4418)) - The current-worker API now reports each task's queue, so you can see which tasks write to a given queue. ([#4525](https://github.com/triggerdotdev/trigger.dev/pull/4525)) ## @trigger.dev/python@4.5.11 ### Patch Changes - Updated dependencies: - `@trigger.dev/core@4.5.11` - `@trigger.dev/sdk@4.5.11` - `@trigger.dev/build@4.5.11` ## @trigger.dev/react-hooks@4.5.11 ### Patch Changes - Updated dependencies: - `@trigger.dev/core@4.5.11` ## @trigger.dev/redis-worker@4.5.11 ### Patch Changes - Updated dependencies: - `@trigger.dev/core@4.5.11` ## @trigger.dev/rsc@4.5.11 ### Patch Changes - Updated dependencies: - `@trigger.dev/core@4.5.11` ## @trigger.dev/schema-to-json@4.5.11 ### Patch Changes - Updated dependencies: - `@trigger.dev/core@4.5.11` ## @trigger.dev/sdk@4.5.11 ### Patch Changes - Chat in the browser now reconnects when the connection drops mid-turn, instead of leaving the reply stuck as if it were still generating. Reports can be fetched as structured data with the `json` format, and the shortest report period is now one minute (`1m`, `30m`, `1h`, `7d`). The `mint-token` command's help is clearer too: a token minted without `--cap` is read-only, and `--ttl` shows the correct maximum lifetime of 7 days. ([#4418](https://github.com/triggerdotdev/trigger.dev/pull/4418)) - Watch-mode chat streams now survive quiet windows and page reloads, and a reply cut off by a lost connection shows an error instead of appearing finished. Aborting a resumed subscription only closes your local stream — call `stopGeneration(chatId)` or pass `stopOnAbort: true` to stop the run. Also fixed a race where quickly restarting a stream could break stop and reconnect, and stopping a chat now hands it back to your other tabs instead of leaving them read-only. ([#4516](https://github.com/triggerdotdev/trigger.dev/pull/4516)) - Updated dependencies: - `@trigger.dev/core@4.5.11` </details> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>	2026-08-13 15:48:29 +01:00
Katia Bulatova	480bede0ad	feat(webapp,sdk): dashboard agent plan enforcement, component gallery — and fixes (#4516) ... Plan enforcement for the dashboard agent — message quota and watch limits — plus the component gallery, fixes and test hardening from the same stack (#4548, #4549, #4550, #4552, #4556 merged here). ## Plan enforcement ([TRI-12863](https://linear.app/triggerdotdev/issue/TRI-12863)) **Agent message quota.** The Free-plan allowance becomes a real server-side limit with a durable counter. New `agent_message_usage` table keyed `(organization_id, period)` — deliberately not joined to chats, so deleting a chat can't free quota within the period. Both send paths count one user message (wakes never count) and refuse at the cap with `403 message_quota_reached`, which the client renders as an upgrade panel, never a silent drop. The refusal code is a single shared constant on both sides. **Watch limits.** A watch whose window exceeds the plan's `agentWatchMaxHours`, or that would push the org past its `agentWatchers` count, is refused with `watch_limit_reached` (409 on the API, an upgrade hint on the card). Plan limits only tighten the existing code ceilings (`min(plan, 24h)`, per-chat cap of 3 still applies). A plan limit of zero means zero, not unlimited. Questions answerable instantly are answered before any plan refusal — a one-shot consumes no slot and never sees an upgrade nag. **Fails open by design.** Cloud ships the actual per-plan numbers separately (TRI-12863 P0). Until then absent limits resolve to the unlimited sentinel and the upgrade UI is gated on billing presence — self-hosted sees no cap, no upsell, with tests proving the fallback. Both quotas are nudges, not security boundaries: a failing limit read never blocks a send. ## Component gallery An admin-only gallery of every agent card state: five `storybook.agent-*` pages (chat UI, view blocks, report, investigation, watch) with their shared shell and manifest, demo fixtures, two demo-only cards, toast examples, and the screenshot script. No LLM and no data — every state renders from fixtures under `dashboard-agent/demo/`, never reachable from a production path. Designers and reviewers can look at every state, including the report states, without seeding anything. ## And fixes **SDK: watch-mode chat subscriptions survive quiet windows** (TRI-13065, TRI-13070) — watch mode keeps reconnecting across empty long-poll windows and only stops on abort or a settled session; a passive subscriber can no longer stop a turn it doesn't own (`stopOnAbort` is explicit, default off). Review findings fixed alongside: a superseded stream's async teardown no longer removes the live successor's abort controller or multi-tab claim, and stopping a generation hands the chat back to the user's other tabs. **Query boundary pinned end-to-end** ([TRI-11165](https://linear.app/triggerdotdev/issue/TRI-11165)) — a route-level test drives `api.v1.query` with a real signed environment JWT (writes refused before ClickHouse, a read passes); `readonly=1` made non-overridable; a per-turn cap stops the model burning a turn rewriting a query it can't fix (deterministic SQL errors only — busy/transport rejections don't count). **chat.agent durability regression suite** ([TRI-11166](https://linear.app/triggerdotdev/issue/TRI-11166)) — testcontainers-backed coverage of the two audit criticals (cross-tenant isolation, no duplicate mid-stream turn, both control-broken) plus crash-resume, cursor-based refresh, clean rollback of a mid-write turn failure (torn by a real constraint violation), and OOM-restart replay. **Investigation sweep backoff** — stale investigations get an attempt counter and backoff so a poison row can't pin the sweep queue head (migration `0005`: `sweep_attempts`, `last_sweep_attempt_at`). ## Screenshots <img width="1440" height="791" alt="Screenshot 2026-08-06 at 00 36 19" src="https://github.com/user-attachments/assets/6a68cd42-8580-469d-afe7-e28d1eef18e1" /> 🤖 Generated with [Claude Code](https://claude.com/claude-code)	2026-08-12 13:59:35 +02:00
Chris Arderne	ed1bb72fb8	feat: implement cron window spread backend (#4566) ... - New DB fields on Schedule and ScheduleInstance - Use `queueTimestamp` for the "effectiveAt" delayed start time, propagate it to Clickhouse TaskRun table - Disable fastpath for delayed jobs - Add schedule timing logic, API endpoints with windows, persistence - Calculate phase for every schedule, only persist when window is non-null - Additional o11y for phased rollout	2026-08-12 12:24:32 +01:00
Chris Arderne	7b390e5984	feat(cli,webapp): allow deploys with environment API keys (#4561)	2026-08-12 10:11:31 +01:00
Katia Bulatova	0b750d00dd	feat(webapp): dashboard agent — Watch (#4525) ... Watch is the agent noticing something later: you ask it to tell you when a condition holds, and it answers when it does — or when it can't any more. A watch is a **durable one-shot promise**. The condition is checked on a schedule by deterministic code (no LLM in the checks), the answer lands in the chat once, and then the watch is over. Ten kinds: three on a run, five on a queue, error recurrence, health recovery. ## Stack Stacked on **#4529** (UI), which is stacked on **#4418** (chat, reports, investigate). Merge those first. **#4516** (storybook gallery) sits on top of this branch. ## How to review [**GUIDEBOOK.md**](https://github.com/triggerdotdev/trigger.dev/blob/feat/dashboard-agent-flows-watch/internal-packages/dashboard-agent/GUIDEBOOK.md) on this branch is the behaviour reference — it states the conditions rather than the code, so you can predict what happens without running anything. "The ten watch kinds, and what makes each fire" and "Creating a watch" describe exactly this PR, and the tables there are the spec the code is written against. ## What's inside - **Ten watch kinds**, one deterministic check each (`dashboardAgentWatch*Checks.ts`), with the spec union in `dashboard-agent-contracts/src/watch.ts`. - **Scheduling** — each watch schedules its own next check; due watches of one `(environment, cadence)` group can be checked together in one batch pass, with a sweep as the backstop for expiry, redelivery and retention. - **Delivery** — the in-chat wake and card, an optional email alert (new `DASHBOARD_AGENT_WATCH` alert channel, so it shows on the project's Alerts page with one-click unsubscribe), and an optional investigation when the outcome needs attention. - **Submission ledger** — `watch_submissions`, keyed `(chat_id, client_request_id)`, so a retried card submission replays the recorded outcome instead of creating a second watch. - **Watch token** — a delayed-execution credential accepted only by the watch endpoints, re-checked against the user's live access on every tick. - **Unread work** — the panel polls for wakes that landed while it was closed, so a chat can go unread and light the launcher dot. ## Key decisions **A check result is a 4-way, and only two of them are verdicts.** `satisfied` / `terminal_unsatisfied` are answers; `pending` and `unavailable` are not. Any exception inside any check is caught in one place and becomes `unavailable` with an unverified observation — a check that failed is never evidence. **A completed window is an answer, and whether it is good or bad news is declared per kind, never inferred.** There is a table for that in the guidebook: `run_failed` completing its window is *good* news ("hasn't failed"), `backlog_drain` completing it is not. One rule overrides the table: a window that completed on an unverified observation is neutral and says only that the watch ended without a confirmed answer. **An unreadable source is never a negative answer** — and, because investigations only open on `attention`, it never starts one either. **Identity is `(chat, project, environment)` plus the condition,** enforced by a partial unique index over active rows (`watches_chat_active_identity_key`), not by the read-then-insert check. Cadence, window, note and `ticks` are deliberately not part of it. Two different chats may watch the same thing — a watch is a promise to a chat. **The server resolves the target's name, whatever the model calls it.** The model can't tell a task queue (`task/<id>`) from a custom queue, so both spellings are tried and the stored one wins — and the rewrite happens **before** identity and before the row is written, so the identity, the checks, the link and the wording all see one spelling. **Freshness fences.** Depth falls back from the live counter to the newest 60 s ClickHouse bucket, which only counts as current within 60 s of now. A non-current reading at or below the *quiet line* is refused as `unavailable` rather than believed, so a stale empty bucket is never read as "drained". The stall streak is the one piece of carried state: it lives in the previous check's facts and *freezes* on an unreadable reading rather than breaking. **Chain reliability.** There is no shared cron — each watch (or batch group) schedules its own next tick, so the failure mode to review is the chain dying. A failed batch check is caught, the next tick is scheduled anyway and the run resolves rather than failing, so the chain survives a check that couldn't run; the sweep re-arms groups and finalizes anything still active past its deadline, even when delivery isn't configured. Wake redelivery is id-deduped rather than conditional, because the sweep can't know whether the user was already told. Access is re-authorized on **every** check against the primary — replica lag would extend access the user has already lost. **Wording lives in one place.** `watch-wording.ts` is read by the card, banner, toast, email and the agent's own narration, and the numbers come from the frozen observation rather than a fresh read, so a retry produces the same sentence. Replay reproduces the **recorded** decision instead of deciding again — the transcript is append-once, so a second decision would contradict it forever. **Cancellation is the ending without an answer** — no resolution, no wake. One exception, decided during testing: a watch the *user* cancelled leaves a single neutral transcript line ("Stopped watching …"), keyed off the watch id so a retry can't repeat it. The other four reasons stay silent. **Email is opt-in and only a fired watch emails.** An expiry is narrated in the chat and nowhere else. Both gates (agent access, a configured email transport) are checked at subscribe time *and* again at delivery, and the subscription outcome is frozen on the ledger row so a retry replays it. Neither gate is a plan check. **One watch offer per turn.** The prompt and the renderer guard this independently — if the turn already proposed a watch card, the action button is dropped, because the card is the better affordance. Two eval cases pin the prompt side: exactly one offer with the line last and the button after it, and zero offers when the rendered card already carries one — deterministic assertions, over a real-model run. ## Testing Unit tests (vitest, testcontainers, no mocks) under `apps/webapp/test/dashboardAgentWatch*.test.ts` and `internal-packages/dashboard-agent/src/watch-*.test.ts` cover the invariants above: the 4-way check results and the freshness fences, identity/dedup and the submission ledger, queue-name resolution, the batch chain surviving a failed check, sweep boundaries and alert-once, tenancy and the watch token's scope, and the wording snapshot. The load-bearing ones were verified by control-breaking the guard first and checking the test goes red. Live-tested end to end against a local stack, following the guidebook: all ten watch kinds firing and expiring, cancellation, the email pair (a fired watch mails, an expired one does not), and watch recovery from a health report.	2026-08-12 09:51:40 +02:00
Katia Bulatova	4569657923	feat(webapp): dashboard agent — chat, reports, investigate (#4418) ... ## What & why This is the system behind the Dashboard Agent — an assistant that answers questions about a project's runs, errors, queues, deploys and health, and can investigate failures end to end. The agent runs as a chat.agent task in its own Trigger project. It has no access to the main database or ClickHouse; all platform data is read through the public API using a delegated, read-only user token. Everything here is behind `canAccessDashboardAgent` and inert with the flag off. The UI that mounts the panel lands in #4529. ## Stack `#4418` (this, base) ← `#4529` UI ← `#4525` Watch ← `#4516` storybook gallery. The scenario/contract reference for the whole stack is `internal-packages/dashboard-agent/GUIDEBOOK.md` (it lands on the Watch branch): it states, per feature, what makes each thing happen and where that is decided. ## What's inside **Agent runtime and tools** — `internal-packages/dashboard-agent`: prompt, tool set (API reads, TRQL query, docs, navigation, evidence/investigations, repo source), conversation compaction, a prompt-prefix token budget pinned by snapshot test, and sampled LLM-judged turn evals. The package cannot import webapp server code, which is what makes the "no DB access" claim structural rather than a convention. **Contracts** — `internal-packages/dashboard-agent-contracts`: `trigger://` URIs, intents, and the block envelope every rendered card travels in. **Conversation store** — `internal-packages/dashboard-agent-db`: drizzle over postgres-js in its own `trigger_dashboard_agent` Postgres schema, plus one additive migration. **Auth boundary** — the user-actor token gains an optional environment claim; one guard (`userActorEnvironment.server.ts`) enforces it so routes don't each re-derive the rule. Token minting, cap ceiling, and the RBAC fallback path for self-hosted. **Transport** — webapp resource routes that mint the token and proxy each turn, and SDK-side mid-turn reconnect. **Public API the agent reads through** — orgs, projects, environments, runs, queue metrics, workers, a run's commit metadata, repo snapshot, reports, and `POST /api/v1/query`. **Reports** — the health report's layout is declared once and shared by the card, the markdown surface and the JSON/MCP surface, so the same report reads the same in the dashboard, the terminal and an editor. **Block renderers** — the report and investigation cards the flows above already emit (`app/components/dashboard-agent/`). The panel that hosts them, and the rest of the chat UI, is #4529. **Query safety and CSP** — see below. ## Key decisions - **The agent is a separate Trigger project, not webapp code.** It reads platform data over the public API with a delegated user-actor token whose `cap` ceilings it to read scopes. No Prisma, no ClickHouse, no webapp imports. - **The PAT-only auth helper now refuses user-actor tokens.** This is an intentional behavioral change: its callers consume only a bare userId and do not enforce delegated-token capabilities. Actor-aware routes continue through the scoped route builders instead. - **RBAC fallback builds a delegated token's ability from its own cap**, never the blanket ability a PAT gets (read-only when the token declares none). Without this, the agent's read-only cap would buy a write JWT on self-hosted. - **Org creation checks RBAC only for user-actor tokens, and only after the env gate**, so an install with `ORG_CREATION_API_ENABLED` off returns 404 rather than 403, and an ordinary PAT never consults an ability the route has no org to scope. Both orderings are pinned by test. - **The query path is read-only in depth.** TRQL rejects write statements at the grammar level (they don't parse, rather than being filtered), ClickHouse runs with `readonly=1`, and the org/project/env filters are injected server-side from the credential — the request body cannot widen scope. An unparseable query denies instead of falling through to the permissive resource. - **Document-wide img-src CSP.** Remote images are an outbound-request/exfiltration surface, so the policy permits only own-origin/data/blob, the required SSO avatar hosts, and the favicon endpoint. Operators can add exact origins through CSP_IMG_SRC_ALLOWLIST; wildcard hosts and bare schemes are intentionally not allowed. - **The chat transport reconnects on a mid-turn EOF** (`@trigger.dev/sdk`). A body that ends without a turn-complete is terminal only when the server says `X-Session-Settled: true`; otherwise the transport resubscribes from `lastEventId` with bounded backoff, and any record re-earns the budget. Previously a closed long-poll window or a proxy restart left the reply stuck as if still generating. - **Conversations live in their own datastore**, schema-scoped and foreign-key-free (it references `organizationId`/`userId` by id, because in cloud it is a different database). It is a display read-model for the History tab and transport resume; `chat.agent`'s object-store snapshot remains the model's source of truth. - **Deterministic first.** Reports and health checks contain no LLM — they are computed from the same data the dashboard shows, and the model only narrates and links them. That is what makes a number in an answer auditable. ## Testing - 63 new test files, run with `pnpm run test --filter webapp` and per-package vitest. Heaviest coverage on the auth boundary (`userActorPatOnlyBoundary`, `userActorTokenClaimsAndScopes`, `contextlessPatRoutes`, `rbacFallbackBranch`), TRQL read-only, the report layout, and the SDK reconnect. - The agent package has a separate eval lane (`pnpm run test:evals`, `vitest.eval.config.ts`) that hits the real model, so it never runs in `pnpm test`. - Live-tested against a local stack scenario by scenario; the GUIDEBOOK lists the condition each behaviour is expected under, which is what those runs were checked against. ## Changelog `.server-changes/dashboard-agent.md`, plus changesets for `@trigger.dev/core` (report schemas), `@trigger.dev/sdk` (chat reconnect) and the CLI's `mint-token` help text.	2026-08-11 18:56:14 +02:00
Eric Allam	6449a644b9	feat(webapp,cli,database): track real dev onboarding progress (#4563) ... ## Summary The dev environment "Get set up" panel used to be a static list of CLI commands that only disappeared once your tasks registered, so nothing ever changed after you ran `init` and people assumed it was stuck. It now tracks real progress: `trigger init` records the project as initialized, so step 1 checks off, and the panel updates live as the dev server connects and your tasks register. It also adds a prominent "Copy AI agent prompt" button, presented as a clear alternative ("or") to the manual CLI steps, that copies a ready-to-paste setup prompt pre-filled with your project reference for Claude Code, Cursor, or any coding agent. ## Notes - Adds a `Project.initializedAt` column (migration `20260811065646_add_project_initialized_at`); the CLI `init` command calls a new project-scoped `POST /api/v1/projects/:ref/init` best-effort at the end of setup. - The `init` scaffold now imports from `@trigger.dev/sdk` instead of the deprecated `/v3` subpath. ## Screenshots <img width="2400" height="1794" alt="v7-redesigned-card" src="https://github.com/user-attachments/assets/c2fb4fa1-9484-4700-8bd3-110d66f5a44e" />	2026-08-11 11:43:33 +01:00
Saadi Myftija	bd8ce4a50f	feat(deployments): split project dependencies and code into separate layers (#4551) ... Deploy images previously shipped node_modules and the bundled task code in a single layer, so every deploy re-pushed and re-pulled the full dependency tree even when nothing in it changed. The generated Containerfile now copies `/app/node_modules` as its own layer and the app files separately. With unchanged dependencies the dependency layer is identical across deploys, so registries and workers already have it and only the code layer moves.	2026-08-10 14:44:11 +02:00
github-actions[bot]	72f50c2dad	chore: release v4.5.10 (#4440)	2026-08-07 14:06:43 +01:00
Matt Aitken	04f9c4e1a5	fix(webapp,run-engine,core): drop the hidden debounce ceiling, fail fast on an unusable maxDelay (#4521) ... Debouncing with a `delay` longer than an hour did nothing at all. The engine applied a server-side ceiling on how long a debounced run could be pushed back, measured from the run's `createdAt` and defaulting to one hour. A run is only pushed back while its new execution time stays inside that ceiling, so a `delay` at or above it could never push anything: the waiting run was released, the trigger started its own run, and the next trigger repeated it. A `delay: "12h"` produced one run per trigger, each correctly delayed by 12h, with no error raised and nothing on the run to show the debounce key had been ignored. The ceiling is now unset by default. A debounce key with no `maxDelay` keeps collapsing triggers for as long as they keep arriving, which is what the docs have always described. Self-hosters who want a bound can still set `RUN_ENGINE_MAXIMUM_DEBOUNCE_DURATION_MS`. That has a consequence worth stating plainly, so the docs now carry a warning for it: with no `maxDelay`, a continuously triggered key never executes. Set `maxDelay` when the work has to happen eventually. **Failing fast on an unusable `maxDelay`.** A caller who sets `maxDelay` no longer than their `delay` hits exactly the dead end described above, so that pair is now rejected at trigger time instead of silently behaving as if no debounce were set: ``` debounce.maxDelay (1h) must be longer than debounce.delay (12h). A debounced run is only pushed back while it stays inside maxDelay, so with these values every trigger would create its own run. ``` An unparseable `maxDelay` is rejected too, rather than quietly falling back to no bound at all, and so is a `delay` given as a date rather than a duration, which could never work because the value is re-applied on every push. The same check runs against a configured server ceiling, so a self-hosted deployment that sets `RUN_ENGINE_MAXIMUM_DEBOUNCE_DURATION_MS` gets the error rather than the silent failure this PR is about. With no `maxDelay` and no configured ceiling, which is the default, there is nothing to conflict with and nothing is rejected. The docs, the `TriggerOptions` JSDoc and the engine option all now state that the room available to push is the gap between `delay` and `maxDelay`. The run engine suite gains the case that motivated this: four triggers on one key with a 12h delay now collapse to a single run.	2026-08-07 07:55:35 +00:00
Matt Aitken	c084fa6e29	fix(sdk,react-hooks): forward debounce when batch triggering with an array (#4520) ... Passing `debounce` in the per-item options of a batch trigger did nothing when the items were an array. The option was accepted by the types and by the API, then dropped before the request went out, so every item created its own run instead of collapsing onto the debounce key. Four public entry points were affected: `task.batchTrigger`, `task.batchTriggerAndWait`, `tasks.batchTrigger`, and `tasks.batchTriggerAndWait`. The streaming (async iterable) forms of the same calls were already correct, as were `batch.trigger`, `batch.triggerAndWait`, `batch.triggerByTask`, and `batch.triggerByTaskAndWait`. `useTaskTrigger` in `@trigger.dev/react-hooks` had the same silent drop on the single-trigger path, so that is fixed here too. It also drops `machine`, `priority`, `region`, `idempotencyKeyTTL`, and `idempotencyKeyOptions`; those are left alone, since forwarding them is a behaviour change beyond this bug. Each batch item builder constructs its options field by field, which is why one of them could fall behind without anything catching it. TypeScript did not help: the literal is returned from a `.map` callback inside `Promise.all`, so excess-property checking never fired against the `BatchItemNDJSON[]` annotation, and the server's schema silently strips unknown keys. A misspelled option name therefore reproduced this bug with no compile error and no server error. Every builder now ends in `satisfies BatchItemNDJSON`, which does catch it: ``` error TS2561: Object literal may only specify known properties, but 'debounceTYPO' does not exist in type '{ ... debounce?: {...} | undefined; }'. Did you mean to write 'debounce'? ``` The new test drives all six public batch surfaces in both array and async-iterable form and asserts on the NDJSON that actually reaches the wire. Each item carries a distinct debounce key so the test catches a wrong item-to-option pairing, not just a wholesale drop. Fixes #3304	2026-08-06 16:54:24 +01:00
Chris Arderne	9409ddf9bc	feat(webapp): add multiple environment API key management (#4390) ... ## Summary Projects can create, inspect, expire, and revoke multiple API keys for each environment. Plaintext values are shown only at creation; stored credentials are hashed and the API keys page displays only an obfuscated suffix afterward. Self-hosted installations support full-access additional keys by default. Authorization extensions can provide additional access presets and optional task selection. Additional keys can also mint scoped public access tokens through the Trigger.dev API without receiving the environment signing key. ## Feature notes - Only admin+ can create API keys (Developer can make in Development branch). - JWT self-signing will be a server call when used with new `_ak_` keys. - JWTs with long expiry can keep working even with api key deleted (gets priveleges from api key, signed with root key) - Unfiltered session listings intentionally preserve the existing broad task-read behavior. Filtered listings enforce task-level scopes for every requested task. - Buffered runs without a task identifier are not safely authorizable, so cancel/replay requests fail closed rather than resolving an unscoped run. - Batch and waitpoint endpoints intentionally return server-minted, narrowly scoped public tokens to all callers. These tokens have bounded lifetimes and may remain valid until expiry after API-key revocation. ## Deployment notes Deploy the management UI and public-token endpoint with new key creation disabled. Enable creation for selected organizations after the authentication path and released SDK have been verified, then expand availability gradually. Revoking an API key prevents new bearer requests and new token minting. Public tokens already minted by that key remain valid until their own expiration because they are signed by the environment signing key. ## TODO - [x] Add "Created by" to the key table - [x] Document that streamed batch ingestion is non-atomic and may partially accept items before a validation or authorization error. ## Follow-ups - [x] Add an organization-level feature flag for the API key management UI and creation action. - [x] Document rollout ordering: enable additional-key lookup before enabling issuance. - [x] Add a system-wide gate that can stop new key issuance without disabling authentication for existing keys. - [x] Replace the generic SDK compatibility warning with the first published compatible version. Old SDK will mint an unusable token if given an `_ak_` key. - [x] Add public documentation covering creation, storage, expiration, revocation, SDK compatibility, and public-token lifetime behavior. - [x] Add observability for key creation, revocation, policy preparation failures, and public-token mint failures. - [ ] Exercise create, copy-once display, authenticate, mint, expire, and revoke flows end to end before broad enablement.	2026-08-06 15:27:10 +01:00
Chris Arderne	1a16d61a37	fix(build): support decorator metadata with TypeScript 7 (#4505) ... ## Summary Allow projects using TypeScript 7 to enable `emitDecoratorMetadata()` without adding the TypeScript 6 compiler to every Trigger.dev CLI installation. Addresses #4500. ## Fix The extension now resolves TypeScript from the project and feature-detects the legacy compiler API. TypeScript 5 and 6 continue using the project's compiler, while TypeScript 7 projects can install Microsoft's optional `@typescript/typescript6` compatibility package alongside TypeScript 7. When no compatible compiler API is available, the build reports an actionable installation error. The extension documentation includes setup commands for npm, pnpm, and Bun. Verified with TypeScript 5, TypeScript 6, TypeScript 7 with and without the compatibility package, emitted decorator metadata, packed ESM and CommonJS consumers, package export checks, and typechecking.	2026-08-05 16:33:32 +01:00
Chris Arderne	85f5b37c68	chore: upgrade to TypeScript 7 (#4318) ... ## Summary Upgrade the monorepo to TypeScript 7.0.2 and update package build tooling for compatibility with the native compiler. ## Design Package builds now use `tshy` 4, while the packages still using `tsup` move to `tsdown`. The few scripts that depend on the legacy TypeScript compiler API use an explicit TypeScript 6 alias; declaration portability coverage invokes the TypeScript 7 CLI directly. Turbo is updated so workspace tasks can read the regenerated pnpm lockfile. --------- Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-08-05 15:49:44 +01:00
nicktrn	c01a4f18f4	feat(supervisor): cancel a resumed run's in-flight checkpoint (#4502) ... A run controller must call the continue route to resume, so the supervisor already knows synchronously that any checkpoint still running for that run is pointless. It only acted on that for the compute path. The continue route now cancels it for the Kubernetes path too, matching what completion already does since #4493. Called after the reply so the runner is never delayed, and skipped when there is no checkpoint client or when the compute path owns the run. The request is bounded by a 5s timeout so a hung call cannot leave the handler pending. `checkpoint_cancel_requests_total{result}` records the outcome, using the same label names as the delete path where they overlap: `sent`, `no_client`, `not_applicable`, `http_error`. No changeset: `CheckpointClient` is a server-only internal API, same as #4493. refs TRI-12915	2026-08-05 12:01:14 +01:00
nicktrn	4f69c43e6b	feat(supervisor): reclaim a run's checkpoint storage when it finishes (#4493) ... When a run reaches a terminal state, ask the checkpoint service to reclaim the storage its checkpoints occupied. Storage for finished runs is not otherwise reclaimed, so nothing frees it today. **Off by default** behind `DELETE_CHECKPOINTS_ON_COMPLETION`, and the service-side handler ships separately, so merging this changes no behaviour. ## Where the tenancy comes from Addressing a run's checkpoints needs org, project, environment, deployment version and run id. All five are already in hand at `attempt.complete`, and three are **signed** by the deployment token: | Value | Source | Trust | | -- | -- | -- | | org | claim `org_id` | signed | | environment | claim `environment_id` | signed | | deployment version | claim `deployment_version` | signed | | project ref | `x-trigger-workload-project-ref` header | runner-supplied | | run | route param | runner-supplied | `authorizeWorkloadRequest` previously returned only `environment_id`, and only in enforce mode, so it now also returns the verified `claims`. That difference is deliberate and documented on the method: claims are used to address a run's **own** resources locally, never to scope the platform, which is why `environmentId` stays enforce-only. The two runner-supplied values are safe because the signed ones are outermost - a runner lying about either can only name something inside its own org and environment, and a project ref that doesn't pair with its signed environment matches nothing. The run id is read from `params.runFriendlyId`, the same value the platform just validated, rather than from the body or a header. Where both a claim and a header exist (`deployment_version`), the claim wins. ## Placement The call sits after `reply.json(...)`, so the runner sees no added latency - the same shape the suspend route already uses. The service enqueues and returns 202, so it is one fast local hop. Terminal means `RUN_FINISHED` **or `RUN_PENDING_CANCEL`** - a run cancelled mid-execution never restores, and skipping it would leave its storage behind. Retries are excluded deliberately: reclamation is per-run, so a retry is covered by the final completion. Also gated on `!snapshotService`, so it stays inert where checkpoints aren't the kind this reclaims. ## Observability `checkpoint_delete_requests_total{result}` counts `sent` **and every reason we decide not to send**: `disabled`, `not_terminal`, `no_claims`, `no_project_ref`, `http_error`. The negative labels are the point - without them, "no requests are happening" looks identical to the feature being switched off. `no_claims` is reachable even under enforcement, since enforce only rejects a *present-but-invalid* token; an absent or legacy id still passes with no claims attached. ## Notes for review - **No changeset**: `CheckpointClient` is `core/v3/serverOnly`, an internal service-to-service API rather than customer-facing surface. - **No `.server-changes/` note**: there is nothing a dashboard user would notice here. Happy to add one if you disagree. - `pnpm run typecheck` can't complete in my checkout - `@trigger.dev/database` fails to build on a missing `tsc` in the pnpm store, unrelated to this diff. Verified with `tsc --noEmit` against the supervisor project instead: **zero errors in `apps/supervisor/src`**. Worth noting it caught a real bug here - the completion response is wrapped, so the status is `data.result.attemptStatus`. refs TRI-12789	2026-08-04 15:01:29 +01:00
Chris Arderne	763b5dc582	feat(webapp): enforce scopes for environment API keys (#4389) ... ## Summary Environment API keys backed by the additional-key table can authenticate API requests using their stored effective scopes. Revoked and expired keys are rejected, branch environments retain their existing routing behavior, and last-used timestamps are updated on a throttled best-effort basis. ## Design API route builders receive the resolved ability and reject restricted keys on routes without an authorization declaration. Existing deployment, environment variable, queue, run, task, batch, session, and waitpoint routes declare the resources they access. Trigger and batch responses return server-signed public access tokens, so additional keys never need access to the environment signing secret. Root-key rotation also keeps public tokens valid for the existing grace window. ## Feature notes - Root environment keys remain unrestricted for backward compatibility. Additional keys enforce their persisted scopes and fail closed on routes without an authorization declaration. - Machine-key requests never exchange one credential for another. Additional keys cannot retrieve the root key, and rotated root keys are not upgraded during their grace window. - Public JWT validation remains host-owned, while installed RBAC plugins continue to supply root-key abilities. - Unfiltered session and run listings preserve existing broad task-read behavior. Filtered requests enforce the supplied task identifiers. - Related-run summaries remain embedded in run retrieval for API compatibility. Retrieving or mutating a related run independently still requires permission for that run. - Queue management authorizes at collection scope, matching the queue permissions currently issued. - Batch responses deliberately include server-signed public access tokens for all clients. Selected-task credentials continue using their original credential for per-item authorization. - Two-phase batches authorize declared task identifiers before creation and authorize every streamed item. Streaming paths that cannot declare the complete task set remain fail closed. - Authentication telemetry records successful credential resolution separately from subsequent resource-authorization failures. - API keys are high-entropy random tokens. SHA-256 is intentionally used for deterministic indexed lookup, not password hashing. ## Deployment notes The schema migration must be present before this code is deployed. Because bearer resolution runs on every authenticated request, deploy the resolver with additional-key lookup disabled, verify root-key and public-token parity, then enable lookup before any additional keys can be issued. The multi-task authorization tightening changes the result for narrowly scoped tokens that request tasks outside their grants. Observe would-deny results before enforcing that check. Request-idempotency keys are also newly isolated by environment and task, so a retry crossing the deployment boundary may execute once more before old cache entries expire. ## Follow-ups - [x] Add a system-wide kill switch for additional-key lookup, defaulted off for the initial deployment. - [x] Add authentication observability by credential kind, result, latency, and lookup path without recording credential values. - [ ] ~Add would-deny observability and an independent enforcement switch for multi-task authorization.~ - [ ] ~Add an independent switch for server-issued batch tokens while root-key parity is verified.~ - [ ] Confirm every API route reachable by a restricted key has an explicit authorization declaration or intentionally fails closed. - [x] Verify root-key rotation, revoked-key grace, and public-token validation through each bearer resolver path.	2026-08-03 14:00:29 +01:00
Eric Allam	db6228dd1e	chore(webapp,core,sdk): upgrade @s2-dev/streamstore to 0.25 and migrate S2 hosts (#4349)	2026-08-01 11:33:34 +01:00
nicktrn	a91c08c731	fix(core): retry run start-attempt on transient connection errors (#4441) ... ## What `startRunAttempt` — the run controller's first call when a run starts — had no retry on transient connection errors. A brief connection blip on that call would abandon the start and send the run back through the queue, delaying its first attempt. This adds a jittered backoff retry, matching the existing `continueRunExecution` path with a shorter budget, so a transient blip is ridden out in place instead of bouncing the run. ## Why a shorter budget The continue path retries generously. Start-attempt keeps a tighter budget (6 attempts, ~25-40s jittered) so it rides out a transient blip but never keeps retrying past the point the run would already have been requeued. ## Safety Retrying is safe: start-attempt is guarded server-side by the snapshot id — a retry after a start has already committed is rejected, so it can never double-start an attempt. A pure connection error (the common case) never reached the server. ## Scope One retry-options object on `startRunAttempt`; no other behavior change. Warm starts share this path and get the same resilience.	2026-07-31 13:57:45 +00:00
claude[bot]	17d849b2d6	feat(cli): expose region option on the MCP trigger_task tool (#4439) ... <!-- ccr-slack-attribution --> _Requested by **Eric Allam** · [Slack thread](https://triggerdotdev.slack.com/archives/C0BEM9Z73TM/p1785491472104199)_ ## Checklist - [ ] I have followed every step in the [contributing guide](https://github.com/triggerdotdev/trigger.dev/blob/main/CONTRIBUTING.md) - [x] The PR title follows the convention. - [ ] I ran and tested the code works --- ## Testing Static checks only, all clean: - `pnpm run typecheck --filter trigger.dev` - `pnpm run format` - `pnpm run lint` No live task was triggered against a running project, so the "ran and tested" box above is left unchecked. --- ## Changelog **Before:** triggering a task through the MCP server always ran it in the project's default region. There was no way to pick one. **After:** the `trigger_task` tool accepts an optional `region` option, so you can choose the region a run executes in. **How:** `region: z.string().optional()` was added to `TriggerTaskInput.options` in `packages/cli-v3/src/mcp/schemas.ts`. No call-site change was needed — `tools/tasks.ts` passes `options` through verbatim, and `TriggerTaskRequestBody.options.region` already existed. The tool description in `docs/mcp-tools.mdx` gained a matching line, and a patch changeset is included. There is no batch-trigger MCP tool, so there is no sibling tool to mirror this change on. --- ## Screenshots N/A — no UI changes. Co-authored-by: Claude <noreply@anthropic.com>	2026-07-31 11:54:42 +01:00
github-actions[bot]	86b948b47a	chore: release v4.5.9 (#4408)	2026-07-30 10:12:14 +01:00
Chris Arderne	2f1734c858	fix(core,webapp): redact sensitive fields in logs by default and cap their size (#4401)	2026-07-29 17:59:47 +01:00
Chris Arderne	8ebc8a41af	fix(webapp,redis-worker): stop logging raw metadata, alert payloads, and job items (#4403)	2026-07-29 17:59:36 +01:00
Chris Arderne	878c15811a	fix(cli): redact environment values from build debug logs (#4420)	2026-07-29 16:36:25 +00:00
Chris Arderne	a81ad4949c	feat(database,rbac): add multiple environment API key foundations (#4388) ... Adds the storage model and authorization contracts needed for multiple environment API keys. Credentials are represented by hashed values, revocation and expiration state, and persisted effective scopes. The built-in authorization fallback exposes full-access policy preparation, while optional authorization extensions can supply additional presets and task-aware scope generation. This change does not create, display, or authenticate additional keys.	2026-07-29 16:24:00 +00:00
Eric Allam	4eb9292cbe	feat(webapp,run-engine): queue metrics and health dashboard (#4131) ... ## Summary Three related changes, each independently gated: **Queue metrics and health.** Per-queue depth, throughput (enqueued, started, completed), concurrency, whether a queue is throttled, and scheduling delay (how long a run waits between becoming eligible and actually starting), plus a per concurrency-key breakdown for keyed queues. Collected from inside the run queue itself, stored in ClickHouse, and surfaced on the Queues list, a new per-queue detail page, the task pages, and the run inspector. The question it answers is "does this queue have enough concurrency to keep up, and if not, which key or which limit is the constraint". **Percent-based queue concurrency limits.** A queue's concurrency override can now be expressed as a percentage of the environment limit, stored as the source of truth and re-materialized whenever the environment limit changes. Absolute overrides above the environment limit are now **rejected with a 400** instead of being silently capped, which is a behavior change on `POST /api/v1/queues/:queue/concurrency/override`. **The `health` report.** A server-computed verdict on whether work is flowing, whether the runs that do start are healthy, and whether telemetry is fresh, rendered as text with sparklines. Available as `GET /api/v1/reports/:key`, `trigger report`, and the `get_report` MCP tool (plus a `report` MCP prompt, which shows up as a slash command in hosts that support prompts). With the flags off, the Queues page renders the pre-metrics component verbatim, nothing is emitted, and nothing is written to ClickHouse. ## Configuration Two independent gates, on purpose. Emission is global so data accrues for everyone before anyone can look at it; the view is per organization so it can be turned on for one org at a time without a deploy. **Runtime flags (no restart)** | Flag | Store | Gates | | --- | --- | --- | | `queue_metrics:enabled` | run-queue Redis key (`"1"`/`"0"`, off by default) | All emission, gauges and counters. Cached in-process for 10s with stale-while-revalidate, warmed eagerly at boot so the first op after a deploy is not dropped. | | `queue_metrics:gauge_sample_rate` | run-queue Redis key, `0..1` | Fraction of queue ops that emit a gauge. Counters are never sampled, so throughput stays exact at any rate. | | `queueMetricsUiEnabled` | feature-flag catalog: global `FeatureFlag` row, per-org `Organization.featureFlags` override wins | Whether an org sees the metrics view at all: the Queues list variant, the queue detail route, the built-in Queues dashboard, the concurrency-keys endpoint, and the metrics blocks on task pages and the run inspector. Off by default; a gated org gets a 404 on the detail route rather than an empty page. | Both Redis keys are readable and writable from `/admin/queue-metrics` (super-admin UI, with a live per-shard stream-health table) and `GET`/`POST /admin/api/v1/queue-metrics` (admin PAT). The admin surface uses its own Redis client, so it works on any instance regardless of whether that instance runs the emitter or the consumer. **Environment variables (boot time)** | Variable | Default | Notes | | --- | --- | --- | | `QUEUE_METRICS_EMIT_ENABLED` | `0` | Constructs the emitter and injects it into the run engine. Without it the run queue has no emitter at all. | | `QUEUE_METRICS_CONSUMER_ENABLED` | `0` | Boots the stream consumer on this instance. Independent of emission, so consumers can be sized separately from the API. | | `QUEUE_METRICS_STREAM_SHARD_COUNT` | `4` | Stream shards, hashed per queue. | | `QUEUE_METRICS_CONSUMER_BATCH_SIZE` | `1000` | Poll batch equals insert batch, so an ack can never outrun a write. | | `QUEUE_METRICS_REDIS_{HOST,PORT,USERNAME,PASSWORD,TLS_DISABLED}` | falls back to the run-queue Redis | Set `HOST` to move the metrics stream onto a dedicated instance so a metrics backlog cannot compete with the run queue for memory. Self-hosters can leave it unset and get a single-Redis deployment. | | `QUEUE_METRICS_COUNTER_STREAM_MAXLEN` | `2000000` shared, `8000000` dedicated | Bound on how much a stalled consumer can hold. The default is deliberately lower when the stream shares the queue-critical Redis. | | `QUEUE_METRICS_COUNTER_ODOMETER_TTL_SECONDS` | `604800` | TTL on the per-queue cumulative counter key, refreshed on every write, so only queues idle for the whole window are purged. | | `QUEUE_METRICS_MAX_QUEUE_NAMES_PER_ENV` | `1000` | Distinct queue names tracked per environment; overflow collapses into `__overflow__`. | | `QUEUE_METRICS_MAX_CONCURRENCY_KEYS_PER_QUEUE` | `10000` | Same idea one level down, per queue. | | `QUEUE_METRICS_GAUGE_SAMPLE_RATE` | `1` | Default for the live sample-rate key above. | | `QUEUE_METRICS_QUERY_TABLES_VISIBLE` | `0` | Lists the queue-metrics tables in the Query page, its schema docs, the schema API and the AI query context. Off keeps them unlisted while the feature is dark; a query naming them still runs either way. | | `QUEUE_METRICS_CLICKHOUSE_URL` | falls back to the shared wiring | Runs queue metrics on their own ClickHouse service: the consumer's inserts and every queue-metrics read go through it, so a metrics-heavy chart refresh never competes with runs-list or trace reads. Unset reproduces the previous split exactly (inserts on `CLICKHOUSE_URL`, reads on the query pool). | | `QUEUE_METRICS_CLICKHOUSE_READER_URL` | the write URL | Reader split, so the consumer's inserts can never land on a read endpoint. | | `QUEUE_METRICS_CLICKHOUSE_{KEEP_ALIVE_ENABLED,KEEP_ALIVE_IDLE_SOCKET_TTL_MS,MAX_OPEN_CONNECTIONS,LOG_LEVEL,COMPRESSION_REQUEST}` | `1`, unset, `10`, `info`, `1` | Pool tuning, matching the other per-workload ClickHouse clients. | Migrations to apply: ClickHouse `036_create_queue_metrics_v1.sql`, and a Postgres migration adding the nullable `TaskQueue.concurrencyLimitOverridePercent`. Both are additive. ## How collection works Queue operations produce two kinds of signal, and they have opposite failure modes, so they are handled differently. **Gauges** (queued, running, queue limit, env queued, env running, env limit, throttled, plus keys-with-backlog and worst-key wait on keyed queues) are read *inside* the same Redis script that performs the enqueue or dequeue, so the reading is atomic with the operation it describes rather than a racy follow-up read. The script returns them on its reply and the app forwards them to the stream. Gauges are sampled and drop-tolerant: they are aggregated with `max`, so a lost reading costs resolution, never correctness. **Counters** (enqueued, started, completed, plus nack and dead-lettered) are cumulative odometers. Each event increments a per-queue key on the metrics Redis and emits the absolute total, and ClickHouse takes the difference across buckets at read time. This is the important property of the design: a summed-delta counter undercounts permanently on any lost event, while a cumulative one self-heals, because the next surviving reading restates the whole total. Only bucket granularity can be lost, never the total. A queue returning after its odometer TTL expired restarts at 1 and reset detection handles it, which is safe precisely because expiry only spans a window with no activity. Both land on one sharded Redis stream. A consumer reads it with a consumer group, reclaims stale pending entries on a 15s interval rather than on every poll, maps one entry to one or two ClickHouse rows (whole-queue and, for keyed queues, per-key), and acks only after the insert lands. Each batch carries a dedup token derived from its stream-entry ids, and the target tables set `non_replicated_deduplication_window`, so a retried batch cannot double-count either the raw rows or the aggregates that hang off them. Consumer and emitter both emit OTel metrics (`queue_metrics.emitter.emitted`, `queue_metrics.consumer.{entries,rows_inserted,insert_errors,insert_duration,stream_depth,group_lag,pending,lag_unknown}`); stream depth and group lag are the two worth alerting on, and `lag_unknown` exists because Redis can report a null lag after a trim, which must not be read as zero. ## Storage and read path `queue_metrics_raw_v1` is a short landing table with a 6 hour TTL. Four aggregate tiers are materialized straight from raw, never cascaded off each other, each with a 30 day TTL: - `queue_metrics_v1`, 10 second buckets per queue, the default read path - `queue_metrics_5m_v1`, 5 minute buckets per queue, for wide ranges and cross-queue ranking - `env_metrics_v1`, 10 second buckets per environment, queue-independent so it stays cheap at any range - `queue_metrics_ck_v1`, 10 second buckets per concurrency key Every tier is an MV from raw because the counter states do not survive a cascade: their merge is order sensitive, so a `-MergeState` chain off the 10s table inflates the result, and the same property means an aggregate state may only be merged inside one queue. That constraint is now enforced by the query engine rather than by reviewer discipline: a column can declare a `mergeGroupKey`, and any query that references it without grouping by, or pinning to a single value of, every named key fails to compile with an actionable message. On the read side, TRQL gains three tables (`queue_metrics`, `env_metrics`, and a `queue_metrics_by_key` that is hidden from the editor, schema docs and schema API but still queryable, so per-key rows can never silently merge into a plain per-queue query), plus `deltaSumTimestampMerge` and `quantilesTDigestMerge`. Two schema-level optimizations ride along: a table can declare coarser rollups, so a query whose bucket interval is 5 minutes or wider is routed to the 5m table with no change to the query itself, and it can opt into the ClickHouse query cache with time bounds floored to a fixed grid, so the auto-refreshing dashboards actually share cache entries instead of missing on every tick. Both are caller-side substitutions, so the printer stays unaware of physical layout. All of this can also live on its own ClickHouse service. A table declares the pool its reads run on, the three queue-metrics tables name the dedicated one, and the ingestion consumer writes through the same client, so both directions move together with one env var and nothing else routes differently. The other engine change is opt-in gap filling: charts can request rows for empty buckets, where counters zero-fill and gauges carry forward. Grouped gauge series are densified per group and carried inside a partition, so a quiet queue's line holds its last value without bleeding another queue's value into it. ## Queue concurrency limits `concurrencyLimitOverridePercent` on `TaskQueue` is the source of truth when an override is set as a percentage; the absolute `concurrencyLimit` is materialized from it (floored, clamped to at least 1 so a percentage can never act as a pause, and never above the environment limit). Every path that changes an environment limit now recalculates the environment's percent-based overrides afterwards, outside the transaction, and pushes changed limits to the engine. The push is attempted even when the stored value did not change, so a previously failed sync self-heals rather than leaving the database and the engine diverged; paused queues are skipped so a recalculation cannot effectively unpause one. The API accepts exactly one of `concurrencyLimit` or `percent`, and the reject-instead-of-clamp change above means a request asking for more than the environment allows now fails loudly. The percent bound (greater than 0, at most 100) is defined once and shared by the zod schema, the dashboard mutation handler and the service, so the three cannot drift. The concurrency-keys table on a queue is now paginated against the ClickHouse per-key tier, ranked by peak backlog with the total on every row from a single scan, and only the keys on the current page are enriched with live counts from Redis. That replaces a hard top-50 cap with something whose cost is a function of page size rather than key cardinality. ## The health report `GET /api/v1/reports/:key?period=&format=markdown|ansi|json`. The verdict is computed on the server and is deterministic, not model-generated. Three independent analyzers run over one input snapshot: flow (is work moving, and if not, is the cause a limit, throttling, one bad queue, or dead-lettering), execution (are the runs that start succeeding, and at what latency), and liveness (how fresh is the telemetry). When telemetry is genuinely stale, the first two are forced to unknown and every actionable field is stripped, so no surface ever advises action off stale data. Authorization is per query table rather than a blanket query grant: a JWT must be scoped to every table the report reads (`runs`, `env_metrics`, `queue_metrics`), so a narrowly scoped token cannot pull a report that reads more than it was granted. `period` is validated as a shorthand with a 90 day ceiling at the edge. The report catalog is a registry of `{ load, interpret }` entries, so the next report is a new entry and no change to the route, the view model, the renderers, the CLI or the MCP tool. `trigger mcp` no longer launches the install wizard when stdout is a TTY, which fixed a real failure: hosts spawn the server over a PTY, so the wizard would open and the client would time out waiting for a server that never started. The wizard now needs `trigger mcp --install`. ## The part that is live regardless of every flag The enqueue and dequeue scripts now return a 2-tuple so a gauge reading can ride back on the reply. Every return site in the eight affected scripts is wrapped, and a `nil` original is converted to `false` on the way out, because a raw `nil` in the first slot would make Lua truncate the multi-bulk reply and silently drop the gauge on the throttled and empty-queue paths. The reply shape and the destructuring on the app side are exercised on every queue operation whether or not metrics are enabled, so that is the part of `run-engine` worth the closest review. One behavior fix in the same area: the scheduling-delay anchor is set only on a run's first entry into the queue. Anchoring it to trigger time on re-enqueues made waitpoint and checkpoint resumes report the entire wait as scheduling delay. Queue ordering is untouched, so a re-enqueued run keeps its position, and nacks deliberately keep the original anchor because a rolled-back dequeue is the same continuous wait. A pending-version promotion still anchors to trigger time, on purpose: that promotion is the run's first real entry into the queue, since the trigger deliberately held it back waiting for a worker version, and the TTL is armed at the same point for the same reason. The consequence is worth naming, because it is a judgement call: a run that waits on a deployment reports that wait as scheduling delay on its queue, which is time unrelated to queue capacity. ## Verification Unit and integration suites across the new package, the run queue, the mapping layer, the query engine and ClickHouse (including a test that applies migration 036 through the same splitter CI uses, and a regression test that inserts the same batch three times to prove the aggregates do not inflate). Beyond that, the whole path was driven end to end against a live stack with real runs: emitter to Redis stream to consumer to ClickHouse to the dashboards, for both the local dev path and the deployed path where a supervisor drives the dequeue, with assertions on exact counter reconstruction per queue and per concurrency key, throttling, environment saturation, scheduling delay, and a deliberate mid-stream reading drop to confirm the cumulative counters still reconstruct the correct total. The gated-off state was checked on every touched surface. The dedicated ClickHouse service was verified against a second, separately-schema'd instance: with it configured, the driven counters reconstruct exactly on the dedicated instance, the shared instance gains no rows for that window, a read through the query API returns the value that exists only on the dedicated instance, and a `runs` query still succeeds (it would fail outright if it were mis-routed to a service without that table). With the variable unset, the full suite passes unchanged. --------- Co-authored-by: Katia Bulatova <katia@trigger.dev> Co-authored-by: Katia Bulatova <katherine.bulatova@gmail.com> Co-authored-by: James Ritchie <james@trigger.dev>	2026-07-29 16:45:24 +01:00
Chris Arderne	38bf82aebe	feat(cli,webapp): target notifications by minimum CLI version (#4407)	2026-07-28 14:23:41 +01:00
github-actions[bot]	d189ce17d3	chore: release v4.5.8 (#4364) ... ## Summary 2 new features, 9 improvements, 3 bug fixes. ## Highlights - Allow additional environment API keys to create scoped public access tokens through the Trigger.dev API. Use server-issued public access tokens for batch operations so environment-scoped API keys can read batch results. ([#4387](https://github.com/triggerdotdev/trigger.dev/pull/4387)) ## Improvements - Preserve the partial assistant message when a chat turn's model stream fails mid-response. `chat.agent` now passes the recovered partial to `onTurnComplete`, and `chat.createSession`'s `turn.complete()` keeps it before rethrowing, instead of dropping the streamed-so-far output. ([#4348](https://github.com/triggerdotdev/trigger.dev/pull/4348)) ## Server changes These changes affect the self-hosted Docker image and Trigger.dev Cloud: - Favorite any dashboard page to a new Favorites section in the side menu, and customize the sidebar by renaming favorites, hiding items, and reordering items and sections. ([#4375](https://github.com/triggerdotdev/trigger.dev/pull/4375)) - List API endpoints now clamp the page size to a maximum of 100. Requests asking for a larger page size return up to 100 items and keep paginating, rather than pulling an unbounded page. ([#4360](https://github.com/triggerdotdev/trigger.dev/pull/4360)) - Organizations without billing alerts now get default spend alert thresholds, so you're notified before usage grows unexpectedly. The billing limit page no longer pre-selects an option before you've set a limit and prompts you to configure one. Alert previews now update immediately after you change your billing limit. ([#4328](https://github.com/triggerdotdev/trigger.dev/pull/4328)) - When you create a Personal Access Token, the generated token now shows its first and last few characters instead of being fully hidden, so you can confirm you copied the right value. ([#4363](https://github.com/triggerdotdev/trigger.dev/pull/4363)) - Add metrics to the realtime backend that measure how often a single changed run is served to multiple subscriptions in one batch. ([#4341](https://github.com/triggerdotdev/trigger.dev/pull/4341)) - Realtime run subscriptions can now be configured to read run data straight from the primary database, so a run's latest state is never served from a lagging replica. Off by default; replica reads are unchanged unless you turn it on. ([#4378](https://github.com/triggerdotdev/trigger.dev/pull/4378)) - SSO and Directory Sync are no longer restricted to Enterprise plans — get in touch and we can turn them on for your organization whatever plan you're on. ([#4393](https://github.com/triggerdotdev/trigger.dev/pull/4393)) - Improved supervisor observability: it now reports metrics for its outbound requests, making failed calls to upstream services easier to monitor. ([#4350](https://github.com/triggerdotdev/trigger.dev/pull/4350)) - The runs list on a task's page now updates live — run statuses change and newly triggered runs appear without a manual refresh, matching the main Runs page. ([#4377](https://github.com/triggerdotdev/trigger.dev/pull/4377)) - Speed up the Batches list page for environments with a large number of batches, which could previously time out while loading. ([#4361](https://github.com/triggerdotdev/trigger.dev/pull/4361)) - Container startup no longer prints database and ClickHouse connection strings (with credentials) to the logs. ([#4346](https://github.com/triggerdotdev/trigger.dev/pull/4346)) - The tasks page no longer runs two queries whose results were never displayed, cutting wasted work on every page load and removing a source of hidden server errors ([#4380](https://github.com/triggerdotdev/trigger.dev/pull/4380)) <details> <summary>Raw changeset output</summary> # Releases ## @trigger.dev/build@4.5.8 ### Patch Changes - Updated dependencies: - `@trigger.dev/core@4.5.8` ## trigger.dev@4.5.8 ### Patch Changes - Updated dependencies: - `@trigger.dev/core@4.5.8` - `@trigger.dev/build@4.5.8` - `@trigger.dev/schema-to-json@4.5.8` ## @trigger.dev/core@4.5.8 ### Patch Changes - Allow additional environment API keys to create scoped public access tokens through the Trigger.dev API. Use server-issued public access tokens for batch operations so environment-scoped API keys can read batch results. ([#4387](https://github.com/triggerdotdev/trigger.dev/pull/4387)) ## @trigger.dev/python@4.5.8 ### Patch Changes - Updated dependencies: - `@trigger.dev/sdk@4.5.8` - `@trigger.dev/core@4.5.8` - `@trigger.dev/build@4.5.8` ## @trigger.dev/react-hooks@4.5.8 ### Patch Changes - Updated dependencies: - `@trigger.dev/core@4.5.8` ## @trigger.dev/redis-worker@4.5.8 ### Patch Changes - Updated dependencies: - `@trigger.dev/core@4.5.8` ## @trigger.dev/rsc@4.5.8 ### Patch Changes - Updated dependencies: - `@trigger.dev/core@4.5.8` ## @trigger.dev/schema-to-json@4.5.8 ### Patch Changes - Updated dependencies: - `@trigger.dev/core@4.5.8` ## @trigger.dev/sdk@4.5.8 ### Patch Changes - Preserve the partial assistant message when a chat turn's model stream fails mid-response. `chat.agent` now passes the recovered partial to `onTurnComplete`, and `chat.createSession`'s `turn.complete()` keeps it before rethrowing, instead of dropping the streamed-so-far output. ([#4348](https://github.com/triggerdotdev/trigger.dev/pull/4348)) - Allow additional environment API keys to create scoped public access tokens through the Trigger.dev API. Use server-issued public access tokens for batch operations so environment-scoped API keys can read batch results. ([#4387](https://github.com/triggerdotdev/trigger.dev/pull/4387)) - Updated dependencies: - `@trigger.dev/core@4.5.8` </details> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>	2026-07-27 16:40:31 +01:00
Chris Arderne	efd0ee8d74	feat(core,sdk): support additional environment API keys (#4387) ... ## Summary Additional environment API keys can use SDK APIs that require public access tokens. The SDK detects the additional-key format and asks the Trigger.dev server to mint scoped tokens instead of attempting to sign them locally. Root environment keys retain their existing local-signing behavior. Trigger and batch clients also prefer server-issued tokens returned in response headers while preserving compatibility with older servers. ## Deployment notes This package update is safe to publish before servers expose additional key creation. Existing root keys continue to use the current path, while an additional key used with an older server fails with an actionable upgrade error.	2026-07-27 15:19:27 +00:00
Matt Aitken	be45cf9e61	fix(sdk): preserve partial assistant message on chat stream failure (#4348) ... ## Summary When a `chat.agent` (or `chat.createSession`) turn's model stream fails mid-response (e.g. a transport timeout like `UND_ERR_BODY_TIMEOUT`), the assistant output that already streamed was dropped: `onTurnComplete` fired with `responseMessage: undefined`, and the manual loop's `turn.complete()` rethrew without keeping the partial. Apps that register `hydrateMessages` are hit hardest, since boot-time tail-replay recovery is off by design. This preserves the streamed-so-far assistant output while still reporting the turn as errored, so persistence and recovery keep it. ## Scope of behavior change Only the **error path** changes. Successful turns are unaffected: the same chunks stream to the client in the same order, and backpressure/cancel behave as before. Everything here is a correctness improvement on a turn that hit a source-stream failure. ## What it does Follow-up to #4304 (`chat.pipeAndCapture`), extending the same partial-recovery to the two loops that lacked it: - **`chat.agent`**: taps the response stream (via a `TransformStream`, so pass-through backpressure and cancel are preserved) to buffer chunks, and on a source-stream failure reconstructs the partial (preferring the `onFinish` message). It's surfaced on the error-path `onTurnComplete` (`responseMessage`, `rawResponseMessage`, `uiMessages`, `newUIMessages`, `newMessages`) and committed to the accumulator so the next turn and the reboot snapshot keep it. - **`chat.createSession` / `turn.complete()`**: the reconstructed partial is accumulated (so `turn.uiMessages` reflects it and the caller can persist after catching) before `turn.complete()` rethrows. `onBeforeTurnComplete` stays skipped on the error path (it hands out a writer for a stream that has already broken). ## Correctness properties (each covered by a regression test) Each test below was confirmed to fail without its fix: - The recovered partial reaches `onTurnComplete` and the next turn's accumulated messages. - An already-committed (possibly enriched) response is not overwritten if a post-response hook then throws. - Incomplete tool parts are cleaned from the recovered partial (text kept), so the UI and model views agree and the next turn isn't poisoned. - A prior turn's model-only compaction survives an errored turn (append only the new tail, don't reconvert the full history). - A reconstructed fragment that reuses an existing message id does not clobber the complete message. - Queued `chat.response` data parts are folded into the recovered partial, matching the success path. - `newMessages` (model delta) stays symmetric with `newUIMessages`. ## Tests New `chat-agent-source-stream-error.test.ts` covers the cases above. The full `@trigger.dev/sdk` unit suite passes and the package build is green across all supported runtimes (Node 20 to 26, Bun, Deno, Cloudflare Workers).	2026-07-24 14:20:20 +01:00
nicktrn	722e240e4d	feat(supervisor): add prometheus metric for outbound http requests (#4350) ... Adds Prometheus metrics so the supervisor's outbound HTTP calls are observable - including client-side failures that previously only surfaced as a log line. - `supervisor_outbound_request_total{name, method, status, outcome}` - counts every outbound request. `outcome` separates a transport failure (`network_error`), an HTTP error response (`http_error`), a response that failed schema validation (`invalid_response`), and success (`ok`). - `supervisor_outbound_request_duration_seconds{name, outcome}` - latency histogram. Leaner labels than the counter (no `status`) to avoid bucket×label cardinality; buckets match the existing dequeue-latency histogram since these calls share the same retrying HTTP client and long-poll envelope. Coverage: - The warm-start request (a one-off `fetch`) - instrumented inline; the response status code is now also included in the failure log (it was previously dropped). - All worker API client calls (`SupervisorHttpClient`: dequeue, run attempt start/complete, heartbeats, snapshots, continue, suspend, debug-log, connect) - routed through a single instrumented `request()` helper that reports via an optional `onHttpRequestComplete` callback on the client, which the supervisor wires into the counter + histogram. Low cardinality by design: `name` is a **static per-endpoint label** (e.g. `dequeue`, `start_run_attempt`), never the interpolated URL - so no run/snapshot IDs land in labels, mirroring the templated `route` labels on the inbound HTTP server. Registered on the existing metrics registry, exposed on `/metrics` with no new wiring. Internal-only change (no package release needed), so the changelog note is a single `.server-changes` entry.	2026-07-23 18:18:58 +01:00
github-actions[bot]	aafc333523	chore: release v4.5.7 (#4319) ... ## Summary 5 improvements, 5 bug fixes. ## Improvements - Add `node-24` and `node-26` as supported `runtime` options in `trigger.config.ts`. The `experimental-node-24` and `experimental-node-26` names are now deprecated aliases and emit a deprecation warning; switch to `node-24` / `node-26` instead. ([#4337](https://github.com/triggerdotdev/trigger.dev/pull/4337)) ```ts import { defineConfig } from "@trigger.dev/sdk"; export default defineConfig({ runtime: "node-24", project: "<your-project-ref>", }); ``` - Avoid logging task run environment variable values at debug level ([#4336](https://github.com/triggerdotdev/trigger.dev/pull/4336)) - Custom chat agent loops get two ergonomic wins for owning the turn loop. ([#4304](https://github.com/triggerdotdev/trigger.dev/pull/4304)) `chat.writeTurnComplete()` now returns the turn boundary's resume cursors (`lastEventId` for the output stream and `sessionInEventId` for the input stream), so you can persist them straight from the task instead of round-tripping them back from the client. ```ts const { lastEventId, sessionInEventId } = await chat.writeTurnComplete(); await db.chats.update(chatId, { lastEventId, sessionInEventId }); ``` `chat.pipeAndCapture()` no longer throws when a stream is stopped or fails. It now returns a `PipeAndCaptureResult` whose `message` holds any partial output captured before the stop or failure, alongside a typed `status` (`"complete" | "aborted" | "error"`) and, on failure, the `error`. Read the message off the result: ```ts const { message, status, error } = await chat.pipeAndCapture(result, { signal, }); if (message) conversation.addResponse(message); if (status === "error") logger.error("turn failed", { error }); ``` Note: `pipeAndCapture` previously resolved to `UIMessage | undefined`. Update call sites to read `.message` from the returned result. - Suppress a build-time warning that could appear in Vite-based projects when the optional `@ai-sdk/otel` package is not installed. ([#4188](https://github.com/triggerdotdev/trigger.dev/pull/4188)) ## Bug fixes - Fixes intermittent `trigger dev` run crashes where a run could fail at boot with a cryptic `Cannot find module .../dev-run-worker.mjs` after a rebuild had cleaned up the build directory the run was launched against. Dev runs now retry cleanly instead of hard-crashing when their build directory is missing, the dev watchdog no longer removes the build tree of a still-running session, and a run assigned to a worker version that was superseded by a rebuild now fails fast with a clear message instead of silently hanging until it times out. ([#4276](https://github.com/triggerdotdev/trigger.dev/pull/4276)) ## Server changes These changes affect the self-hosted Docker image and Trigger.dev Cloud: - Refreshed the side menu: separate organization and account menus, a new project switcher, and the menu is now resizable by dragging its edge. The account Profile page has also been redesigned. ([#4066](https://github.com/triggerdotdev/trigger.dev/pull/4066)) - Allow different organization members to use the same development branch name without sharing or colliding with each other's branch environments. ([#4323](https://github.com/triggerdotdev/trigger.dev/pull/4323)) - Limit account settings email input to 254 characters. ([#4330](https://github.com/triggerdotdev/trigger.dev/pull/4330)) - Prevent duplicate Staging and Preview environments when account setup requests overlap ([#4261](https://github.com/triggerdotdev/trigger.dev/pull/4261)) - Fix the docs link on the empty Prompts page, which pointed to a page that no longer exists. ([#4247](https://github.com/triggerdotdev/trigger.dev/pull/4247)) <details> <summary>Raw changeset output</summary> # Releases ## @trigger.dev/build@4.5.7 ### Patch Changes - Updated dependencies: - `@trigger.dev/core@4.5.7` ## trigger.dev@4.5.7 ### Patch Changes - Fixes intermittent `trigger dev` run crashes where a run could fail at boot with a cryptic `Cannot find module .../dev-run-worker.mjs` after a rebuild had cleaned up the build directory the run was launched against. Dev runs now retry cleanly instead of hard-crashing when their build directory is missing, the dev watchdog no longer removes the build tree of a still-running session, and a run assigned to a worker version that was superseded by a rebuild now fails fast with a clear message instead of silently hanging until it times out. ([#4276](https://github.com/triggerdotdev/trigger.dev/pull/4276)) - Add `node-24` and `node-26` as supported `runtime` options in `trigger.config.ts`. The `experimental-node-24` and `experimental-node-26` names are now deprecated aliases and emit a deprecation warning; switch to `node-24` / `node-26` instead. ([#4337](https://github.com/triggerdotdev/trigger.dev/pull/4337)) ```ts import { defineConfig } from "@trigger.dev/sdk"; export default defineConfig({ runtime: "node-24", project: "<your-project-ref>", }); ``` - Avoid logging task run environment variable values at debug level ([#4336](https://github.com/triggerdotdev/trigger.dev/pull/4336)) - Updated dependencies: - `@trigger.dev/core@4.5.7` - `@trigger.dev/build@4.5.7` - `@trigger.dev/schema-to-json@4.5.7` ## @trigger.dev/core@4.5.7 ### Patch Changes - Add `node-24` and `node-26` as supported `runtime` options in `trigger.config.ts`. The `experimental-node-24` and `experimental-node-26` names are now deprecated aliases and emit a deprecation warning; switch to `node-24` / `node-26` instead. ([#4337](https://github.com/triggerdotdev/trigger.dev/pull/4337)) ```ts import { defineConfig } from "@trigger.dev/sdk"; export default defineConfig({ runtime: "node-24", project: "<your-project-ref>", }); ``` ## @trigger.dev/python@4.5.7 ### Patch Changes - Updated dependencies: - `@trigger.dev/sdk@4.5.7` - `@trigger.dev/core@4.5.7` - `@trigger.dev/build@4.5.7` ## @trigger.dev/react-hooks@4.5.7 ### Patch Changes - Updated dependencies: - `@trigger.dev/core@4.5.7` ## @trigger.dev/redis-worker@4.5.7 ### Patch Changes - Updated dependencies: - `@trigger.dev/core@4.5.7` ## @trigger.dev/rsc@4.5.7 ### Patch Changes - Updated dependencies: - `@trigger.dev/core@4.5.7` ## @trigger.dev/schema-to-json@4.5.7 ### Patch Changes - Updated dependencies: - `@trigger.dev/core@4.5.7` ## @trigger.dev/sdk@4.5.7 ### Patch Changes - Custom chat agent loops get two ergonomic wins for owning the turn loop. ([#4304](https://github.com/triggerdotdev/trigger.dev/pull/4304)) `chat.writeTurnComplete()` now returns the turn boundary's resume cursors (`lastEventId` for the output stream and `sessionInEventId` for the input stream), so you can persist them straight from the task instead of round-tripping them back from the client. ```ts const { lastEventId, sessionInEventId } = await chat.writeTurnComplete(); await db.chats.update(chatId, { lastEventId, sessionInEventId }); ``` `chat.pipeAndCapture()` no longer throws when a stream is stopped or fails. It now returns a `PipeAndCaptureResult` whose `message` holds any partial output captured before the stop or failure, alongside a typed `status` (`"complete" | "aborted" | "error"`) and, on failure, the `error`. Read the message off the result: ```ts const { message, status, error } = await chat.pipeAndCapture(result, { signal, }); if (message) conversation.addResponse(message); if (status === "error") logger.error("turn failed", { error }); ``` Note: `pipeAndCapture` previously resolved to `UIMessage | undefined`. Update call sites to read `.message` from the returned result. - Suppress a build-time warning that could appear in Vite-based projects when the optional `@ai-sdk/otel` package is not installed. ([#4188](https://github.com/triggerdotdev/trigger.dev/pull/4188)) - Updated dependencies: - `@trigger.dev/core@4.5.7` </details> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>	2026-07-22 15:51:29 +01:00
Chris Arderne	55a3bf2858	feat(core,cli): add node-24 and node-26 runtimes, deprecate experimental aliases (#4337) ... ## Summary Adds `node-24` and `node-26` as first-class `runtime` options in `trigger.config.ts`. Previously these Node versions were only reachable via the `experimental-node-24` / `experimental-node-26` names. Those experimental names are now **deprecated aliases**: they still resolve to `node-24` / `node-26` for backwards compatibility, but loading a config that uses them prints a deprecation warning pointing at the new name. ```ts export default defineConfig({ runtime: "node-24", project: "<your-project-ref>", }); ``` ## Details - `ConfigRuntime` (the public config schema) now accepts `node-24` and `node-26` directly; the internal `BuildRuntime` already supported them, so base images and the deploy path are unchanged. - `resolveBuildRuntime` passes the new names straight through and keeps mapping the experimental aliases to their replacements. - Renamed the runtime helper from `isExperimentalConfigRuntime` to `isDeprecatedConfigRuntime` and added `deprecatedRuntimeReplacement` so the CLI can name the replacement in its warning. - Docs snippet updated to list the new versions and flag the deprecated names.	2026-07-22 14:10:27 +01:00
Chris Arderne	509a4597bd	fix(cli): redact task run env values from debug log (#4336)	2026-07-22 12:32:37 +00:00
Eric Allam	a9815f745c	fix(cli): stop dev runs crashing when a rebuild removes an in-use build dir (#4276)	2026-07-22 08:47:58 +01:00
Matt Aitken	e2d3b8388c	feat(sdk): return lastEventId from writeTurnComplete and typed capture result (#4304) ... ## Summary Two ergonomic additions for custom chat-agent loops that own the turn loop (`chat.customAgent`, `chat.createSession`, and the hand-rolled primitives). `chat.writeTurnComplete()` now resolves to `{ lastEventId }`, the resume cursor for the start of the next turn. A custom loop can persist it straight from the task instead of round-tripping it back from the client after the turn ends. The value was already produced internally by the turn-complete write; the public wrapper simply discarded it. `chat.pipeAndCapture()` no longer throws when a stream is stopped or fails. It now resolves to a `PipeAndCaptureResult` carrying any partial `message` captured before the stop or failure, a typed `status` (`"complete" | "aborted" | "error"`), and the `error` on failure. Previously a failed stream threw and the partial was lost, and an abort was captured only when the AI SDK happened to fire `onFinish` in time. ```ts const { message, status, error } = await chat.pipeAndCapture(result, { signal }); if (message) conversation.addResponse(message); if (status === "error") logger.error("turn failed", { error }); const { lastEventId } = await chat.writeTurnComplete(); await db.chats.update(chatId, { lastEventId }); ``` ## Design `pipeAndCapture` wraps the pipe in a `try/catch` and classifies the outcome from the abort signal (a stop drains the source stream cleanly rather than throwing) versus a thrown error. It also races the `onFinish` capture against a timeout so a hard stop that prevents `onFinish` from firing can't hang the caller. This mirrors the capture path `chat.agent` already uses internally. The `finishReason` from `onFinish` is surfaced too, since it was already captured on the built-in path. The internal `turn.complete()` helper keeps its existing contract: it still returns `UIMessage | undefined`, still throws on a genuine stream failure, and still discards output on a full run cancel. ## Breaking change `chat.pipeAndCapture` previously resolved to `UIMessage | undefined`. Call sites now read `.message` off the result. This is a young, low-level API; the docs examples are updated in this PR.	2026-07-21 17:08:30 +02:00
Katia Bulatova	d05f1a7398	chore(webapp): migrate from Remix compiler to Vite (#4188) ... Replaces Remix compiler with the Vite plugin. The Express server (cluster, socket.io, ws) and the Docker image contract are unchanged.	2026-07-21 15:57:13 +02:00
Chris Arderne	dc87b884e7	chore: upgrade to typescript 6 (#4310) ... ## Summary Upgrades the workspace to TypeScript 6.0.3 and applies the compiler, type, and build configuration changes required to preserve package layouts and existing runtime behavior, apart from correcting the HTTP status field used for deployment connection errors. ## Compatibility - Centralizes TypeScript 6.0.3 through the pnpm workspace catalog. - Replaces compiler options and module resolution modes that TypeScript 6 no longer accepts. - Restores explicit Node types where TypeScript 6 no longer includes them transitively. - Adds explicit declaration build roots that preserve each package's existing output layout. - Patches tsup to stop injecting the removed `baseUrl` option during declaration builds. - Uses type-only assertions for stricter typed-array and stream definitions without changing runtime behavior. - Reads the EventSource v3 HTTP status from `code`, so deployment connection errors include it correctly. - Keeps standalone CLI compatibility fixtures pinned to their existing TypeScript version and lockfiles. `turbo run typecheck` and the complete PR test suite are green.	2026-07-21 13:57:52 +01:00
github-actions[bot]	325b906319	chore: release v4.5.6 (#4317) ... ## Summary 5 improvements, 9 bug fixes. ## Breaking changes - Self-hosted deployments no longer ship shared default credentials; fresh installs generate their own. If yours still uses a previously published default, set a unique value before upgrading, or set `ALLOW_INSECURE_DEFAULT_SECRETS=true` to keep booting while you migrate. ([#4316](https://github.com/triggerdotdev/trigger.dev/pull/4316)) ## Improvements - Require explicit browser approval for CLI and MCP login, with resilient polling while approval is pending. ([#4316](https://github.com/triggerdotdev/trigger.dev/pull/4316)) - Deployed task telemetry now reports the deployment identifier (e.g. `deployment_abc123`) in the `worker.id` attribute, instead of an opaque internal value. Upgrade to get the readable identifier in your own OpenTelemetry exporters. ([#4316](https://github.com/triggerdotdev/trigger.dev/pull/4316)) - Prevent prototype pollution when applying run metadata operations or reconstructing nested telemetry attributes, while preserving legitimate `constructor` and `prototype` fields. ([#4316](https://github.com/triggerdotdev/trigger.dev/pull/4316)) - Add helpers to mint and verify the deployment-scoped token used to authenticate run controllers to the platform. ([#4316](https://github.com/triggerdotdev/trigger.dev/pull/4316)) ## Server changes These changes affect the self-hosted Docker image and Trigger.dev Cloud: - Added optional request rate limiting for telemetry ingestion endpoints. ([#4316](https://github.com/triggerdotdev/trigger.dev/pull/4316)) - Background-worker deployment lookups are now scoped to the authenticated environment. ([#4316](https://github.com/triggerdotdev/trigger.dev/pull/4316)) - Updating a GitHub App installation from the callback flow is now scoped to your own organization, so an installation ID belonging to another organization can no longer be used to refresh that organization's installation record. The GitHub App installation session is also now single-use, so completing an installation callback invalidates its state and it can no longer be replayed. ([#4316](https://github.com/triggerdotdev/trigger.dev/pull/4316)) - Scope schedule and environment-variable writes to the caller's project and environment ([#4316](https://github.com/triggerdotdev/trigger.dev/pull/4316)) - Reject compute snapshot callbacks that do not match the snapshot request that created them. ([#4316](https://github.com/triggerdotdev/trigger.dev/pull/4316)) - Require secret-key authentication to initialize the session out (agent→client) stream, matching the append route. ([#4316](https://github.com/triggerdotdev/trigger.dev/pull/4316)) - Live run and trace subscriptions now validate their identifiers more strictly and only return data from your own organization. ([#4316](https://github.com/triggerdotdev/trigger.dev/pull/4316)) - Window-function names in the query compiler are now validated against the allowlist, matching how other function calls are handled. ([#4316](https://github.com/triggerdotdev/trigger.dev/pull/4316)) - Authenticate run controllers to the platform with a signed, deployment-scoped token. ([#4316](https://github.com/triggerdotdev/trigger.dev/pull/4316)) - Verify that worker actions (starting, completing, and continuing a run, and reading its snapshots) target a run belonging to the caller's environment. ([#4316](https://github.com/triggerdotdev/trigger.dev/pull/4316)) <details> <summary>Raw changeset output</summary> # Releases ## @trigger.dev/build@4.5.6 ### Patch Changes - Updated dependencies: - `@trigger.dev/core@4.5.6` ## trigger.dev@4.5.6 ### Patch Changes - Require explicit browser approval for CLI and MCP login, with resilient polling while approval is pending. ([#4316](https://github.com/triggerdotdev/trigger.dev/pull/4316)) - Deployed task telemetry now reports the deployment identifier (e.g. `deployment_abc123`) in the `worker.id` attribute, instead of an opaque internal value. Upgrade to get the readable identifier in your own OpenTelemetry exporters. ([#4316](https://github.com/triggerdotdev/trigger.dev/pull/4316)) - Updated dependencies: - `@trigger.dev/core@4.5.6` - `@trigger.dev/build@4.5.6` - `@trigger.dev/schema-to-json@4.5.6` ## @trigger.dev/core@4.5.6 ### Patch Changes - Prevent prototype pollution when applying run metadata operations or reconstructing nested telemetry attributes, while preserving legitimate `constructor` and `prototype` fields. ([#4316](https://github.com/triggerdotdev/trigger.dev/pull/4316)) - Require explicit browser approval for CLI and MCP login, with resilient polling while approval is pending. ([#4316](https://github.com/triggerdotdev/trigger.dev/pull/4316)) - Add helpers to mint and verify the deployment-scoped token used to authenticate run controllers to the platform. ([#4316](https://github.com/triggerdotdev/trigger.dev/pull/4316)) ## @trigger.dev/python@4.5.6 ### Patch Changes - Updated dependencies: - `@trigger.dev/core@4.5.6` - `@trigger.dev/build@4.5.6` - `@trigger.dev/sdk@4.5.6` ## @trigger.dev/react-hooks@4.5.6 ### Patch Changes - Updated dependencies: - `@trigger.dev/core@4.5.6` ## @trigger.dev/redis-worker@4.5.6 ### Patch Changes - Updated dependencies: - `@trigger.dev/core@4.5.6` ## @trigger.dev/rsc@4.5.6 ### Patch Changes - Updated dependencies: - `@trigger.dev/core@4.5.6` ## @trigger.dev/schema-to-json@4.5.6 ### Patch Changes - Updated dependencies: - `@trigger.dev/core@4.5.6` ## @trigger.dev/sdk@4.5.6 ### Patch Changes - Updated dependencies: - `@trigger.dev/core@4.5.6` </details> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>	2026-07-21 12:07:52 +01:00
Chris Arderne	6997aeb05e	fix: security release 2026-07-08 (#4316)	2026-07-21 12:00:58 +01:00
github-actions[bot]	1cbe25bd1d	chore: release v4.5.5 (#4267) ... ## Summary 5 improvements, 5 bug fixes. ## Improvements - Add experimental Node.js 24 and 26 task runtimes. Set `runtime` to `experimental-node-24` or `experimental-node-26` in `trigger.config.ts`. ([#4085](https://github.com/triggerdotdev/trigger.dev/pull/4085)) - Add `defaultRegion` to the project GET and list API responses; null when unset. ([#4146](https://github.com/triggerdotdev/trigger.dev/pull/4146)) ## Server changes These changes affect the self-hosted Docker image and Trigger.dev Cloud: - Transient internal sync failures are now retried quietly instead of surfacing as errors. ([#4270](https://github.com/triggerdotdev/trigger.dev/pull/4270)) - Optionally route ClickHouse read traffic to a read replica while writes stay on the primary. Set `CLICKHOUSE_READER_URL` to move all reads, or target the busiest paths with `RUNS_LIST_CLICKHOUSE_URL` (runs list) and `EVENTS_READER_CLICKHOUSE_URL` (traces, spans, logs). All optional; unset keeps current behavior. ([#4081](https://github.com/triggerdotdev/trigger.dev/pull/4081)) - Remove the deprecated realtime stream write endpoint used by retired v3 task clients. ([#4250](https://github.com/triggerdotdev/trigger.dev/pull/4250)) - Fix batchTrigger requests that set a per-item idempotency key failing with an error instead of creating and deduplicating the runs ([#4271](https://github.com/triggerdotdev/trigger.dev/pull/4271)) - Speed up idempotency checks on `batchTrigger` calls that use idempotency keys. Large batches against a task with a big run history no longer degrade to multi-second lookups. ([#4255](https://github.com/triggerdotdev/trigger.dev/pull/4255)) - The "Preview branches" usage on the Limits page now counts only preview branches. ([#4283](https://github.com/triggerdotdev/trigger.dev/pull/4283)) - Avoid opening a redundant database connection pool when the legacy and primary databases are the same server, preventing connection usage from doubling. ([#4253](https://github.com/triggerdotdev/trigger.dev/pull/4253)) - Fix pages occasionally loading unstyled or failing to load during a deploy. The dashboard now reloads automatically to recover. ([#4282](https://github.com/triggerdotdev/trigger.dev/pull/4282)) <details> <summary>Raw changeset output</summary> # Releases ## @trigger.dev/build@4.5.5 ### Patch Changes - Updated dependencies: - `@trigger.dev/core@4.5.5` ## trigger.dev@4.5.5 ### Patch Changes - Add experimental Node.js 24 and 26 task runtimes. Set `runtime` to `experimental-node-24` or `experimental-node-26` in `trigger.config.ts`. ([#4085](https://github.com/triggerdotdev/trigger.dev/pull/4085)) - Updated dependencies: - `@trigger.dev/core@4.5.5` - `@trigger.dev/build@4.5.5` - `@trigger.dev/schema-to-json@4.5.5` ## @trigger.dev/core@4.5.5 ### Patch Changes - Add experimental Node.js 24 and 26 task runtimes. Set `runtime` to `experimental-node-24` or `experimental-node-26` in `trigger.config.ts`. ([#4085](https://github.com/triggerdotdev/trigger.dev/pull/4085)) - Add `defaultRegion` to the project GET and list API responses; null when unset. ([#4146](https://github.com/triggerdotdev/trigger.dev/pull/4146)) ## @trigger.dev/python@4.5.5 ### Patch Changes - Updated dependencies: - `@trigger.dev/core@4.5.5` - `@trigger.dev/build@4.5.5` - `@trigger.dev/sdk@4.5.5` ## @trigger.dev/react-hooks@4.5.5 ### Patch Changes - Updated dependencies: - `@trigger.dev/core@4.5.5` ## @trigger.dev/redis-worker@4.5.5 ### Patch Changes - Updated dependencies: - `@trigger.dev/core@4.5.5` ## @trigger.dev/rsc@4.5.5 ### Patch Changes - Updated dependencies: - `@trigger.dev/core@4.5.5` ## @trigger.dev/schema-to-json@4.5.5 ### Patch Changes - Updated dependencies: - `@trigger.dev/core@4.5.5` ## @trigger.dev/sdk@4.5.5 ### Patch Changes - Updated dependencies: - `@trigger.dev/core@4.5.5` </details> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>	2026-07-20 14:01:37 +01:00
Daniel Sutton	ae96b6c175	fix: read-your-writes + global-scope idempotency correctness under the run-ops split (#4284) ... ## What & why Two related correctness fixes for the run-ops DB split. Under the split, run-store reads can route to a **lagging read replica**; a just-written run/waitpoint/batch can then be missed, causing a wrong decision. **1. Read-your-writes → owning primary.** Surfaced first as an intermittent `wait.until({ idempotencyKey })` re-wait on retry. Auditing the run-store read surface found the same class at sibling sites (some gating mutations or returning spurious 404s, others tolerable/self-healing). Reads that must observe their own writes now route to the owning **primary** (`findRun`/`findWaitpoint`/`findBatchTaskRunByFriendlyId` → `*OnPrimary`, a primary re-read on a miss, or a retryable 404 where the SDK polls). Read-view reads stay on the replica. All additive — the happy path is unchanged. **2. Global-scope idempotency across the split.** A `global`-scope key carries no per-run salt, so the same `(env, task, key)` triggered concurrently from parents resident on **different** run-ops DBs could dedup-miss on each DB and create a duplicate (the per-DB unique index can't enforce cross-DB uniqueness). Such triggers (global scope, or scope-absent, while split is active) are serialized through the existing Redis idempotency claim, the loser resolves the winner by id across both DBs, and the claim is reacquired on the expired/failed clear-and-recreate path. `run`/`attempt` scope embed the run id and never contend. ## Stacked for review This is the **base** of a 2-PR stack, split so review is easier: - **This PR** — production code only (34 files). - **Stacked tests PR → https://github.com/triggerdotdev/trigger.dev/pull/4285** — the caller-driven guards (55 test files) on top of this branch. ## Validation Local run-ops split, **both 2-DB and 3-DB**, fresh boot on this branch: SDK canary 64/71 (only the known concurrency/input-streams/s3 failures), quarantine sweep **0 unexpected** (340 pass / 16 known / 4 local) in each topology, dashboard e2e 0 failed. No product regressions.	2026-07-19 17:57:41 +01:00
Chris Arderne	d7ec75d5ad	feat(runtime): add experimental Node.js 24 and 26 task runtimes (#4085) ... ## Summary Adds experimental Node.js 24 and 26 task runtimes through the `experimental-node-24` and `experimental-node-26` config values. Existing runtime defaults and the `node`, `node-22`, and `bun` behavior remain unchanged. The unprefixed `node-24` and `node-26` config values remain unavailable until the runtimes are ready for general use. ## Design Experimental config values normalize to canonical runtime identifiers before build manifests are created, keeping deployment metadata and execution behavior consistent. Kubernetes task pods also use the runtime-default seccomp profile so modern Node.js versions fall back from io_uring to checkpoint-compatible system calls.	2026-07-16 12:19:03 +01:00
Chris Arderne	b902e65dfb	chore: standardise internal node on 24.18.0 (#4254) ... ## Summary Updates the internal development, CI, and runtime-image Node version to 24.18.0. SDK compatibility coverage continues to include Node 20, 22, 24, and 26. The Node type definitions and the package-manager lockfiles now resolve against Node 24 types.	2026-07-15 12:49:12 +01:00
nicktrn	976171ea16	feat(webapp): management API for orgs, projects, members, and settings (#4146) ... ## Summary Adds a set of PAT-authenticated management API endpoints so orgs, projects, members/invites, environment variables, and a few project/environment settings can be managed programmatically (scripting, automation) rather than only through the dashboard. Each route is a thin wrapper over the **existing** service the dashboard already uses, with the same authorization applied at the route layer - no new business logic. ## Endpoints **Organizations** - `POST /api/v1/orgs` - create an org (`createOrganization`) - `PATCH /api/v1/orgs/:orgParam` - rename (title) - `DELETE /api/v1/orgs/:orgParam` - soft-delete (`DeleteOrganizationService`; keeps the active-subscription guard) **Members & invites** - `GET /api/v1/orgs/:orgParam/members` - list members + pending invites - `DELETE /api/v1/orgs/:orgParam/members/:memberId` - remove a member (last-member guarded) - `POST /api/v1/orgs/:orgParam/invites` - invite by email (`inviteMembers`, sends the invite email) - `DELETE /api/v1/orgs/:orgParam/invites/:inviteId` - revoke an invite **Projects** - `PATCH /api/v1/projects/:projectRef` - rename (`ProjectSettingsService`) - `DELETE /api/v1/projects/:projectRef` - soft-delete (`DeleteProjectService`) - `PUT /api/v1/projects/:projectRef/default-region` - set the default region by worker-group name (`SetDefaultRegionService`) - project GET/list now return `defaultRegion` (worker-group name, or null when unset) **Environments** - `POST /api/v1/projects/:projectRef/:env/pause` and `/resume` (`PauseEnvironmentService`) - `POST /api/v1/projects/:projectRef/:env/regenerate-api-key` - rotate the env secret key (`regenerateApiKey`, RBAC `write:apiKeys`) - env var create now accepts an optional `isSecret` flag ## Auth & authorization - All routes authenticate with a **Personal Access Token** (`Authorization: Bearer tr_pat_...`). - Org/project routes are built on the PAT route builders in `apiBuilder.server.ts`: `createLoaderPATApiRoute` (already existed) and **`createActionPATApiRoute`** (added here - the loader builder had no mutation counterpart). The builder runs auth, resolves the org/project role-floor via `context`, and enforces a declarative `authorization` block using the same RBAC actions the dashboard applies (`manage:organization` / `read:members` / `manage:members` / `manage:project`). Handlers keep a membership-scoped query as the floor, so a non-member gets a 404. This also gives these routes `tenantContext` user attribution (Sentry) and `ServiceValidationError`-to-status mapping for free. - **Membership floor (important).** The OSS RBAC fallback grants a permissive ability, so `ability.can(...)` can't reject a non-member on self-hosted. Every handler therefore resolves the target scoped to the caller's membership (`members: { some: { userId } }`) → 404 for non-members. `authorization` is the *role* gate; this is the *tenant* gate. `resolveOrganizationForApiUser` (`organizationApiAccess.server.ts`) is the org-tier version of the existing `findProjectByRef` - org-addressed PAT routes are new, so no such helper existed before. - Env-tier routes reuse the existing `authorizePatEnvironmentAccess` (`write:apiKeys`). ### What `createActionPATApiRoute` gives you A route is pure declaration - the builder handles auth, RBAC, validation, tracing, and error mapping: ```ts export const action = createActionPATApiRoute( { method: "PUT", // one verb, or ["PATCH", "DELETE"] for multi-verb routes params: ParamsSchema, body: SetDefaultRegionRequestBody, // zod-validated context: async ({ projectRef }) => { // resolve the org for the RBAC role-floor const project = await prisma.project.findFirst({ where: { externalRef: projectRef, deletedAt: null }, select: { organizationId: true }, }); return project ? { organizationId: project.organizationId } : {}; }, authorization: { action: "manage", resource: () => ({ type: "project" }) }, }, async ({ params, body, authentication, ability }) => { // auth + authz already enforced. Just do the work. // `throw new ServiceValidationError("Region not found", 400)` → mapped to that status. return json({ ok: true }); } ); ``` Handled for you, so handlers stay thin: - **Method allowlist** - `method` accepts a verb or an array; any other verb → `405` with an `Allow` header, *before* auth runs: ```ts const allowedMethods = method ? (Array.isArray(method) ? method : [method]) : undefined; if (allowedMethods && !(allowedMethods as string[]).includes(request.method.toUpperCase())) { return json({ error: "Method not allowed" }, { status: 405, headers: { Allow: allowedMethods.join(", ") } }); } ``` - **PAT / user-actor auth** in a single roundtrip → `401` on missing/invalid/revoked token. - **RBAC** - `context` computes the caller's role-floor for the target org/project; `authorization` gates it → `403` with a structured error body. - **Sentry attribution** - `tenantContext.enrich({ userId })` so events from the handler carry the acting user. - **Typed errors** - a thrown `ServiceValidationError` is mapped to its `.status` (default 400); anything else → `500`, and expected boundary errors are logged as `warn` (kept out of Sentry). - **Validation** - params / query / headers / body are all zod-checked → `400` with details. ## Notes for reviewers - Everything wraps an existing service; the intent is API parity for things that are currently dashboard-only, not new behaviour. - `createActionPATApiRoute` is new shared infra (the PAT + RBAC mutation builder that didn't exist). It's self-contained - the loader builder and existing routes are untouched. - `@trigger.dev/core` gets one additive field (`defaultRegion` on the project response, optional/nullable for client-server version skew) - changeset included, patch. - `removeTeamMember`'s last-member guard is now atomic (Serializable transaction via the `$transaction` helper, with retry), so the dashboard and API both get it server-side. Added a `## Transactions` rule to `apps/webapp/CLAUDE.md` (always use the `$transaction` helper); migrating the remaining direct usages is tracked in TRI-11698. ## Open questions - ~~Is PAT the right auth (vs OAT for automation)?~~ **Resolved: PAT.** Organization Access Tokens are currently internal-only (used by the image builder) and not user-accessible, so they can't back this yet. - Should any of these be gated behind a flag or scope? - Naming/shape of the routes.	2026-07-15 10:20:08 +01:00
Chris Arderne	313fe03481	test(core): fix flakey run-stream test depending on ordering (#4256)	2026-07-14 12:29:57 +01:00
github-actions[bot]	165955781d	chore: release v4.5.4 (#4228) ... ## Summary 2 new features, 11 improvements, 5 bug fixes. ## Breaking changes - Trigger.dev v3 is no longer supported. For self-hosted deployments, 4.5.0 is the last version we officially support for running v3; stay on 4.5.0 or upgrade to v4. v3 triggers, batch triggers, reschedules, and deploys now return a clear upgrade message instead of running. ([#4236](https://github.com/triggerdotdev/trigger.dev/pull/4236)) ## Improvements - You can now mark environment variables synced via the `syncEnvVars` build extension as secrets. Return `{ name, value, isSecret: true }` from your callback and those variables are stored redacted in the dashboard, just like manually created secret env vars. ([#4203](https://github.com/triggerdotdev/trigger.dev/pull/4203)) - Remove the legacy `--mcp` and `--mcp-port` options from the `dev` command. Run the dedicated `trigger mcp` command to start the Trigger.dev MCP server. ([#4246](https://github.com/triggerdotdev/trigger.dev/pull/4246)) - Removed the unused `ResourceMonitor` export from `@trigger.dev/core/v3/serverOnly`. It was a server-side logging helper with no remaining consumers. ([#4244](https://github.com/triggerdotdev/trigger.dev/pull/4244)) - Removed the unused `@trigger.dev/core/v3/zodNamespace` export and the legacy v3 socket message schemas. These were only used by the now-retired v3 engine and have no v4 consumers. ([#4236](https://github.com/triggerdotdev/trigger.dev/pull/4236)) ## Bug fixes - Fix a `chat.agent` message-loss race where sending a message right after an action (such as an undo) could drop the follow-up's response from the UI until a refresh. ([#4234](https://github.com/triggerdotdev/trigger.dev/pull/4234)) ## Server changes These changes affect the self-hosted Docker image and Trigger.dev Cloud: - Added `EVENT_REPOSITORY_POSTGRES_WRITES_DISABLED` to skip all PostgreSQL task-event writes for deployments that store task events in ClickHouse. Leave it off unless `EVENT_REPOSITORY_DEFAULT_STORE` is `clickhouse_v2`, otherwise task events are lost. ([#4242](https://github.com/triggerdotdev/trigger.dev/pull/4242)) - Promo credits: a /promo signup landing page, redeeming a promo code when a new org selects a plan, and showing remaining credits on the usage page. ([#4138](https://github.com/triggerdotdev/trigger.dev/pull/4138)) - Speed up retrieving a background worker by version. The endpoint no longer runs a slow lookup that scanned the full task table for large deployments; it now reuses data it already loads, so the response is the same but returns much faster. ([#4245](https://github.com/triggerdotdev/trigger.dev/pull/4245)) - Clearer login error when an email address is blocked by the WHITELISTED_EMAILS setting: the message now explains the address isn't allowed on this instance instead of the ambiguous "This email is unauthorized". ([#4220](https://github.com/triggerdotdev/trigger.dev/pull/4220)) - Make the native build server the default in project build settings. It's now opt-out, stored as a new `disableNativeBuildServer` key. Also clarifies in the UI that build settings apply to GitHub-triggered and native build server deployments. ([#3980](https://github.com/triggerdotdev/trigger.dev/pull/3980)) - Optionally process high-volume telemetry ingestion in parallel for higher throughput under heavy load by setting `OTEL_TRANSFORM_WORKER_POOL_ENABLED=1`. Off by default. ([#4232](https://github.com/triggerdotdev/trigger.dev/pull/4232)) - Add a `REALTIME_BACKEND_DEFAULT` env var to choose the default realtime backend (`electric`, `native`, or `shadow`) for environments whose org has no per-org override. Defaults to `electric`, so existing behavior is unchanged. ([#4231](https://github.com/triggerdotdev/trigger.dev/pull/4231)) - Clarified on the Regions page that a region only affects where your runs execute, not where your data is stored. This shows as a tooltip on the Location column and in the confirmation dialog when you change your default region. ([#4226](https://github.com/triggerdotdev/trigger.dev/pull/4226)) - Improved the reliability of how run data is read and written. ([#4237](https://github.com/triggerdotdev/trigger.dev/pull/4237)) - Fixed stale login errors: an error from a previous login attempt (for example a rejected email address) no longer keeps reappearing on the login page and no longer makes later, successful attempts look like they failed. ([#4220](https://github.com/triggerdotdev/trigger.dev/pull/4220)) - The Errors page now shows better details for each error. Errors that don't carry a message — such as errors thrown without a message, or values thrown that aren't `Error` objects — get a meaningful title instead of all reading "Unknown error", and are grouped by their name (or value) rather than collapsed into a single group. The error type now shows the actual error name, and stack traces now appear where previously they were missing. ([#4225](https://github.com/triggerdotdev/trigger.dev/pull/4225)) - Return a clear client error when SSO form submissions use an unsupported content type ([#4238](https://github.com/triggerdotdev/trigger.dev/pull/4238)) - Query page: extracting fields from a run's output with JSON functions (such as JSONExtractString or JSONExtractInt) no longer fails with an "illegal type: JSON" error. ([#4221](https://github.com/triggerdotdev/trigger.dev/pull/4221)) <details> <summary>Raw changeset output</summary> # Releases ## @trigger.dev/build@4.5.4 ### Patch Changes - You can now mark environment variables synced via the `syncEnvVars` build extension as secrets. Return `{ name, value, isSecret: true }` from your callback and those variables are stored redacted in the dashboard, just like manually created secret env vars. ([#4203](https://github.com/triggerdotdev/trigger.dev/pull/4203)) - Updated dependencies: - `@trigger.dev/core@4.5.4` ## trigger.dev@4.5.4 ### Patch Changes - Remove the legacy `--mcp` and `--mcp-port` options from the `dev` command. Run the dedicated `trigger mcp` command to start the Trigger.dev MCP server. ([#4246](https://github.com/triggerdotdev/trigger.dev/pull/4246)) - Updated dependencies: - `@trigger.dev/core@4.5.4` - `@trigger.dev/build@4.5.4` - `@trigger.dev/schema-to-json@4.5.4` ## @trigger.dev/core@4.5.4 ### Patch Changes - Removed the unused `ResourceMonitor` export from `@trigger.dev/core/v3/serverOnly`. It was a server-side logging helper with no remaining consumers. ([#4244](https://github.com/triggerdotdev/trigger.dev/pull/4244)) - Removed the unused `@trigger.dev/core/v3/zodNamespace` export and the legacy v3 socket message schemas. These were only used by the now-retired v3 engine and have no v4 consumers. ([#4236](https://github.com/triggerdotdev/trigger.dev/pull/4236)) ## @trigger.dev/python@4.5.4 ### Patch Changes - Updated dependencies: - `@trigger.dev/sdk@4.5.4` - `@trigger.dev/core@4.5.4` - `@trigger.dev/build@4.5.4` ## @trigger.dev/react-hooks@4.5.4 ### Patch Changes - Updated dependencies: - `@trigger.dev/core@4.5.4` ## @trigger.dev/redis-worker@4.5.4 ### Patch Changes - Updated dependencies: - `@trigger.dev/core@4.5.4` ## @trigger.dev/rsc@4.5.4 ### Patch Changes - Updated dependencies: - `@trigger.dev/core@4.5.4` ## @trigger.dev/schema-to-json@4.5.4 ### Patch Changes - Updated dependencies: - `@trigger.dev/core@4.5.4` ## @trigger.dev/sdk@4.5.4 ### Patch Changes - Fix a `chat.agent` message-loss race where sending a message right after an action (such as an undo) could drop the follow-up's response from the UI until a refresh. ([#4234](https://github.com/triggerdotdev/trigger.dev/pull/4234)) - Updated dependencies: - `@trigger.dev/core@4.5.4` </details> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>	2026-07-14 10:07:58 +01:00
Chris Arderne	64e5d732ad	chore(webapp,core): remove the unused ResourceMonitor server logging helper (#4244) ... The `ResourceMonitor` server-side logging helper is no longer used. It periodically logged the webapp process own memory, disk, and CPU usage behind the `RESOURCE_MONITOR_ENABLED` flag (off by default), and was also exported from `@trigger.dev/core/v3/serverOnly` with no other consumers. This removes the helper, its `@trigger.dev/core` export, the webapp wiring, and the `RESOURCE_MONITOR_ENABLED` env var. The supervisor has its own unrelated `ResourceMonitor` class, which is left untouched.	2026-07-13 20:32:29 +01:00
Chris Arderne	6e943f2421	chore(cli): remove --mcp option from trigger dev (#4246)	2026-07-13 16:23:05 +01:00
Eric Allam	5ba8557a51	chore(webapp,core): remove the end-of-life v3 (engine V1) execution stack (#4236) ... ## Summary v3 (the engine that ran the SDK v3 era, internally `RunEngineVersion.V1`) is end-of-life. Following the removal of the v3 execution apps ([#4194](https://github.com/triggerdotdev/trigger.dev/pull/4194)) and the legacy dev websocket ([#4198](https://github.com/triggerdotdev/trigger.dev/pull/4198)), this removes the remaining v3 execution stack from the server. Clients still on v3 (an old SDK or CLI that has not upgraded) keep getting a clear "upgrade to v4" response. Triggers, batch triggers, reschedules, and deploys that resolve to v3 are rejected with a graceful 4xx pointing at the migration guide, never a 5xx, so a stale client cannot affect server health. Self-hosted instances still running v3 should stay on the 4.5.x release line until they migrate. ## What is removed - The MarQS queue and its shared/dev queue consumers. - The v3 socket.io namespaces (coordinator, provider, shared-queue) and the v3 run lifecycle services (attempt, checkpoint, and batch-resume). - The graphile-worker background job system; all live jobs already run on `@trigger.dev/redis-worker`. - The `DEPRECATE_V3_ENABLED` flag: v3 is now rejected unconditionally, so the flag is gone. - Unused v3 exports from `@trigger.dev/core` (the `v3/zodNamespace` subpath and the legacy socket message catalogs) and the now-dead MarQS environment variables. ## What stays The v4 engine is untouched. The graceful v3 rejection boundary stays, `determineEngineVersion` still detects a v3 project so it can reject it, and the batch service plus batch-completion worker stay for current clients. Live queue concurrency limits and metrics now read from the v4 run engine instead of MarQS, and a brand-new dev environment now defaults to v4. ## Dependency cleanup Removes webapp dependencies left unused by this change: `seedrandom` and `semver` (only the removed v3 code used them) plus a set that was already dead, their orphaned `@types` packages, and two dead files. Adds a `knip:deps` script and a `knip.json` config so unused dependencies can be found the same way going forward.	2026-07-13 11:32:06 +01:00