Commit Graph

1703 Commits

Author SHA1 Message Date
Eric Allam e7fec4097f fix(dev): CLI now properly cleans up the store dir on dev CLI exit (#2744) 2025-12-04 13:13:01 +01:00
Eric Allam d279988e38 fix(workers): prevent ERR_IPC_CHANNEL_CLOSED errors from causing an unhandled exception on TaskRunProcess (#2743) 2025-12-04 12:02:30 +00:00
Eric Allam 9821d02af7 fix(sdk): Re-export schemaTask types to prevent the TypeScript error TS2742 (#2735)
Fixes this type of error when exporting a `schemaTask` in a monorepo:

```
error TS2742: The inferred type of 'helloWorldSchema' cannot be named without a reference to '@trigger.dev/core/v3'. This is likely not portable.
```
2025-12-03 16:12:23 +00:00
Saadi Myftija 255a73a2fe feat(deployments): --native-build-server support for the deploy command (#2702)
This PR adds support for CLI deployments using the native build server.

**Background**

The deployment command currently does the following:
- bundles the code
- submits the build context to our external build provider and waits for
the build
- triggers deployment state transitions using the platform API

Upstream build provider outages cause issue with deployments,
potentially blocking deployments entirely. We recently introduced the
`--force-local-build` flag as a fallback to enable deployment without a
dependency on the upstream build provider, though it requires users to
have docker in their systems. This PR continues that work by providing a
remote build path which uses our own build server and does not rely on
the external provider.

**Changes in this PR**

Introduced the new `--native-build-server` flag, which does the
following:
- scans all files relevant for the Trigger deployment and evaluates
ignore rules
- packages it up in an archive and uploads it as a deployment artifact
- queues the deployment and triggers the build
- streams logs from the build server

This no longer relies on external build services. Also deployment state
transitions happen on the server-side, giving us more flexibility to
evolve the flow and schemas of related deployment API endpoints. In
general it gives us better control of the whole build and deployment
process. This path will eventually become the default.

The `--detach` flag is also new, allowing to trigger deployments without
waiting for the result.

The deployment artifacts are uploaded via pre-signed URLs to avoid
unnecessary load on the platform. The new `/artifacts` endpoint
generates the pre-signed URLs; size limits are enforced on s3. This
endpoint is deliberately generic, we could extend it in the future to
upload other artifacts client-side in a similar way, e.g., large payload
packets.
2025-12-03 16:40:21 +01:00
Eric Allam 5b7dfe23b5 feat(cli): implements content-addressable store for the dev CLI build outputs, reducing disk usage (#2725)
* feat(cli): implements content-addressable store for the dev CLI build outputs, reducing disk usage

* fix a few things
2025-12-03 10:17:35 +00:00
nicktrn af9b3e1c99 fix(cli): header will always print the correct profile (#2728) 2025-12-02 17:45:35 +00:00
Eric Allam 9f27422472 fix(otel): exported logs and spans will now have matching trace IDs (#2724)
When using custom OTLP exporters via `telemetry.exporters` and 

This occurred when tasks were triggered **without** a parent trace 
context (e.g., via API or dashboard). In this scenario: - Spans were 
correctly rewritten to use the generated `externalTraceId` - Logs kept 
their original internal trace ID due to a bug in the early return logic

### Root Cause

In `ExternalLogRecordExporterWrapper.transformLogRecord()`, the early 
return condition incorrectly included `!this.externalTraceContext`:

```typescript
if (!logRecord.spanContext || !this.externalTraceId || 
!this.externalTraceContext) {   return logRecord;  // Bug: Returns early
when externalTraceContext is undefined }

// This fallback logic was never reached:
const externalTraceId = this.externalTraceContext
  ? this.externalTraceContext.traceId
  : this.externalTraceId;
```

### Fix

1. **Reordered logic in `transformLogRecord()`**: Move the 
1. `externalTraceId` calculation before the early return, and check the 
1. culated value instead of `this.externalTraceContext`:

```typescript
const externalTraceId = this.externalTraceContext
  ? this.externalTraceContext.traceId
  : this.externalTraceId;

if (!logRecord.spanContext || !externalTraceId) {
  return logRecord;
}
```

2. **Clarified `_isExternallySampled` logic**: Updated both 
2. `ExternalSpanExporterWrapper` and `ExternalLogRecordExporterWrapper` 
2. explicitly handle the case where there's no external trace context 
2. a generated `externalTraceId` exists:

```typescript
this._isExternallySampled = externalTraceContext
  ? isTraceFlagSampled(externalTraceContext.traceFlags)
  : !!externalTraceId;
```

### Impact

Logs and spans from the same task run will now have matching trace IDs 
when exported to external observability tools, enabling proper trace correlation regardless of whether the task was triggered with or without a parent trace context. 
`telemetry.logExporters` in `trigger.config.ts`, logs and spans were 
exported with **different trace IDs**, breaking trace correlation in
external observability tools like Datadog.
2025-12-02 14:16:06 +00:00
Eric Allam 72e286af2f feat(otel): support for custom resource attributes via config#telemetry.resource and OTEL_RESOURCE_ATTRIBUTES env var (#2704) 2025-11-24 14:33:07 +00:00
Eric Allam f7240a99e7 fix(react): prevent infinite useEffect when passing an array of tags to useRealtimeRunsWithTag (#2705) 2025-11-24 14:32:57 +00:00
github-actions[bot] 6e47377766 chore: Update version for release (#2700)
🚀 Publish Trigger.dev Docker / typecheck (push) Failing after 0s
🚀 Publish Trigger.dev Docker / publish-webapp (push) Has been skipped
🚀 Publish Trigger.dev Docker / publish-worker (push) Has been skipped
🚀 Publish Trigger.dev Docker / publish-worker-v4 (push) Has been skipped
🚀 Publish Trigger.dev Docker / units (push) Failing after 21s
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2025-11-24 10:30:01 +00:00
Eric Allam 5e5c97ea4c fix(cli): stop dev runs stuck in dequeued status (fix #2639) (#2699)
* stop deleting the first dev version files on the first change, prevents system failures

* prevent dev runs getting stuck in dequeued status by deleting workers

* add changeset
2025-11-21 15:24:07 +00:00
github-actions[bot] 6231ddc67a Release v4.1.1
* chore: Update version for release

* chore: Update pnpm-lock.yaml (#2694)

* Initial plan

* Update pnpm-lock.yaml

Co-authored-by: ericallam <534+ericallam@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: ericallam <534+ericallam@users.noreply.github.com>

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: ericallam <534+ericallam@users.noreply.github.com>
2025-11-19 11:09:26 +00:00
Eric Allam 15fef916f6 feat(build): update prisma extension to work with generated clients and rust-free clients (#2689)
* prisma extension fixes WIP

* More prisma stuff

* more prisma stuff

* remove changelog

* upgrade github workflows to use node 20.19 because installing prisma@7 breaks with lower versions

* Don't use generate for the prisma reference projects

* make sure it works if no mode is passed in
2025-11-19 10:48:15 +00:00
github-actions[bot] 78fcba518a chore: Update version for release (#2666)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2025-11-14 09:52:22 +00:00
Saadi Myftija 53047ab648 feat(deployments): external cache support for local builds (#2682)
* Add registry cache support for local builds

* Add changeset
2025-11-14 10:46:31 +01:00
Eric Allam a94a11f44d feat(sdk): replace onStart lifecycle hook with onStartAttempt (#2515)
* fix(sdk): prevent uncaught errors thrown onSuccess, onComplete, and onFailure hooks to fail attempts & in some cases runs

* Add onStartAttempt hook and deprecate onSuccess

* Add onStartAttempt hook and deprecate onStart hook

* Fix onStartAttempt overload types

* Update lifecycle functions diagram
2025-11-13 14:51:13 +00:00
Eric Allam 6137338da9 feat(streams): make v2 streams the default when using 4.1.0+ if they are supported (#2677) 2025-11-13 13:53:42 +00:00
Eric Allam f7cb637b32 fix(streams): scope s2 access token to environment and fix streams v1 appends (#2670)
* fix(streams): scope s2 access token to environment and fix streams v1 appends

* Less stale time
2025-11-12 12:34:15 +00:00
Eric Allam 536d9fa217 feat(realtime): Realtime streams v2 (#2632) 2025-11-11 14:54:00 +00:00
github-actions[bot] d75c3aeadd chore: Update version for release (#2665)
* chore: Update version for release

* Release 4.0.7

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Matt Aitken <matt@mattaitken.com>
2025-11-11 14:29:24 +00:00
Matt Aitken a342332146 Fix for the MCP tool that gets logs for debugging runs (#2653)
* Fix for the MCP tool that gets logs for debugging runs

This was broken when we changed the data on the backend that returns 
log/span data from runs. We changed the data structured and the internal
API that the MCP client uses was failing to parse with the Zod schema

* add changeset

* Revert "add changeset"

This reverts commit 86eca836d5907fa0d0f8ac595d4d5ebade140514.

---------

Co-authored-by: nicktrn <55853254+nicktrn@users.noreply.github.com>
2025-11-11 13:26:07 +00:00
github-actions[bot] 9fdf91a1c4 Release v4.0.6 (#2647)
* chore: Update version for release

* Update lock file

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: myftija <saadi.myftija@gmail.com>
2025-11-03 14:42:02 +01:00
nicktrn 9593a46364 fix(extensions): prevent audiowaveform binary removal (#2643)
* fix(extensions): prevent audiowaveform binary removal

* add changeset
2025-11-03 14:17:38 +01:00
Saadi Myftija 27376df903 fix(cli): show a useful error message when config file is missing (#2650)
* fix(cli): show a useful error message when config file is missing

* Add changeset
2025-10-31 16:34:48 +01:00
Saadi Myftija 2f3f82f3a9 feat(deployments): show local build hint during depot outages (#2646)
* Add an API endpoint to query remote build provider status

* Show local build hint for failed deployments when Depot is down

* Show the local build flag in the help output

* Add changeset

* Fix import

* Fix docs link
2025-10-30 18:09:54 +01:00
github-actions[bot] d4fe71df34 Release v4.0.5 (#2531)
* chore: Update version for release

* Release v4.0.5

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: nicktrn <55853254+nicktrn@users.noreply.github.com>
2025-10-28 14:20:20 +00:00
nicktrn ae8e83b2d0 chore(runner): move max duration logic into parent process (#2637)
* chore(runner): move max duration logic into parent process

* chore(rsc): remove type-marker package.json

* add changeset

* chore(core): remove irrelevant test after our changes

* chore(core): clarify we don't care about the timeout promise
2025-10-28 13:49:59 +00:00
Saadi Myftija 7f25e82299 feat(deployments): support local builds in cloud (#2628)
* Enable skipping image push during deployment finalization step

* Add endpoint to generate registry credentials for a deployment

* Add a --force-local-build flag to the deployment command to skip remote build

* Do not show the new flag in the help output

* Add changeset

* Remove registry login logs from onLog, not useful

* Rename var

* Update platform package to the latest version
2025-10-23 10:57:36 +02:00
James Ritchie fe3fe01fe8 feat(queues): Override queue concurrency limits from the dashboard or API (#2609)
* feat(queues): add ability to override concurrency limit via API and dashboard

* Updates the modal layout and tweaks copy

* Improves the dropdown menu item

* Popover supports both Button and LinkButton

* Right align the columns and fix the dropdown menu item styles

* Organize imports,

* Fix spinner icon in dropdown menu

* Remove unused props

* Adds a tooltip to the Concurrency override badge

* Fixes console error with popover menu

* typo

* Fixes incorrect className

* Minimal buttons to view runs

---------

Co-authored-by: Eric Allam <eallam@icloud.com>
2025-10-17 12:58:25 +01:00
nicktrn 8917478d3c fix(runner): SIGTERM handling during warm start long poll (#2593) 2025-10-08 12:51:10 +01:00
Eric Allam 692316e82a fix(realtime): Upgrade to @electric-sql/client@1.0.14 to prevent cached 409 Conflict errors from breaking realtime updates (#2588) 2025-10-07 14:26:03 +01:00
Eric Allam 128bc437f6 feat(otel): Add support for storing run spans and log data in Clickhouse (#2567) 2025-10-01 12:41:18 -07:00
Saadi Myftija 700a6ea598 feat: enable canceling deployments (#2545)
* Add canceledAt to the deployment db schema

* Expose an api endpoint to cancel deployments

* Show the canceled status description in the dashboard

* Enable canceling deployments from the dashboard

* Show cancelation reason in the deployment details

* Make verifyProjectMembership a function for consistency

* Apply some good 🐰 suggestions
2025-09-24 11:14:29 +02:00
Saadi Myftija cc94d121f2 feat: installing status for deployments (#2544)
* Add installing status to the deployment db schema

* Replace the deployments /start endpoint with /progress

* Show the installing status in the dashboard

* Add installing status to the api schema and cli

* Add changeset
2025-09-24 10:27:43 +02:00
Eric Allam 87b3603b23 feat(webapp): completing spans server-side no longer write-after-read, improving efficiency and perf (#2530)
* Cancel run events which then propogate cancellation status to span ancestors

* WIP

* convert closing cached run spans to new system

* converted expired complete span event to new method

* move v3 over to new methods

* Convert getDetailedTraceSummary to use the new ancestor override stuff

* remove debug logs

* Don't return UNSPECIFIED task events in getRunEvents

* fix the call site for cancelling run event in v3

* Add changeset

* remove methods
2025-09-19 13:39:48 +01:00
github-actions[bot] a03783d1a0 Release v4.0.4
🚀 Publish Trigger.dev Docker / typecheck (push) Failing after 19s
🚀 Publish Trigger.dev Docker / publish-webapp (push) Has been skipped
🚀 Publish Trigger.dev Docker / publish-worker (push) Has been skipped
🚀 Publish Trigger.dev Docker / publish-worker-v4 (push) Has been skipped
🚀 Publish Trigger.dev Docker / units (push) Failing after 19s
* chore: Update version for release

* Release v4.0.4

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Eric Allam <eallam@icloud.com>
2025-09-18 23:18:59 +01:00
Eric Allam 0178bdbb00 fix(packages): remove effect from optional peer dependencies (#2527)
Fixes ERROR: Could not resolve "effect"
2025-09-18 23:09:37 +01:00
github-actions[bot] ad51168181 Release v4.0.3 (#2486)
* chore: Update version for release

* Release v4.0.3

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: nicktrn <55853254+nicktrn@users.noreply.github.com>
2025-09-18 21:28:30 +01:00
nicktrn db87295049 fix(runner): reduce restore recovery time and deprecated runner false positives (#2523)
* fix(runner): improve restore detection

* chore(supervisor): skip schema parsing when debug logs disabled

* fix(runner): deprecation race condition

* add changeset
2025-09-18 16:59:44 +01:00
Saadi Myftija a3ef6ea236 feat: separate deployment initialize and start steps (#2522)
* Enable setting the initial status on deployment creation

* Expose endpoint to start deployments

* Extend build timeout on deployment start

* Use separate timeout value for queued deployments

* Add startedAt to the deployment schema

* Show the new startedAt instead of createdAt in the dashboard

* Show github user tag also in the deployment details page

* Show `pending` deployment status as `queued` in the dashboard

* Apply some good 🐰 suggestions

* Add missing return
2025-09-18 16:12:17 +02:00
nicktrn 885ae5e06f fix(runner): immediate poll to decrease restore time (#2516)
* fix(runner): immediate poll to decrease restore time

* chore: bump default prerelease tag

* fix(cli): s is not a function

* add changeset
2025-09-16 16:17:04 +01:00
nicktrn 3c199e6d9c chore(supervisor): remove deprecated route (#2513) 2025-09-16 13:47:07 +01:00
nicktrn a55294b7dd fix(run-engine): retry SIGSEGV errors (#2514) 2025-09-16 13:46:58 +01:00
Eric Allam 08702cd710 chore(mcp): Add our MCP server to the official MCP registry (#2510)
See: 
https://blog.modelcontextprotocol.io/posts/2025-09-08-mcp-registry-preview/
2025-09-16 10:44:41 +01:00
Eric Allam 0f9b83db09 fix(core): prettyPrintingPacket will now do a structuredClone on non-circular references instead of outputting [Circular] (#2508)
* Mo-Stashed changes

* fix(core): prettyPrintingPacket will now do a structuredClone on non-circular references instead of outputting [Circular]

This also fixes an issue with replaying of runs that include 
non-circular references
2025-09-15 17:43:46 +01:00
Saadi Myftija 7d333e5b3c feat(cli): attach to existing deployments in the build server (#2501)
* Add external build data and image platform to the get deployment endpoint

* If provided, attach to an existing deployment in the deploy command

* Check status for existing deployments

* Add changeset
2025-09-15 17:16:26 +02:00
Eric Allam 6483a0f1c6 fix(core): Improves our schema to JSON Schema conversion (fix for zod 4) (#2483) 2025-09-15 14:11:40 +01:00
Eric Allam f077d49291 feat(engine): Improve execution stalls troubleshooting, align dev and prod behavior, adding heartbeats.yield utility (#2489)
* feat(engine): Improve execution stalls troubleshooting, align dev and prod behavior, adding heartbeats.yield utility

* A few improvements via the 🐇 review

* Allow treating EXECUTION stalls as OOM errors, improve the error message, add more information to the docs, improve resource monitor and add it to the docs

* Add changeset
2025-09-12 14:54:39 +01:00
Saadi Myftija 12bef0a938 feat: env command in the cli (#2485)
* Add a CLI command to list and view env vars

* Add changeset

* Restrict pemissions on env files created with `env pull`

* Escape env vars when exporting to file

* Switch changeset to patch
2025-09-09 13:13:24 +02:00
nicktrn cf9398b56e fix(run-engine): retry SIGTERM errors (#2468) 2025-09-02 15:18:16 +01:00