Commit Graph

1721 Commits

Author SHA1 Message Date
github-actions[bot] caa40ce925 chore: release v4.3.1 (#2788)
🚀 Publish Trigger.dev Docker / typecheck (push) Failing after 0s
🚀 Publish Trigger.dev Docker / publish-webapp (push) Has been skipped
🚀 Publish Trigger.dev Docker / publish-worker (push) Has been skipped
🚀 Publish Trigger.dev Docker / publish-worker-v4 (push) Has been skipped
🚀 Publish Trigger.dev Docker / units (push) Failing after 4s
This PR was opened by the [Changesets
release](https://github.com/changesets/action) GitHub action. When
you're ready to do a release, you can merge this and publish to npm
yourself or [setup this action to publish
automatically](https://github.com/changesets/action#with-publishing). If
you're not ready to do a release yet, that's fine, whenever you add more
changesets to main, this PR will be updated.


# Releases
## @trigger.dev/build@4.3.1

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.3.1`

## trigger.dev@4.3.1

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.3.1`
    -   `@trigger.dev/build@4.3.1`
    -   `@trigger.dev/schema-to-json@4.3.1`

## @trigger.dev/core@4.3.1

### Patch Changes

- Added support for idempotency reset
([#2777](https://github.com/triggerdotdev/trigger.dev/pull/2777))

## @trigger.dev/python@4.3.1

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/sdk@4.3.1`
    -   `@trigger.dev/core@4.3.1`
    -   `@trigger.dev/build@4.3.1`

## @trigger.dev/react-hooks@4.3.1

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.3.1`

## @trigger.dev/redis-worker@4.3.1

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.3.1`

## @trigger.dev/rsc@4.3.1

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.3.1`

## @trigger.dev/schema-to-json@4.3.1

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.3.1`

## @trigger.dev/sdk@4.3.1

### Patch Changes

- feat: Support for new batch trigger system
([#2779](https://github.com/triggerdotdev/trigger.dev/pull/2779))
- feat(sdk): Support debouncing runs when triggering with new debounce
options
([#2794](https://github.com/triggerdotdev/trigger.dev/pull/2794))
- Added support for idempotency reset
([#2777](https://github.com/triggerdotdev/trigger.dev/pull/2777))
-   Updated dependencies:
    -   `@trigger.dev/core@4.3.1`

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2025-12-22 15:29:18 +00:00
Mihai Popescu 7574c69c2d feat(webapp): Add support for resetting idempotency keys (#2777)
Add support for resetting idempotency keys both from ui and sdk

##  Checklist

- [x] I have followed every step in the [contributing
guide](https://github.com/triggerdotdev/trigger.dev/blob/main/CONTRIBUTING.md)
- [x] The PR title follows the convention.
- [x] I ran and tested the code works

---

## Testing
- Created a new run with a idempotency idempotencyKey.
- Started a new run with the same task and got redirected to the first
run.
- Deleted the key from the UI on the run details
- Started a new run with the same task  and it created a new one
- Did the above steps using the SDK


---

## Changelog

- Add new action route for resetting idempotency keys via UI
- Add reset button in Idempotency section of run detail view
- Added API and SDK for resetting imdepotency
- Updated docs page for this feature


---

## Screenshots

_[Screenshots]_
<img width="438" height="363" alt="Screenshot 2025-12-11 at 11 56 37"
src="https://github.com/user-attachments/assets/30b8ef5e-8aac-4d04-b57a-9bf30d085dcb"
/>
2025-12-19 11:55:04 +00:00
Eric Allam 3875bb292a feat(engine): run debounce system (#2794)
Adds support for **debounced task runs** - when triggering a task with a
debounce key, subsequent triggers with the same key will reschedule the
existing delayed run instead of creating new runs. This continues until
no new triggers occur within the delay window.

## Usage

```typescript
await myTask.trigger({ userId: "123" }, {
  debounce: {
    key: "user-123-update",
    delay: "5s",
    mode: "leading", // default
  }
});
```

- **key**: Scoped to the task identifier
- **delay**: How long to wait before executing (supports duration
strings like `"5s"`, `"1m"`)
- **mode**: Either `"leading"` or `"trailing"`. Leading debounce will
use the payload and options from the first run created with the debounce
key. Trailing will use payload and options from the last run.

### "trailing" mode overrides

When using `mode: "trailing"` with debounce, the following options are
updated from the **last** trigger:

- **`payload`** - The task input data
- **`metadata`** - Run metadata
- **`tags`** - Run tags (replaces existing tags)
- **`maxAttempts`** - Maximum retry attempts
- **`maxDuration`** - Maximum compute time
- **`machine`**  - Machine preset (cpu/memory)

## Behavior

- **First run wins**: The first trigger creates the run, subsequent
triggers push its execution time later
- **Idempotency keys take precedence**: If both are specified,
idempotency is checked first
- **Max duration**: Configurable via `DEBOUNCE_MAX_DURATION_MS` env var
(default: 10 minutes)

Works with `triggerAndWait` - parent runs correctly block on the
debounced run.
2025-12-18 16:04:43 +00:00
Eric Allam a999d9ea3f feat(engine): Batch trigger reloaded (#2779)
New batch trigger system with larger payloads, streaming ingestion,
larger batch sizes, and a fair processing system.

This PR introduces a new `FairQueue` abstraction inspired by our own
`RunQueue` that enables multi-tenant fair queueing with concurrency
limits. The new `BatchQueue` is built on top of the `FairQueue`, and
handles processing Batch triggers in a fair manner with per-environment
concurrency limits defined per-org. Additionally, there is a global
concurrency limit to prevent the BatchQueue system from creating too
many runs too quickly, which can cause downstream issues.

For this new BatchQueue system we have a completely new batch trigger
creation and ingestion system. Previously this was a single endpoint
with a single JSON body that defined details about the batch as well as
all the items in the batch.

We're introducing a two-phase batch trigger ingestion system. In the
first phase, the BatchTaskRun record is created (and possibly rate
limited). The second phase is another endpoint that accepts an NDJSON
body with each line being a single item/run with payload and options.

At ingestion time all items are added to a queue, in order, and then
processed by the BatchQueue system.

## New batch trigger rate limits

This PR implements a new batch trigger specific rate limit, configured
on the `Organization.batchRateLimitConfig` column, and defaults using
these environment variables:

- `BATCH_RATE_LIMIT_REFILL_RATE` defaults to 10
- `BATCH_RATE_LIMIT_REFILL_INTERVAL` the duration interval, defaults to
`"10s"`
- `BATCH_RATE_LIMIT_MAX` defaults to 1200

This rate limiter is scoped to the environment ID and controls how many
runs can be submitted via batch triggers per interval. The SDK handles
the retrying side.

## Batch queue concurrency limits

The new column `Organization.batchQueueConcurrencyConfig` now defines an
org specific `processingConcurrency` value, with a backup of the env var
`BATCH_CONCURRENCY_LIMIT_DEFAULT` which defaults to 10. This controls
how many batch queue items are processed concurrently per environment.

There is also a global rate limit for the batch queue set via the
`BATCH_QUEUE_GLOBAL_RATE_LIMIT` which defaults to being disabled. If
set, the entire batch queue system won't process more than
`BATCH_QUEUE_GLOBAL_RATE_LIMIT` items per second. This allows
controlling the maximum number of runs created per second via batch
triggers.

## Batch trigger settings

- `STREAMING_BATCH_MAX_ITEMS` controls the maximum number of items in a
single batch
- `STREAMING_BATCH_ITEM_MAXIMUM_SIZE` controls the maximum size of each
item in a batch
- `BATCH_CONCURRENCY_DEFAULT_CONCURRENCY` controls the default
environment concurrency
- `BATCH_QUEUE_DRR_QUANTUM` how many credits each environment gets each
round for the DRR scheduler
- `BATCH_QUEUE_MAX_DEFICIT` the maximum deficit for the DRR scheduler
- `BATCH_QUEUE_CONSUMER_COUNT` how many queue consumers to run
- `BATCH_QUEUE_CONSUMER_INTERVAL_MS` how frequently they poll for items
in the queue

### Configuration Recommendations by Use Case

**High-throughput priority (fairness acceptable at 0.98+):**

```env
BATCH_QUEUE_DRR_QUANTUM=25
BATCH_QUEUE_MAX_DEFICIT=100
BATCH_QUEUE_CONSUMER_COUNT=10
BATCH_QUEUE_CONSUMER_INTERVAL_MS=50
BATCH_CONCURRENCY_DEFAULT_CONCURRENCY=25
```

**Strict fairness priority (throughput can be lower):**

```env
BATCH_QUEUE_DRR_QUANTUM=5
BATCH_QUEUE_MAX_DEFICIT=25
BATCH_QUEUE_CONSUMER_COUNT=3
BATCH_QUEUE_CONSUMER_INTERVAL_MS=100
BATCH_CONCURRENCY_DEFAULT_CONCURRENCY=5
```
2025-12-16 14:32:49 +00:00
github-actions[bot] 7d34817473 chore: release v4.3.0 (#2770)
This PR was opened by the [Changesets
release](https://github.com/changesets/action) GitHub action. When
you're ready to do a release, you can merge this and publish to npm
yourself or [setup this action to publish
automatically](https://github.com/changesets/action#with-publishing). If
you're not ready to do a release yet, that's fine, whenever you add more
changesets to main, this PR will be updated.


# Releases
## trigger.dev@4.3.0

### Minor Changes

- feat(cli): deterministic image builds for deployments
([#2778](https://github.com/triggerdotdev/trigger.dev/pull/2778))
- feat(cli): enable zstd compression for deployment images
([#2773](https://github.com/triggerdotdev/trigger.dev/pull/2773))

### Patch Changes

- The new `triggeredVia` field is now populated in deployments via the
CLI. ([#2767](https://github.com/triggerdotdev/trigger.dev/pull/2767))
- fix(dev): stop max listeners exceeded warning messages when running
more than 10 runs concurrently
([#2771](https://github.com/triggerdotdev/trigger.dev/pull/2771))
- Upgrade @modelcontextprotocol/sdk to 1.24.3
([#2768](https://github.com/triggerdotdev/trigger.dev/pull/2768))
-   Updated dependencies:
    -   `@trigger.dev/core@4.3.0`
    -   `@trigger.dev/build@4.3.0`
    -   `@trigger.dev/schema-to-json@4.3.0`

## @trigger.dev/core@4.3.0

### Minor Changes

- feat(cli): deterministic image builds for deployments
([#2778](https://github.com/triggerdotdev/trigger.dev/pull/2778))

### Patch Changes

- The new `triggeredVia` field is now populated in deployments via the
CLI. ([#2767](https://github.com/triggerdotdev/trigger.dev/pull/2767))

## @trigger.dev/build@4.3.0

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.3.0`

## @trigger.dev/python@4.3.0

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.3.0`
    -   `@trigger.dev/build@4.3.0`
    -   `@trigger.dev/sdk@4.3.0`

## @trigger.dev/react-hooks@4.3.0

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.3.0`

## @trigger.dev/redis-worker@4.3.0

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.3.0`

## @trigger.dev/rsc@4.3.0

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.3.0`

## @trigger.dev/schema-to-json@4.3.0

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.3.0`

## @trigger.dev/sdk@4.3.0

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.3.0`

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2025-12-12 13:11:56 +01:00
Saadi Myftija 6d6ed471d1 feat(cli): deterministic image builds for deployments (#2778)
⚒️ Publish Worker (v4) / build (supervisor) (push) Has been cancelled
This PR makes our image builds deterministic and reproducible by
ensuring that identical source code always produces the same image
layers and image digest. This means that deployments where nothing has
changed will no longer invalidate the image cache in our worker cluster
nodes, thus avoid making the cold starts for runs worse.

**Context**
New deployments currently increase the cold start times for runs, as
they generate a new image which needs to be pulled in the worker cluster
where runs are executed. It happens also when the source code for the
deployment has not changed due to non-deterministic steps in our build
system. This addresses the latter issue by making builds reproducible.

**Main changes**
- Avoided baking `TRIGGER_DEPLOYMENT_ID` and
`TRIGGER_DEPLOYMENT_VERSION` in the image, we now pass these via the
supervisor instead.
- Used `json-stable-stringify` for consistent key ordering in the files
we generate for the build, e.g., `package.json`, `build.json`,
`index.json`.
- Removed `metafile.json` from the image contents as it is not actually
used in the container. This is only relevant for the `analyze` command.
- Added `SOURCE_DATE_EPOCH=0` and `rewrite-timestamp=true` to Docker
builds to normalize file timestamps.
- Removed some `timings` and `outputHashes` from build outputs and
manifests.

The builds are now reproducible for both native build server and Depot
paths. This should also lead to better image layer cache reuse in
general.
2025-12-12 09:48:04 +01:00
nicktrn 74e9246bfa feat(cli): enable zstd compression for deployment images (#2773)
This will speed up ice cold starts (*) for two reasons:
- better compression ratio
- faster decompression

This is a minor release because zstd compression will now be enabled by
default for all deployments.

(*) ice cold starts happen when deploy images are not cached on the
worker node yet. These cold start durations are highly dependent on
image size and as it turns out, also the type of compression used.
2025-12-10 14:49:44 +00:00
Eric Allam c021d1db63 fix(dev): stop max listeners exceeded warning messages when running more than 10 runs concurrently (#2771)
<!-- CURSOR_SUMMARY -->
> [!NOTE]
> Centralizes SIGTERM handling in `DevSupervisor` and removes per-run
SIGTERM listeners in `DevRunController` to avoid
MaxListenersExceededWarning under high concurrency.
> 
> - **Dev runtime**:
> - **SIGTERM handling**: Add centralized handler in
`packages/cli-v3/src/dev/devSupervisor.ts` to gracefully stop all run
controllers; unregisters on `shutdown()`.
> - **Cleanup**: Remove per-controller `SIGTERM` listener and handler
from `packages/cli-v3/src/entryPoints/dev-run-controller.ts` to reduce
event listeners and warnings.
> - **Changeset**: Add patch note in
`.changeset/fuzzy-ghosts-admire.md`.
> 
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
5ad2f5341829cebf6fd37a3c616a2db5e4ad936a. This will update automatically
on new commits. Configure
[here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->
2025-12-09 09:54:28 +00:00
nicktrn 66c6da7114 security: dependabot alert triage (#2768)
- Upgrade @modelcontextprotocol/sdk to 1.24.3
- Override jws to 3.2.3
2025-12-08 21:02:13 +00:00
Saadi Myftija 7fba9e9f6b feat(deployments): add build server meta and trigger source info (#2767)
This PR applies a small change to the deployments table to keep track
of:
- where the deployment was triggered from
- build server metadata, if the build server was involved
2025-12-08 16:29:26 +00:00
github-actions[bot] 185f4ecaf9 chore: release v4.2.0 (#2713)
🚀 Publish Trigger.dev Docker / units (push) Failing after 0s
🚀 Publish Trigger.dev Docker / typecheck (push) Failing after 19s
🚀 Publish Trigger.dev Docker / publish-webapp (push) Has been skipped
🚀 Publish Trigger.dev Docker / publish-worker (push) Has been skipped
🚀 Publish Trigger.dev Docker / publish-worker-v4 (push) Has been skipped
This PR was opened by the [Changesets
release](https://github.com/changesets/action) GitHub action. When
you're ready to do a release, you can merge this and publish to npm
yourself or [setup this action to publish
automatically](https://github.com/changesets/action#with-publishing). If
you're not ready to do a release yet, that's fine, whenever you add more
changesets to main, this PR will be updated.


# Releases
## trigger.dev@4.2.0

### Minor Changes

- feat(cli): upgrade bun deployments to v1.3.3
([#2756](https://github.com/triggerdotdev/trigger.dev/pull/2756))

### Patch Changes

- fix(otel): exported logs and spans will now have matching trace IDs
([#2724](https://github.com/triggerdotdev/trigger.dev/pull/2724))
- The `--force-local-build` flag is now renamed to just `--local-build`
([#2702](https://github.com/triggerdotdev/trigger.dev/pull/2702))
- fix(cli): header will always print the correct profile
([#2728](https://github.com/triggerdotdev/trigger.dev/pull/2728))
- feat: add ability to set custom resource properties through
trigger.config.ts or via the OTEL_RESOURCE_ATTRIBUTES env var
([#2704](https://github.com/triggerdotdev/trigger.dev/pull/2704))
- feat(cli): implements content-addressable store for the dev CLI build
outputs, reducing disk usage
([#2725](https://github.com/triggerdotdev/trigger.dev/pull/2725))
- Added support for native build server builds in the deploy command
(`--native-build-server`)
([#2702](https://github.com/triggerdotdev/trigger.dev/pull/2702))
-   Updated dependencies:
    -   `@trigger.dev/build@4.2.0`
    -   `@trigger.dev/core@4.2.0`
    -   `@trigger.dev/schema-to-json@4.2.0`

## @trigger.dev/build@4.2.0

### Patch Changes

- syncVercelEnvVars to skip API and read env vars directly from
env.process for Vercel build environments. New syncNeonEnvVars build
extension for syncing environment variablesfrom Neon database projects
to Trigger.dev. The extension automatically detects branches and builds
appropriate PostgreSQL connection strings for non-production, non-dev
environments (staging, preview).
([#2729](https://github.com/triggerdotdev/trigger.dev/pull/2729))
-   Updated dependencies:
    -   `@trigger.dev/core@4.2.0`

## @trigger.dev/core@4.2.0

### Patch Changes

- fix: prevent ERR_IPC_CHANNEL_CLOSED errors from causing an unhandled
exception on TaskRunProcess
([#2743](https://github.com/triggerdotdev/trigger.dev/pull/2743))
- Added support for native build server builds in the deploy command
(`--native-build-server`)
([#2702](https://github.com/triggerdotdev/trigger.dev/pull/2702))

## @trigger.dev/python@4.2.0

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/build@4.2.0`
    -   `@trigger.dev/sdk@4.2.0`
    -   `@trigger.dev/core@4.2.0`

## @trigger.dev/react-hooks@4.2.0

### Patch Changes

- fix: prevent infinite useEffect when passing an array of tags to
useRealtimeRunsWithTag
([#2705](https://github.com/triggerdotdev/trigger.dev/pull/2705))
-   Updated dependencies:
    -   `@trigger.dev/core@4.2.0`

## @trigger.dev/redis-worker@4.2.0

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.2.0`

## @trigger.dev/rsc@4.2.0

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.2.0`

## @trigger.dev/schema-to-json@4.2.0

### Patch Changes

-   Updated dependencies:
    -   `@trigger.dev/core@4.2.0`

## @trigger.dev/sdk@4.2.0

### Patch Changes

- fix(sdk): Re-export schemaTask types to prevent the TypeScript error
TS2742: The inferred type of 'task' cannot be named without a reference
to '@trigger.dev/core/v3'. This is likely not portable.
([#2735](https://github.com/triggerdotdev/trigger.dev/pull/2735))
- feat: add ability to set custom resource properties through
trigger.config.ts or via the OTEL_RESOURCE_ATTRIBUTES env var
([#2704](https://github.com/triggerdotdev/trigger.dev/pull/2704))
-   Updated dependencies:
    -   `@trigger.dev/core@4.2.0`

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2025-12-05 16:27:21 +01:00
nicktrn 0d764d4c46 feat(cli): upgrade bun deployments to v1.3.3 (#2756)
New deployments with `runtime: "bun"` will now use Bun v1.3.3

Link to Bun release: https://bun.com/blog/bun-v1.3.3
2025-12-05 13:31:27 +00:00
Saadi Myftija 4dc956470d fix(deployments): misc fixes for the native build server deployment flow (#2748)
Improved a couple of error messages.

Also fixed an issue with the s2 token caching.
2025-12-04 17:01:47 +01:00
Oskar 6e038d4d1e chore: Improve Vercel/Neon syncEnvVars extensions docs 2025-12-04 16:32:15 +01:00
Oskar Otwinowski 05b53967ea feat(build): Add NeonDB branch resolution for Vercel preview environments (#2729)
Vercel's NeonDB integration renders database connection environment
variables at runtime, which means Trigger.dev cannot directly sync these
values during the build process. This change adds support for fetching
branch-specific NeonDB connection strings via the Neon API.

feat(build): Add syncNeonEnvVars extension and improve Vercel env var
syncing

Add a new `syncNeonEnvVars` build extension for syncing environment
variables
from Neon database projects to Trigger.dev. The extension automatically
detects
branches and builds appropriate PostgreSQL connection strings for
non-production
environments (staging, dev, preview).

Features of `syncNeonEnvVars`:
- Fetches branch-specific database credentials from Neon API
- Generates all standard Postgres connection strings (DATABASE_URL,
POSTGRES_URL,
  POSTGRES_PRISMA_URL, etc.) with both pooled and unpooled variants
- Supports custom database name, role name, and env var prefix options
- Skips automatically in Vercel environments (Neon's Vercel integration
handles this)
- Skips for production environments (designed for preview/staging/dev
branches)

Improvements to `syncVercelEnvVars`:
- When running in a Vercel build environment (detected via VERCEL env
var),
values are now read from process.env instead of the Vercel API response
- This ensures the build uses the actual runtime values Vercel provides
- Removed embedded Neon-specific logic (now handled by separate
extension)
- Simplified and cleaned up the extension code

Documentation updates for both extensions with usage examples and
configuration
options.

Closes #2714

##  Checklist

- [x] I have followed every step in the [contributing
guide](https://github.com/triggerdotdev/trigger.dev/blob/main/CONTRIBUTING.md)
- [x] The PR title follows the convention.
- [x] I ran and tested the code works

---

## Testing

Set up Vercel + Trigger.dev envs, used Vercel's chat-bot-ai template.
2025-12-04 16:26:35 +01:00
Oskar 7cbf82a4ae fix: PR feedback, doc improvements for neon/vercel syncEnvVars 2025-12-04 14:07:31 +01:00
Eric Allam e7fec4097f fix(dev): CLI now properly cleans up the store dir on dev CLI exit (#2744) 2025-12-04 13:13:01 +01:00
Eric Allam d279988e38 fix(workers): prevent ERR_IPC_CHANNEL_CLOSED errors from causing an unhandled exception on TaskRunProcess (#2743) 2025-12-04 12:02:30 +00:00
Eric Allam 9821d02af7 fix(sdk): Re-export schemaTask types to prevent the TypeScript error TS2742 (#2735)
Fixes this type of error when exporting a `schemaTask` in a monorepo:

```
error TS2742: The inferred type of 'helloWorldSchema' cannot be named without a reference to '@trigger.dev/core/v3'. This is likely not portable.
```
2025-12-03 16:12:23 +00:00
Saadi Myftija 255a73a2fe feat(deployments): --native-build-server support for the deploy command (#2702)
This PR adds support for CLI deployments using the native build server.

**Background**

The deployment command currently does the following:
- bundles the code
- submits the build context to our external build provider and waits for
the build
- triggers deployment state transitions using the platform API

Upstream build provider outages cause issue with deployments,
potentially blocking deployments entirely. We recently introduced the
`--force-local-build` flag as a fallback to enable deployment without a
dependency on the upstream build provider, though it requires users to
have docker in their systems. This PR continues that work by providing a
remote build path which uses our own build server and does not rely on
the external provider.

**Changes in this PR**

Introduced the new `--native-build-server` flag, which does the
following:
- scans all files relevant for the Trigger deployment and evaluates
ignore rules
- packages it up in an archive and uploads it as a deployment artifact
- queues the deployment and triggers the build
- streams logs from the build server

This no longer relies on external build services. Also deployment state
transitions happen on the server-side, giving us more flexibility to
evolve the flow and schemas of related deployment API endpoints. In
general it gives us better control of the whole build and deployment
process. This path will eventually become the default.

The `--detach` flag is also new, allowing to trigger deployments without
waiting for the result.

The deployment artifacts are uploaded via pre-signed URLs to avoid
unnecessary load on the platform. The new `/artifacts` endpoint
generates the pre-signed URLs; size limits are enforced on s3. This
endpoint is deliberately generic, we could extend it in the future to
upload other artifacts client-side in a similar way, e.g., large payload
packets.
2025-12-03 16:40:21 +01:00
Oskar 53f21e1330 feat(build): Add syncNeonEnvVars extension and improve Vercel env var syncing
Add a new `syncNeonEnvVars` build extension for syncing environment variables
from Neon database projects to Trigger.dev. The extension automatically detects
branches and builds appropriate PostgreSQL connection strings for non-production
environments (staging, dev, preview).

Features of `syncNeonEnvVars`:
- Fetches branch-specific database credentials from Neon API
- Generates all standard Postgres connection strings (DATABASE_URL, POSTGRES_URL,
  POSTGRES_PRISMA_URL, etc.) with both pooled and unpooled variants
- Supports custom database name, role name, and env var prefix options
- Skips automatically in Vercel environments (Neon's Vercel integration handles this)
- Skips for production environments (designed for preview/staging/dev branches)

Improvements to `syncVercelEnvVars`:
- When running in a Vercel build environment (detected via VERCEL env var),
  values are now read from process.env instead of the Vercel API response
- This ensures the build uses the actual runtime values Vercel provides
- Removed embedded Neon-specific logic (now handled by separate extension)
- Simplified and cleaned up the extension code

Documentation updates for both extensions with usage examples and configuration
options.
2025-12-03 16:03:41 +01:00
Eric Allam 5b7dfe23b5 feat(cli): implements content-addressable store for the dev CLI build outputs, reducing disk usage (#2725)
* feat(cli): implements content-addressable store for the dev CLI build outputs, reducing disk usage

* fix a few things
2025-12-03 10:17:35 +00:00
Oskar 49b2f683f4 feat(build): Add NeonDB branch resolution for Vercel preview environments
Vercel's NeonDB integration renders database connection environment
variables at runtime, which means Trigger.dev cannot directly sync
these values during the build process. This change adds support for
fetching branch-specific NeonDB connection strings via the Neon API.

Changes:
- Discover NEON_PROJECT_ID from incoming Vercel environment variables
- Call NeonDB API to search for branches matching the git branch name
- Filter branches to find exact matches with Vercel environment prefix
  (e.g., "preview/branch-name") to avoid false positives from partial
  string matches
- Retrieve branch endpoints and select the write endpoint (or first
  available)
- Build connection strings (DATABASE_URL, POSTGRES_URL, etc.) using
  the branch endpoint host while preserving user/password credentials

Safety measures for non-production environments:
- Filter out all Neon-related env vars (DATABASE_URL, PGHOST, etc.)
  before calling the Neon API to prevent accidental use of production
  database credentials
- Only add branch-specific database env vars if a matching Neon branch
  is found and the API call succeeds
- If neonDbAccessToken is not provided or the API fails, non-production
  environments will not receive any database connection env vars

Usage:
Users must provide a NEON_ACCESS_TOKEN (via options or env var) to
enable automatic branch resolution for preview deployments. Production
environments continue to use Vercel's standard env var sync without
modification.
2025-12-02 19:52:45 +01:00
nicktrn af9b3e1c99 fix(cli): header will always print the correct profile (#2728) 2025-12-02 17:45:35 +00:00
Eric Allam 9f27422472 fix(otel): exported logs and spans will now have matching trace IDs (#2724)
When using custom OTLP exporters via `telemetry.exporters` and 

This occurred when tasks were triggered **without** a parent trace 
context (e.g., via API or dashboard). In this scenario: - Spans were 
correctly rewritten to use the generated `externalTraceId` - Logs kept 
their original internal trace ID due to a bug in the early return logic

### Root Cause

In `ExternalLogRecordExporterWrapper.transformLogRecord()`, the early 
return condition incorrectly included `!this.externalTraceContext`:

```typescript
if (!logRecord.spanContext || !this.externalTraceId || 
!this.externalTraceContext) {   return logRecord;  // Bug: Returns early
when externalTraceContext is undefined }

// This fallback logic was never reached:
const externalTraceId = this.externalTraceContext
  ? this.externalTraceContext.traceId
  : this.externalTraceId;
```

### Fix

1. **Reordered logic in `transformLogRecord()`**: Move the 
1. `externalTraceId` calculation before the early return, and check the 
1. culated value instead of `this.externalTraceContext`:

```typescript
const externalTraceId = this.externalTraceContext
  ? this.externalTraceContext.traceId
  : this.externalTraceId;

if (!logRecord.spanContext || !externalTraceId) {
  return logRecord;
}
```

2. **Clarified `_isExternallySampled` logic**: Updated both 
2. `ExternalSpanExporterWrapper` and `ExternalLogRecordExporterWrapper` 
2. explicitly handle the case where there's no external trace context 
2. a generated `externalTraceId` exists:

```typescript
this._isExternallySampled = externalTraceContext
  ? isTraceFlagSampled(externalTraceContext.traceFlags)
  : !!externalTraceId;
```

### Impact

Logs and spans from the same task run will now have matching trace IDs 
when exported to external observability tools, enabling proper trace correlation regardless of whether the task was triggered with or without a parent trace context. 
`telemetry.logExporters` in `trigger.config.ts`, logs and spans were 
exported with **different trace IDs**, breaking trace correlation in
external observability tools like Datadog.
2025-12-02 14:16:06 +00:00
Eric Allam 72e286af2f feat(otel): support for custom resource attributes via config#telemetry.resource and OTEL_RESOURCE_ATTRIBUTES env var (#2704) 2025-11-24 14:33:07 +00:00
Eric Allam f7240a99e7 fix(react): prevent infinite useEffect when passing an array of tags to useRealtimeRunsWithTag (#2705) 2025-11-24 14:32:57 +00:00
github-actions[bot] 6e47377766 chore: Update version for release (#2700)
🚀 Publish Trigger.dev Docker / typecheck (push) Failing after 0s
🚀 Publish Trigger.dev Docker / publish-webapp (push) Has been skipped
🚀 Publish Trigger.dev Docker / publish-worker (push) Has been skipped
🚀 Publish Trigger.dev Docker / publish-worker-v4 (push) Has been skipped
🚀 Publish Trigger.dev Docker / units (push) Failing after 21s
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2025-11-24 10:30:01 +00:00
Eric Allam 5e5c97ea4c fix(cli): stop dev runs stuck in dequeued status (fix #2639) (#2699)
* stop deleting the first dev version files on the first change, prevents system failures

* prevent dev runs getting stuck in dequeued status by deleting workers

* add changeset
2025-11-21 15:24:07 +00:00
github-actions[bot] 6231ddc67a Release v4.1.1
* chore: Update version for release

* chore: Update pnpm-lock.yaml (#2694)

* Initial plan

* Update pnpm-lock.yaml

Co-authored-by: ericallam <534+ericallam@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: ericallam <534+ericallam@users.noreply.github.com>

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: ericallam <534+ericallam@users.noreply.github.com>
2025-11-19 11:09:26 +00:00
Eric Allam 15fef916f6 feat(build): update prisma extension to work with generated clients and rust-free clients (#2689)
* prisma extension fixes WIP

* More prisma stuff

* more prisma stuff

* remove changelog

* upgrade github workflows to use node 20.19 because installing prisma@7 breaks with lower versions

* Don't use generate for the prisma reference projects

* make sure it works if no mode is passed in
2025-11-19 10:48:15 +00:00
github-actions[bot] 78fcba518a chore: Update version for release (#2666)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2025-11-14 09:52:22 +00:00
Saadi Myftija 53047ab648 feat(deployments): external cache support for local builds (#2682)
* Add registry cache support for local builds

* Add changeset
2025-11-14 10:46:31 +01:00
Eric Allam a94a11f44d feat(sdk): replace onStart lifecycle hook with onStartAttempt (#2515)
* fix(sdk): prevent uncaught errors thrown onSuccess, onComplete, and onFailure hooks to fail attempts & in some cases runs

* Add onStartAttempt hook and deprecate onSuccess

* Add onStartAttempt hook and deprecate onStart hook

* Fix onStartAttempt overload types

* Update lifecycle functions diagram
2025-11-13 14:51:13 +00:00
Eric Allam 6137338da9 feat(streams): make v2 streams the default when using 4.1.0+ if they are supported (#2677) 2025-11-13 13:53:42 +00:00
Eric Allam f7cb637b32 fix(streams): scope s2 access token to environment and fix streams v1 appends (#2670)
* fix(streams): scope s2 access token to environment and fix streams v1 appends

* Less stale time
2025-11-12 12:34:15 +00:00
Eric Allam 536d9fa217 feat(realtime): Realtime streams v2 (#2632) 2025-11-11 14:54:00 +00:00
github-actions[bot] d75c3aeadd chore: Update version for release (#2665)
* chore: Update version for release

* Release 4.0.7

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Matt Aitken <matt@mattaitken.com>
2025-11-11 14:29:24 +00:00
Matt Aitken a342332146 Fix for the MCP tool that gets logs for debugging runs (#2653)
* Fix for the MCP tool that gets logs for debugging runs

This was broken when we changed the data on the backend that returns 
log/span data from runs. We changed the data structured and the internal
API that the MCP client uses was failing to parse with the Zod schema

* add changeset

* Revert "add changeset"

This reverts commit 86eca836d5907fa0d0f8ac595d4d5ebade140514.

---------

Co-authored-by: nicktrn <55853254+nicktrn@users.noreply.github.com>
2025-11-11 13:26:07 +00:00
github-actions[bot] 9fdf91a1c4 Release v4.0.6 (#2647)
* chore: Update version for release

* Update lock file

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: myftija <saadi.myftija@gmail.com>
2025-11-03 14:42:02 +01:00
nicktrn 9593a46364 fix(extensions): prevent audiowaveform binary removal (#2643)
* fix(extensions): prevent audiowaveform binary removal

* add changeset
2025-11-03 14:17:38 +01:00
Saadi Myftija 27376df903 fix(cli): show a useful error message when config file is missing (#2650)
* fix(cli): show a useful error message when config file is missing

* Add changeset
2025-10-31 16:34:48 +01:00
Saadi Myftija 2f3f82f3a9 feat(deployments): show local build hint during depot outages (#2646)
* Add an API endpoint to query remote build provider status

* Show local build hint for failed deployments when Depot is down

* Show the local build flag in the help output

* Add changeset

* Fix import

* Fix docs link
2025-10-30 18:09:54 +01:00
github-actions[bot] d4fe71df34 Release v4.0.5 (#2531)
* chore: Update version for release

* Release v4.0.5

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: nicktrn <55853254+nicktrn@users.noreply.github.com>
2025-10-28 14:20:20 +00:00
nicktrn ae8e83b2d0 chore(runner): move max duration logic into parent process (#2637)
* chore(runner): move max duration logic into parent process

* chore(rsc): remove type-marker package.json

* add changeset

* chore(core): remove irrelevant test after our changes

* chore(core): clarify we don't care about the timeout promise
2025-10-28 13:49:59 +00:00
Saadi Myftija 7f25e82299 feat(deployments): support local builds in cloud (#2628)
* Enable skipping image push during deployment finalization step

* Add endpoint to generate registry credentials for a deployment

* Add a --force-local-build flag to the deployment command to skip remote build

* Do not show the new flag in the help output

* Add changeset

* Remove registry login logs from onLog, not useful

* Rename var

* Update platform package to the latest version
2025-10-23 10:57:36 +02:00
James Ritchie fe3fe01fe8 feat(queues): Override queue concurrency limits from the dashboard or API (#2609)
* feat(queues): add ability to override concurrency limit via API and dashboard

* Updates the modal layout and tweaks copy

* Improves the dropdown menu item

* Popover supports both Button and LinkButton

* Right align the columns and fix the dropdown menu item styles

* Organize imports,

* Fix spinner icon in dropdown menu

* Remove unused props

* Adds a tooltip to the Concurrency override badge

* Fixes console error with popover menu

* typo

* Fixes incorrect className

* Minimal buttons to view runs

---------

Co-authored-by: Eric Allam <eallam@icloud.com>
2025-10-17 12:58:25 +01:00
nicktrn 8917478d3c fix(runner): SIGTERM handling during warm start long poll (#2593) 2025-10-08 12:51:10 +01:00
Eric Allam 692316e82a fix(realtime): Upgrade to @electric-sql/client@1.0.14 to prevent cached 409 Conflict errors from breaking realtime updates (#2588) 2025-10-07 14:26:03 +01:00
Eric Allam 128bc437f6 feat(otel): Add support for storing run spans and log data in Clickhouse (#2567) 2025-10-01 12:41:18 -07:00