Daniel Sutton
920892bc11
Gives read-through and idempotency their gen-2 shard arms, so an id that names its own shard is read there and nowhere else. #4764 has landed, so this now targets `main` directly and no longer depends on an unmerged branch. It builds on what that PR supplied: `resolveShard`, `runOpsShardHandles` and the keyed router. TRI-13431 ## What changes **Read-through routes by `resolveShard`, not by the binary residency classifier.** A gen-2 id reads its own shard's replica once and probes no other store. A gen-1 v1 id still reads new only. **Callers now declare `idKind`.** A cuid gives no way to tell a run id from a waitpoint id, and the two must route differently: - a legacy-classified **run** id reads the legacy replica only — there is no cuid run migration, so the new-store probe cannot find it; - a cuid **waitpoint** keeps the new-first pair probe, which is load-bearing because a cuid waitpoint can be co-located with its run on the new store. There is no default, because a default would pick one of those arms silently. The field `runId` is renamed to `id`, since it carried both kinds already. **`ReadThroughResult` carries `found`.** `source` is an open-ended union once shards exist, so a consumer testing found-ness by listing the hit sources reads a gen-2 hit as a miss. One consumer did exactly that. Discriminating on `found` makes that class of bug a compile error rather than something a reviewer has to spot. **Idempotency resolves its client through one shard-keyed map.** Both call sites go through `clientForShardKey`, so they cannot disagree about which store owns an id. An absent key takes an explicit logged branch to the fallback, not a silent legacy default. The `classify` seam is retyped to return a `ShardKey`: `Residency` (`"NEW"`) and the reserved shard keys (`"new"`) differ only by case, and `ShardKey` collapses to `string`, so the compiler would not have caught feeding one into the other. The dead `isMigrated` branch is deleted. Nothing implemented it, and the one production comment recorded that omitting it was deliberate. **`PostgresRunStore._residency` widens to `ShardKey`.** Still unused; the store stays unaware of its siblings. ## Two behaviour fixes found while doing the above **An unconfigured shard key logs and returns not-found instead of throwing.** The waitpoint route takes the id from a URL parameter, and any base32hex core plus `[a-z0-9]` plus `"2"` parses as gen-2. The route turns a throw into a 500, so throwing here would let any authenticated client generate 500s and error logs by guessing shard chars, of which there are 36. An error-logged not-found is neither silent nor a misroute. Throwing stays correct on the router path, where ids are minted rather than received. **The two cross-seam batch hydration sites were gen-2 blind.** `hydrateRunsAcrossSeam` and `ApiBatchResultsPresenter` classified with the binary `ownerEngine`, so a gen-2 run id joined the gen-1 `new` group, missed there, and — classifying dedicated-family — never reached the legacy probe either. The id was dropped from a bulk-action page and from batch results with no error. Both now partition ids by shard key and read each configured shard once. Also: a gen-2 waitpoint that missed its shard replica fell back to the gen-1 new writer, a different database, silently disabling read-your-writes for the freshly minted token that fallback exists to serve. It now falls back to its own shard's writer. ## Merge safety Inert while `RUN_OPS_SHARDS` is unset: the shard maps are empty, so every gen-2 arm is unreachable, and gen-2 minting is not live yet. The one live change is the gen-1 run arm, and it removes work rather than adding it. `RoutingRunStore.findRun` never forwards the caller's client object — it routes by id and reads only the client's presence and replica brand — so `readRunForEvent`'s "new" closure already resolved a legacy-classified run id to the legacy store. The arm removes a duplicated read of the legacy replica. A test pins this, because a future caller passing a raw client and a run id would lose the pre-cutover 27-char case, which is new-resident but classifies legacy. ## Testing 14 tests added, testcontainers throughout, no mocks. 22 affected test files pass; typecheck, lint, format and knip are clean. Both arms were verified by neutralising them and confirming the new tests fail. The batch-results test needed rewriting after that check: the first version passed with the fix neutralised, because it used one container as both the gen-1 new client and the shard replica, so it was not testing what it claimed. Note for review: run testcontainer suites in small batches. Sixteen at once starves Docker and everything times out at 60 seconds. The run-ops legacy-guard baseline is refreshed in its own commit. The baseline is keyed by line number, so partitioning the batch-results read shifted four pre-existing entries and added one. Baselined violations in that file go from four to five, all reads; the new one is the shard read beside two gen-1 reads already there. No changeset and no `.server-changes` entry: a user notices nothing while the flag is unset.
Build and deploy fully‑managed AI agents and workflows
Website | Docs | Issues | Example projects | Feature requests | Public roadmap | Self-hosting
About Trigger.dev
Trigger.dev is the open-source platform for building AI workflows in TypeScript. Long-running tasks with retries, queues, observability, and elastic scaling.
The platform designed for building AI agents
Build AI agents using all the frameworks, services and LLMs you're used to, deploy them to Trigger.dev and get durable, long-running tasks with retries, queues, observability, and elastic scaling out of the box.
-
Long-running without timeouts: Execute your tasks with absolutely no timeouts, unlike AWS Lambda, Vercel, and other serverless platforms.
-
Durability, retries & queues: Build rock solid agents and AI applications using our durable tasks, retries, queues and idempotency.
-
True runtime freedom: Customize your deployed tasks with system packages – run browsers, Python scripts, FFmpeg and more.
-
Human-in-the-loop: Programmatically pause your tasks until a human can approve, reject or give feedback.
-
Realtime apps & streaming: Move your background jobs to the foreground by subscribing to runs or streaming AI responses to your app.
-
Observability & monitoring: Each run has full tracing and logs. Configure error alerts to catch bugs fast.
Key features:
- JavaScript and TypeScript SDK - Build background tasks using familiar programming models
- Long-running tasks - Handle resource-heavy tasks without timeouts
- Durable cron schedules - Create and attach recurring schedules of up to a year
- Trigger.dev Realtime - Trigger, subscribe to, and get real-time updates for runs, with LLM streaming support
- Build extensions - Hook directly into the build system and customize the build process. Run Python scripts, FFmpeg, browsers, and more.
- React hooks - Interact with the Trigger.dev API on your frontend using our React hooks package
- Batch triggering - Use batchTrigger() to initiate multiple runs of a task with custom payloads and options
- Structured inputs / outputs - Define precise data schemas for your tasks with runtime payload validation
- Waits - Add waits to your tasks to pause execution for a specified duration
- Preview branches - Create isolated environments for testing and development. Integrates with Vercel and git workflows
- Waitpoints - Add human-in-the-loop judgment at critical decision points without disrupting workflow
- Concurrency & queues - Set concurrency rules to manage how multiple tasks execute
- Multiple environments - Support for DEV, PREVIEW, STAGING, and PROD environments
- No infrastructure to manage - Auto-scaling infrastructure that eliminates timeouts and server management
- Automatic retries - If your task encounters an uncaught error, we automatically attempt to run it again
- Checkpointing - Tasks are inherently durable, thanks to our checkpointing feature
- Versioning - Atomic versioning allows you to deploy new versions without affecting running tasks
- Machines - Configure the number of vCPUs and GBs of RAM you want the task to use
- Observability & monitoring - Monitor every aspect of your tasks' performance with comprehensive logging and visualization tools
- Logging & tracing - Comprehensive logging and tracing for all your tasks
- Tags - Attach up to ten tags to each run, allowing you to filter via the dashboard, realtime, and the SDK
- Run metadata - Attach metadata to runs which updates as the run progresses and is available to use in your frontend for live updates
- Bulk actions - Perform actions on multiple runs simultaneously, including replaying and cancelling
- Real-time alerts - Choose your preferred notification method for run failures and deployments
Write tasks in your codebase
Create tasks where they belong: in your codebase. Version control, localhost, test and review like you're already used to.
import { task } from "@trigger.dev/sdk";
//1. You need to export each task
export const helloWorld = task({
//2. Use a unique id for each task
id: "hello-world",
//3. The run function is the main function of the task
run: async (payload: { message: string }) => {
//4. You can write code that runs for a long time here, there are no timeouts
console.log(payload.message);
},
});
Deployment
Use our SDK to write tasks in your codebase. There's no infrastructure to manage, your tasks automatically scale and connect to our cloud. Or you can always self-host.
Environments
We support Development, Staging, Preview, and Production environments, allowing you to test your tasks before deploying them to production.
Full visibility of every job run
View every task in every run so you can tell exactly what happened. We provide a full trace view of every task run so you can see what happened at every step.
Getting started
The quickest way to get started is to create an account and project in our web app, and follow the instructions in the onboarding. Build and deploy your first task in minutes.
Useful links:
- Quick start - get up and running in minutes
- How it works - understand how Trigger.dev works under the hood
- Guides and examples - walk-through guides and code examples for popular frameworks and use cases
Self-hosting
If you prefer to self-host Trigger.dev, you can follow our self-hosting guides:
- Docker self-hosting guide - use Docker Compose to spin up a Trigger.dev instance
- Kubernetes self-hosting guide - use our official Helm chart to deploy Trigger.dev to your Kubernetes cluster
Support and community
We have a large active community in our official Discord server for support, including a dedicated channel for self-hosting.
Development
To setup and develop locally or contribute to the open source project, follow our development guide.