Eric Allam
60d71da90e
Cuts CPU on the `engine/v1/worker-actions/*` routes a managed supervisor calls, and adds the benchmark harness the numbers come from. Measured on a local stack: **on-CPU per completed run 9.07ms → 6.59ms (−27%)**, busy fraction 45.6% → 33.8%, with every worker-action p50 down 23–27%. Load was 5,000 runs / 24 virtual supervisors / 90s window / 30,120 requests / 0 errors. Query-count work from the same investigation is deliberately **not** here — it will follow as a separate PR. ## The three changes **1. Split the event-loop monitor in two (~14% of on-CPU, plus ~5pp of GC).** `eventLoopMonitor.server.ts` installs a global `async_hooks` hook: `init` writes a `Map` entry for *every* async resource the process creates, `before` calls `process.hrtime()` and `context.active()` on every one. Enabling any async hook also puts V8 on the slow path for promise instrumentation process-wide. `EVENT_LOOP_MONITOR_ENABLED` defaulted to `"1"`, so this was the shipping configuration. The blocked-loop detector is now opt-in (`EVENT_LOOP_MONITOR_ENABLED`, default `0`). The event-loop *utilization* gauge — a single interval timer with no per-request cost — moves to its own flag (`EVENT_LOOP_UTILIZATION_MONITOR_ENABLED`, default `1`) and stays on, so the useful half survives without the expensive half. A/B under identical load: | | monitor on | monitor off | change | |---|---|---|---| | on-CPU per run | 9.08ms | 7.25ms | −20% | | GC self time | 9.80% | 5.05% | −4.75pp | | dequeue p50 | 76.6ms | 62.8ms | −18% | | attempts/start p50 | 56.3ms | 43.5ms | −23% | **2. Bucket route matching by first static path segment (10.4% → 3.9% of on-CPU).** `patches/@remix-run__router@1.23.3.patch` already memoized flattened branches and compiled path regexes. What remained was the linear scan: `matchRouteBranch` walked the ranked branch list calling `matchPath` per branch across 521 route files, so every worker-action request paid a scan proportional to the whole route table. Branches are now indexed by their lowercased leading segment, with one always-considered list for branches whose leading segment is dynamic, splat or optional (and for root/pathless paths). A request walks only its own bucket merged with that list. Route-matching self time dropped 64% (3.6s → 1.3s over a 90s window). Ordering is preserved exactly: both lists hold indexes into the already rank-sorted branch array and are walked in ascending-index order, so the first match found is the same branch the full scan would have found. Bucketing lowercases on both sides, so case-insensitive matching still resolves and `caseSensitive: true` routes are still rejected by `matchPath` itself. A pathname whose own leading segment can't be bucketed falls back to the full scan. Verified equivalent to the unpatched matcher over 20,050 pathnames (literal, dynamic, splat, optional, case variants, basenames, percent-encoded) with zero mismatches. `apps/webapp/test/routeMatchingPatch.test.ts` pins the matching semantics rather than the optimisation, so it still passes without the patch. **3. Demote per-heartbeat and per-dequeue `info` logs to `debug`.** These are the two highest-rate engine calls and each wrote a synchronous structured log line on every request. Synchronous `console` writes can block the loop when stdout backs up, which costs more than the ~1.3% CPU share suggests. ## The harness Two benchmarks, neither in the default suite (they run for minutes, attach the V8 profiler, and report numbers rather than assert on them). See `apps/webapp/test/bench/README.md`. - `apps/webapp/test/bench/engineHttp.bench.test.ts` — spawns a real webapp against throwaway Postgres/Redis containers, seeds a production environment with a promoted managed deployment, and drives a closed-loop supervisor pool through the full lifecycle. Profiling runs over CDP rather than `--cpu-prof` so it covers only the measured window instead of being swamped by boot, and `performance.eventLoopUtilization()` is sampled *inside* the webapp process. - `internal-packages/run-engine/src/engine/bench/runEngineLifecycle.bench.test.ts` — drives `RunEngine` directly, profiling enqueue and lifecycle separately so engine cost isn't mixed with request-stack overhead. - `apps/webapp/test/bench/analyzeProfile.ts` — dependency-free `.cpuprofile` analyzer that symbolicates through the build's source maps and ranks CPU by package, self time and total time. Percentages are shares of on-CPU time (V8's `(idle)`/`(program)` excluded). `startWebapp` gains `overrideEnv`, applied after the worker-disable defaults, so the HTTP bench can re-enable the run engine worker that drains the master queue into the worker queues a supervisor dequeues from. The local OTel collector gains a traces pipeline. It only defined a metrics pipeline, so pointing `INTERNAL_OTEL_TRACE_EXPORTER_URL` at it locally failed and the webapp silently fell back to the console span logger. ## Configuration For operators upgrading: - `EVENT_LOOP_MONITOR_ENABLED` (now defaults to `0`) — the per-async-resource blocked-loop detector. Set to `1` to restore the previous behaviour and keep emitting `event-loop-blocked` spans. - `EVENT_LOOP_UTILIZATION_MONITOR_ENABLED` (new, defaults to `1`) — the `nodejs.event_loop.utilization` gauge. Unchanged in behaviour; it just has its own flag now so it survives turning the detector off. ## Notes for review - `pnpm-lock.yaml` changes only because the router patch content changed, which changes its patch hash. - One thing the profile ruled out: with a real OTLP collector receiving spans, tracing costs ~1.7% of on-CPU at 100% sampling and ~0.8% at the production rate. Span shipping is not a hidden cost, so nothing here touches it. - Caveats on the numbers: a laptop, not production hardware, so DB and Redis *latency* are unrepresentative (client-side CPU is what's ranked); single webapp process; throughput varies ~5% run to run, which is why the claims rest on on-CPU per run rather than req/s. ## Verification - 20,050-pathname router equivalence check vs the unpatched matcher, zero mismatches - `apps/webapp/test/routeMatchingPatch.test.ts` (12 cases) passes - webapp e2e smoke suite (68 tests) passes through the patched router - run-engine suites covering the snapshot/attempt paths pass - `typecheck`, `format`, `lint`, `knip` clean
Build and deploy fully‑managed AI agents and workflows
Website | Docs | Issues | Example projects | Feature requests | Public roadmap | Self-hosting
About Trigger.dev
Trigger.dev is the open-source platform for building AI workflows in TypeScript. Long-running tasks with retries, queues, observability, and elastic scaling.
The platform designed for building AI agents
Build AI agents using all the frameworks, services and LLMs you're used to, deploy them to Trigger.dev and get durable, long-running tasks with retries, queues, observability, and elastic scaling out of the box.
-
Long-running without timeouts: Execute your tasks with absolutely no timeouts, unlike AWS Lambda, Vercel, and other serverless platforms.
-
Durability, retries & queues: Build rock solid agents and AI applications using our durable tasks, retries, queues and idempotency.
-
True runtime freedom: Customize your deployed tasks with system packages – run browsers, Python scripts, FFmpeg and more.
-
Human-in-the-loop: Programmatically pause your tasks until a human can approve, reject or give feedback.
-
Realtime apps & streaming: Move your background jobs to the foreground by subscribing to runs or streaming AI responses to your app.
-
Observability & monitoring: Each run has full tracing and logs. Configure error alerts to catch bugs fast.
Key features:
- JavaScript and TypeScript SDK - Build background tasks using familiar programming models
- Long-running tasks - Handle resource-heavy tasks without timeouts
- Durable cron schedules - Create and attach recurring schedules of up to a year
- Trigger.dev Realtime - Trigger, subscribe to, and get real-time updates for runs, with LLM streaming support
- Build extensions - Hook directly into the build system and customize the build process. Run Python scripts, FFmpeg, browsers, and more.
- React hooks - Interact with the Trigger.dev API on your frontend using our React hooks package
- Batch triggering - Use batchTrigger() to initiate multiple runs of a task with custom payloads and options
- Structured inputs / outputs - Define precise data schemas for your tasks with runtime payload validation
- Waits - Add waits to your tasks to pause execution for a specified duration
- Preview branches - Create isolated environments for testing and development. Integrates with Vercel and git workflows
- Waitpoints - Add human-in-the-loop judgment at critical decision points without disrupting workflow
- Concurrency & queues - Set concurrency rules to manage how multiple tasks execute
- Multiple environments - Support for DEV, PREVIEW, STAGING, and PROD environments
- No infrastructure to manage - Auto-scaling infrastructure that eliminates timeouts and server management
- Automatic retries - If your task encounters an uncaught error, we automatically attempt to run it again
- Checkpointing - Tasks are inherently durable, thanks to our checkpointing feature
- Versioning - Atomic versioning allows you to deploy new versions without affecting running tasks
- Machines - Configure the number of vCPUs and GBs of RAM you want the task to use
- Observability & monitoring - Monitor every aspect of your tasks' performance with comprehensive logging and visualization tools
- Logging & tracing - Comprehensive logging and tracing for all your tasks
- Tags - Attach up to ten tags to each run, allowing you to filter via the dashboard, realtime, and the SDK
- Run metadata - Attach metadata to runs which updates as the run progresses and is available to use in your frontend for live updates
- Bulk actions - Perform actions on multiple runs simultaneously, including replaying and cancelling
- Real-time alerts - Choose your preferred notification method for run failures and deployments
Write tasks in your codebase
Create tasks where they belong: in your codebase. Version control, localhost, test and review like you're already used to.
import { task } from "@trigger.dev/sdk";
//1. You need to export each task
export const helloWorld = task({
//2. Use a unique id for each task
id: "hello-world",
//3. The run function is the main function of the task
run: async (payload: { message: string }) => {
//4. You can write code that runs for a long time here, there are no timeouts
console.log(payload.message);
},
});
Deployment
Use our SDK to write tasks in your codebase. There's no infrastructure to manage, your tasks automatically scale and connect to our cloud. Or you can always self-host.
Environments
We support Development, Staging, Preview, and Production environments, allowing you to test your tasks before deploying them to production.
Full visibility of every job run
View every task in every run so you can tell exactly what happened. We provide a full trace view of every task run so you can see what happened at every step.
Getting started
The quickest way to get started is to create an account and project in our web app, and follow the instructions in the onboarding. Build and deploy your first task in minutes.
Useful links:
- Quick start - get up and running in minutes
- How it works - understand how Trigger.dev works under the hood
- Guides and examples - walk-through guides and code examples for popular frameworks and use cases
Self-hosting
If you prefer to self-host Trigger.dev, you can follow our self-hosting guides:
- Docker self-hosting guide - use Docker Compose to spin up a Trigger.dev instance
- Kubernetes self-hosting guide - use our official Helm chart to deploy Trigger.dev to your Kubernetes cluster
Support and community
We have a large active community in our official Discord server for support, including a dedicated channel for self-hosting.
Development
To setup and develop locally or contribute to the open source project, follow our development guide.