- Adds an end-to-end OTEL metrics pipeline: task workers collect and
export metrics via OpenTelemetry, the webapp ingests them into
ClickHouse, and they're queryable through the existing dashboard query
engine
- Workers emit process CPU/memory metrics (via
`@opentelemetry/host-metrics`) and Node.js runtime metrics (event loop
utilization, event loop delay, heap usage)
- Users can create custom metrics in their tasks via
`otel.metrics.getMeter()` from `@trigger.dev/sdk`
- Metrics are automatically tagged with run context (run ID, task slug,
machine, worker version) so they can be sliced per-run, per-task, or
per-machine
- The TSQL query engine gains metrics table support with typed attribute
columns, `prettyFormat()` for human-readable values, and per-schema time
bucket thresholds
- Includes reference tasks
(`references/hello-world/src/trigger/metrics.ts`) demonstrating
CPU-intensive, memory-ramp, bursty workload, and custom metrics patterns
## What changed
### Metrics collection (packages/core, packages/cli-v3)
- **Metrics export pipeline** — `TracingSDK` now sets up a
`MeterProvider` with a `PeriodicExportingMetricReader` that chains
through `TaskContextMetricExporter` (adds run context attributes) and
`BufferingMetricExporter` (batches exports to reduce overhead)
- **Host metrics** — Enabled `@opentelemetry/host-metrics` for process
CPU, memory, and system-level metrics
- **Node.js runtime metrics** — New `nodejsRuntimeMetrics.ts` module
using `performance.eventLoopUtilization()`, `monitorEventLoopDelay()`,
and `process.memoryUsage()` to emit 6 observable gauges
- File system and diskio metrics
- **Custom metrics** — Exposed `otel.metrics` from `@trigger.dev/sdk` so
users can create counters, histograms, and gauges in their tasks
- **Machine ID** — Stable per-worker machine identifier for grouping
metrics
- **Dev worker** — Drops `system.*` metrics to reduce noise, keeps
sending metrics between runs in warm workers
### Metrics ingestion (apps/webapp)
- **OTEL endpoint** — `otel.v1.metrics.ts` accepts OTEL metric export
requests (JSON and protobuf), converts to ClickHouse rows
- **ClickHouse schema** — `017_create_metrics_v1.sql` with 10-second
aggregation buckets, JSON attributes column, 60-day TTLs
### Query engine (internal-packages/tsql, apps/webapp)
- **Metrics query schema** — Typed columns for metric attributes
(`task_identifier`, `run_id`, `machine_name`, `worker_version`, etc.)
extracted from the JSON attributes column
- **`prettyFormat()`** — TSQL function that annotates columns with
format hints (`bytes`, `percent`, `durationSeconds`) for frontend
rendering without changing the underlying data
- **Per-schema time buckets** — Different tables can define their own
time bucket thresholds (metrics uses tighter intervals than runs)
- **AI query integration** — The AI query service knows about the
metrics table and can generate metric queries
- **Chart improvements** — Better formatting for byte values,
percentages, and durations in charts and tables
### Reference project
- **`references/hello-world/src/trigger/metrics.ts`** — 6 example tasks:
`cpu-intensive`, `memory-ramp`, `bursty-workload`, `sustained-workload`,
`concurrent-load`, `custom-metrics`
## Test plan
- [ ] Build all packages and webapp
- [ ] Start dev worker with hello-world reference project
- [ ] Run `cpu-intensive`, `memory-ramp`, and `custom-metrics` tasks
- [ ] Verify metrics in ClickHouse: `SELECT DISTINCT metric_name FROM
metrics_v1`
- [ ] Query via dashboard AI: "show me CPU utilization over time"
- [ ] Verify `prettyFormat` renders correctly in chart tooltips and
table cells
- [ ] Confirm dev worker drops `system.*` metrics but keeps `process.*`
and `nodejs.*`
This will prevent internal logs to be added to the
task_events_search_table
Closes #<issue>
## ✅ Checklist
- [x] I have followed every step in the [contributing
guide](https://github.com/triggerdotdev/trigger.dev/blob/main/CONTRIBUTING.md)
- [x] The PR title follows the convention.
- [x] I ran and tested the code works
---
## Testing
Ran the migration, deleted the old invalid rows and ran new tasks.
The undesired logs are not added to the table.
---
## Changelog
Updated the MATERIALIZED VIEW to also filter for `trace_id != ''`
---------
Co-authored-by: Matt Aitken <matt@mattaitken.com>
Summary
- Implemented metrics dashboards with a built-in dashboard and custom
dashboards
- Added a "Big number” display type
What changed
- New data format for metric layouts and saving/editing layouts
(editing, saving, cancel revert)
- QueryWidget usable on Query page and Metrics dashboards
- Time filtering, auto-reloading and timeBucket() auto-bin support
- Filters added to metrics; widget popover/improved history and blank
states
- Side menu:
- Metrics/Insights section with icons, colors, padding, collapsible
behavior and reordering of custom dashboards
- Move action logic into service for reuse and API querying; refactor
reordering for reuse
<!-- devin-review-badge-begin -->
---
<a
href="https://app.devin.ai/review/triggerdotdev/trigger.dev/pull/3019"
target="_blank">
<picture>
<source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-open-in-devin-review-dark.svg?v=1">
<img
src="https://static.devin.ai/assets/gh-open-in-devin-review-light.svg?v=1"
alt="Open with Devin">
</picture>
</a>
<!-- devin-review-badge-end -->
---------
Co-authored-by: James Ritchie <james@trigger.dev>
Closes#2798
When a run finished the logs UI could get stuck and so be pending and
never update again. If you did a hard reload it would be correct.
This happened because when we insert a log/span we ping Redis which
causes a reload of the UI. However there was a race condition – the
insert into ClickHouse can take a while so we were refreshing the UI too
early. Then never refreshing it again.
Changes
- Send refresh pings every 5s to keep run page logs live
- Throttle updates so the run UI is never updated more than once per
second
- Stop auto-reloading when a run has been completed for >= 30s
- Add type inference improvements for the throttle function
<!-- devin-review-badge-begin -->
---
<a
href="https://app.devin.ai/review/triggerdotdev/trigger.dev/pull/2971">
<picture>
<source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-open-in-devin-review-dark.svg?v=1">
<img
src="https://static.devin.ai/assets/gh-open-in-devin-review-light.svg?v=1"
alt="Open with Devin">
</picture>
</a>
<!-- devin-review-badge-end -->
## ✨ Changes
### UI & UX
- Normalized log level display across table and detail view
- Fixed table header scroll behavior and sidebar positioning
- Improved loading state with taller segment and disabled resizing
- Added "no more logs" message with count
- Enhanced keyboard shortcuts
### Filtering & Search
- Streamlined filters: RunId and Task only (removed run filters)
- Side panel closes when filters change
- Fixed logs from previous search remaining in table
- Fixed table scroll position when changing filters
### Backend
- Added performance indexes on message and attributes
(`014_add_task_runs_v2_search_indexes.sql`)
- Added DEBUG level logging by default
- Removed internal logs from display
- Fixed ServiceValidationError forwarding to frontend
- Removed v1 logs API support
Closes #<issue>
## ✅ Checklist
- [ ] I have followed every step in the [contributing
guide](https://github.com/triggerdotdev/trigger.dev/blob/main/CONTRIBUTING.md)
- [ ] The PR title follows the convention.
- [ ] I ran and tested the code works
---
## Testing
_[Describe the steps you took to test this change]_
---
## Changelog
_[Short description of what has changed]_
---
## Screenshots
_[Screenshots]_
💯
---------
Co-authored-by: Matt Aitken <matt@mattaitken.com>
Closes #<issue>
## ✅ Checklist
- [x] I have followed every step in the [contributing
guide](https://github.com/triggerdotdev/trigger.dev/blob/main/CONTRIBUTING.md)
- [x] The PR title follows the convention.
- [x] I ran and tested the code works
---
## Testing
- Verified log detail view displays correctly with message, metadata,
and attributes
- Tested search highlighting functionality in log messages (escapes
special regex characters)
- Confirmed tabs (Details/Run) switch properly with keyboard shortcuts
(d/r)
- Verified run information loads via async fetcher in Run tab
- Tested close button and Escape key for dismissing the panel
- Verified log details display correct information: level badges, kind
badges, timestamps, trace IDs, span IDs
- Confirmed links to parent spans and run pages work correctly
- Tested with various log levels (ERROR, WARN, INFO, DEBUG, TRACE) and
kinds (SPAN, SPAN_EVENT, LOG_*)
- Verified admin-only fields display correctly when user has admin
access
- Tested data loading states and error states (log not found, run not
found)
---
## Changelog
Created new Logs page.
The information shown is gathered from the spans from each run.
The feature supports all run filters with two new filters for level and
logs text search.
---
## Screenshots
<img width="2059" height="1196" alt="Logs page preview"
src="https://github.com/user-attachments/assets/70b667b4-98cc-4728-855a-2766dd5c1aa5"
/>
💯
---------
Co-authored-by: James Ritchie <james@trigger.dev>
TRQL (pronounced Treacle like the delicious British dark sweet syrup) is
the TRiggerQueryLanguage. It allows users to safely write queries on
their data. The queries are safely turned into ClickHouse queries which
are tenant-safe and not SQL injectable.
https://github.com/user-attachments/assets/bbfca473-b3fc-4150-8fe6-79e8840a2d29
This started out as a translation of HogQL by PostHog from Python to
TypeScript.
Features
- Tenant safe queries.
- Many underlying ClickHouse features including functions and
aggregations.
- Virtual columns, which are exposed to users as real columns but are
actually expressions.
- Transformations of data types and where clauses.
- Simple JSON path querying.
- Limits on execution time.
- Reporting of query statistics.
## Query page
There’s a new Query page (currently behind a feature flag) where you can
write TRQL queries and execute them against your environment, project or
organization.
Features
- Executing TRQL queries
- Syntax highlighting and errors
- Autocomplete
- AI generation/editing of queries
- Help and examples
- Table with auto-inferred data types from the table schema
- Table cell renderers for our special types like Run ids, environments,
machines, tasks, queues, etc.
- Copy/export as CSV/JSON
- Line and bar graphs with grouping and stacking
- History of queries
Add support for resetting idempotency keys both from ui and sdk
## ✅ Checklist
- [x] I have followed every step in the [contributing
guide](https://github.com/triggerdotdev/trigger.dev/blob/main/CONTRIBUTING.md)
- [x] The PR title follows the convention.
- [x] I ran and tested the code works
---
## Testing
- Created a new run with a idempotency idempotencyKey.
- Started a new run with the same task and got redirected to the first
run.
- Deleted the key from the UI on the run details
- Started a new run with the same task and it created a new one
- Did the above steps using the SDK
---
## Changelog
- Add new action route for resetting idempotency keys via UI
- Add reset button in Idempotency section of run detail view
- Added API and SDK for resetting imdepotency
- Updated docs page for this feature
---
## Screenshots
_[Screenshots]_
<img width="438" height="363" alt="Screenshot 2025-12-11 at 11 56 37"
src="https://github.com/user-attachments/assets/30b8ef5e-8aac-4d04-b57a-9bf30d085dcb"
/>
New batch trigger system with larger payloads, streaming ingestion,
larger batch sizes, and a fair processing system.
This PR introduces a new `FairQueue` abstraction inspired by our own
`RunQueue` that enables multi-tenant fair queueing with concurrency
limits. The new `BatchQueue` is built on top of the `FairQueue`, and
handles processing Batch triggers in a fair manner with per-environment
concurrency limits defined per-org. Additionally, there is a global
concurrency limit to prevent the BatchQueue system from creating too
many runs too quickly, which can cause downstream issues.
For this new BatchQueue system we have a completely new batch trigger
creation and ingestion system. Previously this was a single endpoint
with a single JSON body that defined details about the batch as well as
all the items in the batch.
We're introducing a two-phase batch trigger ingestion system. In the
first phase, the BatchTaskRun record is created (and possibly rate
limited). The second phase is another endpoint that accepts an NDJSON
body with each line being a single item/run with payload and options.
At ingestion time all items are added to a queue, in order, and then
processed by the BatchQueue system.
## New batch trigger rate limits
This PR implements a new batch trigger specific rate limit, configured
on the `Organization.batchRateLimitConfig` column, and defaults using
these environment variables:
- `BATCH_RATE_LIMIT_REFILL_RATE` defaults to 10
- `BATCH_RATE_LIMIT_REFILL_INTERVAL` the duration interval, defaults to
`"10s"`
- `BATCH_RATE_LIMIT_MAX` defaults to 1200
This rate limiter is scoped to the environment ID and controls how many
runs can be submitted via batch triggers per interval. The SDK handles
the retrying side.
## Batch queue concurrency limits
The new column `Organization.batchQueueConcurrencyConfig` now defines an
org specific `processingConcurrency` value, with a backup of the env var
`BATCH_CONCURRENCY_LIMIT_DEFAULT` which defaults to 10. This controls
how many batch queue items are processed concurrently per environment.
There is also a global rate limit for the batch queue set via the
`BATCH_QUEUE_GLOBAL_RATE_LIMIT` which defaults to being disabled. If
set, the entire batch queue system won't process more than
`BATCH_QUEUE_GLOBAL_RATE_LIMIT` items per second. This allows
controlling the maximum number of runs created per second via batch
triggers.
## Batch trigger settings
- `STREAMING_BATCH_MAX_ITEMS` controls the maximum number of items in a
single batch
- `STREAMING_BATCH_ITEM_MAXIMUM_SIZE` controls the maximum size of each
item in a batch
- `BATCH_CONCURRENCY_DEFAULT_CONCURRENCY` controls the default
environment concurrency
- `BATCH_QUEUE_DRR_QUANTUM` how many credits each environment gets each
round for the DRR scheduler
- `BATCH_QUEUE_MAX_DEFICIT` the maximum deficit for the DRR scheduler
- `BATCH_QUEUE_CONSUMER_COUNT` how many queue consumers to run
- `BATCH_QUEUE_CONSUMER_INTERVAL_MS` how frequently they poll for items
in the queue
### Configuration Recommendations by Use Case
**High-throughput priority (fairness acceptable at 0.98+):**
```env
BATCH_QUEUE_DRR_QUANTUM=25
BATCH_QUEUE_MAX_DEFICIT=100
BATCH_QUEUE_CONSUMER_COUNT=10
BATCH_QUEUE_CONSUMER_INTERVAL_MS=50
BATCH_CONCURRENCY_DEFAULT_CONCURRENCY=25
```
**Strict fairness priority (throughput can be lower):**
```env
BATCH_QUEUE_DRR_QUANTUM=5
BATCH_QUEUE_MAX_DEFICIT=25
BATCH_QUEUE_CONSUMER_COUNT=3
BATCH_QUEUE_CONSUMER_INTERVAL_MS=100
BATCH_CONCURRENCY_DEFAULT_CONCURRENCY=5
```
## ✅ Checklist
- [x] I have followed every step in the [contributing
guide](https://github.com/triggerdotdev/trigger.dev/blob/main/CONTRIBUTING.md)
- [x] The PR title follows the convention.
- [x] I ran and tested the code works
---
## Testing
Manual testing of the task run pages
---
## Changelog
- Add previous/next run navigation buttons to run detail page header
- Support [ and ] keyboard shortcuts to jump between adjacent runs
- Preserve runs table state (filters, pagination) when navigating
- Preload adjacent page runs at boundaries for seamless navigation
- Add actions prop to PageTitle component
- Document shortcut in keyboard shortcuts panel
- Store current filter state from runs table as `tableState` search
param when navigating to individual run pages
- Restore filters when navigating back from run detail view to runs list
- Update `v3RunPath` and `v3RunSpanPath` helpers to accept optional
searchParams
- Use `useOptimisticLocation` to capture current search params in
TaskRunsTable
- Parse `tableState` param in run detail route and pass filters to back
button
- This improves UX by remembering filter selections (task, status, date
range, etc.) when users click into a run and then navigate back to the
runs list
- Add new text-below variant that shows "Click to copy" tooltip on hover
and "Copied" on click. Also add controlled open/onOpenChange props to
SimpleTooltip for managing tooltip visibility.
---
## Screenshots
https://github.com/user-attachments/assets/5067bbe0-1bcd-4e75-80a7-f56dabd5ed69
* Don't use the organization max concurrency anymore
* Early draft of the concurrency page
* WIP adding a new stepper input component
* Move stepper to be alphabetical
* When max value is reached, disabled the + button
* Show placeholder if you delete all numbers
* Make all the html input values available to the component
* Adds size variants
* Move stepper into its own component
* Work on showing the extra concurrency
* The purchase form styling and functionality (minus actually purchasing)
* New style for outline input fields
* Concurrency purchasing working
* Purchasing concurrency and quota emails working
* Improvements to the modal
* Show cost breakdown in the modal
* Fix for allocated concurrency including DEV
* Improved types
* Allocating concurrency is working
* Live updates total env concurrency
* Implemented reset
* Fix for concurrency allocation editing across multiple projects
* Tabular numbers
* Added an error from allocating concurrency
* Fixes for allocating concurrency where it didn't calculate correctly
* "Increase limit" link to concurrency page
* Indent environments
* Added Preview limit when updating concurrency for an org
* Show error when changing plan fails
* Added maximumProjectCount column to Org
* Limit project count and display a rich error toast (with title and button now)
* Added title and button to toasts. Use it for new project error
* @trigger.dev/platform 1.0.20
* Allow submitting zero concurrency so you can downgrade back to nothing
* Use the server as the truth for omitted environments
* Updated the pricing panels
---------
Co-authored-by: James Ritchie <james@trigger.dev>
* Fix settigns page delete project width issue
* Apply a couple of touch-ups to the project settings page
* Add UI flow to connect gh repos
* Enabling adding another gh account in the ui
* Enable connecting a repo to a project
* Enable updating git settings
* Enable disconnecting gh repos from a project
* Remove prisma migration drifts
* Hide git settings when github app is disabled
* Fix migration order
* Avoid using `location` to avoid SSR issues
* Make branch tracking optional
* Disable save buttons when there are no field changes
* Disable delete project button unless the input matches the project slug
* Show connected repo connectedAt date
* Check that tracking branch exists when updating git settings
* Show tracking branch hint in the deployments page
* Fix positioning issue of the pagination pane in the deployments page
* Use mono font for branch names
* Add link to git settings
* Show tracking branch hint for the preview env too
* Add a confirmation prompt on repo disconnect
* Add link to configure repo access in gh
* Add rel prop to github links
* Automatically open repo connection modal after app installation
* Apply some fixes suggested by mr rabbit
* Fix flash cookie issue
* Extract project settings actions into a service
* Extract project settings loader into a presenter service
* Introduce neverthrow for error handling
* Try out neverthrow for error handling in the project setting flows
* Move env gh branch resolution to the presenter service
* Initial work on upgrading to 6.14.0
Set the output to node_modules still to make it easier
* Use ./generated Prisma folder, update types to fix issues
* Docker compose restart Clickhouse
* Prisma instrumentation update
* Docker
* Removed database dockerignore file, add generated prisma client to the top-level one
* Delete v3-catalog package.json
* Resolved pnpm lock file
* Log errors for very slow queries
* Create schema and migration for organization access tokens
* Add helpers for creating and authenticating OATs
* Adapt the auth service to also accept OATs
* Accept OATs in the whoami v2 endpoint
* Enable deployments with the CLI using OATs
* Avoid reading env variables directly in the token utils
* Remove duplicate cli token utils
* Validate ENCRYPTION_KEY length when parsing env vars
* Make token utils a server-only module
* Disallow revoking already revoked OATs
* Simplify generics in authenticateRequest
* Use 32 bytes mock encryption key in the test setup
* Update dummy encryption key values in tests and templates
* Add a column in the OATs table to differentiate between user and system generated
* Simplify args for v3ProjectPath
Co-authored-by: Matt Aitken <matt@mattaitken.com>
* Add index on org id and createdAt
* Avoid storing the encrypted oat token and its obfuscated version in the DB at all
It is a safer approach. Also we do not need to ever read the decrypted token value after creation.
* Fix prisma update condition
* Add token type to the OAT table index
* Accept OATs in the mcp auth flow
* Simplify env auth flow around the /projects endpoints
---------
Co-authored-by: Matt Aitken <matt@mattaitken.com>
* add tier scheduling support to supervisor
* add billing info to dequeued message w/o cache
* add cache with best effort invalidation
* fix invalidate circular dep
* add changeset
* use new plan type on runs as fallback during dequeue
* tidy up
* be more explicit with plan type fallback
* remove additional billing check from hot path
* switch to placement tags
* update changeset
* update platform package
* start using new entitlement response
* ensure skipChecks optimization validates at batch level
* add optional items to add to queue manager limits
* make the bool env helper only accept boolean defaults
* remove redundant private field
* update placement tag helper to prevent unsupported tags
* Initial Regions page
* Fix for bad attribute name
* Switching regions is working. Some style improvements
* Use a dialog for confirmation, not great yet
* Added allowedMasterQueues
* Improves flag icons
* Improves the region switch modal with more info
* Adds “suggest a region” table row
* New icons for the buttons
* Improved the tooltip information
* New “small” badge style
* Make the default badge live in its column
* Better DO icon size
* Remove unused export of regions options
* Show upgrade message for free users to get static IPs
* Admins can view all regions and switch at will
---------
Co-authored-by: James Ritchie <james@trigger.dev>
* First draft billing alerts page
* Budget alert form working
* Don't let free plan users change the billing alert amount
* Fix missing key in map in the form
* Disable queues/org from admin API endpoint
* Don't allow resuming if runsEnabled is false
* Refer to "Billing alerts" not "Plans"
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
* Form missing dependencies fix
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
* Deal with thrown errors, fix for duplicating email fields
* Added a RuntimeEnvironment organizationId index
---------
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
* Queue in run table and filtering
* Debounce the filter changes
* Remove console log
* Added machine filtering
* Added version filtering
* Filter by version in the db
* Removed duplicate classes
* Version filtering hasFilters consistency
* Added queues and machines to the bulk action summary
* runs.list filtering for queue and machine
* Fix for machine errors
* useSearchParams has
* useSearchParams has
* useSearchParams has
* Consistent way to get the run filters
* Consistent way to get the run filters
* Consistent way to get the run filters
* Initial work on the new bulk actions
* Initial work on the new bulk actions
* Initial work on the new bulk actions
* WIP actions and filtering
* WIP actions and filtering
* WIP actions and filtering
* Empty filter arrays are set to undefined
* Empty filter arrays are set to undefined
* Empty filter arrays are set to undefined
* WIP prisma schema
Removed extra runtimeEnvironmentId
* WIP prisma schema
Removed extra runtimeEnvironmentId
* WIP prisma schema
Removed extra runtimeEnvironmentId
* Migrations
* Migrations
* Migrations
* BulkActionGroup changed some columns around
* BulkActionGroup changed some columns around
* BulkActionGroup changed some columns around
* New badge variant, removed unused ones
* New badge variant, removed unused ones
* New badge variant, removed unused ones
* Bulk action button
* Bulk action button
* Bulk action button
* Make the next runs page the default now
* Make the next runs page the default now
* Make the next runs page the default now
* Improved the RadioButton style
* Improved the RadioButton style
* Improved the RadioButton style
* Remove the old bulk action bar
* Remove the old bulk action bar
* Remove the old bulk action bar
* More UI progress
* More UI progress
* More UI progress
* Lots of UI changes to the Runs page
* Lots of UI changes to the Runs page
* Lots of UI changes to the Runs page
* Fixed period filter resetting everything
* Fixed period filter resetting everything
* Fixed period filter resetting everything
* Improved the Switch secondary style
* Improved the Switch secondary style
* Improved the Switch secondary style
* Buggy filter fixes
* Buggy filter fixes
* Buggy filter fixes
* Improved the filter display and fixed a bug with search param from object
* Improved the filter display and fixed a bug with search param from object
* Improved the filter display and fixed a bug with search param from object
* Clear button is minimal
* Clear button is minimal
* Clear button is minimal
* Using a presenter now
* Using a presenter now
* Using a presenter now
* Bulk actions are created, but not actually processed (yet)
* Bulk actions are created, but not actually processed (yet)
* Bulk actions are created, but not actually processed (yet)
* Bulk replay/cancel is working
* Bulk replay/cancel is working
* Bulk replay/cancel is working
* Multiple fixes, added bulk column to PG
* Multiple fixes, added bulk column to PG
* Multiple fixes, added bulk column to PG
* Bulk action run filtering working using CH
* Bulk action run filtering working using CH
* Bulk action run filtering working using CH
* Replay setting the bulk id on the runs
* Replay setting the bulk id on the runs
* Replay setting the bulk id on the runs
* Properly cap the time when doing a bulk action
* Properly cap the time when doing a bulk action
* Properly cap the time when doing a bulk action
* If the bulk action isn't recent, add it to the dropdown anyway
* If the bulk action isn't recent, add it to the dropdown anyway
* If the bulk action isn't recent, add it to the dropdown anyway
* Blank version of the bulk actions page
* Blank version of the bulk actions page
* Blank version of the bulk actions page
* Individually selected runs working
* Individually selected runs working
* Individually selected runs working
* Use selected mode if runs are checked
* Use selected mode if runs are checked
* Use selected mode if runs are checked
* Added the modal
* Added the modal
* Added the modal
* Marked the old bulk actions stuff as deprecated
* Marked the old bulk actions stuff as deprecated
* Marked the old bulk actions stuff as deprecated
* Renamed bulk action file
* Renamed bulk action file
* Renamed bulk action file
* Bulk run filter with the name and a default
* Bulk run filter with the name and a default
* Bulk run filter with the name and a default
* WIP on bulk actions page
* WIP on bulk actions page
* WIP on bulk actions page
* Updated panel, added new truncated id component
* Updated panel, added new truncated id component
* Updated panel, added new truncated id component
* Style improvements to the radio buttons
* Style improvements to the radio buttons
* Style improvements to the radio buttons
* Added an option action completion email
* Added an option action completion email
* Added an option action completion email
* Adds a blank state for the bulk actions page
* Adds a blank state for the bulk actions page
* Adds a blank state for the bulk actions page
* Nicer completed email
* Nicer completed email
* Nicer completed email
* Don't open the bulk action panel if there are no runs
* Don't open the bulk action panel if there are no runs
* Don't open the bulk action panel if there are no runs
* Runs blank state and bulk action accordion
* Runs blank state and bulk action accordion
* Runs blank state and bulk action accordion
* Updates secondary/small switch style
* Updates secondary/small switch style
* Updates secondary/small switch style
* Pagination buttons no longer split in twain (WIP)
* Pagination buttons no longer split in twain (WIP)
* Pagination buttons no longer split in twain (WIP)
* Aborting working
* Aborting working
* Aborting working
* Bulk action live reloading
* Bulk action live reloading
* Bulk action live reloading
* ListPagination works correctly in all states
* ListPagination works correctly in all states
* ListPagination works correctly in all states
* Run page, show friendlyId instead of number
* Run page, show friendlyId instead of number
* Run page, show friendlyId instead of number
* Bulk action help open by default if you have none
* Bulk action help open by default if you have none
* Bulk action help open by default if you have none
* Extra status filtering step because of replication delay
* Extra status filtering step because of replication delay
* Extra status filtering step because of replication delay
* Wider bulk action onboarding
* Wider bulk action onboarding
* Wider bulk action onboarding
* More sensible widths on the bulk action side panel
* More sensible widths on the bulk action side panel
* More sensible widths on the bulk action side panel
* Border color tweak to the RadioButton
* Border color tweak to the RadioButton
* Border color tweak to the RadioButton
* Improved the accordion component hover states
* Improved the accordion component hover states
* Improved the accordion component hover states
* Updates the bulk action blank state images to the latest UI
* Updates the bulk action blank state images to the latest UI
* Updates the bulk action blank state images to the latest UI
* Added R and C shortcuts back in
* Added R and C shortcuts back in
* Added R and C shortcuts back in
* Fix for selecting a single run
* Fix for selecting a single run
* Fix for selecting a single run
* Improved exit icon, added shortcut to modal
* Improved exit icon, added shortcut to modal
* Improved exit icon, added shortcut to modal
* Tidy imports
* Tidy imports
* Tidy imports
* Tidy imports
* Tidy imports
* Tidy imports
* Tidy imports
* Tidy imports
* Tidy imports
* Tidy imports
* Fix for grid layout when 1 page of bulk actions visible
* Fix for grid layout when 1 page of bulk actions visible
* Fix for grid layout when 1 page of bulk actions visible
* Removed the ... on the abort button
* Removed the ... on the abort button
* Removed the ... on the abort button
* Removed the ... on the abort button
* Animate the progress bar
* Set TZ="UTC" in the env example
* Filter summary in the bulk inspector
* Improves the pagination styling
* Improves the pagination styling
* Delete old bulk action routes
* Removed old Postgres RunListPresenter
* Retry any replication error where the message contains "timeout"
* Increase wait to make test less flaky
* The test was using run id instead of friendly id
* Safer array access
* Remove error log if there's a bad status
* Nicer frontend type safety with the bulk action and mode
* Switched a log to a debug log
* Retry replication unless the error is a known non-retry error
Flip the strategy to retry by default
* Make ClickHouse required
* Backfill run replication admin API endpoint
* Set a CLICKHOUSE_URL for unit tests
---------
Co-authored-by: James Ritchie <james@trigger.dev>
* update node-22 image
* update bun image
* disable io_uring
* fix fallback bun path
* prevent duplicate warnings
* add runtime and version to deployments
* runtime icons
* fallback to nodejs
* prevent empty table cell menu
* log if local build on deploy
* pass io_uring env var to child
* denormalize runtime and version, display on run details
* add changesets
* disable pr checks for changeset commits..
* add runtime data to deployed bg workers
* Introduce request idempotency to prevent duplicate triggers
* Implement request idempotency on trigger
* Use x-trigger-request-idempotency-key header instead
Plus cleanup shared logic in the routes
* Add changeset
* Oops, lets not hardcode a 408
* A couple of improvements
* Adds a new route for logging in with mfa
* New path for security page
* Adds “Security” link to account side menu
* Update the Switch component to allow label positions left and right
* Optionally hide the Close button in the Dialog title bar
* Installs `qrcode` react package for generating QR codes.
* CopyButton component now takes children
* New Security route for setting up MFA
* Adds new OTP package for the chadcn InputOTP component
* Adds new InputOTP chadcn component
* Adds InputOTP chadcn component to the MFA login screen
* InputOTP component supports variant styles
* Improvements to form handling
* Show a confirmation modal before you can disable MFA
* Revert redirect back to the dashboard for now
* Implement MFA enabling and disabling
* Refactor and cleanup mfa management code
* More cleanup
* Handle errors in the management action
* Implement mfa login flow
* recovery code input should be password
* Implement rate limiting on the mfa validation endpoint
* Better error ux
* Implement mfa emails and apply James' updates
* Use latest @better-auth/utils
* Improvements via CodeRabbit review
---------
Co-authored-by: James Ritchie <james@trigger.dev>
* Add retry logic for insert operations
Add a generic retry mechanism for task run and payload inserts to handle
transient connection errors. The new #insertWithRetry method retries up to
three times with exponential backoff and jitter on retryable connection
errors such as connection resets or timeouts. Errors are logged and
recorded in tracing spans to improve observability and robustness of the
replication service.
* Replication settings are configurable
* Log out the runIds for failed batches
* Detecting bad JSON in run replication and ignoring it
* Reproduced split unicode error
* Move output file
* Massively improved the performance
* Minor performance improvements
* Unskip tests
* Remove unused test in CH package
* Fix for the ClickHouse UI explorer
* RunReplication keepAlive defaults to false
* Add concurrency_key and bulk_action_group_ids to ClickHouse task runs
* ClickHouse package doesn't need to be built anymore for the webapp
* Set the concurrency_key from the run replication service
* Initial preview migrations
* Modified the staging endpoint to create preview environments
* Added isBranchableEnvironment to RuntimeEnvironment
* Staging = yellow Preview = orange
* Changed the env sort order
* Set isBranchableEnvironment correctly. Create preview for new projects
* Very basic branch menu
* Creating branches from the dashboard
* Fix for string icons on project delete page
* Don’t show branch API keys
* WIP on the manage branches page
* RuntimeEnvironment added projectId index
* Only create the parentEnvironmentId column if it doesn’t exist already
* Improved the limit wording
* Add search to the branch list
* contains in both places
* Many style improvements
* Branch dropdown and v4 badge
* Arching/unarchive branches working in the dashboard
* Tidied imports
* Change preview slug from `prev` to `preview`
* Use correct color for side menu preview branch icon
* Upsert the branch and use the shortcode as a unique constraint
* Upserting working with nice messages in the dashboard
* Better errors when upserting branches
* Button shortcut, don’t allow event to propagate
* Better duplicate error message
* Filter out archived branches from the env selector
* Archiving/creating tweaked some more
* Add an archived banner to the app, fixes for archived branches and upsells
* Fixed pagination
* Disable editing schedules, pausing queues, testing tasks
* Don’t allow replaying if the env is archived
* When deploying detect the correct environment
* Get the projectClient when there’s a branch
* createGitMeta function, most code from the vercel CLI repo
* Deploy, getting the correct environment client
* Added git column to WorkerDeployment
* Add GitMeta to core schemas
* Create branch when deploying
* WIP on branch support in the API
* Delete old createTaskRunAttempt fn
* apiAuth remove export from internal functions
* Rename env var to “TRIGGER_PREVIEW_BRANCH”
* Add TRIGGER_PREVIEW_BRANCH to resolved env vars for runs
* First preview deploy and run working
* Set the preview branch in the main SDK
* Added git links to the preview branches table
* Better errors when replaying/testing archived branches
* Don’t dequeue archived environments
* Env var resolution with parent environment
* Hello world default machine small-2x to save my memory
* Fix for more env var functions
* Only return non-archived envs
* Switch to controlled state for the checkboxes
* Uncheck everything when PREVIEW is checked
* WIP on branch UI
* Show the preview branch label on the env vars list
* Fix for overriding env vars
* Adding preview branch env vars working
* Progress on new env vars
* Only allow selecting a single branch
* Layout fix when there are errors
* Set the defaultValue so there are some fields
* Conform fix for team invite page
* Archived environments don’t run scheduled tasks
* Added Git data to deployments
* Added git data to the deployment inspector
* Don’t allow upserting schedules when archived
* Deduplicate and blacklist some env vars
* Fix for wrong conform function being used
* Show a better error if all vars were blacklisted
* Added environment variable search (by key and value)
* Improved preview branch icon
* Replay now supports branches
* Schedule page render branches properly
* Show the env icon in bottom-left of the test page
* When editing older schedules (that have multi-env) show preview branches correctly
* Fix for incorrect disallowed branch name character
* Extract and improve the directory verification code
* WIP for CLI preview archive command
* Improved the preview branch action buttons
* Redirect to the project if we don’t find a matching env
* Archiving branch via the CLI working
* Fix for archiving branches
* Public access token test task
* JWTs working are with preview branches
* Add branch and git data to the Run ctx
* Updated GitMeta functions to work in CI
* Added pullRequestState
* Archive when deploying if the PR is closed/merged
* Fix for the changesets guide
* Fix for CLI dev bug introduced
* CLI promote now supports preview branches
* Add PR title. Reordered them and added tooltips
* syncEnvVars working with branches
* Added preview branch support to syncVercelEnvVars()
* Detect the branch from Vercel env var (set during build)
* Allow passing a branch in
* Use process.env.VERCEL_TOKEN as well… this used in Vercel CI
* Temp delete
* Improved regenerate api key modal
* Added Accordion component (with styles)
* Redesigned the API keys page
* Revert "Temp delete"
This reverts commit 177b92cd935a6161456bde65d01294e23ecfd47f.
* Changeset
* Fixed docs link
* The new branch panel closes when a branch is created
* Update apps/webapp/app/services/upsertBranch.server.ts
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
* Removed findUniques from WorkerGroupTokenService
* Made the parentEnvironmentId migrations safe
* Latest lockfile
* Update packages/cli-v3/src/commands/workers/build.ts
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
* Move isValidGitBranchName to a separate file
* Move the sanitize fn too
* removeBlacklistedVariables moved to a separate file
* Moved deduplicateVariableArray to a separate file…
* Fix broken sanitizeBranchName import
* Another import fix…
* Improved blacklisted error message
* SImplified migration to use `ADD COLUMN IF NOT EXISTS "parentEnvironmentId" TEXT`
---------
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
* WIP clickhouse package with test containers setup
* More clickhouse client setup now with otel and real tests, and the v1 of raw run events
* Add some additional columns to raw_run_events_v1
* WIP runs dashboard service
* Create a new run engine event bus event for the runs dashboard to hook into
* Track run events in the run engine
* make sure engine v1 runs get synced to CH
* Update the attemptNumber of v3 task runs
* Restructure the run events to be more sparse
* emit more stuff
* Setup replication package
* scaffold the replication package
* replication wip
* resolve conflicts
* more replication stuff
* Add ability to drop the replication slot completely on teardown
* Use the new single replacingmergetree task events table for replication
* get it working
* insert payloads into their own table only on insert and then join
* prepare for using clickhouse cloud and now running ch migrations during boot in the entrypoint.sh
* Handover WIP and tests
* Testing the replication service
* Remove the runs dashboard stuff that we aren't using anymore
* Added a test for large payloads
* hacky typecheck fix
* Fix new internal package typecheck issues and start adding telemetry to the replication service
* tracing over spans, some other improvements
* Improvements to the runs replication service, now ready for testing
* Some fixes and cleanups
* Don't need this code anymore
* move transaction types into the runs replication service
* only send spans where there are transaction events
* A couple of suggested tweaks
* set correct worker name on create
* dev: kill run process on failed cancel attempt
* add missing prod timeline events
* add more debug logs after api call failures
* import events are always visible
* set dequeued at on new runners
* skip fork event test
* fix timeline test for import event
* v4: eagerly fork child process before warm start
Also fixes an issue where the attempt span events weren't coming through in the partial spans
* Making flushing a safer operation
* Add changeset
* Improve attempt metrics when in a warm start, and pass the STYLE_VARIANT for a warm or cold start on attempt spans
* New time period filter (permanently displayed)
* Batches, and fix for blank state
* Waitpoint token filtering
* Tags query: remove things we’re not using
* Run tag and waitpoint tags use startsWith for faster search
* Fix for run page on wrong env. Added schedule last triggered column
* Removed the Redis pubsub, just use the presence key
* Improve the dev presence responsiveness
* The CLI presence connection recovers when the webapp is restarted
* Dev schedules are now working for v4
* Refactored to make the dev presence stuff
* Got rid of stupid extra /dev, added a connecting state with icon
* Remove unused Redis client
* Added waitpoints/tokens to the sidebar
* Added indexes to the Waitpoint time for filtering
* Begun work on `WaitpointTokenListPresenter`, the pag is a copy of the Queues page for now
* MVP of waitpoint token page
* Added status
* Expiry of timeout/ttl
* Improvements to the waitpoint table
* Improved columns and icon
* Changes from the RunTag copy on hover branch
* Fix for nested button error
* Added waitpoint tags to the DB/table
* Applied Eric’s task run tag fix (it’s live on prod in the legacy run engine branch)
* Added tags to waitpoints
* Removed todos that have been done
* Added token support for releaseConcurrency. Also added a ton of JSDocs
* Added releaseConcurrency to the API token endpoint…
* WIP on waitpoint page filters
* Fix for tags filtering
* Waitpoint filters working
* Fix for badly named function
* WaitpointPresenter used from SpanPresenter
* Waitpoint detail panel WIP
* Fix for server client hydration issue with CodeBlock
* Selected waitpoint panel
* Added a blank state
* Added waitpoint docs link
* Fix for animated number going past the target
* Fix for the queue list pagination and upgrade status
* Engine version error for waitpoint token list
* RunTag component doesn’t get squished and hover behaviour is nicer
* Associating runs with waitpoints
* Added triggered icon
* Link directly to the waitpoint
* Fix for TS error on waitpoint retrieve
* Added CopyableText component, used for waitpoint id in the table
* Removed the confetti 🎊
* Deleted some old images
* Moved some schemas/types to core. Use `id` instead of `friendlyId`
* Added wait.listTokens() function. Made some changes to the types to make it nicer
* WIP wait.retrieveToken()
* wait.retrieveToken working
* Added data to retrieve token
* Separate ApiWaitpointPresenter completely
* Added completed time to the waitpoint detail panel
* Fix for the Avatar component having SSR issues. Specify the size in rems and removed the useLayoutEffect
* Fix for applied idempotency key filter dropdown showing the id field
* Use parentheses to make sure the token list query respects idempotency key correctly
* Use the proper logger, and have a decent message with info to track the bug down
* Pass the org title into the Avatar
* Better error when failing to creating a manual waitpoint after X attempts
* Delete the proxy app (was v2)
* Delete RunPresenterElectric
* Select the best proj/org/env
* Storing current proj/env in DB. Initial selection logic working with tasks page
* 2sm needed to be in the Tailwind merge list
* Move the task stream route (although we don’t actually use the env for now)
* Alerts moved from /v3
* API keys page moved from /v3
* Concurrency page moved from /v3
* WIP on side menu sections
* Improved the accordion animation
* Moved schedules from /v3
* More pages moved
* Move pages working
* Run page working
* Schedules working
* Moved deployments
* Alert pages moved
* Delete electric hooks, not used
* Started setting up blank states
* Test page working
* Removed “Select task” from the test page
* Some work on deployment page
* Style tweaks
* Redirect from project root to approriate env
* Improved env selector styling
* Fix for jsx errors
* Better min width on env selector
* Improved the env switching logic
* Added deployments to env routing
* Redirect deployments to the correct env
* Redirect run from proj to env
* JSX icon fix
* Only allow single env schedules from now on
* Remove env var count from the API keys page
* Move improvements and redirects
* Project settings moved
* Fix for scroll area on test page
* Tweaked the test design
* Made recent payloads column narrower
* Improved the test layout some more
* Added org icon, new project selector menu
* WIP on org switching menu
* Org switching is working
* New menu working well, removed old side menu items
* Buttons can now have a component name or an actual component for their icons
* Removed the Projects page, instead redirect appropriately
* Fix for broken blank states
* Minor run table improvements
* Removed unused switcher log and logic
* Concurrency page fix for invalid html, improved layout
* Minor improvements
* Moved the side menu to the project level
* Improved account styling
* Moved org settings pages (with redirects)
* Add current plan to billing side menu link
* Upgrade to get staging from env dropdown
* New env badge on concurrency limits page
* Show Run Engine version in span presenter
* New promote icon
* Concurrency limits page is the sum of engine v1 + v2 queues
* Fix for missing batch import
* Added currentConcurrencyOfEnvQueue function
* Basic avatar setting working
* Avatar setting is working
* You can change the color of your icon
* Avatar improvements
* Bugfix for mising prop
* Removed some old env badges
* Fixed replaying
* Removed EnvironmentLabel
* Old env badge deleted, changed everywhere to the new one
* Fix for Slack integration paths
* Fix for waitpoint completion form moving
* Bulk replay/cancel env fix
* Fix for alert webhook path
* Redirect projects/v3/* to project/*
* Fixes for CLI redirect routes
* Remove welcome email (unused)
* Change how we count schedules towards your limits
* Use new schedules limits when checking a schedule
* Added projectId back in to task queries (indexes)
* WIP dev presence
* CLI modal
* Moved things around and use Context
* Fix for p inside p
* Dev connected status on run page
* Correct dev env (not a teammates)
* Show disconnected message at the end
* Minor tweak on project dropdown icon padding
* Fix for inconsistent date format for presence
* Added a message when pushing to the billing page
* Center the team page
* Project settings page centered
* Improvements to the dev presence
* Delete v2 Stripe routes
* Delete v2 billing/usage pages
* Delete v2 integration pages
* Delete v2 project pages
* Deleted a load of components and services
* Deleted a load more components, presenters and services
* Deleted another 100 files or so…
* Removed old v2 paths
* Removed named icons from form titles
* Removed more string icons
* Delete NamedIcon
* Fixed some type errors
* Delete endpointApi
* Removed v2 from core/sdk
* Post merge fixes
* added explicit return types
* using the new sdk export without v3
* Delete old v2 file
* Added explicit return types because TS was complaining…
* Don’t export RuntimeEnvironmentType from two core files. Was causing TS issue
* Fix for removal of NamedIcon in new route
* Removed strange eslintrc rule
* Use the new redis client
---------
Co-authored-by: James Ritchie <james@trigger.dev>
* re2: fix @trigger.dev/core exports
* re2: WIP env based queue selection algo
* more wip
* WIP
* Get run engine tests to pass
* Adding tests for the fair dequeueing strat in the run engine
* Configure the new queue selection strategy in the webapp and get it all building and typechecks passing
* webapp now uses built packages, building redis-worker, run-engine, database, using better tsconfig setups for tests, moving isomorphic code into core/v3/isomorphic
* Fixed webapp typechecks
* dev now depends on build, fixed supervisor typecheck
* Fixed run engine tests
* Fixed e2e tests
* In dev, the worker group is optional when triggering tasks (the master queue is defined by the environment). Also deprecated the TaskEvent.isDebug column and using TaskEventKind.LOG instead for debug events
* Fixed a couple of type issues
* More type fixes
* bump worker version
* Suggested glossary for the RunEngine, TBC
* Removed BatchTaskRun changes from this branch, they were done in main
* Set the BatchTaskRun status to completed when all runs are completed
* When dequeuing respect passed in maxResources
* Ported over the new run props: idempotencyKeyExpiresAt, versions, oneTimeUseToken, maxDurationInSeconds
* Didn’t hit save… the new props when triggering tasks passed through
* Idempotency expiration + waitpoint edge case
* WIP on creating checkpoint, parking for now
* fix worker routes
* upgrade webapp node types to support generic event emitter
* separate event bus handler singleton and run failure alerts
* duration waits
* fix execution snapshot debug spans
* task waits
* fix event bus types
* temporary fix for react hook run handle type
* disable run notifications for now
* convert any typecasts to expect errors to more easily fix later
* fix webapp types after node types upgrade
* updateEnvConcurrencyLimits across marqs and the runqueue
* Pass proper values into the run engine
* RunQueue settings and removed unused rebalancing workers
* Remove rebalancing prop
* Tidied more things up
* Update/remove queue limits for MARQS and RunQueue
* taskQueue/concurrencyLimit changes ported back into the RunEngine
* Reworked completing waitpoints to improve performance and reduce race conditions
* Improved test robustness
* Down to a single run lock only when a run is totally unblocked and ready to continue
* warm starts, worker notifications, wait fixes
* Fix for Run Engine poll interval env var
* Expect the waitpoint to be completed quickly
* If a run is locked then it’s too late to expire it
* Added VALKEY_ env vars and plugged them into the run engine
* Extracted and updated the guard queue function so it can be used when batching
* Added logging and universal concurrency changes to trigger task v1
* Added notes back in
* Bump @trigger.dev/worker to 3.3.7
* reportInvocationUsage for the runAttemptStarted event
* improve execution snapshot span debug span start times
* Unfriendly IDs
* update lockfile
* Created a shared determineEngineVersion function
* disable unfinished commands
* save new cli config to different location, misc fixes
* add basic engine version check via current deploy
* new run engine will default to node 22 runtime
* block some actions for projects on previous run engine
* fix worker group tests
* fix triggerAndWait test
* one typescript version to rule them all
* redlock type patch
* fix type issues caused by ts-reset
* improve cleanup scripts
* add missing socket.io dep
* fix run notification handler type
* fix worker group test again
* generate prisma client for e2e tests
* remove worker group tests for now
* prevent image pull rate limits during unit tests
* increase timeout for queue concurrency limit test
* generate prisma client for preview release
* same node types everywhere
* Updated engine readme, removed legacy system notes
* use default machine preset from platform package
* worker instances plural in schema
* disable pnpm update notifications
* return worker group details from connect call
* add workers admin route
* fix heartbeat route return type
* move deployment labels to core apps
* refactor run controller env schema
* Add firstAttemptStartedAt to TaskRun
* RunEngine 2.0 batch trigger support (#1581)
* Make it clear when BatchTriggerV2Service is used
* Copy of BatchTriggerV2Service
* WIP batch triggering
* Allow blocking a run with multiple waitpoints at once. Made it atomic
* Removed unused param
* New batch service
* Pass through the parentRunId and resumeParentOnCompletion
* Use the new batch service, and correct trigger task version
* Force V1 engine if using BatchTriggerV2Service, we’ve already done the check at this point
* Removed the $transaction and early exit if nothing changed
* Adedd a simple batch task to the hello world reference catalog
* Fix for batch waits not working
* Added parentRunId in a couple more places
* Removed waitForBatch log
* Added another parentRunId
* Expanded the example to include all the different triggers
* More changes to blocking to support continuing after idempotent completed runs
* Fix for the wrong type when blocking a run
* remove @map
* optimise worker auth query
* add engine version header to core api client requests
* remove unique constraint for default group id
* consolidate migrations
* the first managed worker becomes the global default
* Debug events off by default, added an admin toggle to show them
* worker group name can't be an empty string
* add exec helper to core
* move machine resources to core
* add pre-dequeue callback to determine max resources
* optionally skip dequeue
* bump worker package
* move worker to core
* fix ReadableStream type error
* fix another type issue
* update a few more tsconfigs
* add metadata changes introduced in #1563
* Run Engine 2.0 trigger idempotency (#1613)
* Return isCached from the trigger API endpoint
* Fix for the wrong type when blocking a run
* Render the idempotent run in the inspector
* Event repository for idempotency
* Debug events off by default, added an admin toggle to show them
* triggerAndWait idempotency span
* Some improvements to the reference idempotency task
* Removed the cached tracing from the SDK
* Server-side creating cached span
* Improved idempotency test task
* Create cached task spans in a better way
* Idempotency span support inc batch trigger
* Simplified how the spans are done, using more of the existing code
* Improved the idempotency test task
* Added Waitpoint Batch type, add to TaskRunWaitpoint with order
* Pass batch ids through to the run engine when triggering
* Added batchIndex
* Better batch support in the run engine
* Added settings to batch trigger service, before major overhaul
* Allow the longer run/batch ids in the filters
* Changed how batching works, includes breaking changes in CLI
* Removed batch idempotency because it gets put on the runs instead
* Added `runs` to the batch.retrieve call/API
* Set firstAttemptStartedAt when creating the first attempt
* Do nothing when receiving a BATCH waitpoint
* Some fixes in the new batch trigger service… mostly just passing missing optional params through
* Tweaked the idempotency test task for more situations
* Only block with a batch if it’s a batchTriggerAndWait… 🤦♂️
* Added another case to the idempotency test task: multiple of the same idempotencyKey in a single batch
* Support for the same run multiple times in the same batch
* Small tweaks
* Make sure to complete batches, even if they’re not andWait ones
* Export RunDuplicateIdempotencyKeyError from the run engine
* Latest lockfile
* Trigger with a machine (old run engine)
* RE2, allow setting machine when triggering
* Fix for new glob patterns
* add max run count to dequeue from version route
* add worker instance name env var and header
* queue consumer pre skip callback
* poll for more runs after final execution errors
* fix dequeue search param schema
* add shortcut to debug switch
* expose run engine timeouts as env vars
* make warm start durations configurable
* add optional status to json reply helper
* fix preSkip hook, add debug logs
* BLOCKED_BY_WAITPOINTS -> SUSPENDED
* exit controller when run suspended
* check if already replied before http reply
* run controller will wait for next run after the current one is suspended
* cancel run button shortcut
* minimal event repository environment type
* fix update metadata call
* run suspension and misc fixes wip
* change debug shortcut to shift + D
* Started work on the Dev supervisor
* Formatting
* Fix for bad imports
* Before rebuilding SSE
* Presence updating from the CLI working via SSE
* add worker notification debug logs
* send run:stop when exiting run phase
* skip current snapshot poll on worker notification
* add more logs and route to submit run debug logs
* add worker and runner ids to snapshots
* improve run notification debug logs
* add workload debug log route
* misc run controller fixes and refactor
* prevent parallel execution of critical functions
* update bun to 1.2.1
* WIP with dev dequeuing
* Method to convert friendlyIds to non-friendly, do nothing with actual ids
* Set the engine on BackgroundWorker, lazily upgrade projects to engine V2
* Runs with ttls were getting immediately expired… oops.
* Pass the Waiting for deploy reason through, so we have it on the execution snapshots
* Fixed the logic for getting the right background worker for a run
* Use the correct ID when dequeuing…
* determineEngineVersion is now fully functional
* Rate limiter ignores the dev endpoints
* Retrieving a batch gives you the runIds
* Set a unique version for the RE2 BatchTaskRun
* add provisional changeset
* The start of dev run execution is working
* First dev run working
* Moved the dev run controller closer to what Nick did with the managed one
* export exec output type
* Heartbeat fix: don’t heartbeat if _isHeartbeating == false
* Dev runs get notifications, some dev bug fixes
* Improved logging or dequeuing
* We need to dequeue runs from the latest version too, for triggerAndWait
* Ported Eric’s validateWorkerManifest with nicer errors
* When flattening an idempotency key if part is undefined, return undefined
* Dev logging fixes
* Remove sigterm listener
* Deprecating workers. Don’t specify a BackgroundWorker when dequeuing an environment
* Deleted some old files. Renamed “managed” to “deploy”
* When a build finishes, always copy the build dir (otherwise the first one gets trampled on by the 2nd)
* Dev master queues should work differently
* Deleting old workers
* Added debounce function to core
* Improvement to canceling
* WIP on debounce canceling on socket disconnection
* Added environment data to execution snapshots
* Dev runs that have stalled get “Canceled” with a reason explaining why
* Show CLI messaged when a connection to the platform is lost/restored
* Fix TriggerTask after merge
* Add trigger task v2 max attempts, replace some findUniques
* Port the new queue logic to the run engine
* More fixes post-merge
* We weren’t setting a `retryConfig` up for the tests… it’s now required
* Start the Redis worker inside the Run Engine… 🤦♂️
* Trying to make the testcontainers more reliable
* Added keyPrefix: "engine:”
* Badly placed bracket in trigger task
* Better Redis namespacing
* Fix for expired run not getting removed from the queue
* Don’t create a redis client in the testcontainers, return the redisOptions instead
* Cleanup redis client in the run lock tests
* Fix for the RunQueue not supporting keyPrefix
* Updated more of the RunQueue scripts rebalancing
* Trying to make Redis more robust in the tests…
* Improved test resiliciency more
* Fix for delays (checkpoint check)
* Increase the timeout slightly to fix ttl test
* Added priority support when triggering
* More wip trying to make test containers more reliable
* batchTriggerAndWait test is still failing… some wip to try fix it
* Fixed redis tests now we’re not providing a client
* Separate Redis clients for the run engine worker/queue/runlock
* Made the wait for duration test more resilient
* Added idempotencyKeyExpiresAt to Waitpoints
* Waitpoint timeouts and idempotency expiry
* Use finishWaitpoint, removed extra worker job
* Added waitpoint idempotency tests
* Creating resume tokens is working
* Some improvements to the resume tokens
* Moved resumeTokens to just be wait functions 🥳
* Delete old RuntimeManagers
* Wait for token is working
* Better test for the wait tokens
* Improved the test task some more
* Hide the accessories in the span inspector
* WIP on waitpoint inspector
* WIP on complete waitpoint form
* Span overview panel can be changed based on the entity type
* Improved the waitpoint display
* WIP on completing waitpoint form
* Use the existing CodeBlock for the tip
* Style improvements
* Complete waitpoint
* All waitpoint sidebar variants
* Waits now use a pause icon
* Durations waits use the API to create/block with a waitpoint, not the runtime
* Fix for engine.blockRunWithWaitpoint required org id
* Removed old wait code from the run controllers/task run process
* Form action for skipping a datetime waitpoint
* Move testDockerCheckpoint to a separate core package export (it can’t be bundled on the client)
* Fix for glitchy hourglass animation
* Completed waitpoints display better
* Increase Redis maxRetriesPerRequest to 20 (default)
* Completing and skipping waitpoints is working
* Remove the database prisma dev command, since we need to use create only now. Updated docs
* Added skip timeout, reworked the UI
* Tweaked spacing
* Added payload limit to waitpoint token completion from dashboard
* Test idempotency works on wait.for and wait.until
* Moved the worker-actions to /engine/ from /api/
* Moved dev engine endpoints to /engine/ from /api/
* Separate /engine/ rate limiter
* Added parallel wait prevention, it’s working for duration waits but not well for triggerAndWait yet
* WIP post-merge conflicts
* Set taskEventStore column in the new engine
* Remove duplicate keys
* Post-merge fixes
* Fix for span merge layout
* Use executedAt instead of firstAttemptStartedAt
---------
Co-authored-by: Matt Aitken <matt@mattaitken.com>
James Ritchie
