Commit Graph

4112 Commits

Author SHA1 Message Date
Saadi Myftija cc94d121f2 feat: installing status for deployments (#2544)
* Add installing status to the deployment db schema

* Replace the deployments /start endpoint with /progress

* Show the installing status in the dashboard

* Add installing status to the api schema and cli

* Add changeset
2025-09-24 10:27:43 +02:00
Saadi Myftija 412e80fdde fix(webapp): hide outdated connected repos from deleted installations (#2538) 2025-09-23 13:24:58 +01:00
Saadi Myftija 49728b5a5f feat(api): defer remote build creation for pending deployments (#2536)
Depot builds have short-lived tokens and their TTL is not exposed in the SDK. As queued deployments can stay in the queue for an arbitrary amount of time, deferring the remote build creation helps avoid expired Depot token issues.
2025-09-22 13:01:56 +02:00
Saadi Myftija d45696c000 fix(api): 204 response issue in deployment start endpoint (#2534) 2025-09-19 19:05:18 +02:00
Eric Allam 8313800746 fix(webapp): don't override spans from ancestors unless the span is partial (#2532) 2025-09-19 14:58:53 +01:00
James Ritchie 28f8cee3a4 Fix for errors returned from searching time specific queries (#2525)
* Fix for errors returned from searching time specific queries

* Adds prompt patterns
2025-09-19 13:49:43 +01:00
Eric Allam 87b3603b23 feat(webapp): completing spans server-side no longer write-after-read, improving efficiency and perf (#2530)
* Cancel run events which then propogate cancellation status to span ancestors

* WIP

* convert closing cached run spans to new system

* converted expired complete span event to new method

* move v3 over to new methods

* Convert getDetailedTraceSummary to use the new ancestor override stuff

* remove debug logs

* Don't return UNSPECIFIED task events in getRunEvents

* fix the call site for cancelling run event in v3

* Add changeset

* remove methods
2025-09-19 13:39:48 +01:00
James Ritchie 365adc24a6 Chore(webapp): adds more copy buttons (#2529)
* Pass up asChild

* Include padding when asChild specified

* Adds copy buttons to useful data in Details tab

* Improve view batch tooltip message
2025-09-19 13:02:10 +01:00
Saadi Myftija 7d17730b52 feat(api): handle build server deployment init gracefully for older cli versionsi (#2526)
This PR adapts the deployment initialization endpoint to handle build server deployments with older CLI versions gracefully.

When we introduced automatic deployments via the build server, we slightly changed the deployment flow
mainly in the initialization and starting step: now deployments are first initialized in the `PENDING` status
and updated to `BUILDING` once the build server dequeues the build job.
Newer versions of the `deploy` command in the CLI will automatically attach to the existing deployment
and continue with the build process. For older versions, we can't change the command's client-side behavior,
so we need to handle this case here in the initialization endpoint. As we control the env variables which
the git meta is extracted from in the build server, we can use those to pass the existing deployment ID
to this endpoint. This doesn't affect the git meta on the deployment as it is set prior to this step using the
/start endpoint. It's a rather hacky solution, but it will do for now as it enables us to avoid degrading the
build server experience for users with older CLI versions. We'll eventually be able to remove this workaround
once we stop supporting 3.x CLI versions.
2025-09-19 10:40:59 +02:00
Saadi Myftija e4982bfd6d feat(webapp): deployments page live reloading (#2524)
* Fix `current` badge inconsistency in the deployment details page

* Add custom hook for auto revalidation based on an interval and/or focus change

* Use the autoRevalidate hook for live reloading of the deployments page

* Extract autoReloadPollIntervalMs to an env var

* Replace the sse-based autoreload in bulk actions and queues page with the simpler autoRevalidate hook
2025-09-19 10:40:07 +02:00
nicktrn db87295049 fix(runner): reduce restore recovery time and deprecated runner false positives (#2523)
* fix(runner): improve restore detection

* chore(supervisor): skip schema parsing when debug logs disabled

* fix(runner): deprecation race condition

* add changeset
2025-09-18 16:59:44 +01:00
Saadi Myftija a3ef6ea236 feat: separate deployment initialize and start steps (#2522)
* Enable setting the initial status on deployment creation

* Expose endpoint to start deployments

* Extend build timeout on deployment start

* Use separate timeout value for queued deployments

* Add startedAt to the deployment schema

* Show the new startedAt instead of createdAt in the dashboard

* Show github user tag also in the deployment details page

* Show `pending` deployment status as `queued` in the dashboard

* Apply some good 🐰 suggestions

* Add missing return
2025-09-18 16:12:17 +02:00
James Ritchie ae22000409 chore(webapp): pricing page improvements (#2457)
* Improve styling of onboarding pricing plans

* type only import

* Improve the onboarding plan page so it scrolls on smaller screens

* Adds additional pricing for bolt-ons for the Pro plan

* fix text wrapping issue

* add bg color back in
2025-09-18 14:31:06 +01:00
James Ritchie 2bbf8ec6cd chore(webapp): a new cancelation reason (#2465)
* Adds a new cancelation option

* Better wording
2025-09-18 14:30:25 +01:00
James Ritchie c8858edf0a New jump to parent or root run buttons (#2067)
* Change the color to indigo

* Pro tier pricing information now matches the marketing site

* Update the button styles to secondary

* WIP adding separate links to Parent and Root runs

* TextLink now supports optional shortcuts

* Adds shortcut keys to the root and parent links + the shortcut help panel

* Adds new icons for root and parent

* root friendlyId works

* Updates icons for jump to root and parent

* Copy tweak

* Improve how the Free tier shows no preview branches

* Improve the wording in the tooltip

* Align the x icon better

* Show price for additional preview branches

* Change the shortcut key

* Fixes button alignment

* Adds nested dependencies task hello-world

* Fixes typo “Cancelled”

* Removes taskIdentifier, not needed

* Removes unused taskIdentifier
2025-09-18 14:29:47 +01:00
Eric Allam 9c087646bf fix(engine): prevent race condition that prevents triggerAndWait runs from resuming by atomically creating associated waitpoint records (#2519) 2025-09-17 13:58:17 +01:00
Saadi Myftija 691903cf58 feat(api): accept OATs in preview branch related endpoints (#2517)
Adjusts the authentication in a couple of endpoints to accept OATs too.
2025-09-17 14:10:25 +02:00
Saadi Myftija 501a383bcd feat: expose project build settings (#2507)
This PR enables setting project build settings in the settings page:
root directory, install command and trigger config file path.

For most cases there should be no need to set these explicitly.
2025-09-16 15:02:09 +02:00
nicktrn 3c199e6d9c chore(supervisor): remove deprecated route (#2513) 2025-09-16 13:47:07 +01:00
Eric Allam 0f9b83db09 fix(core): prettyPrintingPacket will now do a structuredClone on non-circular references instead of outputting [Circular] (#2508)
* Mo-Stashed changes

* fix(core): prettyPrintingPacket will now do a structuredClone on non-circular references instead of outputting [Circular]

This also fixes an issue with replaying of runs that include 
non-circular references
2025-09-15 17:43:46 +01:00
Saadi Myftija 7d333e5b3c feat(cli): attach to existing deployments in the build server (#2501)
* Add external build data and image platform to the get deployment endpoint

* If provided, attach to an existing deployment in the deploy command

* Check status for existing deployments

* Add changeset
2025-09-15 17:16:26 +02:00
nicktrn 7c4ce6f76b feat(supervisor): add optional memory limit overhead (#2506) 2025-09-15 16:09:05 +01:00
Saadi Myftija b0b88f1e05 fix(webapp): show deployments where triggeredById is missing (#2504)
Small fix for the sql query used in the deployments list page.
2025-09-15 14:12:48 +01:00
Eric Allam 2eddda1233 fix(webapp): worker actions now catch service validation errors and respond properly (#2481)
This also stops all the unnecessary error logging when throwing 
ServiceValidationErrors
2025-09-12 14:56:54 +01:00
HUORT Louis 49f2c54031 fix: webapp crash on clickhouse data corruption (#2491) 2025-09-12 14:55:29 +01:00
Eric Allam f077d49291 feat(engine): Improve execution stalls troubleshooting, align dev and prod behavior, adding heartbeats.yield utility (#2489)
* feat(engine): Improve execution stalls troubleshooting, align dev and prod behavior, adding heartbeats.yield utility

* A few improvements via the 🐇 review

* Allow treating EXECUTION stalls as OOM errors, improve the error message, add more information to the docs, improve resource monitor and add it to the docs

* Add changeset
2025-09-12 14:54:39 +01:00
Saadi Myftija 5567f49846 feat(webapp): expose project git settings (#2464)
* Fix settigns page delete project width issue

* Apply a couple of touch-ups to the project settings page

* Add UI flow to connect gh repos

* Enabling adding another gh account in the ui

* Enable connecting a repo to a project

* Enable updating git settings

* Enable disconnecting gh repos from a project

* Remove prisma migration drifts

* Hide git settings when github app is disabled

* Fix migration order

* Avoid using `location` to avoid SSR issues

* Make branch tracking optional

* Disable save buttons when there are no field changes

* Disable delete project button unless the input matches the project slug

* Show connected repo connectedAt date

* Check that tracking branch exists when updating git settings

* Show tracking branch hint in the deployments page

* Fix positioning issue of the pagination pane in the deployments page

* Use mono font for branch names

* Add link to git settings

* Show tracking branch hint for the preview env too

* Add a confirmation prompt on repo disconnect

* Add link to configure repo access in gh

* Add rel prop to github links

* Automatically open repo connection modal after app installation

* Apply some fixes suggested by mr rabbit

* Fix flash cookie issue

* Extract project settings actions into a service

* Extract project settings loader into a presenter service

* Introduce neverthrow for error handling

* Try out neverthrow for error handling in the project setting flows

* Move env gh branch resolution to the presenter service
2025-09-09 13:03:43 +02:00
nicktrn 99660112bd feat(supervisor): add configurable resource requests (#2474) 2025-09-04 17:09:34 +01:00
Eric Allam 00d32ed4ee feat(webapp): add support for running web services (api, engine, webapp) in cluster mode for better perf (#2472)
* feat(webapp): add support for running web services (api, engine, webapp) in cluster mode for better perf

* cleaned up signal handling and resolved some valid 🐇 issues
2025-09-04 09:53:15 +01:00
nicktrn a1e9738faa fix(webapp): prevent duplicate preview env image tags (#2475)
* fix(webapp): prevent duplicate preview env image tags

* use deploy shortcode instead of new nanoid

* replace regexps in tests
2025-09-04 09:37:22 +01:00
Eric Allam ed23615aa4 perf(webapp): add event loop utilization metric (#2471)
* perf(webapp): add event loop utilization metric

* add event loop utilization logging as well
2025-09-03 11:39:03 +01:00
Saadi Myftija 436d951b65 feat(webapp): github app installation flow (#2463)
* Add schemas for gh app installations

* Implement gh app installation flow

* Make the gh app configs optional

* Add additional org check on gh app installation callback

* Save account handle and repo default branch on install

* Do repo hard deletes in favor of simplicity

* Disable github app by default

* Fix gh env schema union issue

* Use octokit's iterator for paginating repos

* Parse gh app install callback with a discriminated union

* Remove duplicate env vars

* Use bigint for github integer IDs

* Sanitize redirect paths in the gh installation and auth flow

* Regenerate migration after rebase on main to fix ordering

* Handle gh install updates separately from new installs
2025-09-02 16:35:33 +02:00
nicktrn 0d1eac9406 feat(supervisor): dynamic queue consumer pool (#2461)
* feat(supervisor): dynamic queue consumer pool

* add changeset

* fix: correctly handle zero median and even samples

* feat(supervisor): consumer pool metrics

* fix tests

* more tests and fixes

* decrease default scaling cooldowns

* don't treat initial pool size as scale up

* handle scale down when queue length drops to zero

* remove changeset, supervisor changes only

* add damping factor env var
2025-09-02 13:52:42 +01:00
nicktrn 2b095b1072 chore(webapp): upgrade otel packages and add more metrics (#2458)
* feat(webapp): upgrade otel packages and add more metrics

* add env var to disable additional detectors

* expose more prisma metrics

* chore(webapp): drop node 16 support
2025-09-01 16:24:12 +01:00
James Ritchie fee31f2dc0 chore(webapp): update region message to say we are GDPR compliant (#2455)
* Updates the Region message to say we are GDPR compliant

* Hide the message if not on cloud
2025-08-29 14:33:46 +01:00
James Ritchie 27bc2ad232 chore(webapp): show a regions info panel to explain about data residency (#2428) 2025-08-29 11:35:54 +01:00
James Ritchie 3a3e863daf chore(webapp): bulk action cancel as default (#2437)
* Switches “replay” and “cancel” options in the list

* Fixed incorrect cancel icon being used

* Fixes the alignment of the radiobuttons
2025-08-29 11:30:46 +01:00
Eric Allam f5f14c4206 chore(webapp): fix @trigger.dev/database imports so it doesn't end up in the frontend bundle (#2452) 2025-08-29 11:27:01 +01:00
nicktrn 73e7378459 feat(webapp): add ecs otel resource attributes (#2454)
* add attributes to runlock dequeue span

* feat(webapp): add ecs otel resource attributes
2025-08-29 11:05:47 +01:00
Eric Allam b0b0df69be fix(run-engine): prevent stalled SUSPENDED snapshots from endlessly retrying (#2448) 2025-08-28 14:25:50 +01:00
Saadi Myftija e629810e96 fix: dev env connection issue (#2449)
This PR fixes an issue introduced in #2391 where the dev runtime env was not being resolved correctly.
2025-08-28 14:05:10 +01:00
Eric Allam 82401ec4fe fix(run-engine): dequeue performance improvements and improved telemetry (#2446)
* more eager dequeuing, queue cooloff periods, return workerQueueLength when dequeueing

* Cache worker group authentication and remove old self-hosted worker code (only managed is currently supported)

* add additional spans during dequeue

* Add env vars and additional spans
2025-08-27 21:22:10 +01:00
Matt Aitken 24a915133e Prisma 6.14.0 upgrade (#2444)
* Initial work on upgrading to 6.14.0

Set the output to node_modules still to make it easier

* Use ./generated Prisma folder, update types to fix issues

* Docker compose restart Clickhouse

* Prisma instrumentation update

* Docker

* Removed database dockerignore file, add generated prisma client to the top-level one

* Delete v3-catalog package.json

* Resolved pnpm lock file

* Log errors for very slow queries
2025-08-27 16:52:58 +01:00
Saadi Myftija 1cc62230ab feat: introduce organization access tokens (#2391)
* Create schema and migration for organization access tokens

* Add helpers for creating and authenticating OATs

* Adapt the auth service to also accept OATs

* Accept OATs in the whoami v2 endpoint

* Enable deployments with the CLI using OATs

* Avoid reading env variables directly in the token utils

* Remove duplicate cli token utils

* Validate ENCRYPTION_KEY length when parsing env vars

* Make token utils a server-only module

* Disallow revoking already revoked OATs

* Simplify generics in authenticateRequest

* Use 32 bytes mock encryption key in the test setup

* Update dummy encryption key values in tests and templates

* Add a column in the OATs table to differentiate between user and system generated

* Simplify args for v3ProjectPath

Co-authored-by: Matt Aitken <matt@mattaitken.com>

* Add index on org id and createdAt

* Avoid storing the encrypted oat token and its obfuscated version in the DB at all

It is a safer approach. Also we do not need to ever read the decrypted token value after creation.

* Fix prisma update condition

* Add token type to the OAT table index

* Accept OATs in the mcp auth flow

* Simplify env auth flow around the /projects endpoints

---------

Co-authored-by: Matt Aitken <matt@mattaitken.com>
2025-08-27 13:23:45 +02:00
Eric Allam ad26cded99 fix(v3): prevent saturated queues from dominating dequeue attempts by adding a cooloff period of successive failed dequeues (#2439)
* fix(v3): prevent saturated queues from dominating dequeue attempts by adding a cooloff period of successive failed dequeues

* Add env vars and add a marqs shutdown on SIGTERM/INT
2025-08-22 14:37:21 +01:00
Eric Allam 5e4756f8b7 fix(v3): eagerly dequeue messages from a queue when that queue is added to or removed from (v4 backport) (#2438)
* fix(v3): eagerly dequeue messages from a queue when that queue is added to or removed from (v4 backport)

* fixed configuration
2025-08-22 13:23:34 +01:00
Eric Allam c16cc577b0 fix(v3): adopt v4 dequeue strategy to increase perf (#2436) 2025-08-22 07:40:59 +01:00
Eric Allam c7e3811794 fix(webapp): correctly associate event-loop-blocked spans with parent span context (#2435) 2025-08-21 16:21:26 +01:00
Eric Allam 895e24a0d7 fix(cli): various mcp server fixes (#2430)
* fix: span events are optional, as logs don't have them

* remove CI: true env var to prevent massive tool responses

* fix github resolver relative path

* prevent the dev command from asking for user input when not interactive

* Remove lodash.get because it's deprecated

* span output can be object other than an object, like a string

* prevent large traces from causing get_run_details failures

* Decreased max log lines, made it a tool input property

* Some install mcp tweaks
2025-08-21 13:57:14 +01:00
Matt Aitken 58ff034325 Add attemptDequeue span to marqs (#2433)
* Add attemptDequeue span to marqs

More telemetry to figure out why dequeuing is slow

* More spans

* use "receive" for all operation names
2025-08-21 13:09:08 +01:00