Commit Graph

1164 Commits

Author SHA1 Message Date
github-actions[bot] a49edd3251 chore: Update version for release (beta) (#1237)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-08-01 12:57:39 +01:00
nicktrn 8cae1d0875 v3: fix triggering tasks with custom queues (#1242)
* add missing dotenv requires in catalog scripts

* pass task queue options to all task trigger functions

* remove unnecessary include

* fallback to background worker task queue options

* add changeset
2024-08-01 11:26:34 +01:00
Matt Aitken 994ea7c3c8 Update the docs to reflect the new tags limit of 5 (up from 3) 2024-07-31 17:56:45 +01:00
Matt Aitken 09413a62a4 Improved run page and replaying (#1240)
* PropertyTable component changed to use sub components

* Separate run span component

* Early WIP on tabs that use search query

* Shortcut key tabs for the span panel

* The span timeline is working

* When runs get expired, update the OTEL event with an error

* Improved the expired error message

* Reveal env vars when editing

* Tightened things up a bit

* Added detail tab properties

* Progress dashed line

* Move the env label next to the Run number title

* Top level cancel/replay buttons

* Replay with a different payload and environment

* Fix for non json payloads

* Hide the clear/copy buttons

* UI improvements with large payloads

* Close the panels when you replay/cancel

* Added the new timeline to spans

* Use the u-turn left icon for replay

* Added an index for spanId on TaskRun

* Remove replay/cancel buttons the span view

* Replay shortcut works inside the code editor

* Split the log/span inspector between Overview and Detail as well

* More improvements to the inspector

* Context and output improvements

* Focus on run working

* Added version to run.ctx

* Added some padding to the detail view

* Added context tab with shortcut

* Only load the replay data when the dialog is open

* Replaying uses the tags from the original run

* Links are now text links

* Removed version links for now because we don’t have dropdown filters for them yet

* Tabs are now outside of the scrollview

* The inspector is now 30% of the width by default

* Allow replaying and editing SuperJSON payloads

* Deleted unused CodeGroup file

* Increase the tags limit to 5, do the limiting on the server

* The admin tooltip now always shows basic org, project and user info

* Fix for schedule inspector disabled state layout

* Remove new unused span metadata and context

* Removed unused import
2024-07-31 15:11:50 +01:00
Quetzalcoatl c1d4c04e89 add xdg command check for linux servers (#1225)
* add xdg command check for linux

* bump version of cli-v3, add changeset

* Don't bump the version

* lint and log

* prettier log message

---------

Co-authored-by: Matt Aitken <matt@mattaitken.com>
Co-authored-by: nicktrn <55853254+nicktrn@users.noreply.github.com>
2024-07-30 15:15:06 +01:00
nicktrn 9882d66f87 v3: pre-pull deployments for faster startups from the first run (#1236)
* pre-pull deployed images on all workers

* automatically clean up pre-pull resources

* rename pre-pull image to deployment

* add changeset
2024-07-26 13:12:58 +01:00
github-actions[bot] 53cf5c089d chore: Update version for release (beta) (#1233)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-07-24 13:55:32 +01:00
Matt Aitken 8614ea2107 Tags docs and a couple of minor fixes (#1235)
* runs.list and runs.retrieve OpenAPI docs updated

* Added docs for how to use runs.retrieve and runs.list with usage data

* SDK reference for triggering with tags

* Added Tags docs page to the sidenav, it’s a copy of usage right now

* Fix for type “users” -> “uses”

* Docs for the tags feature

* Fix for bad import

* Added docs how propagating tags to child runs

* Better error messages when Zod parsing fails for triggering and batch triggering

* Tags are now a string or array of strings, this makes it a lot easier to use
2024-07-24 11:22:03 +01:00
Eric Allam 979bee50d0 Fix return type of runs.retrieve, and allow passing the type of the task to runs.retrieve 2024-07-23 23:28:14 +01:00
Eric Allam 086a0f95c5 Extract common trigger code into internal functions and add a tasks.batchTriggerAndWait function 2024-07-23 22:48:59 +01:00
Matt Aitken 55264657d7 Run tags (#1232)
* TaskRunTag migration. Removed unused TaskTag

* Show tags for a run in the run list

* API endpoint for searching tags

* Let’s not expose an API endpoint for tags at the moment

* Filter by tags working from the URL

* Tag async filter working

* We don’t need a “None” option because it’s multi-select

* When triggering a run you can add tags

* SDK runs.retrieve and runs.list with tag support

* Tidied imports

* Changed the run list query so we show all the tags even if a run only matches one of them

* Fix for dealing with weird characters in tags

* Run tags changeset

* Creating a project has a proper loading state (and blocks multiple)

* Improved the error message for tag length

* Added a tags icon for display on the run screen

* Convenient functions for creating and getting run tags

* tags.set() from inside the run function

* Replay a run passes tags through

* Less ridiculous tags for the catalog example

* Allow passing just a string for the tags

* Order by id because there’s an index on the primary key already

* Trim the tags earlier so we don’t accidentally error if a blank string is passed

* Renamed some thing from setTags to addTags

* Use findFirst for the tags project lookup

* Added an index for "TaskRunTag"("name", "id")

It massively improves the performance of run filtering based on tags

* Use `array_agg` for the run list tags so pagination works and we get a single result for each run

* Tidied imports

* More comprehensive test of tags with all triggering functions

* Support tags with tasks.trigger, tasks.batchTrigger and tasks poll variants

* Added tasks.batchTrigger tags

* Sort the tags in the UI so they’re always in the same order

* Added tooltips in the run table for delay, ttl and tags

* Added costInCents, baseCostInCents and durationMs to runs.retrieve and runs.list

* Added support for displaying a split tag if you use key_value or key:value format

* Fix code comment

* Tweaked the JSDoc to make the prefixing clearer

* Added tasks.triggerAndWait to the tags test task
2024-07-23 16:48:09 +01:00
Eric Allam be3157ace7 Don’t yet auto-bundle monorepo dependencies 2024-07-23 12:39:45 +01:00
Émile Ré ae9a8b0ceb Monorepo packages automatic handling (#1208)
* Add fixture

* Update fixture name

* Update fixture to work with npm & pnpm

* Update setup/teardown and README

* Add support for monorepo to e2e suite utilities

* Use direct dependencies info in esbuild plugin

* Renaming

* Add support for yarn

* Add fixture for npm

* Remove console.log call

* Edge cases

* Remove unused options

* Fix error unknown yarn in CI

* Fix logger debug call's error field

* Add changeset

* Fix missing span end
2024-07-23 11:02:48 +01:00
Émile Ré 6e65591e87 Fix cli v3 e2e suite issues (#1207)
* Fix resolve-legacy-peer-deps when PM=pnpm

* Fix setup/teardown

* Fix skip test before installing dependencies

* Re-enable e2e suite in CI

* Add changeset
2024-07-22 16:53:45 +01:00
github-actions[bot] 8b10a592f3 chore: Update version for release (beta) (#1229)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-07-19 11:33:36 +01:00
Matt Aitken 8ba9987944 Declarative schedules (#1226)
* Added type (STATIC or DYNAMIC) to TaskSchedule. Defaults to dynamic

* WIP with dev indexing of static schedules

* Added a code comment

* First stab at deleting unused static schedules

* Dashboard changes for the static schedules

* Generate the description. Upsert the instances when editing. Fix for the friendlyId

* Don’t allow deleting of static schedules

* Don’t allow enabling/disabling of static schedules

* Added filtering for schedule types

* Syncing of schedule for deployed tasks

* Static schedules are now created for each environment

* Added a second static schedule for testing

* Add the type to the schedule task run payload and the object you get back from the SDK

* Changed static/dynamic to declarative/imperative

* Timezone example

* Changeset

* Updated scheduled docs to include declarative

* When you test a schedule it set the type to “IMPERATIVE”

* Improved the tooltip

* Fix for queue time continuing to rise when a run is canceled/expired etc

* Update the info panel on a selected declarative schedule

* Check if there are no instances. This should never happen but log an error if it does

* Throw errors and push them through to the CLI dev command

* Fail deployments if creating the background tasks or schedules fails

* Format the deployment error so it gets displayed

* Changed the maxed out schedules error message to remove bit about support
2024-07-18 20:24:54 +01:00
github-actions[bot] e8993ee882 chore: Update version for release (beta) (#1211)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-07-17 14:18:40 +01:00
Matt Aitken d6c6dc993d try/catch opening the v3 CLI login URL. Fix for Ubuntu 2024-07-17 13:15:42 +01:00
nicktrn 27de884f93 Remove unused ping message schema 2024-07-17 13:01:43 +01:00
nicktrn c2707800aa Improve prisma errors for missing generate step 2024-07-17 09:53:38 +01:00
nicktrn dbda820a71 v3: fix websocket timeouts (#1220)
* prevent uncaught websocket exceptions

* dev cli sends regular pings to keep connection alive

* add changeset

* fix for prod message forwarding

* add server-side websocket ping

* Revert "dev cli sends regular pings to keep connection alive"

This reverts commit 9a6a7a21433b9f0665f72be015d0ba5a8d83eba4.

* add dev cli websocket debug logs, always print errors

* actually start the ping service

* update changeset

* catch remaining dev cli message handler errors
2024-07-17 09:05:15 +01:00
Gautam Singh e417aca879 v3: add extraCACerts config (#1214)
* Added extraCACerts config to help with self signed/private ca cert chain

* changeset

* Update changeset

* resolved pr comments

---------

Co-authored-by: nicktrn <55853254+nicktrn@users.noreply.github.com>
2024-07-17 08:56:52 +01:00
nicktrn 9971de6a1e Increase span attribute value length limit to 2048 2024-07-16 08:15:05 +01:00
Jacob Paris d934feb02a feature: add no-trigger-core-import (#1175)
* feature: add no-trigger-core-import

* fix: handle aliases and limit to exported modules

* fix: use longest export map name

* fix: support export type and interface

* fix: simplify message when export map is needed

* feature: fine grained modules for core

* fix: add prerequisite lint plugins

* fix: why do I have to fix these components again???

* fix: tests

* Add eslint config and vscode settings

* Create yellow-roses-arrive.md

* Remove the vscode linting

This is temporary so no one fixes these until we merge the remix branch

---------

Co-authored-by: Eric Allam <eric@trigger.dev>
Co-authored-by: Eric Allam <eallam@icloud.com>
Co-authored-by: Matt Aitken <matt@mattaitken.com>
2024-07-11 15:14:55 +01:00
nicktrn 946efd1fb4 v3: deployment index retry (#1205)
* add built at date to worker deployments

* stay on correct deployment page when clicking row

* expose built at via presenters

* more specific deployment statuses

* add deployment indexing retry

* improve deploy timeout logs and add dequeue on completion

* unify deployment indexing retryable status

* only allow index retries of the latest deployment

* improve jsxfu
2024-07-10 17:54:36 +01:00
github-actions[bot] e341df806c chore: Update version for release (beta) (#1209)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-07-09 14:05:49 +01:00
Eric Allam ecf1110ab7 v3: Export AbortTaskRunError from @trigger.dev/sdk/v3 2024-07-09 14:00:44 +01:00
github-actions[bot] 2f14e5136d chore: Update version for release (beta) (#1203)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-07-09 12:24:09 +01:00
Eric Allam 16ad595338 v3: update @depot/cli to latest 0.0.1-cli.2.71.0 2024-07-08 19:33:15 +01:00
Eric Allam 46789292b5 Add back in the run link when calling trigger 2024-07-08 17:59:48 +01:00
Eric Allam e04d44866f v3: better handle null unicode characters when creating task events and completing a failed attempt 2024-07-08 15:59:19 +01:00
Matt Aitken 72f3a4b128 Usage and billing (v3) (#1201)
* v3 subscription endpoints

* Use pnpm linked billing package during development

* Moved v2 billing components into a subfolder

* Select plan using the real data

* Improved v3 plan display

* Use new api response that doesn’t require a Stripe call

* Free flow is working

* Added GitHub modal and verified badge

* Deleted old request v3 access component/route

* Allow setting classes on the Tooltip button

* Paid plans working

* Redirect from select plan if you’ve got v3 enabled

* Loading state improvements

* New admin API endpoint to set concurrency across multiple environments

* When projects are created, conditionally create staging based on the plan

* New billing page working with side menu and stripe portal

* Layout, formatting and some plan state improvements/fixes

* Don’t show the period if you’re on the free plan

* Temporary upgrade callout

* Refactored the platform code so it’s easier to call and doesn’t require a isManagedCloud check

* Send taskIdentifier to OpenMeter

* Side menu

* Upgrade prompts

* More improvements to the app-wide usage indicators

* Early work on usage graphs

* Added the usage bar for v3

* Moved code to presenter and now using defer

* Added the tasks table to usage

* Improved the v3 usage bar if theres’ no usage on a paid plan

* If no run data, still render a graph

* Usage page errors when defered loading fails

* Don’t show the public API key for v3, they’re not used and probably never will be

* Improved the upgrade callout and API keys page layout

* Only show the “reveal all” toggle if you have environment variables in the table

* Replaced Upgrade callout with a more generic InfoPanel component

* better panel width

* Show conditional upgrade prompts based on plan and number of schedules used

* Removed duplicate class

* Wider blank state panels for the scheduled page

* Wider info panel for the env var page

* Blank state now using the info panel

* Platform alerts prompt now using the InfoPanel

* Deploy blank state uses InfoPanel

* Github verified badge padding adjustment

* Improved the layout of the page, some style tweaks, organized imports

* Better default tooltip style

* could be undefined fix

* text fix + style updates

* Changed the billing icon in the side menu

* Billing page layout and style improvements

* plan tooltips don’t use dark variant

* Don’t highlight the plan on the billing page

* Tooltip underlines stand out more

* Fixed padding in the PageTitle

* Tooltips use the correct cursor

* Improved the plan banner on the billing page

* Fixed Header1 inconsistent font weight

* Fixed issue where input field focus states were being clipped

* Fixed large button not having large text size

* Added a link to the Get in touch copy and improved the connect to GitHub modal

* Select plan page uses the MainCenteredContainer

* Better logging from the Loops endpoint because this error finally got hit

* Move the ingestion of compute to the platform

* Reporting usage of invocations moved to the platform

* Get the entitlement before triggering a non-dev task

* Contact us enterprise plan button opens the feedback form

* Removed Github discussions link from the Feedback panel

* Swapped billing icon for credit card

* Show a Unlock staging panel on the env var page

* Updated staging environment colour

* Show a prompt to upgrade to get staging in the new env var modal

* Improved the edit env var modal

* Implement ability to disable org concurrency

* Use common logic for the plans

* Use the billing server to get the schedule limits

* Some schedules page fixes

* More convenient way of getting a limit

* Use the new schedule limit

* Team member limiting

* Made the limit visible on the team page

* Limit alerts

* Added an index for TaskRun.scheduleId

* Remove console.log on schedules page

* Added durations to the run table

* Tabular numbers

* Improved the usage page formatting

* Only admins see the compute column on the run table

* Include the base cost on the usage stats

* Moved the status to the sidebar

* Optional table header tooltip

* Allow InfoIconTooltips to have customizable content styles

* Added a tooltip to the duration header, changed no test to a dash

* Removed all references to signing up to v3 from the docs

* Switched @trigger.dev/billing to @trigger.dev/platform

* Passing up the variant for the InfoIconTooltip

* table tooltip max-width fixed

* Switch to the published @trigger.dev/platform 1.0.11

* v2 usage page title changed to include “v2"

* code theme has a transparent background so it works on any background

* duration columns now grouped together nicely at wide screen size

* Last duration column fills the width properly

* Fix for the per run price being in cents not dollars

* Show the total cost with 8 decimal places

* Show 8 decimal places in the usage graph tooltip

* Moved the UpgradePrompt to the v3 folder

* Prepare to use Shadcns chart helpers

* Much nicer chart

* Small tweaks to the graph

* Fix run table col spans for empty/loading messages

* We don’t need isManagedCloud in createProject

* Hide v3 usage/billing pages if there aren’t v3 projects in your org

* Removed unused tooltipStyle

* Usage bar now says “Included usage” instead of “Tier limit” if you’re paying

* Get the plan/usage data in parallel

* The usage page now has a month dropdown and all data is for that calendar month

* Ensure the passed date is the 1st of the month

* Use the machine presets from the platform package

---------

Co-authored-by: James Ritchie <james@jamesritchie.co.uk>
Co-authored-by: Eric Allam <eallam@icloud.com>
2024-07-08 14:42:19 +01:00
Eric Allam 305a6cebd4 Add tasks.triggerAndWait SDK function 2024-07-08 11:48:33 +01:00
Eric Allam 4f95c9de4e v3: Cancel awaited subtasks and reliable rate-limit recovery (#1200)
* v3: cancel subtasks when parent task runs are cancelled

* v3: recover from server rate limiting errors in a more reliable way

- Changing from sliding window to token bucket in the API rate limiter, to help smooth out traffic
- Adding spans to the API Client core & SDK functions
- Added waiting spans when retrying in the API Client
- Retrying in the API Client now respects the x-ratelimit-reset
- Retrying ApiError’s in tasks now respects the x-ratelimit-reset
- Added AbortTaskRunError that when thrown will stop retries
- Added idempotency keys SDK functions and automatically injecting the run ID when inside a task
- Added the ability to configure ApiRequestOptions (retries only for now) globally and on specific calls
- Implement the maxAttempts TaskRunOption (it wasn’t doing anything before)

* Adding some docs about the request options

* Fix type error

* Remove context propagation through graphile jobs

* Remove logger

* only select a subset of task run columns

* limit columns selected in batchTrigger as well

* added idempotency doc

* allow scoped idempotency keys, and fixed an issue with the unique index on BatchTaskRun and TaskRun

* Removed old cancel task run children code
2024-07-05 10:30:52 +01:00
github-actions[bot] ad68a3cc05 chore: Update version for release (beta) (#1199)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-07-04 14:07:42 +01:00
nicktrn 6fd7560d5c Add core timers export 2024-07-03 19:14:06 +01:00
nicktrn 14c2bdf89b v3: checkpoint and reliability improvements (#1198)
* only checkpoint retries with delays greater than threshold

* rename checkpoint threshold env var

* log task monitor ignores

* crash runs with unbounded attempts

* fix retry check in shared queue consumer

* add missing stop for env var sync spinner

* prod entry point refactor

* missing awaits

* more verbose prod flush and exit logs

* reduce checkpoint support logs

* heartbeat while checkpointing between retries

* dynamic coordinator config

* measure lazy attempt creation time in prod

* simplify delay threshold

* heartbeat clarifications

* crash run if it doesn't reach checkpointable state

* require dynamic config threshold

* fix retry prep, await previous worker kill

* unify wait mechanics

* fix prod worker without tasks error

* ensure worker is ready to be checkpointed for dependency waits

* improve worker attempt creation logging

* prevent crashes caused by failed socket schema parsing

* fix dynamic imports in v3 catalog

* clarify attempt retry mechanics

* move backoff helper to core-apps

* remove core-apps barrel file

* add backoff execute with callback

* deprecate non-lazy attempt messages

* update socket.io-client to v4.7.5

* fix socket.io types for emits with timeout

* retry all the things

* remove todo

* fix retry restores

* improve index failure logs

* retry incomplete dependency waits

* fix checkpoint in-progress detection

* prevent losing messages during reconnect

* checkpoint when greater or equal to threshold

* improve handling of duration wait edge cases

* add ready for lazy attempt replay

* retry attempt completion

* allow failing runs with unfriendly run id

* fix min max jitter

* cancel checkpoints on run failure

* improve attempt creation errors

* prevent crashing run on failed cleanup

* handle at-least-once execute lazy attempt delivery

* log exit code on prepare for retry

* fix timeout promise

* mark some things

* chaos monkey superpowers

* refactor checkpointer

* set chaos monkey defaults

* less chaos

* fix backoff

* handle uncaught entry point exceptions

* only replay rpcs on true reconnects

* allow resume unless final run status

* add changeset

* small fixes
2024-07-03 16:54:09 +01:00
github-actions[bot] b9b86c89a7 chore: Update version for release (beta) (#1188)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-07-01 17:16:11 +01:00
Eric Allam 0e77e7ef7d v3: Delayed runs and run ttl expiration (#1193)
* v3: Trigger delayed runs and reschedule them

* Create a `@trigger.dev/core/v3/schemas` export

* fixed the `@trigger.dev/core/v3/schemas` export

* Small docs tweak

* Add ttl option when triggering tasks, expire runs after ttl

Dev runs expire in 10m by default
2024-07-01 12:01:35 +01:00
nicktrn b171fde483 Fix index failure completion override 2024-06-29 19:23:32 +01:00
nicktrn 5ae3da6b4e v3: various small fixes (#1192)
* Await file watcher cleanup in dev

* Fix artifact detection logs

* Fix next runs table when schedule disabled

* Improve OOM error messages

* Add test link to completed deployment message

* Fix OOM detection, again

* Add changeset
2024-06-29 16:48:01 +01:00
Eric Allam f565829959 v3: Copy over more of the project's package.json keys into the deployed package.json (support for custom config like zenstack) 2024-06-28 20:13:54 +01:00
Eric Allam 75ec4ac6a6 v3: postInstall config option now replaces the postinstall script found in package.json 2024-06-28 15:59:41 +01:00
nicktrn 374b6b9c0c Increase dev worker timeout 2024-06-28 15:56:51 +01:00
nicktrn d0d3a64bd6 v3: misc CLI improvements (#1173)
* prevent downgrades during update check

* detect bun and use npm instead

* detect missing tsconfig during init and print helpful error

* add changeset

* add links to dev worker started message

* allow users to add custom pkg manager args during init

* update changeset

* fix links in unsupported terminals

* deprecate terminalLink

* update changeset
2024-06-28 15:51:17 +01:00
nicktrn 568da01785 v3: improve non-zero exit errors (#1179)
* docker provider will optionally enforce machine presets

* update task monitor oom message

* add oom task to v3 catalog

* improve handling of non-zero exit errors

* add changeset
2024-06-28 15:50:04 +01:00
nicktrn c75e29a9a7 v3: add audio binaries to worker images (#1183)
* add sox

* add audiowaveform

* add changeset
2024-06-28 15:49:24 +01:00
Émile Ré e5d26bd12d Fix infisical-sdk fixture, use in config instead (#1190) 2024-06-28 09:50:24 +01:00
Émile Ré 52b6f48a94 Cli v3 e2e: fixtures (#1184)
* Run tests

* Remove useless comments

* Fix readme

* Add fixtures for config step

* Add fixture for additionalDependencies name parsing scope issue

* Add URL to commit fix

* Some fixtures will only use 1 package manager

* Add resolve-trigger-deps fixture

* use rimraf in test setup

* Add legacy-peer-deps fixture

* Use vitest fixtures

* Add nested peer dep locking issue

* Add missing peer dep resolving fixture

* Get rid of global namespace, fix parallelism

* Add console logs to debug log level

* Add changeset

* Remove comments

* Update README.md
2024-06-27 13:55:54 +01:00
github-actions[bot] 9ba2a217a4 chore: Update version for release (beta) (#1187)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-06-27 13:38:57 +01:00