Commit Graph

267 Commits

Author SHA1 Message Date
Matt Aitken 8ba9987944 Declarative schedules (#1226)
* Added type (STATIC or DYNAMIC) to TaskSchedule. Defaults to dynamic

* WIP with dev indexing of static schedules

* Added a code comment

* First stab at deleting unused static schedules

* Dashboard changes for the static schedules

* Generate the description. Upsert the instances when editing. Fix for the friendlyId

* Don’t allow deleting of static schedules

* Don’t allow enabling/disabling of static schedules

* Added filtering for schedule types

* Syncing of schedule for deployed tasks

* Static schedules are now created for each environment

* Added a second static schedule for testing

* Add the type to the schedule task run payload and the object you get back from the SDK

* Changed static/dynamic to declarative/imperative

* Timezone example

* Changeset

* Updated scheduled docs to include declarative

* When you test a schedule it set the type to “IMPERATIVE”

* Improved the tooltip

* Fix for queue time continuing to rise when a run is canceled/expired etc

* Update the info panel on a selected declarative schedule

* Check if there are no instances. This should never happen but log an error if it does

* Throw errors and push them through to the CLI dev command

* Fail deployments if creating the background tasks or schedules fails

* Format the deployment error so it gets displayed

* Changed the maxed out schedules error message to remove bit about support
2024-07-18 20:24:54 +01:00
Ray Amjad 5a3b4a450b Update github-actions.mdx (#1222) 2024-07-17 08:55:14 +01:00
Matt Aitken 91d45e5ba6 Added a link to the pricing page for machines 2024-07-10 17:04:29 +01:00
Matt Aitken e59dd465d8 Documented the task usage functions 2024-07-10 15:19:45 +01:00
Matt Aitken 98d5f005e4 Added delay to the docs and improved the ttl 2024-07-10 10:28:49 +01:00
Matt Aitken a6b5d2edc5 Added ttl to trigger docs 2024-07-10 09:41:09 +01:00
Matt Aitken f4a1d5f36d Removed broken links from writing tasks page 2024-07-10 06:35:41 +01:00
Matt Aitken 6fb6cb6aaf Docs: added alerts link from feature matrix 2024-07-10 06:35:41 +01:00
Eric Allam 0b6417f1d8 Improve the section on adding trigger.dev to dev deps 2024-07-09 21:37:37 +01:00
Matt Aitken 62532d5ce1 Fix for package name mistake 2024-07-09 15:53:03 +01:00
Matt Aitken 786d5508f0 Cover upgrading dev dependency 2024-07-09 15:52:42 +01:00
Matt Aitken 187f02780b Added upgrading packages guide 2024-07-09 15:49:46 +01:00
Matt Aitken 72f3a4b128 Usage and billing (v3) (#1201)
* v3 subscription endpoints

* Use pnpm linked billing package during development

* Moved v2 billing components into a subfolder

* Select plan using the real data

* Improved v3 plan display

* Use new api response that doesn’t require a Stripe call

* Free flow is working

* Added GitHub modal and verified badge

* Deleted old request v3 access component/route

* Allow setting classes on the Tooltip button

* Paid plans working

* Redirect from select plan if you’ve got v3 enabled

* Loading state improvements

* New admin API endpoint to set concurrency across multiple environments

* When projects are created, conditionally create staging based on the plan

* New billing page working with side menu and stripe portal

* Layout, formatting and some plan state improvements/fixes

* Don’t show the period if you’re on the free plan

* Temporary upgrade callout

* Refactored the platform code so it’s easier to call and doesn’t require a isManagedCloud check

* Send taskIdentifier to OpenMeter

* Side menu

* Upgrade prompts

* More improvements to the app-wide usage indicators

* Early work on usage graphs

* Added the usage bar for v3

* Moved code to presenter and now using defer

* Added the tasks table to usage

* Improved the v3 usage bar if theres’ no usage on a paid plan

* If no run data, still render a graph

* Usage page errors when defered loading fails

* Don’t show the public API key for v3, they’re not used and probably never will be

* Improved the upgrade callout and API keys page layout

* Only show the “reveal all” toggle if you have environment variables in the table

* Replaced Upgrade callout with a more generic InfoPanel component

* better panel width

* Show conditional upgrade prompts based on plan and number of schedules used

* Removed duplicate class

* Wider blank state panels for the scheduled page

* Wider info panel for the env var page

* Blank state now using the info panel

* Platform alerts prompt now using the InfoPanel

* Deploy blank state uses InfoPanel

* Github verified badge padding adjustment

* Improved the layout of the page, some style tweaks, organized imports

* Better default tooltip style

* could be undefined fix

* text fix + style updates

* Changed the billing icon in the side menu

* Billing page layout and style improvements

* plan tooltips don’t use dark variant

* Don’t highlight the plan on the billing page

* Tooltip underlines stand out more

* Fixed padding in the PageTitle

* Tooltips use the correct cursor

* Improved the plan banner on the billing page

* Fixed Header1 inconsistent font weight

* Fixed issue where input field focus states were being clipped

* Fixed large button not having large text size

* Added a link to the Get in touch copy and improved the connect to GitHub modal

* Select plan page uses the MainCenteredContainer

* Better logging from the Loops endpoint because this error finally got hit

* Move the ingestion of compute to the platform

* Reporting usage of invocations moved to the platform

* Get the entitlement before triggering a non-dev task

* Contact us enterprise plan button opens the feedback form

* Removed Github discussions link from the Feedback panel

* Swapped billing icon for credit card

* Show a Unlock staging panel on the env var page

* Updated staging environment colour

* Show a prompt to upgrade to get staging in the new env var modal

* Improved the edit env var modal

* Implement ability to disable org concurrency

* Use common logic for the plans

* Use the billing server to get the schedule limits

* Some schedules page fixes

* More convenient way of getting a limit

* Use the new schedule limit

* Team member limiting

* Made the limit visible on the team page

* Limit alerts

* Added an index for TaskRun.scheduleId

* Remove console.log on schedules page

* Added durations to the run table

* Tabular numbers

* Improved the usage page formatting

* Only admins see the compute column on the run table

* Include the base cost on the usage stats

* Moved the status to the sidebar

* Optional table header tooltip

* Allow InfoIconTooltips to have customizable content styles

* Added a tooltip to the duration header, changed no test to a dash

* Removed all references to signing up to v3 from the docs

* Switched @trigger.dev/billing to @trigger.dev/platform

* Passing up the variant for the InfoIconTooltip

* table tooltip max-width fixed

* Switch to the published @trigger.dev/platform 1.0.11

* v2 usage page title changed to include “v2"

* code theme has a transparent background so it works on any background

* duration columns now grouped together nicely at wide screen size

* Last duration column fills the width properly

* Fix for the per run price being in cents not dollars

* Show the total cost with 8 decimal places

* Show 8 decimal places in the usage graph tooltip

* Moved the UpgradePrompt to the v3 folder

* Prepare to use Shadcns chart helpers

* Much nicer chart

* Small tweaks to the graph

* Fix run table col spans for empty/loading messages

* We don’t need isManagedCloud in createProject

* Hide v3 usage/billing pages if there aren’t v3 projects in your org

* Removed unused tooltipStyle

* Usage bar now says “Included usage” instead of “Tier limit” if you’re paying

* Get the plan/usage data in parallel

* The usage page now has a month dropdown and all data is for that calendar month

* Ensure the passed date is the 1st of the month

* Use the machine presets from the platform package

---------

Co-authored-by: James Ritchie <james@jamesritchie.co.uk>
Co-authored-by: Eric Allam <eallam@icloud.com>
2024-07-08 14:42:19 +01:00
Eric Allam 4f95c9de4e v3: Cancel awaited subtasks and reliable rate-limit recovery (#1200)
* v3: cancel subtasks when parent task runs are cancelled

* v3: recover from server rate limiting errors in a more reliable way

- Changing from sliding window to token bucket in the API rate limiter, to help smooth out traffic
- Adding spans to the API Client core & SDK functions
- Added waiting spans when retrying in the API Client
- Retrying in the API Client now respects the x-ratelimit-reset
- Retrying ApiError’s in tasks now respects the x-ratelimit-reset
- Added AbortTaskRunError that when thrown will stop retries
- Added idempotency keys SDK functions and automatically injecting the run ID when inside a task
- Added the ability to configure ApiRequestOptions (retries only for now) globally and on specific calls
- Implement the maxAttempts TaskRunOption (it wasn’t doing anything before)

* Adding some docs about the request options

* Fix type error

* Remove context propagation through graphile jobs

* Remove logger

* only select a subset of task run columns

* limit columns selected in batchTrigger as well

* added idempotency doc

* allow scoped idempotency keys, and fixed an issue with the unique index on BatchTaskRun and TaskRun

* Removed old cancel task run children code
2024-07-05 10:30:52 +01:00
nicktrn be58638904 Documentation edits made through Mintlify web editor 2024-07-04 13:34:30 +01:00
Matt Aitken 50506dce9f Documentation edits made through Mintlify web editor 2024-07-03 16:58:57 +01:00
Eric Allam 0e77e7ef7d v3: Delayed runs and run ttl expiration (#1193)
* v3: Trigger delayed runs and reschedule them

* Create a `@trigger.dev/core/v3/schemas` export

* fixed the `@trigger.dev/core/v3/schemas` export

* Small docs tweak

* Add ttl option when triggering tasks, expire runs after ttl

Dev runs expire in 10m by default
2024-07-01 12:01:35 +01:00
Eric Allam 50d46a8513 Fixed lock file of "Update docs theme (#1189)"
This reverts commit 4cc61ac0ec.
2024-06-27 20:39:24 +01:00
Eric Allam 4cc61ac0ec Revert "Update docs theme (#1189)"
This reverts commit a696359c3e.
2024-06-27 20:31:25 +01:00
James Ritchie a696359c3e Update docs theme (#1189)
* Changed theme to venus and updated primary colours

* Replaced ‘v3 (Developer Preview)’ with ‘v3’

* Removed webhooks and zod from tasks overview and intro

* pnpm lock

* Changed the theme and background + removed the tabs

---------

Co-authored-by: D-K-P <8297864+D-K-P@users.noreply.github.com>
2024-06-27 18:31:47 +01:00
Eric Allam 34ca7667d3 Various perf improvements to prevent event loop lag (#1186)
* WIP

* Handle tasks that have failed but are being auto yielded

* Limit trace view to 25k event records, add a download run logs button

Also added two new indexes to TaskEvent:

```
/// Used on eventRepository.getTraceSummary()
  @@index([traceId, startTime])
  // Used for getting all logs for a run
  @@index([runId])
```

* perf improvements on eventRepository.getSpan()

* v2: Add a 5 minute timeout for run execution requests in dev

* v3: Include presigned urls for downloading large payloads and outputs when using runs.retrieve

* v3: better handle large task payloads and outputs

* Change to 512KB

* v2: paginate trigger schedules endpoint

* v3: add 3MB limit on batch and single payloads

* Update task payload and output limits
2024-06-26 15:22:35 +01:00
Eric Allam 7c36a1a4b0 v3: Adding SDK functions for triggering tasks in a typesafe way (#1177)
* v3: Adding SDK functions for triggering tasks in a typesafe way, without importing task file

* Add type usages
2024-06-24 12:37:34 +01:00
Eric Allam 225effb599 Add task endpoints to the endpoint authentication table 2024-06-21 15:11:50 +01:00
Eric Allam 98eb6ed4f9 Add trigger and batch trigger API docs 2024-06-21 15:08:06 +01:00
Eric Allam 098932ea96 v3: edge runtime support (#1172)
* v3: remove node:stream and simplify env var upload API to better work with non-node runtimes

* Remove file/Response envvars upload docs

* Add changeset
2024-06-21 13:33:38 +01:00
Eric Allam 85a543d8ec v3: usage tracking (#1163)
* Starting to measure wall time and cpu time in the workers, and reporting that via otel and to completed task run attempts

* Move usage tracking outside of the executor

* WIP prod usage tracking

* WIP

* WIP custom fetch to openmeter

* Create a usage client

* WIP

* WIP

* Implement new machine preset stuff and send usage reports to OpenMeter from webapp

* WIP

* Expose usage info to the client

* Add usage and cost to TaskEvent

* Add ability to globally configure the task machine preset

* Report start run usage

* Change the machine docs to use presets

* setExpirationTime to 24h

* Removed logs

* Update machines.mdx

* Removed console.logs

* Handle revalidating JWT tokens

* Couple tweaks

---------

Co-authored-by: Matt Aitken <matt@mattaitken.com>
2024-06-18 09:40:23 +01:00
Matt Aitken c405ae7117 Schedule limits and timezone support (#1165)
* Added maximumScheduleInstancesLimit column to Org, default to 20

* Docs on the schedule limits and improved soft-limit communication

* Added limit info to the schedules list page

* Created a task that creates schedules, useful for testing

* Make deduplicationKey required when creating/updating a schedule using the SDK

* New schedule button shows an alert if you’re over the limit

* Added timezone to the form and db

* WIP on the timezone dropdown for the create/edit schedule form

* Use the new filter search for timezones

* Made the timezone dropdown faster by fixing the virtualization

* The preview table is working and added a nice message about daylight savings

* Created a page where you can view the full list of timezones

The URL is included in the error message if you send an invalid time using the SDK

* Creating tasks with the timezone

* Added timezone support the the scheduler and the schedules list

* Added timezone support to more of the schedules UI

* The timezone comes through to scheduled runs with nice JSDocs

* Allow setting the timezone from the SDK

* Always have a timezone on a schedule

* Updated jsdocs

* Updated catalog example

* Changed the column to be a string, not null. Added the timezone across the SDK

* API endpoint for getting the timezones

* Added an SDK function to get the list of timezones

* Added timezones to the docs

* Changeset: Added timezone support to schedules

* Added support for testing timezone

* Tidied up imports

* Imports

* Imports

* Update limits.mdx

* Fixed a couple type issues and use the already exported zodfetch

---------

Co-authored-by: Eric Allam <eallam@icloud.com>
2024-06-14 13:31:27 +01:00
nicktrn 68d32429b6 v3: checkpoint failover and misc fixes (#1157)
* configurable checkpoint registry namespace

* add missing task create await

* remove unused messages

* changeset

* update self-hosting docs

* capture and display stderr for failed deploys

* add missing lockfile changes

* stderr changeset

* fix cli stderr message

* update error logs label
2024-06-11 14:06:50 +01:00
Matt Aitken 52112c3bfc Docs: added v3 limits page 2024-06-10 16:07:08 +01:00
nicktrn 35dbaedf69 v3: self-hosting (#1147)
* add amin email regex env var

* fix displayed init command for self-hosted setups

* shared env var to disable telemetry in cli and webapp

* pin sdk version during init

* if specified, add api url to dev command shown after init

* improve checkpoint support detection

* control forced checkpoint simulation via env var

* add public init to providers

* better checkpoint support check for coordinator

* add docker to coordinator image

* update docker provider containerfile

* bump remaining containers to node 20

* add infra image build to default publish workflow

* lockfile

* remove concurrency group from infra workflow

* add docker provider to build matrix

* fix var subst

* checkpoint test is docker specific

* enable v3 projects by default on self-hosted instances

* fix v3 setup command again

* add default posthog key

* self-hosting docs

* add latest tags to versioned infra and webapp builds

* some checkpoint errors should skip retrying

* add changeset

* shorten paragraph

* some docs updates

* update tunnelling section

* add registry setup section

* use correct cli push flag

* add checkout to v3 branch

* update the worker machine setup steps

* fix infra build

* small docs update

* remove unused feature function

* Revert "remove unused feature function"

This reverts commit cfe07887a12b6893dca8ce499964481a9b3dc9db.

* fix self-hosted v3 feature gate

* add note about missing arm support

* simplify helper script syntax
2024-06-10 14:13:04 +01:00
Eric Allam 9ebd91ccec Fixed auto-pagination example 2024-06-06 13:26:16 +01:00
Matt Aitken c24a23b551 Docs: env vars SDK docs 2024-06-05 11:41:14 +01:00
Matt Aitken ee1ae1fca6 Docs: improved trigger.config docs for instrumentations, env vars and troubleshooting 2024-06-05 11:41:14 +01:00
Eric Allam ba71f959e2 v3: Runs List Management SDK and SDK improvements (#1133)
* Improved the existing runs API

* WIP next runs API

* Improve the returned ApiPromise to add ability to return response

* More WIP

* WI{

* Added offset/limit pagination stuff like the cursor one, and converted all API methods to use ApiPromise

* More run API stuff

- Adding schedule  output from the retrieveRun endpoint
- Ability to filter by schedule and isTest

* Remove env from retrieve run in openAPI

* prefer duplication over merge

* WIP docs

* Use spread to DRY up some run API schemas

* Finish the overview docs

* Adding changeset

* Fixed typecheck errors

* Typo fix

* Re-export zodfetch from core so the v3 CLI can use it

* Fixed type errors

---------

Co-authored-by: Matt Aitken <matt@mattaitken.com>
2024-05-31 11:02:28 +01:00
Matt Aitken 5fe23e4b3f Troubleshooting guide (docs) (#1139)
* Started troubleshooting guide

* More work on the troubleshooting guide

* NestJS troubleshooting and using await when triggering

* Fleshed out more

* Added Prisma

* Tweaks
2024-05-30 22:03:54 +01:00
Parker 782d4f75ae Added tip for setting up github actions using npm run to simplify version pinning (#1132)
* added arg to install puppeteer deps to base image

* added tip on deploy setup

* removed docker changes
2024-05-28 12:10:24 +01:00
Matt Aitken 0dd3447c31 Improved the migration from v2 to v3 guide 2024-05-28 09:39:21 +01:00
Eric Allam 3a1b0c486a v3: env var management API (#1116)
* WIP env var management API

* Add import env var API endpoint

* Adding docs and support for using both API keys and PATs when interacting with the env var endpoints

* WIP envvar SDK

* Uploading env vars in a variety of formats now works

* Finish env var endpoints and add resolveEnvVars hook

* Add changeset
2024-05-23 16:11:25 +01:00
Alec Dilanchian 8a263c8ab9 Adding note for supabase integration (#1101)
* Update management.mdx

Adding note for custom domains and initializing db object.

* Update management.mdx

---------

Co-authored-by: Matt Aitken <matt@mattaitken.com>
2024-05-17 15:11:29 +01:00
James Ritchie 89eaef495f New alerts platform status + Alerts docs (#1098)
* debug tooltip now scrolls

* Added a link to subscribe to alerts and improved the project alerts blank state

* Better external link icon

* Docs: Removed webhook tasks

* Docs: removed limits performance

* Docs: removed FAQs

* Docs: Removed Architecture section

* Docs: Removed API reference: CLI

* Docs: Removed API reference: Objects

* Docs: Removed API reference: Functions

* Docs: removed automated tests

* Docs: removed Middleware

* Docs: removed Using APIs

* Docs: removed Rollbacks

* Docs: removed Trigger Filters

* Docs: removed Webhook Tasks

* Docs: removed Zod Tasks

* Docs: Renamed Community page

* Docs: Added a new Troubleshooting section and Alerts docs page

* Hide the New Alerts button again if list is greater than 10 items

* Customers now have to contact us for Slack Connect Support.

* Fix the alerts docs link

* Added env vars for a different alert email address, and whether the feature is enabled or not

* Only show the alerts sidemenu item if the feature is enabled

* Use a separate email client for sending alerts

* Removed the link to Context from v3 docs

* Removed obvious docs links that are now missing pages

---------

Co-authored-by: Matt Aitken <matt@mattaitken.com>
2024-05-16 16:03:25 +01:00
Matt Aitken 2081cb15b8 Removed docs for dashboard as they’re empty at the moment. Moved reference docs for Runs and schedules up 2024-05-12 18:29:46 +01:00
Matt Aitken ff551c5cbd Ticked off alerts (in preparation for putting it live) 2024-05-12 18:27:39 +01:00
Matt Aitken 794b25a988 Some fixes for the v3 docs showing v2 items in the sidebar 2024-05-12 18:27:26 +01:00
Matt Aitken 194f336b84 Added an image of where to set the v3 GitHub action secret 2024-05-12 18:01:27 +01:00
Zafer Cesur ff04c986a1 Add notice on sideEffects in deployment guide (#680)
Co-authored-by: Matt Aitken <matt@mattaitken.com>
2024-05-07 22:58:27 +01:00
Eric Allam 6d9dfbc75d Add configure function to be able to configure the SDK manually
And move the ApiClient configuration away from AsyncLocalStorage and use our globals system instead
2024-05-06 12:12:30 +01:00
Eric Allam 203e002087 Add runs.retrieve management API method to get info about a run by run ID 2024-05-04 22:24:42 +01:00
Eric Allam 2dfe191c55 Fixed some docs that were missing imports 2024-05-01 14:47:50 +01:00
Andreas Thomas cac3c32f6a docs: add 'retry' import in code snippet (#1071)
* docs: add 'retry' import in code snippet

* docs: import all primitives
2024-04-29 17:42:36 +01:00
Praveen Pendyala 29e9e372ee Added example of github workflow for deploy to staging (#1068) 2024-04-26 18:39:34 +01:00