Commit Graph

12 Commits

Author SHA1 Message Date
Eric Allam 086a0f95c5 Extract common trigger code into internal functions and add a tasks.batchTriggerAndWait function 2024-07-23 22:48:59 +01:00
Eric Allam 4f95c9de4e v3: Cancel awaited subtasks and reliable rate-limit recovery (#1200)
* v3: cancel subtasks when parent task runs are cancelled

* v3: recover from server rate limiting errors in a more reliable way

- Changing from sliding window to token bucket in the API rate limiter, to help smooth out traffic
- Adding spans to the API Client core & SDK functions
- Added waiting spans when retrying in the API Client
- Retrying in the API Client now respects the x-ratelimit-reset
- Retrying ApiError’s in tasks now respects the x-ratelimit-reset
- Added AbortTaskRunError that when thrown will stop retries
- Added idempotency keys SDK functions and automatically injecting the run ID when inside a task
- Added the ability to configure ApiRequestOptions (retries only for now) globally and on specific calls
- Implement the maxAttempts TaskRunOption (it wasn’t doing anything before)

* Adding some docs about the request options

* Fix type error

* Remove context propagation through graphile jobs

* Remove logger

* only select a subset of task run columns

* limit columns selected in batchTrigger as well

* added idempotency doc

* allow scoped idempotency keys, and fixed an issue with the unique index on BatchTaskRun and TaskRun

* Removed old cancel task run children code
2024-07-05 10:30:52 +01:00
Eric Allam 0e77e7ef7d v3: Delayed runs and run ttl expiration (#1193)
* v3: Trigger delayed runs and reschedule them

* Create a `@trigger.dev/core/v3/schemas` export

* fixed the `@trigger.dev/core/v3/schemas` export

* Small docs tweak

* Add ttl option when triggering tasks, expire runs after ttl

Dev runs expire in 10m by default
2024-07-01 12:01:35 +01:00
Eric Allam 34ca7667d3 Various perf improvements to prevent event loop lag (#1186)
* WIP

* Handle tasks that have failed but are being auto yielded

* Limit trace view to 25k event records, add a download run logs button

Also added two new indexes to TaskEvent:

```
/// Used on eventRepository.getTraceSummary()
  @@index([traceId, startTime])
  // Used for getting all logs for a run
  @@index([runId])
```

* perf improvements on eventRepository.getSpan()

* v2: Add a 5 minute timeout for run execution requests in dev

* v3: Include presigned urls for downloading large payloads and outputs when using runs.retrieve

* v3: better handle large task payloads and outputs

* Change to 512KB

* v2: paginate trigger schedules endpoint

* v3: add 3MB limit on batch and single payloads

* Update task payload and output limits
2024-06-26 15:22:35 +01:00
Eric Allam 7c36a1a4b0 v3: Adding SDK functions for triggering tasks in a typesafe way (#1177)
* v3: Adding SDK functions for triggering tasks in a typesafe way, without importing task file

* Add type usages
2024-06-24 12:37:34 +01:00
Eric Allam 374edef020 Updates the trigger, batchTrigger and their *AndWait variants to use the first parameter for the payload/items, and the second parameter for options (#1045)
Also always returns a `TaskRunResult` object from `triggerAndWait` instead of rethrowing subtask errors in the parent
2024-04-19 14:51:51 +01:00
nicktrn c9e1a3e9c5 v3: fix dependency trigger and wait (#1030)
* fix dev dependecy resumes

* add catalog entry with multiple trigger waits

* update docker provider lifecycle command

* switch to x for clearing run filters

* Revert "fix dev dependecy resumes"

This reverts commit b4061f2ae71f21889adcf061928e2e77c436f0ed.

* fix dependency resumes, properly this time

* add catalog entry for dependency waits in loops

* advice in docs re parallel dependency waits

* fix link from v3 to v2 docs

* move lifecycle command logging to debug only

* Removed batchOptions from the trigger options and the docs

---------

Co-authored-by: Matt Aitken <matt@mattaitken.com>
2024-04-18 14:48:43 +01:00
Matt Aitken acd7681e58 Scheduled task docs (#1039)
* Scheduled docs v1

* Improved copy

* Rejigged the docs some more. Added a dynamic section
2024-04-17 17:41:16 +01:00
Matt Aitken eb050f6730 Fix for typo on triggering page. “yourTask.batchTriggerAndWait” -> “yourTask.batchTrigger” 2024-04-04 09:12:29 +01:00
Matt Aitken afe7f410c7 v3 docs improvements (#993)
* WIP on adding versioning to the docs to make it less confusing

* Link to v3 from v2 and vice versa. Also added a warning to the v2 docs

* Make v3 the default

* Trigger config docs about bundling

* Next stpe to triggering guide from tasks overview

* Server actions guidance for triggering

* Moved the trigger.config docs to the end of fundamentals
2024-04-02 14:32:28 +01:00
Matt Aitken f1c768a255 Docs for API keys, fixed links to docs from app (#981)
* Add missing API keys docs and link from the web app to them. Renamed Server API keys to Secret keys

* Correct link to env var docs

* Duplicate image fixed on environment variables docs
2024-03-28 14:00:42 +00:00
Matt Aitken de302858f8 v3 docs (#929)
* Started building out the docs navigation

* Drafted out a lot of the docs pages. With just titles and descriptions with a coming soon message for now.

* A ton more scaffolded pages with titles, descriptions, coming soon/incomplete docs messages

* Created the remaining page stubs

* Introduction page improvements

* Lots of docs changes, mostly the task overview page

* Moved some bits around

* Task overview fleshed out

* Triggering docs page

* Regular task page

* CLi dev docs

* Docs for debugging locally

* Environment variables docs

* Added env var images

* Deployment guides

* Updated the title of “Integrations” to “Deployment integrations” to make it less confusing

* Lots more docs pages

* Changed the language to JavaScript

* Fixed typos about queues and added a simple code sample

* Moved the local debugger docs into the CLI dev page

* Added the CLI deploy options

* Delete the local debugger page

* Updated the logging page with structured logging

* Added the new CLI deploy options

* Change to h4 for one of the options

* Errors page

* WIP on retrying

* Retrying guide

* Wait docs pages

* Queuing docs simplified, partial written

* Queuing and concurrency docs

* Run tests docs

* New combined error and retrying page

* Errors and retrying page updated

* More docs

* Added a possible configurations coming soon table

* Created a limits page

* Added warnings about how you need to get early access

* Minor fixes for the docs
2024-03-22 14:09:27 +00:00