Compare commits

...

419 Commits

Author SHA1 Message Date
Eric Allam 4acfb8f4bb Release 3.0.0-beta.41 2024-06-24 17:30:01 +01:00
github-actions[bot] 2ef278db67 chore: Update version for release (beta) (#1178)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-06-24 17:29:32 +01:00
Eric Allam c7a55804d9 v3: fix jsonc-parser import in cli
Lock jsonc-parser to 3.2.1

Revert that change
2024-06-24 17:24:23 +01:00
Eric Allam da6a66efff v3: fix task queues with concurrency = 0 getting cleared on task trigger 2024-06-24 16:37:30 +01:00
Eric Allam 7c36a1a4b0 v3: Adding SDK functions for triggering tasks in a typesafe way (#1177)
* v3: Adding SDK functions for triggering tasks in a typesafe way, without importing task file

* Add type usages
2024-06-24 12:37:34 +01:00
Eric Allam 225effb599 Add task endpoints to the endpoint authentication table 2024-06-21 15:11:50 +01:00
Eric Allam 98eb6ed4f9 Add trigger and batch trigger API docs 2024-06-21 15:08:06 +01:00
Eric Allam 3069ebf0d8 Release 3.0.0-beta.40 2024-06-21 14:29:04 +01:00
github-actions[bot] e133e628ca chore: Update version for release (beta) (#1170)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-06-21 14:28:25 +01:00
Eric Allam 098932ea96 v3: edge runtime support (#1172)
* v3: remove node:stream and simplify env var upload API to better work with non-node runtimes

* Remove file/Response envvars upload docs

* Add changeset
2024-06-21 13:33:38 +01:00
Eric Allam 65f960e883 Move the max live reload setting into an env var instead of hardcoding it 2024-06-21 12:55:33 +01:00
Eric Allam ccbeff47e6 Improve trace view performance 2024-06-21 12:46:46 +01:00
Matt Aitken 7c8f2df105 After 500 logs we stop live reloading on the run page 2024-06-21 11:26:02 +01:00
Eric Allam fd44dabfe0 Disable the event loop monitor 2024-06-20 23:56:29 +01:00
Eric Allam 5daed3f69d Turn off rebalancing 2024-06-20 23:42:54 +01:00
Eric Allam 596bf78e55 Add an event loop monitor 2024-06-20 23:26:38 +01:00
Eric Allam 6ca66b76f4 Remove unnecessary transaction from the runtime environment session handling 2024-06-20 22:47:12 +01:00
nicktrn 29ef0395ce Only import types from db in task status 2024-06-20 15:25:18 +01:00
nicktrn 55d1f8c677 v3: fix dependency checkpoint race (#1171)
* consolidate task statuses and utils

* check dependency completion when creating checkpoints

* add changeset
2024-06-20 10:10:39 +01:00
Eric Allam 9835f4ec55 Adding changeset 2024-06-19 15:11:50 +01:00
Eric Allam 7fae10db23 v3: make sure force flush timeout is set so flushing doesn’t hang forever 2024-06-19 14:56:10 +01:00
Eric Allam 8cf1f0a37d v3: don’t set a default forceFlushTimeoutMillis 2024-06-19 14:29:50 +01:00
Eric Allam dba4313c5c v3: more visibility into flushing worker otel data in prod 2024-06-19 14:22:06 +01:00
Eric Allam 506613dc92 v3: prod workers should always log everything from forked node process 2024-06-19 14:05:30 +01:00
Eric Allam 764df23d19 Release 3.0.0-beta.39 2024-06-19 12:22:19 +01:00
github-actions[bot] 4b961a6ae2 chore: Update version for release (beta) (#1169)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-06-19 12:21:30 +01:00
Eric Allam 8757fdceef v3: [prod] force flush timeout should be 1s 2024-06-19 12:16:11 +01:00
Eric Allam 2404e88ac5 Add a IMPORTANT note to the ProdTaskRunExecution 2024-06-19 11:16:40 +01:00
Eric Allam 88b36f5090 Add a default on machine preset 2024-06-19 11:07:57 +01:00
Eric Allam b73ae3f927 Release 3.0.0-beta.38 2024-06-19 10:40:29 +01:00
github-actions[bot] b605b892ac chore: Update version for release (beta) (#1166)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-06-19 10:39:24 +01:00
Eric Allam 233316f7e8 TaskRun.startedAt will now be lazily migrated 2024-06-18 15:49:14 +01:00
Eric Allam 1b90ffbb8c Add a changeset for the usage tracking PR (I forgot) 2024-06-18 14:56:25 +01:00
Matt Aitken b45ca4e146 If the timezone has changed then we need to reschedule the next scheduled run 2024-06-18 14:49:16 +01:00
Eric Allam 25d15578f7 Remove the update migration 2024-06-18 13:51:29 +01:00
Eric Allam fe865a0f49 Make startedAt backwards compat by giving it a default 2024-06-18 13:16:36 +01:00
Eric Allam 0ed93a748e v3: Remove aggressive otel flush timeouts in dev/prod 2024-06-18 11:48:35 +01:00
Michael Li e02320f65d fix: allow command login to read api url from cli args (#1168)
* fix: allow login to read api url from args

* Create strong-phones-smoke.md

---------

Co-authored-by: Eric Allam <eallam@icloud.com>
2024-06-18 09:43:42 +01:00
Eric Allam 85a543d8ec v3: usage tracking (#1163)
* Starting to measure wall time and cpu time in the workers, and reporting that via otel and to completed task run attempts

* Move usage tracking outside of the executor

* WIP prod usage tracking

* WIP

* WIP custom fetch to openmeter

* Create a usage client

* WIP

* WIP

* Implement new machine preset stuff and send usage reports to OpenMeter from webapp

* WIP

* Expose usage info to the client

* Add usage and cost to TaskEvent

* Add ability to globally configure the task machine preset

* Report start run usage

* Change the machine docs to use presets

* setExpirationTime to 24h

* Removed logs

* Update machines.mdx

* Removed console.logs

* Handle revalidating JWT tokens

* Couple tweaks

---------

Co-authored-by: Matt Aitken <matt@mattaitken.com>
2024-06-18 09:40:23 +01:00
Ryan Lee 10ceb85a92 fix: cloudfront timeouts not being caught (#1162) 2024-06-14 15:11:39 +01:00
Matt Aitken c405ae7117 Schedule limits and timezone support (#1165)
* Added maximumScheduleInstancesLimit column to Org, default to 20

* Docs on the schedule limits and improved soft-limit communication

* Added limit info to the schedules list page

* Created a task that creates schedules, useful for testing

* Make deduplicationKey required when creating/updating a schedule using the SDK

* New schedule button shows an alert if you’re over the limit

* Added timezone to the form and db

* WIP on the timezone dropdown for the create/edit schedule form

* Use the new filter search for timezones

* Made the timezone dropdown faster by fixing the virtualization

* The preview table is working and added a nice message about daylight savings

* Created a page where you can view the full list of timezones

The URL is included in the error message if you send an invalid time using the SDK

* Creating tasks with the timezone

* Added timezone support the the scheduler and the schedules list

* Added timezone support to more of the schedules UI

* The timezone comes through to scheduled runs with nice JSDocs

* Allow setting the timezone from the SDK

* Always have a timezone on a schedule

* Updated jsdocs

* Updated catalog example

* Changed the column to be a string, not null. Added the timezone across the SDK

* API endpoint for getting the timezones

* Added an SDK function to get the list of timezones

* Added timezones to the docs

* Changeset: Added timezone support to schedules

* Added support for testing timezone

* Tidied up imports

* Imports

* Imports

* Update limits.mdx

* Fixed a couple type issues and use the already exported zodfetch

---------

Co-authored-by: Eric Allam <eallam@icloud.com>
2024-06-14 13:31:27 +01:00
nicktrn 3687fcb61e Make pod cleaner interval configurable 2024-06-14 12:12:42 +01:00
Émile Ré d4ccdf7105 v3 CLI compiling E2E test suite (#1135)
* Boilerplate server-only use case

* wip: integration suite instrumentation setup

* Working poc testing compileProject

* Add pnpm script to run e2e tests only

* Use vitest globals

* Remove commented line

* Remove useless export

* Add modifier to test only one fixture project

* Handle package manager and log level choice

* Update server-only example

* Setup / teardown + split compile for package manager capabilities

* Ignore yarn files

* Fix issue with corepack, store version in engines field

* Rename test file

* Fix npm updates yarn.lock

* Move typecheking in a dedicated test

* Stop bundling the compile command to allow for more granular testing

* Put config resolving in separate test

* Add no-config test case and add test case expected errors configuration

* Add wantCompilationError option

* Add dependencies handling

* Use packageManager passed as option to resolve required deps

* Remove unused guard clauses

* Add postinstall & hash handling step

* Add worker start test

* Handle yarn.lock copy renaming on sigterm and sigkill

* Update vitest and use concurrent option

* Add a readme file

* Add CI workflow

* Fix handle cli deps

* Run cli v3 e2e tests on publish action

* Increase timeout on deps resolving step

* Add changeset

* Remove .pnp.cjs as we use yarn with nodeLinker node-modules

* Add missing .yarnrc.yml file

* No need to build CLI to run E2E tests

* Remove bun.lockb files

* Update beige-pears-explode.md

---------

Co-authored-by: Eric Allam <eallam@icloud.com>
2024-06-14 11:28:36 +01:00
nicktrn e08b4569e5 Improve checkpoint restore logging 2024-06-13 10:18:41 +01:00
nicktrn 79da0ca9b5 Update lockfile 2024-06-12 17:37:58 +01:00
github-actions[bot] 3aca603a33 chore: Update version for release (beta) (#1153)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-06-12 17:34:47 +01:00
Eric Allam c9e97d6b78 Add debug logging when we skip importing a resource span 2024-06-12 17:27:12 +01:00
nicktrn 01633c9c03 Print error logs on dev worker failure 2024-06-12 13:08:50 +01:00
nicktrn 691990d79e Remove catalog example that was causing typecheck issues 2024-06-12 12:48:19 +01:00
nicktrn b2ba403dd3 Improve error logs cli message 2024-06-12 12:04:59 +01:00
nicktrn 1d47cab69f Ensure rollback triggers execution of tasks waiting for deploy 2024-06-12 11:51:51 +01:00
nicktrn e23047f9ad update pre-release script 2024-06-12 09:49:47 +01:00
nicktrn 68d32429b6 v3: checkpoint failover and misc fixes (#1157)
* configurable checkpoint registry namespace

* add missing task create await

* remove unused messages

* changeset

* update self-hosting docs

* capture and display stderr for failed deploys

* add missing lockfile changes

* stderr changeset

* fix cli stderr message

* update error logs label
2024-06-11 14:06:50 +01:00
Eric Allam 36ac79ac66 Make sure users cannot revoke invites for other orgs 2024-06-11 11:09:52 +01:00
Eric Allam ca94f0cac3 cli v3: fix deploys by using sub-path imports (fixes execa require ESM error) 2024-06-11 11:08:15 +01:00
nicktrn a5d8e453a5 v3: deploy rollbacks (#1154)
* add deployment rollbacks

* update test presenter

* update task list presenter

* fix test page for dev env

* read replica for the test presenter
2024-06-11 10:09:29 +01:00
Eric Allam c332519e72 v3: Add a much needed index on TaskEvent.spanId 2024-06-10 20:54:13 +01:00
Matt Aitken 52112c3bfc Docs: added v3 limits page 2024-06-10 16:07:08 +01:00
Eric Allam eae294a332 Add back in the v2 timeout task thing 2024-06-10 16:06:21 +01:00
Eric Allam 465cd0335c v2: No longer eagerly timeout runs when no tasks are created 2024-06-10 14:54:22 +01:00
nicktrn 35dbaedf69 v3: self-hosting (#1147)
* add amin email regex env var

* fix displayed init command for self-hosted setups

* shared env var to disable telemetry in cli and webapp

* pin sdk version during init

* if specified, add api url to dev command shown after init

* improve checkpoint support detection

* control forced checkpoint simulation via env var

* add public init to providers

* better checkpoint support check for coordinator

* add docker to coordinator image

* update docker provider containerfile

* bump remaining containers to node 20

* add infra image build to default publish workflow

* lockfile

* remove concurrency group from infra workflow

* add docker provider to build matrix

* fix var subst

* checkpoint test is docker specific

* enable v3 projects by default on self-hosted instances

* fix v3 setup command again

* add default posthog key

* self-hosting docs

* add latest tags to versioned infra and webapp builds

* some checkpoint errors should skip retrying

* add changeset

* shorten paragraph

* some docs updates

* update tunnelling section

* add registry setup section

* use correct cli push flag

* add checkout to v3 branch

* update the worker machine setup steps

* fix infra build

* small docs update

* remove unused feature function

* Revert "remove unused feature function"

This reverts commit cfe07887a12b6893dca8ce499964481a9b3dc9db.

* fix self-hosted v3 feature gate

* add note about missing arm support

* simplify helper script syntax
2024-06-10 14:13:04 +01:00
Eric Allam c11a77f50b cli v3: increase otel force flush timeout to 30s from 500ms 2024-06-07 20:46:03 +01:00
Eric Allam fb52b9efea Remove redundant log (you’re welcome baselime) 2024-06-07 20:41:34 +01:00
Matt Aitken 0896b9fffc Use the read replica more (#1152)
* Switch to read replica: getEvent API endpoint

* Switch to read replica: v2 run list presenter

* Switch to read replica: Job presenter

* Switch to read replica: Job list presenter

* Switch to read replica: billing client

* Switch to read replica: OrgUsagePresenter

* Switch to read replica: OrgBillingPlanPresenter

* Switch to read replica: ScheduleListPresenter

* Switch to read replica: EventRepository taskEvent.findMany
2024-06-07 15:47:54 +01:00
Matt Aitken 3a2dd983c5 Fix for sendEvent same id causing multiple runs (#1151)
* Proof of concept

* When ingesting events, if it’s already been delivered then don’t continue

* DeliverEvent: throw AlreadyDeliveredError and don’t retry if that’s thrown

* Test for duplicate event ids

* Return the original event so sendEvent doesn’t fail, don’t enqueue

* Add AlreadyDeliveredError to the logged out message

---------

Co-authored-by: Eric Allam <eallam@icloud.com>
2024-06-07 15:10:53 +01:00
Eric Allam a627ca67d1 Use a different logger key for the marqs version 2024-06-07 14:19:05 +01:00
Eric Allam afc180aa70 marqs: Concurrency monitor that runs periodically and vacuums completed runs (#1150) 2024-06-07 13:36:02 +01:00
Eric Allam 393af1b7c5 Add ability to enable v2 marqs requeuing (and remove deprecated marqs visibility requeuing) 2024-06-06 17:14:15 +01:00
Eric Allam 6a91fb89b8 Release 3.0.0-beta.36 2024-06-06 16:00:07 +01:00
github-actions[bot] df7d1de16d chore: Update version for release (beta) (#1142)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-06-06 15:59:21 +01:00
Eric Allam 8e8ed4a3bf Remove queueName option from ZodWorker 2024-06-06 14:49:45 +01:00
Eric Allam 8fc8f57b39 v2: MarQS powered job executions (#1149)
* WIP

* Allow marqsv2 and v2 graphile to run in parallel

* Fix missing GraphileLogger import

* Fixed heartbeat after rebase

* Replace postgres based run counters with redis ones with a backfill

* Add back in the graphile logger

* Remove duplicate visibility timeout calls

* Clamp simple weighted strategy to max of 5
2024-06-06 13:56:27 +01:00
Eric Allam 9ebd91ccec Fixed auto-pagination example 2024-06-06 13:26:16 +01:00
Eric Allam 665f7c9756 v3: runs.retrieve fixed payload value 2024-06-06 13:21:59 +01:00
Matt Aitken 928a632e23 The IngestSendEvent rate limiter is now a singleton 2024-06-06 09:18:27 +01:00
Eric Allam 74db2de1bc Use graphile strategy 0 (no named queues) and remove all named queues 2024-06-05 19:08:00 +01:00
Eric Allam 93acca6c3c maxPoolSize should be +1 from concurrency 2024-06-05 17:50:15 +01:00
Eric Allam ebe079d83c Patch graphile-worker to log out the getJob query 2024-06-05 17:45:24 +01:00
Matt Aitken d272996de3 Use the env.server import 2024-06-05 17:20:44 +01:00
Matt Aitken 531bd4970d VERBOSE_GRAPHILE_LOGGING env var to control logging (defaults to “false”) 2024-06-05 17:18:13 +01:00
Matt Aitken 5c9eb25b5a Added very verbose Graphile logging 2024-06-05 16:59:21 +01:00
Matt Aitken c970e892a7 Added GraphileLogger 2024-06-05 16:20:19 +01:00
Matt Aitken a867b6e5ae Log out Graphile worker:getJob:error events 2024-06-05 15:24:00 +01:00
Matt Aitken d44abbd0fc Only auto-index v2 endpoints every hour. Timeout endpoint indexes after 5s 2024-06-05 14:39:17 +01:00
Matt Aitken 1cc680ac1e Fail runs if they’re disabled for an org 2024-06-05 13:54:01 +01:00
Matt Aitken 9b049bc480 Set more Graphile worker job priorities to zero, including endpoint indexing 2024-06-05 12:28:30 +01:00
Matt Aitken c24a23b551 Docs: env vars SDK docs 2024-06-05 11:41:14 +01:00
Matt Aitken ee1ae1fca6 Docs: improved trigger.config docs for instrumentations, env vars and troubleshooting 2024-06-05 11:41:14 +01:00
nicktrn 8e5ef176a4 v3: registry maintenance (#1146)
* retry checkpoints with backoff and optional failover registry for deploys

* never abort checkpoint cleanup

* simulate checkpoint failure for 5 minutes

* add flag to simulate checkpoint push failure

* add flag to control push failure simulation duration

* backoff with helper

* handle all coordinator errors

* improve stop retrying

* increase cleanup ipc timeout

* improve webapp socket.io handler error logging

* remove unused backoff function
2024-06-05 11:06:18 +01:00
nicktrn 58b6b1aa0d Schedule task requeue when dequeueing message in shared queue 2024-06-04 11:02:15 +01:00
Eric Allam 9c0ae1459f v2: Add ability to manually pause certain performRunExecutionV3 jobs in graphile 2024-06-03 22:22:48 +01:00
Eric Allam 2f15a84320 Fixed continueRun service 2024-06-03 21:51:06 +01:00
Matt Aitken a49a0ff416 Set the priority for deliverRunSubscriptions & deliverRunSubscription to zero (the highest) 2024-06-03 20:13:39 +01:00
Eric Allam b703ffed29 v2: decrease performRunExecutionV3 reschedulings 2024-06-03 14:32:17 +01:00
Eric Allam b4f9b70ae2 v3: Support triggering tasks with non-URL friendly characters in the ID 2024-05-31 11:39:17 +01:00
Matt Aitken 51bb4c887a Support custom queue when triggering a task (#1138)
* Created a v3-catalog test script for queues

* SDK: Fix for calling trigger and passing a custom queue

* Support custom queue in TriggerTaskService

* Improved the script in the catalog so it’s clearer what’s going on

* Remove the concurrencyLimit from a queue if the limit is null

* Fix for the test code… stupid
2024-05-31 11:24:13 +01:00
Eric Allam ba71f959e2 v3: Runs List Management SDK and SDK improvements (#1133)
* Improved the existing runs API

* WIP next runs API

* Improve the returned ApiPromise to add ability to return response

* More WIP

* WI{

* Added offset/limit pagination stuff like the cursor one, and converted all API methods to use ApiPromise

* More run API stuff

- Adding schedule  output from the retrieveRun endpoint
- Ability to filter by schedule and isTest

* Remove env from retrieve run in openAPI

* prefer duplication over merge

* WIP docs

* Use spread to DRY up some run API schemas

* Finish the overview docs

* Adding changeset

* Fixed typecheck errors

* Typo fix

* Re-export zodfetch from core so the v3 CLI can use it

* Fixed type errors

---------

Co-authored-by: Matt Aitken <matt@mattaitken.com>
2024-05-31 11:02:28 +01:00
Matt Aitken bc7bbd4576 v2 IngestSendEvent rate limit (#1134)
* Easier to create a rate limiter, use it in the ApiRateLimiter. Upgraded the Upstash package

* Always prefix any rate limiter in Redis with “ratelimit:”

* By default log when the rate limit is hit

* Added rate limiting to IngestSendEvent

* Log out the EventRecord id

* Increase events.deliverScheduled attempts

* INGEST_EVENT_RATE_LIMIT_MAX is optional

* Removed old API rate limit code

* IngestSendEvent rate limiter is optional. Moved outside of the DB transaction

* Log a message out when the rate limiter is created

* Return undefined if the rate limit has been crossed
2024-05-31 10:00:03 +01:00
Matt Aitken 5fe23e4b3f Troubleshooting guide (docs) (#1139)
* Started troubleshooting guide

* More work on the troubleshooting guide

* NestJS troubleshooting and using await when triggering

* Fleshed out more

* Added Prisma

* Tweaks
2024-05-30 22:03:54 +01:00
Matt Aitken 7b3b2e0d8e Deal with Dev environments where the team member has been removed (#1141)
* Test page: don’t show deleted dev environments

* Function to filter out environments that are dev and have no orgMember or orgMemberId

* Don’t show tasks from deleted members (or the associated environment labels)

* Show “Dev: Deleted” if the user has been removed.

* Only show your environment in the env vars table

* Renamed function

* Use the displayableEnvironment function
2024-05-30 17:47:09 +01:00
Matt Aitken 3900ddadce Release 3.0.0-beta.35 2024-05-30 12:17:23 +01:00
github-actions[bot] ca9e827bd3 chore: Update version for release (beta) (#1130)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-05-30 12:15:10 +01:00
Matt Aitken 04e936b69b Set the server-side indexing timeout to 180s (from 50s) 2024-05-30 12:08:52 +01:00
Matt Aitken 98ef170299 Set the CLI deploy timeout to 3mins from 1min 2024-05-30 12:08:14 +01:00
Eric Allam e69ffd314a v3: Refactor attempt creation to be worker requested (#1077)
* WIP worker TaskRunAttempt creation

* Handling failing task runs that cannot create an attempt for whatever reason

* Move the visibility queue stuff into a graphile job

* Fixed task runs with unsanitized queue names

* “Borrow” the code from alerts PR to get self hosted deployments working

* Add an admin API endpoint to get info about the shared marqs queue

* Allow admins to view any project metrics

* start adding lazy attempts to prod

* lazy attempt creation for prod workers

* resurrect prod stack traces

* add exception event to failed run spans

* simplify dependency resumes

* fix typecheck

* fix merge

* fresh process for all attempts

* always try sigterm first

* stop heartbeat timeout on non-inplace replace message

* add missing ack on checkpoint creation service failure

* bypass dequeue for retries with running worker

* respect retry delays

* crash runs with invalid run status for execution

* remove debug logs

* fix nack message

* fix version locking

* fresh attempt processes in dev and prod

* improve handling of ipc timeouts

* consider checkpoint failures on cancellation

* add basic chaos monkey to checkpointer

* changeset

* control forced checkpoint simulation via env var

* fix merge

* kill old attempt processes before checkpointing

* detailed perf logging for checkpointing

* add coordinator otlp endpoint example

* improve prod run cancellation

* rename supports lazy attempts migration

* fix graceful exit

* fix retry mechanics

* clear paused state before retry

* remove checkpoint image after push

* crash worker on unrecoverable errors

* refactor unrecoverable error emit

* switch to do hosted busybox image

* increase wait for duration ipc timeout

* add changeset for misc fixes

* fix merge

* fix retry delay span runId

* fix dev retries

* improve prod worker logging

* log checkpoint sizes

* add lazy attempts catalog entries

* Fixed merge issue: use zodFetch, not wrapZodFetch

* Revert "Fixed merge issue: use zodFetch, not wrapZodFetch"

This reverts commit d137e4e1fe.

* importEnvVars uses wrapZodFetch now

* add backwards compat for retries without checkpoints

* handle more cases of unrecoverable runs

* don't kill the child process if it shouldn't be killed

---------

Co-authored-by: nicktrn <55853254+nicktrn@users.noreply.github.com>
Co-authored-by: Matt Aitken <matt@mattaitken.com>
2024-05-30 11:05:52 +01:00
Parker 782d4f75ae Added tip for setting up github actions using npm run to simplify version pinning (#1132)
* added arg to install puppeteer deps to base image

* added tip on deploy setup

* removed docker changes
2024-05-28 12:10:24 +01:00
Matt Aitken b6de469d07 Proxy rate limit (#1131)
* Updated worker types

* Cloudflare rate limiter applied to sendEvent/sendEvents

* Latest wrangler

* Updated to the latest compatibility_date and nest the unsafe bindings properly

* Added some types from a Discord members

* Better logging and added a reset header so the SDK can use it from inside the run function

* Set staging proxy rate limit to 100/60s
2024-05-28 12:09:05 +01:00
Matt Aitken 0dd3447c31 Improved the migration from v2 to v3 guide 2024-05-28 09:39:21 +01:00
Matt Aitken a5a5d3ae21 We could already disable the queue by not setting AWS_SQS_QUEUE_URL
This reverts commit ee3619bbb1.
2024-05-27 11:13:55 +01:00
Matt Aitken ee3619bbb1 Added env var to enable/disable the v2 SqsEventConsumer 2024-05-27 11:04:12 +01:00
Eric Allam d9ad72446e Abort v2 runs when the job version they’re associated with is disabled 2024-05-26 21:30:05 +01:00
Eric Allam a56f9af9fe Safeguard against out of control v2 run executions 2024-05-26 20:06:10 +01:00
Eric Allam ece6ca678a Fix issue when using SDK in non-node environments by scoping the stream import with node: 2024-05-25 21:43:16 +01:00
Matt Aitken 6243ae30bb Send a “sign-up” event to Loops (#1129) 2024-05-24 18:05:37 +01:00
Matt Aitken cec20b96fd Release 3.0.0-beta.34 2024-05-24 15:32:04 +01:00
github-actions[bot] ea4d15536c chore: Update version for release (beta) (#1115)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-05-24 14:55:09 +01:00
Matt Aitken 2fa5780076 At the root path, if you have no projects yet but you do have an org then go to the new project page 2024-05-24 12:35:33 +01:00
Matt Aitken 914a394d15 Fix for create/edit schedule form blank page when an error is thrown from the service 2024-05-24 10:54:04 +01:00
Matt Aitken 80b1d8c6b1 Task list page: if there’s a newer version then show the icon/type etc for that 2024-05-24 10:54:04 +01:00
Matt Aitken 129c6022a0 Make the resolveEnvVars fn optional in the v3-catalog/trigger.config file 2024-05-24 10:54:04 +01:00
Eric Allam 1ca3b99ace Remove “PAUSED” status from being filterable 2024-05-24 09:44:17 +01:00
Matt Aitken dbbc7760f3 When logging out clear the projectId from the session if there is one 2024-05-23 18:27:31 +01:00
Matt Aitken ff3cd97a3b Fixed the syntax highlighting for JSON and TypeScript with the new Prisma renderer 2024-05-23 17:41:25 +01:00
Eric Allam 3a1b0c486a v3: env var management API (#1116)
* WIP env var management API

* Add import env var API endpoint

* Adding docs and support for using both API keys and PATs when interacting with the env var endpoints

* WIP envvar SDK

* Uploading env vars in a variety of formats now works

* Finish env var endpoints and add resolveEnvVars hook

* Add changeset
2024-05-23 16:11:25 +01:00
Jacob Paris 1f462eaa2f fix: secret store options was a circular import (#1117) 2024-05-23 16:04:21 +01:00
Jacob Paris f537778dc7 Separate classes from client code (#1119)
* fix: move route to route folder

* fix: move service class to its own file
2024-05-23 16:03:45 +01:00
Matt Aitken 974bcc49e8 Speed up the Run log queries (#1126)
* Added traceId index to TaskEvent

* Get less data and get the trace from the run

* Use the read replica in the EventRepository

* Use the regular client to get the run because the redirect from test can be very fast
2024-05-23 15:32:43 +01:00
Matt Aitken 60cbb87b08 New TaskRunNumberCounter that uses taskIdentifier + environmentId (#1124)
* Switch to new TaskRunNumberCounter that uses taskIdentifier + environmentId

* Drop old TaskRunCounter table (not used anymore)

* Revert "Drop old TaskRunCounter table (not used anymore)"

This reverts commit d7311a3d03cd8020df02105da155ceec4fa24b1b.
2024-05-23 13:28:23 +01:00
Matt Aitken 1281d40e4b v2: When a run hits the rate limit reschedule the re-execution (#1125)
* Fix: API rate limit error has the correct seconds until reset

* When a v2 run hits the rate limit, reschedule using the reset timestamp

* Still throw AutoYieldRateLimitErrors

* Reschedule runs from the rate limit

* The stress test timeout should be inside the task

* If the rate limit error is thrown, don’t retry the API request
2024-05-23 13:26:03 +01:00
Matt Aitken 116766f398 Request v3 access (from the app) and disable v2 projects by default (#1123)
* Added v2Enabled and hasRequestedV3 columns to Organization

* Don’t create a project when you create an org

* Form for requesting v3 access

* Reworked the new project form with the different version states. Refined copy on early access

* If the project isn’t in the org then redirect to the new project page

* Better message for existing users

* Tidy imports

* If it’s not the managed cloud then allow them to create v2 projects
2024-05-22 19:38:42 +01:00
Émile Ré c815f28c84 Update contributing instructions (#1121)
* Add OPEN_API_KEY env var in example file

* Update prerequisites and first-time setup
2024-05-22 16:12:40 +01:00
Eric Allam c092c0f9db v3: Prevent legacy-peer-deps=true from breaking deploys 2024-05-22 15:46:13 +01:00
Alexandre Costa a86f36cefa Fix TypeScript inclusion in tsconfig.json for cli-v3 init (#1105)
* Fix TypeScript inclusion in tsconfig.json for cli-v3 init

Fixed an issue where TypeScript files were included in the project directory when no include directive was present in tsconfig.json. Previously, the CLI added trigger.config.ts to the inclusion list by default, causing TypeScript compilation errors for other files. The fix ensures that trigger.config.ts is only added to the inclusion list if there's an existing include directive present in tsconfig.json

* Create hot-fishes-retire.md

---------

Co-authored-by: Eric Allam <eric@trigger.dev>
2024-05-21 14:40:49 +01:00
Eric Allam 5a6e79e0c0 Fix missing logs when client includes their own @opentelemetry/api package (#1106) 2024-05-21 14:38:06 +01:00
Jacob Paris 7cb20fbead fix: upgrade prism react renderer (#1111)
* fix: upgrade prism react renderer

* Update CodeBlock.tsx
2024-05-21 14:36:21 +01:00
Eric Allam a561bca315 Merge branch 'jacobparis-vite-inline-prisma-values' 2024-05-21 14:26:59 +01:00
Eric Allam 3f45738612 Fixed typescript errors 2024-05-21 14:19:33 +01:00
Eric Allam 05d3b08ae5 Merge branch 'vite-inline-prisma-values' of github.com:jacobparis/trigger.dev into jacobparis-vite-inline-prisma-values 2024-05-21 14:12:13 +01:00
Eric Allam 3f8b6d8fce v2: Better handle recovering from platform communication errors by auto-yielding back to the platform in case of temporary API failures 2024-05-21 14:07:16 +01:00
Jacob Paris 0a7ad232fb fix: reuse taskrunattemptstatus 2024-05-21 03:49:58 -06:00
Jacob Paris 04f863c42a Merge branch 'main' into vite-inline-prisma-values 2024-05-21 03:46:39 -06:00
Jacob Paris ae5ec86d0b fix: ensure enums satisfy prisma types 2024-05-21 03:44:22 -06:00
nicktrn f243eab9c9 v3: fix version locking (#1113)
* fix prod version locking

* fix worker version display
2024-05-21 08:35:35 +01:00
Matt Aitken a5cba375ae Bulk replaying and canceling from the runs list (#1109)
* WIP on multi-select

* WIP on simple checkbox

* CheckboxWIthLabel and Checkbox

* Multi-selection of runs across pages is working

* Fix for selection on seconds page

* Focus the run filter on page load

* Don’t focus the checkbox

* BulkActionBar now shows/hides and has buttons

* Some state to stop escape clearing the selection when the modals are open

* Delete unused formData util

* Improvements to the page

* Created the replay resource action. It doesn’t do anything useful yet.

* Database schema created for BulkActionGroup/BulkActionItem

* The BulkActionService is creating the right data, now we need to process it

* WIP on bulk processing

* Added failed state and made the sourceRun required

* Bulk replaying is working

* WIP on bulk action filtering

* Fixed bulk filters displaying

* Filtering by batch is working

* Some fixes for the bulk id filtering

* Style tweaks

* Load the extra info in parallel

* Bulk canceling working

* Get the most recent 20 bulk actions to display in the filter menu

* Even if the run isn’t cancelable add it to the final list

* Maximum of 250 runs can be bulk actioned

* Don’t let them select more than the maximum (250 currently)

* Separate each bulk item action into it’s own separate graphile job to increase resiliency

---------

Co-authored-by: Eric Allam <eallam@icloud.com>
2024-05-20 17:17:33 +01:00
Eric Allam c9ebe7f114 Update contributing guide for local v3 development 2024-05-20 15:43:31 +01:00
Eric Allam f4f87ad573 Defer the enqueuing of events.deliverScheduled events, to keep the graphile_jobs table smaller 2024-05-20 14:27:20 +01:00
Jacob Paris a98b62652c fix: vite cant resolve real values from prisma, only types are ok 2024-05-18 01:33:47 -06:00
Jacob Paris 38965d8921 fix: vite cant resolve real values from prisma, only types are ok 2024-05-18 01:31:22 -06:00
Matt Aitken 14cffd4e5c Fix for the EnabledStatus component icons and improved the style of the private Slack channel callout 2024-05-17 16:09:59 +01:00
Eric Allam 68107565a4 v3: Fix issues with posting alerts to public slack channels (#1108)
* Fix issues with posting alerts to public slack channels

* Use the actual values in the new environmentTypes column to display the environment type labels in the alerts list

* Implement environment alert options
2024-05-17 15:24:47 +01:00
Alec Dilanchian 8a263c8ab9 Adding note for supabase integration (#1101)
* Update management.mdx

Adding note for custom domains and initializing db object.

* Update management.mdx

---------

Co-authored-by: Matt Aitken <matt@mattaitken.com>
2024-05-17 15:11:29 +01:00
James Ritchie 6fecf2dbc0 More info about environments on the alerts page and alerts modal (#1107) 2024-05-17 14:22:40 +01:00
Matt Aitken a8db8ce4e4 Latest lockfile 2024-05-16 19:13:16 +01:00
github-actions[bot] a7ca1222d7 chore: Update version for release (beta) (#1104)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-05-16 19:00:06 +01:00
Eric Allam c9733f357f Fix issues with special characters in queue/task names causing runs to get stuck in queued 2024-05-16 17:35:50 +01:00
Matt Aitken ba61bfe3b9 Fixed feedback panel links (status page and docs) 2024-05-16 17:30:44 +01:00
James Ritchie 89eaef495f New alerts platform status + Alerts docs (#1098)
* debug tooltip now scrolls

* Added a link to subscribe to alerts and improved the project alerts blank state

* Better external link icon

* Docs: Removed webhook tasks

* Docs: removed limits performance

* Docs: removed FAQs

* Docs: Removed Architecture section

* Docs: Removed API reference: CLI

* Docs: Removed API reference: Objects

* Docs: Removed API reference: Functions

* Docs: removed automated tests

* Docs: removed Middleware

* Docs: removed Using APIs

* Docs: removed Rollbacks

* Docs: removed Trigger Filters

* Docs: removed Webhook Tasks

* Docs: removed Zod Tasks

* Docs: Renamed Community page

* Docs: Added a new Troubleshooting section and Alerts docs page

* Hide the New Alerts button again if list is greater than 10 items

* Customers now have to contact us for Slack Connect Support.

* Fix the alerts docs link

* Added env vars for a different alert email address, and whether the feature is enabled or not

* Only show the alerts sidemenu item if the feature is enabled

* Use a separate email client for sending alerts

* Removed the link to Context from v3 docs

* Removed obvious docs links that are now missing pages

---------

Co-authored-by: Matt Aitken <matt@mattaitken.com>
2024-05-16 16:03:25 +01:00
Eric Allam 6a379e4e91 Fix 3rd party otel propagation from breaking our Task Events data from being properly correlated to the correct trace 2024-05-16 12:07:21 +01:00
Eric Allam c0b815c363 Check for an existing restore event before trying to restore a checkpoint 2024-05-16 09:27:44 +01:00
Matt Aitken 598906fc4c Fix for typo in v3 CLI login command 2024-05-16 09:23:45 +01:00
Eric Allam b77ece1567 Log when disabling an endpoint 2024-05-15 21:51:08 +01:00
Eric Allam a73dc80177 Failing jobs shouldn’t log as an error 2024-05-15 21:49:52 +01:00
Eric Allam 2496917a7a v2: Graphile auto-cleanup and auto-endpoint disabling (#1103)
* Auto-cleanup failed graphile jobs instead of keeping them around

* Disable endpoint after a period of sequential indexing failures

* Remove log
2024-05-15 21:15:06 +01:00
Matt Aitken dc53f0f432 Added steps on how to test local tasks 2024-05-15 18:56:48 +01:00
Matt Aitken 87788f29c9 A v3 catalog readme 2024-05-15 18:53:38 +01:00
Eric Allam 6ede68648a Remove alerts side menu (again) 2024-05-15 17:25:41 +01:00
Eric Allam f24b5b770d Protect against doWork getting called mulitple times per consumer 2024-05-15 17:01:03 +01:00
Eric Allam cb81d6c835 Release 3.0.0-beta.32 2024-05-15 16:23:06 +01:00
github-actions[bot] 72748603ab chore: Update version for release (beta) (#1102)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-05-15 16:21:58 +01:00
Eric Allam f96f1e91a2 Adding changeset 2024-05-15 13:49:11 +01:00
Eric Allam 21c8b847e5 Better handle resolving dependencies when there are conflicts 2024-05-15 13:26:03 +01:00
Matt Aitken b39f79f0ab Use an env var for the Graphile cleanup ttl, set the default to 3 days (was 7) 2024-05-14 19:07:50 +01:00
Eric Allam 90b8797fbb Release 3.0.0-beta.31 2024-05-13 16:15:00 -04:00
github-actions[bot] 107f2e7bd5 chore: Update version for release (beta) (#1099)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-05-13 16:14:14 -04:00
Eric Allam b8477ea2b0 Fixes an issue with scoped packages in additionalPackages option 2024-05-13 15:31:19 -04:00
Matt Aitken 864498ea85 Latest lockfile 2024-05-13 17:46:02 +01:00
github-actions[bot] 2e21bc1563 chore: Update version for release (beta) (#1094)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-05-13 17:38:50 +01:00
Matt Aitken 1477a2e309 Set the timeout for canceling checkpoints 2024-05-13 16:50:53 +01:00
Matt Aitken 2081cb15b8 Removed docs for dashboard as they’re empty at the moment. Moved reference docs for Runs and schedules up 2024-05-12 18:29:46 +01:00
Matt Aitken ff551c5cbd Ticked off alerts (in preparation for putting it live) 2024-05-12 18:27:39 +01:00
Matt Aitken 794b25a988 Some fixes for the v3 docs showing v2 items in the sidebar 2024-05-12 18:27:26 +01:00
Matt Aitken 5fdb43ecd1 Bring alerts side menu item back 2024-05-12 18:05:01 +01:00
Matt Aitken 80997bcd6d .env.example Redis env vars are uncommented 2024-05-12 18:02:42 +01:00
Matt Aitken 194f336b84 Added an image of where to set the v3 GitHub action secret 2024-05-12 18:01:27 +01:00
Eric Allam 2e02743170 Resume rate limiting run executions, but reschedule throttled jobs 10 seconds into the future instead of their original run_at, which will help with database load 2024-05-10 15:44:20 -04:00
nicktrn 1642fd7baf perf: migrate to graphile worker v0.16.6 (#1097)
* migrate to graphile worker v0.16.6

* remove stale docs link

* fix jobs cleanup query
2024-05-10 18:07:16 +01:00
Matt Aitken c6449126b1 Don’t start runs if they have runs disabled 2024-05-10 16:11:06 +01:00
Matt Aitken 1ccf38e440 Return a 404 so the webhook provider doesn’t retry 2024-05-10 16:00:42 +01:00
Matt Aitken 29b773cd36 Prevent ingesting events if the org has runs disabled 2024-05-10 15:59:04 +01:00
Matt Aitken 73fb7cfeb3 Prevent HTTP source requests if the org has runs disabled 2024-05-10 15:58:51 +01:00
nicktrn 5431638926 Revert "Revert "Revert "Revert "disable advisory locks""""
This reverts commit 933a14b44c.
2024-05-10 15:09:35 +01:00
Eric Allam 933a14b44c Revert "Revert "Revert "disable advisory locks"""
This reverts commit 4ff07f8a94.
2024-05-10 09:35:53 -04:00
Eric Allam 4ff07f8a94 Revert "Revert "disable advisory locks""
This reverts commit af9957c085.
2024-05-10 09:30:32 -04:00
Eric Allam 0afdb4cc7c Help out the DB 2024-05-10 09:15:20 -04:00
Eric Allam af9957c085 Revert "disable advisory locks"
This reverts commit 892fd9f212.
2024-05-10 09:12:33 -04:00
Eric Allam 5c4275619b v3.1 experiments 2024-05-10 09:11:54 -04:00
nicktrn 892fd9f212 disable advisory locks 2024-05-10 10:21:48 +01:00
James Ritchie cbe9317a86 Added padding to the email body 2024-05-09 17:39:39 +01:00
James Ritchie 6baf9e5294 Dismiss the dev user tooltip in the task page when your mouse leaves the target 2024-05-09 17:05:17 +01:00
James Ritchie a823421324 Small style improvements to the emails 2024-05-09 16:49:45 +01:00
Eric Allam 0e919f56f2 Alerts v1 (#1065)
* Alerts v1

* Encrypt alert webhook secrets and allow them to be generated by the server

* Alert v1 UI

* Remove unnecessary emails

* Move to using `@react-email/components`

* WIP slack alerts

* More slack alerts WIP

* Update pnpm lock after rebase

* Finish implementing Slack alerts

* Use a more error like emoji

* New secondary variant for the segmented control

* Added a simple checkbox style variant to storybook

* Style tweak to the segmented control

* UI improvements to the alert modal

* Use searchable Select for alerts. Changed default variant for SegmentedControl

* Secondary button now using secondary colour

* segmented control style tweak

* Improved the channel column in the alerts table

* Updated logo-mono.png

* Updated email styles

* Email templates updated to new styles

* Don’t log the decrypted secret

* await enqueing the deployment alert when an index fails

* await enqueing the timeout alert

---------

Co-authored-by: James Ritchie <james@jamesritchie.co.uk>
Co-authored-by: Matt Aitken <matt@mattaitken.com>
2024-05-09 12:12:00 +01:00
Matt Aitken f90960fc13 Better display when there are lots of dev environment (#1093)
* Created an EnvironmentLabels component that puts extra dev environments behind a +

* Added new environment labels components to schedules pages
2024-05-09 10:32:38 +01:00
nicktrn 2306217697 v3: automatic pod cleanup (#1092)
* add automatic pod cleaner

* fix task monitor namespace override
2024-05-09 10:16:00 +01:00
nicktrn cfe4b5409a Fix lockfile 2024-05-08 17:27:34 +01:00
github-actions[bot] 223768a025 chore: Update version for release (beta) (#1091)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-05-08 17:25:22 +01:00
Ramadan Omar 1fd26ff98b Improvement: Added more context to Shopify webhook registering errors (#1090)
* Improvement: Added more context to Shopify webhook registering errors

Co-authored-by: Ahmed Ramadan <ahmedramadan1337@gmail.com>

* revert: shopify reference jobs

* fix: added error handling to some shopify client edge cases

* small changes to crud error handling

---------

Co-authored-by: Ahmed Ramadan <ahmedramadan1337@gmail.com>
Co-authored-by: nicktrn <55853254+nicktrn@users.noreply.github.com>
2024-05-08 17:22:59 +01:00
Eric Allam 8d34c63c2d Fix issue when concurrency limit is set to 0 on a queue 2024-05-08 10:19:06 +01:00
Zafer Cesur ff04c986a1 Add notice on sideEffects in deployment guide (#680)
Co-authored-by: Matt Aitken <matt@mattaitken.com>
2024-05-07 22:58:27 +01:00
Matt Aitken 7d6430dc46 New filtered dropdown and run filters (#1089)
* Extract out useFilterTasks

* WIP on Combobox

* Revert "WIP on Combobox"

This reverts commit f0a9b2e57c67f24d5e0876630e17a54252ceb86b.

* Added ariakit

* Started adding a Listbox component

* Listbox WIP

* More WIP on the list box

* More work on the Listbox

* WIP on filterable Listbox using headless… but it’s going to be a nightmare to make it good. Wrong approach

* WIP using Ariakit

* Multiple selection working

* WIP in reworking it so search is built in and it’s easier to use

* Nicer API for listbox

* Allow sections to be rendered with filtering

* More progress on the styling of the menu

* The empty state

* Removed focus outline

* Allow filtering based on the section title

* Render the shortcut correctly

* Allow passing static children to Select and render a separator if a section has no title

* Static example and added subtle focus

* Allow static data with no items array passed in

* Allow shortcuts on select items

* Only how heading if true

* Tooltip for keyboard shortcut

* useShortcutKeys fixes

- Can be passed an undefined definition (so we render the same number of hooks always).
- Fix for enabledOnInputElements not being used…

* Shortcut keys working with lists

* Don’t render shortcut numbers higher than 0

* More improvements

* WIP on a Link row

* Nicer API for the children function

* Much nicer shortcut key support

* Separated SelectTrigger out

* Heading separated out

* More styled elements created

* Removed unused element

* Reordered statuses

* Added search context

* Fixes for default values

* Removed tab elements

* Combo box

* Simplified it more

* Started work on Filter menu

* Experiments with filter menu

* Filter menu proof of concept working

* Nice hook for using a search param

* The escape key goes back to the previous menu

* Added filter page to storybook

* Added other variants

* Renamed Select file to OldSelect, implemented Feedback form

* Added placeholder support

* Added an optional dropdown icon (defaults to a chevron down)

* Feedback new Select

* Feedback just use static items

* New project page

* Renamed storybook pages

* Lots of progress on the new run filters

* Added new tasks filters

* Show the applied filters

* Added period filtering

* Fix for escape resetting the period value to the previous value when escape is pressed

* Show the Clear button sooner

* Removed unused code

* Reworked useSearchParams so multiple values can be set at once…

* Renamed file: OldSelect -> SImpleSelect

* Fix for new Select not working in forms because props weren’t being passed through

* New Select on the schedules page

* Better layout when there are lots of run filters

* Many style improvements to run filters

* Status filter allows you to select as well

* Rolled out the menus for the other files

* Minor improvements to schedule page and page padding

* Latest lockfile
2024-05-07 22:55:31 +01:00
Dan b289aa31da Readme updates - removed old roadmap and added dev preview EA link (#1078)
* Removed outdated roadmap

* Added link to the dev preview article
2024-05-07 22:55:00 +01:00
Eric Allam 0508de6258 Release 3.0.0-beta.28 2024-05-06 12:35:27 +01:00
github-actions[bot] 56a063e586 chore: Update version for release (beta) (#1087)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-05-06 12:34:43 +01:00
Eric Allam d490bc5cbf Add the "log" level back in as an alias to "info" 2024-05-06 12:28:17 +01:00
Eric Allam 6d9dfbc75d Add configure function to be able to configure the SDK manually
And move the ApiClient configuration away from AsyncLocalStorage and use our globals system instead
2024-05-06 12:12:30 +01:00
Eric Allam 6406924b02 Ensure @trigger.dev/sdk and @trigger.dev/core are always in the list of deployed dependencies 2024-05-06 11:01:49 +01:00
Eric Allam 2156e15266 Adding some additional telemetry during deploy to help debug issues 2024-05-06 10:34:54 +01:00
Eric Allam 49b4d47726 Release 3.0.0-beta.27 packages 2024-05-05 08:20:07 +01:00
github-actions[bot] 2e4f5d893c chore: Update version for release (beta) (#1086)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-05-05 08:19:13 +01:00
Eric Allam 203e002087 Add runs.retrieve management API method to get info about a run by run ID 2024-05-04 22:24:42 +01:00
Eric Allam 1670c4c41d Remove "log" Log Level, unify log and info messages under the "info" log level 2024-05-03 14:55:24 +01:00
Eric Allam 9d6c424236 Add deploy admin debug tooltips 2024-05-03 11:22:51 +01:00
nicktrn cffc88aeac Increase indexing specs 2024-05-03 09:45:18 +01:00
Eric Allam ae13757703 Release 3.0.0-beta.26 (fix server-only imports) 2024-05-03 09:40:32 +01:00
github-actions[bot] 0ee39e6101 chore: Update version for release (beta) (#1085)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-05-03 09:39:34 +01:00
nicktrn bce398b892 increase indexing specs 2024-05-03 09:36:16 +01:00
Eric Allam e667028d4a Strip out server-only package from worker builds 2024-05-02 21:28:41 +01:00
Eric Allam ae99807047 Allow admin tooltips while impersonating 2024-05-02 16:39:01 +01:00
Eric Allam bf4e505c70 Release 3.0.0-beta.25 2024-05-02 16:00:43 +01:00
github-actions[bot] c8538c236c chore: Update version for release (beta) (#1080)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-05-02 15:58:42 +01:00
Eric Allam ff6e2461d5 Add admin tooltips to make support/debugging easier for admin users 2024-05-02 14:12:16 +01:00
Eric Allam 3ed016bc2b Fixed SharedQueueConsumer internal otel logging 2024-05-02 12:31:55 +01:00
Eric Allam 68dd3804a7 Improve the fair queue selection strategy by paginating through all the queues in the parent queue 2024-05-02 11:56:35 +01:00
Eric Allam 95f0c7b780 increase queue selection count to 36 2024-05-01 22:41:50 +01:00
Eric Allam 339c39d739 Fix type issues and unit tests 2024-05-01 22:23:38 +01:00
Eric Allam bc0ca01319 Fix fair queue priority strategy off-by one error 2024-05-01 22:13:52 +01:00
Eric Allam c37c822314 Use locked package versions when resolving dependencies in deployed workers 2024-05-01 15:51:39 +01:00
Eric Allam 2dfe191c55 Fixed some docs that were missing imports 2024-05-01 14:47:50 +01:00
Eric Allam e337b21650 Add a postInstall option to allow running scripts after dependencies have been installed in deployed images 2024-05-01 14:16:52 +01:00
Eric Allam 9e5382951b Improve the display of non-object return types in the run trace viewer 2024-05-01 13:52:05 +01:00
nicktrn ed03f4bc15 fix lockfile 2024-05-01 11:33:10 +01:00
github-actions[bot] 9feb0f70b0 chore: Update version for release (beta) (#1079)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-05-01 11:32:15 +01:00
nicktrn 25de9e59af run workflows 2024-05-01 10:57:09 +01:00
nicktrn 83dc871550 add changeset 2024-05-01 10:48:27 +01:00
nicktrn 62700245a3 v3: fix consecutive waits (#1073)
* fix spacing for delete hints

* don't try to resume deps on wait resume

* sending duration wait resumes is not an error anymore

* set correct status with new wait resume flow

* cancel checkpoint schema v2

* don't mix messages and schemas

* prevent unintended case fallthrough in tree view

* completely switch to platform-led duration wait resumes

* prevent infinite restores

* some entries for the catalog

* add pg to additional packages

* add checkpoint safe timeout

* prevent duplicate spans after restore

* wait for post start

* add sdk version to deploy tab

* fail on impossible checkpoint scenarios

* remove debug logs
2024-05-01 10:33:30 +01:00
Matt Aitken 6ce820cb45 Test tasks that return different types 2024-04-30 19:07:44 +01:00
Matt Aitken 0f0a6884e8 Environment variables pasting uses dotenv (#1075)
* Add dotenv package to the webapp frontend, required some polyfills

* Use dotenv to parse the pasted env vars. Make the panel wider on larger screens
2024-04-30 14:23:49 +01:00
Matt Aitken 7ff8f0ebab Task and run page improvements (#1076)
* TaskListPresenter: if there are no tasks then don’t do stats queries

* RunListPresenter, use BasePresenter and the read replica

* Added populate script

* Simplified the Runs list query, added live timer

* Added TaskRun indexes for the RunList

* Status can’t be null now we’re using the TaskRun status

* Use defer so the page loads and shows a spinner

* Improved the loading style

* Get rid of latest run info from the tasks table super slow

* Fix for the activity graph tooltip getting clipped

* Added a code comment crediting the GitHub issue with the portal fix

* Add search to the tasks list

* Padding

* Fix for the schedules columns not being UTC

* Remove unused function
2024-04-30 14:19:50 +01:00
Matt Aitken 68455c796a Fixes/run filtered keyboard nav (#1074)
* Ensure each switch condition returns a state

* Fix for scrolling when filtered

* Fix for up/down navigation when filtered
2024-04-29 18:52:46 +01:00
Eric Allam b0a2c42e0e Add some additional logging around nacking messages 2024-04-29 17:53:38 +01:00
Andreas Thomas cac3c32f6a docs: add 'retry' import in code snippet (#1071)
* docs: add 'retry' import in code snippet

* docs: import all primitives
2024-04-29 17:42:36 +01:00
Matt Aitken ed8d24fd3d Run page performance improvements (#1072)
* lotsOfLogs task now outputs much larger logs

* Removed tree view collapse/expand animation

* JSDocs for useDebounce

* WIP moving filtering into the state

* Reworked the reducer to do the filtering
2024-04-29 17:05:26 +01:00
nicktrn d0ef36260a stop trying to pull init image when already present 2024-04-29 15:33:46 +01:00
Eric Allam 8eb68dd852 Fix pnpm lock file 2024-04-29 14:50:28 +01:00
github-actions[bot] a42037da03 chore: Update version for release (beta) (#1070)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-04-29 14:48:01 +01:00
Eric Allam 43bc7ed94e Hoist uncaughtException handler to the top of workers to better report error messages 2024-04-29 14:22:55 +01:00
Matt Aitken 4fdb7f8288 Bulk insert environment variables (#1069)
* Fix for #1066. Correct environment username if dev

* WIP changing the form

* WIP on the form

* WIP on repository

* Adding environment variables en masse is working

* WIP on pasting

* useList hook with reducer

* Got bulk insert working with pasting… it was a pain

* Allow overwriting of values

* Set a max height on the new env var form
2024-04-28 18:31:31 +01:00
Matt Aitken 37b9b056c4 Upload payload packets outside of the db transaction 2024-04-28 18:18:33 +01:00
Matt Aitken 801c86bf73 Added search to the test tasks list 2024-04-28 16:10:46 +01:00
Matt Aitken a1de11a001 Default the test page to the “DEV” tab 2024-04-28 15:59:45 +01:00
Praveen Pendyala 29e9e372ee Added example of github workflow for deploy to staging (#1068) 2024-04-26 18:39:34 +01:00
Eric Allam affc128161 Try to fix prisma generate errors 2024-04-26 18:28:17 +01:00
Eric Allam 40ba8ad0ee Fix the test page when a scheduled task has recent run payloads that aren’t scheduled payloads 2024-04-26 18:20:04 +01:00
Matt Aitken 96168eb383 Fixes for test page problems 2024-04-26 17:45:06 +01:00
Eric Allam 2c68473cc8 Fix pnpm lock file 2024-04-26 17:14:11 +01:00
github-actions[bot] 469c8a2532 chore: Update version for release (beta) (#1067)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-04-26 17:13:20 +01:00
Eric Allam ebeb790522 Add typescript as a dependency so the esbuild-decorator will work even when running in npx 2024-04-26 17:00:14 +01:00
Matt Aitken e765240370 Latest lockfile 2024-04-26 14:37:46 +01:00
github-actions[bot] 38770a91c7 chore: Update version for release (beta) (#1061)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-04-26 14:35:30 +01:00
Matt Aitken de188a22c5 Option for users to opt-in to a staging environment (#1064) 2024-04-26 11:29:20 +01:00
Matt Aitken fae93ac747 New tasks page (with activity) and multi-user fixes (#1063)
* Fix for “overridden” spelling mistake

* Make the task function names grey instead of yellow

* The task page now shows each task once

* Blankline

* WIP on activity bars

* Correct size of the graphs

* Add data for all days, even if there are no runs

* Activity graph is working and has a tooltip

* No activity label

* Got the bar bg working

* Activity chart now has a bg

* Added a compound id to make the activity graphs appear faster

* Improvements to the activity graph

* Tried to disable the bg from animating

* Added read replica support to the app, using Nick’s code from his batching PR

* Added the read replica to BasePresenter

* Added the queued and running columns to the Tasks list

* Added avg duration column

* Included completed but failed runs in the average duration

* Get rid of the old useDevEnvironment hook and remove API keys from the projects query

* Unified getting environment info to using a function and doing it in presenters

* Removed the path from the tasks table

* Onboarding for dev for 2nd+ user. Improved environment sorting.

* Removed log

* Improved the spacing

* Enable staging for new v3 projects
2024-04-26 10:44:39 +01:00
Eric Allam c3f6557eb6 Create publish preprelease script 2024-04-25 11:26:14 +01:00
Eric Allam 9491a1649c v3: Support decorators (#1060)
* Adds support for `emitDecoratorMetadata: true` and `experimentalDecorators: true` in your tsconfig

* Implement task.onSuccess/onFailure and config.onSuccess/onFailure

* Added onStart and more docs for lifecycle functions

* Use onStart instead of init for TypeORM
2024-04-25 11:17:13 +01:00
Eric Allam 0a5aa2dc15 Fix slow job retries by optimizing how errored tasks are deleted 2024-04-24 22:07:12 +01:00
Eric Allam 4a1a5b2e4a fix pnpm lock file 2024-04-24 15:07:22 +01:00
github-actions[bot] 503882762c chore: Update version for release (beta) (#1059)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-04-24 15:06:25 +01:00
Eric Allam e3db257397 Fix error stack traces 2024-04-24 14:18:53 +01:00
Eric Allam de1cc868e3 Fix dev CLI output when not printing update messages 2024-04-24 14:06:49 +01:00
Eric Allam ff7fa9e19a Clear out errored tasks when re-attempting a job run (instead of using the cached error) 2024-04-24 13:00:02 +01:00
Eric Allam ba4f04db37 Fix pnpm lock file 2024-04-24 12:49:06 +01:00
github-actions[bot] d7911892ee chore: Update version for release (beta) (#1058)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-04-24 12:48:00 +01:00
Eric Allam f888f85a09 Make the update trigger packages not print anything in dev 2024-04-24 12:16:49 +01:00
Eric Allam e9a63a4868 Lock SDK and CLI deps on exact core version 2024-04-24 12:12:24 +01:00
Eric Allam 93ecbc5e05 Fix pnpm lock file 2024-04-24 11:52:12 +01:00
Eric Allam 60a8a5777b Merge pull request #1057 from triggerdotdev/changeset-release/main
chore: Update version for release (beta)
2024-04-24 11:50:13 +01:00
github-actions[bot] ed1cd16753 chore: Update version for release (beta) 2024-04-24 10:43:28 +00:00
Eric Allam 6bb4dd7b6f Fixed messaging when trying to login with an existing invalid access token 2024-04-24 11:41:38 +01:00
nicktrn 0b5a0be807 update CI deploy error 2024-04-24 11:23:20 +01:00
nicktrn 74d1e61e42 v3: fix for fresh CLI logins after CLI token revoke (#1056)
* prettier login command output

* fix for trying to login again after revoking cli token

* changeset

* improve errors when logging in with revoked or invalid token

* fix builds.. again
2024-04-24 10:44:34 +01:00
Eric Allam 12c83a56af Skip e2e github workflows for now because playwright is broken 2024-04-24 10:33:11 +01:00
nicktrn 75a54540a4 use full image refs for init containers 2024-04-24 10:13:28 +01:00
Eric Allam b68012f81c v3: Various fixes for Next.js projects and projects that use v2 and v3 together (#1051)
* Fixes an issue that was treating v2 trigger directories as v3

* Make msw a normal dependency (for now) to fix Module Not Found error in Next.js.

* Extract out all the zod* stuff from core so the SDK does not import it

* Add a changeset

* Fixing typecheck errors in the webapp

* Export the Task and TaskOptions types

* Extract additional exports from core/v3 that aren’t used in the SDK

* Move to our global system from AsyncLocalStorage for the current task context storage

* Update the esbuild core bundling plugin for the new core v3 exports

* Fix v3 CLI telemetry

* Add support for tasks located in subdirectories inside trigger dirs

* Remove the env var check during deploy (too many false negatives)
2024-04-24 10:09:36 +01:00
nicktrn fb83d58703 fix builds 2024-04-24 08:48:32 +01:00
nicktrn 52b2a8289c fix private package installs (#1055) 2024-04-24 07:58:14 +01:00
nicktrn 1e93ec4216 Revert "change action runner location" (resolved)
This reverts commit cb27b7278a.
2024-04-24 07:45:08 +01:00
nicktrn cb27b7278a change action runner location ⬆️⬆️⬇️⬇️⬅️➡️⬅️➡️🅱️🅰️ 2024-04-23 18:13:04 +01:00
Eric Allam e85fc501a6 Fix pnpm lock file 2024-04-23 17:03:53 +01:00
github-actions[bot] 0bfeb0816f chore: Update version for release (beta) (#1054)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-04-23 16:58:48 +01:00
nicktrn b207601732 v3: semi-automatic package update command (#1052)
* fix package manager detection

* fix cli update check while in beta

* add update command and warn on mismatches

* update fixes and better integration with dev and deploy

* fix for ci

* update docs

* changeset

* be more specific about prompts before updates

* fail on version mismatch during ci

* update github actions docs

* fix package manager copypasta

* remove ncu

* ignore companyicons

* improve warnings and errors
2024-04-23 16:08:46 +01:00
Matt Aitken 3913e57ef4 Trace view fixes and improvements (#1046)
* Query param for span using history.replaceState is working

* When clicking again on a node, don’t collapse it

* Close the span view using the same replacing of the search param

* Conditional rendering of the resize panels was causing the tree view re-rendering and collapsing…

* Live reloading moved to where the parent label is

* Span action bar is now deeper

* WIP on trace view navigation changes with shortcuts

* Shortcuts for expanding and collapsing en masse

* Number keys expand/collapse levels

* Changed duration toggle to a shortcut key

* Option + click expands/collapse at that level

* Option/alt left/right expands/collapse at that level

* Removed unused imports

* Link from the runs table to the specific span

* Latest lockfile

* Sorted imports

* When doing a test link directly to a span

* Replay links to the span

* CLI log links go directly to a span

* Keyboard shortcuts are in a popover if the width is narrow

* If holding alt only collapse level

* Don’t expand the individual node if you’re holding alt
2024-04-22 13:46:08 +01:00
James Ritchie 26f310397a Fixed the grid layout for v2 runs page 2024-04-22 13:06:56 +01:00
github-actions[bot] 0a845767a0 chore: Update version for release (beta) (#1049)
* chore: Update version for release (beta)

* update lockfile

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: nicktrn <55853254+nicktrn@users.noreply.github.com>
2024-04-21 15:07:11 +01:00
nicktrn ed2a26c865 v3: fix additionalFiles, uncaught exceptions, and improve errors (#1048)
* fix additionalFiles for paths up the tree

* warn if additionalFiles pattern produced no matches

* improve deploy warnings and errors, fail early

* changeset
2024-04-21 15:03:00 +01:00
Eric Allam 4a68e71583 Fix pnpm lock file 2024-04-19 19:20:01 +01:00
github-actions[bot] b657eb6555 chore: Update version for release (beta) (#1044)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-04-19 19:19:11 +01:00
Eric Allam 62c9a5b712 Fix restoring after waiting for task/batch 2024-04-19 18:59:02 +01:00
Matt Aitken f339b41ef3 Added some basic Defer migration details for CRON 2024-04-19 18:30:58 +01:00
Eric Allam ae40ce3995 Fix the management file 2024-04-19 16:14:57 +01:00
Eric Allam 374edef020 Updates the trigger, batchTrigger and their *AndWait variants to use the first parameter for the payload/items, and the second parameter for options (#1045)
Also always returns a `TaskRunResult` object from `triggerAndWait` instead of rethrowing subtask errors in the parent
2024-04-19 14:51:51 +01:00
Eric Allam b82db67b81 Add additional logging around cleaning up dev workers, and always kill them after 5 seconds if they haven't already exited 2024-04-19 11:14:20 +01:00
Eric Allam 26093896d2 v3: Fixes for using (batch)triggerAndWait with idempotency keys (#1043)
* Fixes various issues with triggerAndWait and batchTriggerAndWait

When using idempotency keys, triggerAndWait and batchTriggerAndWait will still work even if the existing runs have already been completed (or even partially completed, in the case of batchTriggerAndWait)

- TaskRunExecutionResult.id is now the run friendlyId, not the attempt friendlyId
- A single TaskRun can now have many batchItems, in the case of batchTriggerAndWait while using idempotency keys
- A run’s idempotencyKey is now added to the ctx as well as the TaskEvent and displayed in the span view
- When resolving batchTriggerAndWait, the runtimes no longer reject promises, leading to an error in the parent task

* Remove the default queue concurrency limit as we now have env and org concurrency limits

* Use the run friendlyId in the completion result id

* Added some error logging
2024-04-19 10:54:43 +01:00
github-actions[bot] e7bd1ee676 chore: Update version for release (beta) (#1042)
* chore: Update version for release (beta)

* update lockfile

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: nicktrn <55853254+nicktrn@users.noreply.github.com>
2024-04-18 17:53:33 +01:00
nicktrn 584c7da5df v3: prod worker graceful shutdown (#1034)
* graceful exit with timeout

* handle and display graceful timeout errors

* fix for very long waits

* changeset

* increase termination grace period to an hour
2024-04-18 15:46:14 +01:00
nicktrn c9e1a3e9c5 v3: fix dependency trigger and wait (#1030)
* fix dev dependecy resumes

* add catalog entry with multiple trigger waits

* update docker provider lifecycle command

* switch to x for clearing run filters

* Revert "fix dev dependecy resumes"

This reverts commit b4061f2ae71f21889adcf061928e2e77c436f0ed.

* fix dependency resumes, properly this time

* add catalog entry for dependency waits in loops

* advice in docs re parallel dependency waits

* fix link from v3 to v2 docs

* move lifecycle command logging to debug only

* Removed batchOptions from the trigger options and the docs

---------

Co-authored-by: Matt Aitken <matt@mattaitken.com>
2024-04-18 14:48:43 +01:00
nicktrn 2f5b4a8471 v3: fix raw queries for custom schemas (#1033)
* add custom validation to db url env vars

* extract schema from db url and use in all raw queries

* use qualified names in scheduling raw queries

* cook a few raw queries

* Added missing raw query schema specifier to DeploymentListPresenter

---------

Co-authored-by: Matt Aitken <matt@mattaitken.com>
2024-04-18 12:01:38 +01:00
Eric Allam 69f6891687 Fixed rebalancing when message is acked
I’ve also added a “rebalancer worker” that will periodically rebalance parent queues, in case have been any issues with rebalancing (basically a “sweeper of entropy” process)
2024-04-17 21:00:39 +01:00
Matt Aitken acd7681e58 Scheduled task docs (#1039)
* Scheduled docs v1

* Improved copy

* Rejigged the docs some more. Added a dynamic section
2024-04-17 17:41:16 +01:00
Eric Allam 180a5ef01d Fix pnpm lock file 2024-04-17 16:34:14 +01:00
github-actions[bot] 62544d3234 chore: Update version for release (beta) (#1038)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-04-17 16:33:23 +01:00
Eric Allam 9b045071d8 Better errors when trying to delete a schedule that doesn’t exist 2024-04-17 16:05:51 +01:00
Eric Allam 618d2207c6 Fixed the deployed scheduled task check by looking at the slug, not the task.id 2024-04-17 15:38:39 +01:00
Eric Allam 36d8bee14a Don’t delete task runs if the attached task schedule or instance is deleted 2024-04-17 15:26:35 +01:00
Eric Allam 44e1b87547 v3: various schedule fixes (#1040)
* Improve the SDK function types and expose a new APIError instead of the APIResult type

* Skip triggering scheduled tasks if the task isn’t found in the current deployment

Also fixes an issue when editing the environments of a schedule
2024-04-17 15:10:23 +01:00
Matt Aitken 71b0ef8f77 Don’t allow editing the Deduplication key when updating a schedule 2024-04-17 14:42:18 +01:00
Matt Aitken f66543c9c7 When updating a schedule in the UI, it now says “Schedule updated” 2024-04-17 14:33:11 +01:00
Matt Aitken 03b104a3d5 Added JSDocs to the schedule options in the SDK 2024-04-17 14:25:40 +01:00
Eric Allam fde939a30e Fixed empty strings making their way into schedule externalId 2024-04-17 11:59:29 +01:00
Matt Aitken 4986bfda2e Scheduled tasks (#1036)
* Database schema and migrations for schedules

* Added schedules to the side menu

* The pagination can optionally hide the page numbers for a compactive mode

* Filters for the schedule page

* Added triggerSource (“STANDARD”, “SCHEDULED”) to BackgroundWorkerTask

* Added the ability to disabled a LinkButton

* Started work on the schedule page

* Environment buttons

* The new schedule form styles

* Added a cxouple of extra fields

* Allow a checkbox to have a rich label

* Added cronstrue package to the webapp

* WIP creating tasks using the form

* Improved the form styling

* Creating schedules is working in the UI

* Minor improvements

* Basic schedule table is displaying

* Creating tasks with triggerSource = scheduled. Refactored how task metadata is stored and accessed to be cleaner

* Resource route

* Fixes in the form

* Added a gap between the environments

* WIP on OpenAI generating CRON expressions

* AI generated CRON expressions is working

* Fix for the CRON field being uneditable after an AI generation

* Improvements

* Table padding

* useThrottle now behaves correctly

* Added filtering to the schedules list

* Improved the layout and fixed CRON search

* Fixed pagination for the schedules list. Just use a regular Prisma query

* Page size of 20

* Added links to the schedule rows

* Get rid of the Last run column for now

* Implement triggered scheduled tasks

Also implemented superjson payloads and dev runtime environment “presence” with RuntimeEnvironmentSession

* Move CronPattern and CreateSchedule into a common client-accessible file

* Latest UI changes

* Fix for creating a task schedule with a blank dedup key

* Refactor the human to cron stuff into a separate file and use json_object OpenAI response format

* Fix for trying to use a hook on the server-side…

* A couple of fixes to the new schedule form

* WIP on viewing a scheduled run

* Make the filters all optional

* Use the RunListPresenter from the schedule presenter

* Display a table of runs… the wrong runs but still

* Runs from the schedule

* Deleting schedules from the UI

* Tidied imports and fixed name of options object

* Disabling a schedule

* Editing schedules

* Tidied imports

* Added icons to the task list, needs some design love

* Added a tooltip for CRON pattersn

* Show the last run in the schedules table

* Some tweaks

* Added a placeholder to the CRON AI field

* Improved the trigger source icon

* Dim out disabled schedules

* Scheduled tasks have the correct icon in a run

* Added the task source icon to the test task list

* Added the date field component to storybook

* Style improvements to the date field

* Implement Task Schedule API

* added a medium sized variant to the date field

* Fixed replay run for superjson payload types

Also now linking from the replaying run to the original run (using span links)

Also added a project metrics prometheus endpoint to detect the state of the queues

* WIP on allowing different forms for testing

* If you pass a string to prettyPrintPacket which is json or superjson, safeParse it first

* Test page, deal json and superjson. Started splitting UI for schedules

* WIP on schedule form

* Removed the Label from the DateField

* WIP on test schedule form

* Fix for the runs page showing the wrong message when there are no runs from time filtering

* Removed labels from the DateField

* Fixes for the form

* Test runs are working for schedules

* Fix for nextScheduledTimestamps in triggerScheduledTask

* Implement idempotency key support and fix issue with cancelled runs causing concurrency usage

* Added API documentation for the schedule API

* Remove log

* Removed console log from runs page

* Transform the recent runs test data on the server

* Fix for hydration mismatch

* Current date as the default for the test form

* Recent payloads working

* Delete schedule modal

* Deal with empty strings from the form

* Set the initial value for the scheduled test form

* Add option to print console logs in the dev CLI locally (issue #1014)

* Export queue from the SDK

* Fix for schedules list when you have no schedule tasks

* Blank states improved

* Make task schedules more generic, to support additional schedule generators in the future

* Removed log from maqrs

* Removed “v3/schedules” export from the SDK

---------

Co-authored-by: Eric Allam <eallam@icloud.com>
Co-authored-by: James Ritchie <james@jamesritchie.co.uk>
2024-04-17 11:14:23 +01:00
Matt Aitken a3abe4ca08 Documentation edits made through Mintlify web editor 2024-04-17 09:08:08 +01:00
Matt Aitken fad32a79dd Documentation edits made through Mintlify web editor 2024-04-17 09:07:16 +01:00
Matt Aitken 9d843caf83 Documentation edits made through Mintlify web editor 2024-04-16 14:33:07 +01:00
nicktrn 5eaf7bca34 don't require git for build 2024-04-12 19:02:38 +01:00
nicktrn 4a0368dec3 lockfile 2024-04-12 18:18:49 +01:00
github-actions[bot] f10f120e55 chore: Update version for release (beta) (#1028)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-04-12 18:18:03 +01:00
nicktrn d3a18fbdf6 v3: cli and misc fixes for windows (#1027)
* fix: otlp-importer script

* update pluginPath

* Refactor script to ensure compatibility with both Windows and Unix platforms

* Revert command changes

* Fix: Init command was failing on Windows because of bad template paths

* Removed resolveInternalFilePath

* Path fixes for the dev command

* Fix for the import paths being wrong on Windows because the contain backslashes

* The metaOutputKey should have forward slashes in it

* Print the banner immediately, otherwise with bad internet you get a blank console for a long time

* Try normalizing the trigger.config import path

* Allow tsx and jsx files

* Improved task file names and paths

* Removed closing bracket for the upgrade message

* Log out the metafile outputs for debugging

* Log out the entryPointContents before esbuild

* Log the metafile out

* ballmerize the cli

* replace npm-watch with plain old nodemon

* fix text inputs on windows

* changeset

---------

Co-authored-by: Kritik Jiyaviya <kritikjiyaviya07@gmail.com>
Co-authored-by: Matt Aitken <matt@mattaitken.com>
2024-04-12 18:12:01 +01:00
nicktrn b82a07ad1c add missing ca certs to prod worker 2024-04-12 18:07:54 +01:00
Eric Allam fa81bf356a Fix pnpm lock file 2024-04-12 12:34:32 +01:00
github-actions[bot] fec4dc3bef chore: Update version for release (beta) (#1025)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-04-12 12:34:02 +01:00
Eric Allam a80089c88f Fix the package version of trigger.dev with the rest of the @trigger.dev/* packages 2024-04-12 11:19:04 +01:00
nicktrn 63a643b7c9 v3: fix digest extraction (#1024)
* fix image digest extraction

* fix project ref cli arg override

* Create late-steaks-behave.md

---------

Co-authored-by: Eric Allam <eric@trigger.dev>
2024-04-12 11:10:52 +01:00
nicktrn efd970a901 v3: correctly handle triggering tasks prior to deploy (#1019)
* add waiting for deploy status

* sort tasks filter

* marqs: optional timestamp on enqueue

* add new status to consts

* detect and execute tasks waiting for deploy

* display runs list even without any deploys

* ensure we only execute after the run creation tx has completed

* ensure correct execution order after waiting for deploy
2024-04-12 11:03:22 +01:00
Eric Allam 89a5e9f7da Move gc to it’s own admin endpoint 2024-04-11 16:24:40 +01:00
James Ritchie 404931e224 Made the message when you’re in a developer preview project punchier 2024-04-11 14:34:45 +01:00
Eric Allam 57bf98307a Run gc before taking heapsnapshot 2024-04-11 13:53:14 +01:00
Eric Allam 7551adfeb6 Add ability to disable tracing to test if it’s the cause of the memory leak 2024-04-11 13:14:02 +01:00
Matt Aitken 69117e8ba1 Re-arrange the mint.json file so the v3 docs become the default 2024-04-11 11:22:18 +01:00
Eric Allam b580d53b59 Better identifiy heap snapshots 2024-04-11 11:13:36 +01:00
Eric Allam d81e21d2ec Remove admin config endpoint 2024-04-11 10:44:26 +01:00
Matt Aitken 826a64fe6f If your dev run hasn’t started, display an info panel saying you need to run the CLI 2024-04-11 10:36:44 +01:00
Eric Allam d1849b0ea9 Adding admin config endpoint (temp) 2024-04-11 10:30:45 +01:00
Eric Allam 3aa634d2b8 Add a metric for number of event repo subscribers and project subscribers 2024-04-11 10:30:37 +01:00
Matt Aitken 7de3fb01c7 Added more docs for replaying a run 2024-04-10 18:35:43 +01:00
github-actions[bot] 4ae9a3feac chore: Update version for release (beta) (#1023)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-04-10 18:04:20 +01:00
Matt Aitken 5b18298fef Multiple machine CLI support (#1022)
* If a cli PAT exists, associate the authorization code with it

* Improved the error messages
2024-04-10 17:59:21 +01:00
nicktrn 7a9bd18ba2 v3: stop swallowing deployment errors and display them better (#1020)
* exit worker with code 111 for handled errors

* task monitor will ignore previously handled errors

* fix logging of index errors

* deployment error component

* changeset
2024-04-10 17:58:51 +01:00
Matt Aitken 03af44e545 Improved the title of one of the debugger scripts 2024-04-10 11:23:17 +01:00
Matt Aitken 5776257663 Changed the Discord link to use the trigger.dev one. Mainly to bump the GitHub action 2024-04-10 11:08:56 +01:00
Matt Aitken d3997c9fc6 Merge branch 'main' into changeset-release/main 2024-04-10 10:20:56 +01:00
github-actions[bot] 0676ea9668 chore: Update version for release (beta) (#1013)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-04-10 10:14:00 +01:00
github-actions[bot] 7a4721e122 chore: Update version for release (beta) 2024-04-09 16:44:12 +00:00
Eric Allam a5aed0d139 Fix memory leak by calling punsubscribe instead of unsubscribe
Also adding a /metrics endpoint with a few custom metrics and the built-in prom-client ones
2024-04-09 17:42:28 +01:00
Eric Allam f1480595d5 Stop swallowing webapp logs when the span isn’t being recorded 2024-04-09 17:11:09 +01:00
Eric Allam 328947dbfd Use the dashboard url instead of the API url for the View logs link 2024-04-09 15:15:16 +01:00
Eric Allam 279717b092 v3: pnpm deploy & various fixes for dependency resolution (#1012)
* Don’t swallow some error messages when deploying

* Try and resolve dependency versions using pnpm/npm/yarn

* Adding some debug logs around dependency resolution

* Use npm list instead of npm show to resolve deps

* Improve the dependency resolution
2024-04-09 15:03:31 +01:00
Matt Aitken 702d198445 Latest lockfile 2024-04-09 09:25:28 +01:00
github-actions[bot] 803f3c15ab chore: Update version for release (beta) (#1010)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-04-09 09:24:46 +01:00
nicktrn 1c24348f7d v3: add openssl to prod image and fix deploy command auth via env var (#1011)
* add openssl to prod worker

* make deploy command accept auth token via env var
2024-04-08 19:28:35 +01:00
Matt Aitken f854cb90eb Replay\Cancel a run (v3) (#1006)
* WIP on replaying a task from the run page

* Don’t pass the existing runs idempotency key, it will cause the replay to always return the original run

* Replay from the run list

* Don’t use fetchers in the replay/cancel dialogs

* API endpoint for replaying a run

* REST API docs (mostly coming soon) but added replay run

* replayRun function added to the SDK

* Cancel run added to the SDK

* v3-catalog file to test canceling and replaying

* Changed the SDK to be runs.replay and runs.cancel

* Removed comment

* Latest lockfile
2024-04-08 19:08:44 +01:00
github-actions[bot] 7268f17b00 chore: Update version for release (beta) (#1009)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-04-08 14:11:17 +01:00
nicktrn 624ddce32f v3: fix prod worker node_modules permissions (#1008)
* set node user prior to npm install

* changeset

* add ffmpeg and ffprobe catalog entries
2024-04-08 14:05:05 +01:00
Matt Aitken 9be1557bb7 CLI v3 binary changed from trigger.dev to trigger.dev, to fix a Windows issues (#1007)
* Changed the v3 CLI binary name from trigger.dev to triggerdev to fix a Windows issue

* Changed the exec to triggerdev
2024-04-08 14:03:12 +01:00
nicktrn 6ce6f8e3ad fix lockfile 2024-04-05 16:51:10 +01:00
github-actions[bot] d462b7a51b chore: Update version for release (beta) (#1001)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-04-05 16:45:38 +01:00
nicktrn f2894c177a switch to busybox and only call post start hook for restore (#1005) 2024-04-05 15:47:58 +01:00
Matt Aitken e35f29764a We want retrying enabled in dev by default, it’s confusing otherwise (#1004) 2024-04-05 14:45:19 +01:00
nicktrn 1207efbbad v3: prod image upgrade and fixes (#1003)
🚢 Publish Infra Images / build (coordinator) (push) Has been cancelled
🚢 Publish Infra Images / build (kubernetes-provider) (push) Has been cancelled
* fix shutdown after final attempt

* fix workdir permissions

* correctly handle self-hosted deploy errors

* switch to node:20-bookworm-slim

* add binaries catalog file

* changeset

* skip recommended packages

* post start hook retries

* comment and improve lifecycle retry
2024-04-05 13:09:07 +01:00
nicktrn 7ea8532cce v3: crash observability (#1002)
* indexing resource limits

* refactor and storage limits

* handle and display worker crashes

* enable prod cancellation

* fix reconnect delay

* improve crash messages

* log and display crash events

* changeset
2024-04-05 13:00:20 +01:00
Eric Allam 6642228f26 Fixing the dev & shared queue consumer telemetry so we can see what’s going on 2024-04-04 17:54:33 +01:00
Matt Aitken d39145d810 Defer migration guide: string not a date in the payload 2024-04-04 17:03:22 +01:00
Matt Aitken 8886bb76e0 Defer migration guide v1 2024-04-04 17:02:15 +01:00
Matt Aitken 4b72726078 defer.run migration guide 2024-04-04 17:02:15 +01:00
Eric Allam 6dcfeadaca v3: pnpm support (#1000)
* WIP fixing pnpm nested install strat

* Add changeset

* Pass (and log) cwd during background worker initialization

* Log out the env

* manually add the .pnpm/node_modules to NODE_PATH if in a pnpm repo

* Remove supports color dep
2024-04-04 16:13:04 +01:00
Matt Aitken ae839ebe11 Renamed Limits to “Limits & Performance” 2024-04-04 11:17:02 +01:00
Matt Aitken 5d0d71c2ae Added a section on upgrading your project 2024-04-04 11:17:02 +01:00
James Ritchie 1239a3ceb9 V3 project labels (#998)
* Added a side menu label to indicate your project version

* Added a link to request access to v3

* Toggle password visiblility of the env var inputs
2024-04-04 09:53:59 +01:00
Matt Aitken dd31b1e668 Fix for absolute URL from card 2024-04-04 09:23:40 +01:00
James Ritchie a707446989 Fixed Triggers pages not scrolling (#999) 2024-04-04 09:18:18 +01:00
Matt Aitken eb050f6730 Fix for typo on triggering page. “yourTask.batchTriggerAndWait” -> “yourTask.batchTrigger” 2024-04-04 09:12:29 +01:00
Matt Aitken 56d9bf7c67 Started the v2 -> v3 docs 2024-04-03 19:40:19 +01:00
Matt Aitken 73e469daf5 Added a known issues page to the docs 2024-04-03 14:30:43 +01:00
Matt Aitken 0382cf8719 Fix for bad tests link 2024-04-03 14:16:46 +01:00
Matt Aitken 0fe835492d Added more to the feature matrix 2024-04-02 19:47:55 +01:00
James Ritchie 29b69160e1 Fix for run page not scrolling the full runs content 2024-04-02 19:02:35 +01:00
Eric Allam ae9efe3d8b Added an error property to the problem JSON response 2024-04-02 17:46:40 +01:00
Eric Allam 3feb5ffb5f whitelist more api paths in the rate limiter 2024-04-02 17:29:53 +01:00
Eric Allam 7fb482de64 Allow OPTIONS requests in the rate limiter 2024-04-02 17:08:43 +01:00
Matt Aitken 6f11584aaa Remove our PostHog analytics from the docs 2024-04-02 15:35:36 +01:00
Matt Aitken 43e240cd50 Absolute link to v3 docs from v2 docs home, to fix nav issue 2024-04-02 15:30:44 +01:00
Matt Aitken afe7f410c7 v3 docs improvements (#993)
* WIP on adding versioning to the docs to make it less confusing

* Link to v3 from v2 and vice versa. Also added a warning to the v2 docs

* Make v3 the default

* Trigger config docs about bundling

* Next stpe to triggering guide from tasks overview

* Server actions guidance for triggering

* Moved the trigger.config docs to the end of fundamentals
2024-04-02 14:32:28 +01:00
nicktrn 28837f39b3 Prevent crash on upgrade errors 2024-04-02 13:58:40 +01:00
Eric Allam 5c64cefaf0 Fix pnpm lock file 2024-04-02 13:25:05 +01:00
github-actions[bot] dc5ed0a0cd chore: Update version for release (beta) (#995)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-04-02 13:24:32 +01:00
Eric Allam e7e7397ed9 Force CLI change 2024-04-02 13:20:26 +01:00
Eric Allam 49184c7189 Add trigger.dev changeset 2024-04-02 13:03:37 +01:00
Eric Allam eb60126284 Fixed batched otel in dev and made prod configurable to be batched as well 2024-04-02 12:58:56 +01:00
Eric Allam d876c358d2 Update the default env var concurrency limit and set them when orgs and envs are created 2024-04-02 11:55:00 +01:00
Eric Allam 6e9c8ab555 Fix pnpm lock file 2024-04-02 11:13:31 +01:00
github-actions[bot] 9a3eb289cd chore: Update version for release (beta) (#992)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-04-02 11:12:56 +01:00
nicktrn 51315fc3c8 v3: shared queue pool (#994)
* configurable retry delay checkpoint threshold

* shared queue consumer pool

* add more attributes to marqs spans

* inject trace context into queued messages
2024-04-02 10:22:52 +01:00
Eric Allam 4047f00562 Allow each service to define their own name 2024-04-01 22:01:36 +01:00
Matt Aitken 545f85b44e Fix for shared queryKey between useRunDetails and useRunStatuses 2024-04-01 13:20:39 +01:00
Eric Allam abe202e7b7 Fix pnpm lock file 2024-04-01 10:54:29 +01:00
github-actions[bot] ae27fd83af chore: Update version for release (beta) (#987)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-04-01 10:53:45 +01:00
Eric Allam c702d6a9ca v3: better task metadata errors (#991)
* v3: better handle task metadata parse errors, and display nicely formatted errors (dev, deploy, UI)

* Add changeset
2024-03-31 21:47:05 +01:00
Eric Allam 9af2570da6 Retry 429, 500, and connection error API requests to the trigger.dev server 2024-03-31 16:06:12 +01:00
Eric Allam a946797d95 v3: Upgrades to MarQS (#989)
* Upgrades to MarQS

- It’s “queue choosing” algorithm is now MUCH better (pretty much didn’t work before) and should be performant even when we have a bunch of prod queues
- There are now concurrency limits at the environment and organization level, as well as the task/queue level. So if any of them are at capacity the message won’t be dequeued. This means we can have an org wide concurrency limit, as well as different limits for dev/prod/staging
- I’ve added an admin API that can be used to update the org/env concurrency limits

* More MarQS upgrades

- Extract the queue priortity choosing strategy into an interface
- Implement a much better weighted average strategy
- “Slide the window” of queue candidates if the parent queue sends all at-capacity queues (so we won’t get stuck attempting to choose the same 12 full queues)
- Added some unit tests for the priority stuff
- Added some ideas for expanding the priority choosing strategy with more dynamic features

* Fixes an issue with the shared queue consumer relying on the queue name to get the env id
2024-03-31 15:58:00 +01:00
Eric Allam 8c4df326cc v3: Better dev/deploy errors and handle image build errors (#988)
* Better error output when an ESM only package is required

* Add better errors for npm package errors, and bring deploy errors to the dev CLI

* Handle depot build errors and write out a temporary file to view the build logs

* Add changeset
2024-03-29 12:46:11 +00:00
Eric Allam 11b997d2bf Make env var value input fields passwords 2024-03-29 09:05:53 +00:00
Eric Allam 8694e573f5 Fix CLI logout and add list-profiles command 2024-03-28 22:18:15 +00:00
Eric Allam b271742dca v3: implement configurable log levels via config file and env var (#985)
* v3: implement configurable log levels via config file and TRIGGER_LOG_LEVEL

Also, test runs automatically set the TRIGGER_LOG_LEVEL to debug

* Fix type error and changeset

* Added a Node.js runtime check for the CLI dev command
2024-03-28 22:00:38 +00:00
974 changed files with 74534 additions and 11944 deletions
+5
View File
@@ -0,0 +1,5 @@
---
"trigger.dev": patch
---
Fixes an issue with scoped packages in additionalPackages option
+9
View File
@@ -0,0 +1,9 @@
---
"trigger.dev": patch
"@trigger.dev/core": patch
---
- Fix additionalFiles that aren't decendants
- Stop swallowing uncaught exceptions in prod
- Improve warnings and errors, fail early on critical warnings
- New arg to --save-logs even for successful builds
+5
View File
@@ -0,0 +1,5 @@
---
"trigger.dev": patch
---
Add an e2e suite to test compiling with v3 CLI.
+5
View File
@@ -0,0 +1,5 @@
---
"trigger.dev": patch
---
v3 CLI update command and package manager detection fix
+5
View File
@@ -0,0 +1,5 @@
---
"trigger.dev": patch
---
Hoist uncaughtException handler to the top of workers to better report error messages
+5
View File
@@ -0,0 +1,5 @@
---
"@trigger.dev/shopify": patch
---
improved error messages when a shopify webhook fails to register
+6
View File
@@ -0,0 +1,6 @@
---
"trigger.dev": patch
"@trigger.dev/core": patch
---
better handle task metadata parse errors, and display nicely formatted errors
+5
View File
@@ -0,0 +1,5 @@
---
"trigger.dev": patch
---
cli v3: increase otel force flush timeout to 30s from 500ms
+5
View File
@@ -0,0 +1,5 @@
---
"@trigger.dev/core": patch
---
Fix error stack traces
+7
View File
@@ -0,0 +1,7 @@
---
"@trigger.dev/sdk": patch
"trigger.dev": patch
"@trigger.dev/core": patch
---
Implement task.onSuccess/onFailure and config.onSuccess/onFailure
+6
View File
@@ -0,0 +1,6 @@
---
"trigger.dev": patch
"@trigger.dev/core": patch
---
Remove "log" Log Level, unify log and info messages under the "info" log level
+2 -1
View File
@@ -4,7 +4,8 @@
"commit": false,
"fixed": [
[
"@trigger.dev/*"
"@trigger.dev/*",
"trigger.dev"
]
],
"linked": [],
+5
View File
@@ -0,0 +1,5 @@
---
"trigger.dev": patch
---
Fixing missing logs when importing client @opentelemetry/api
+7
View File
@@ -0,0 +1,7 @@
---
"@trigger.dev/sdk": patch
"trigger.dev": patch
"@trigger.dev/core": patch
---
Configurable log levels in the config file and via env var
+5
View File
@@ -0,0 +1,5 @@
---
"trigger.dev": patch
---
Dont swallow some error messages when deploying
+6
View File
@@ -0,0 +1,6 @@
---
"trigger.dev": patch
"@trigger.dev/core": patch
---
Add option to print console logs in the dev CLI locally (issue #1014)
+5
View File
@@ -0,0 +1,5 @@
---
"trigger.dev": patch
---
Strip out server-only package from worker builds
+5
View File
@@ -0,0 +1,5 @@
---
"@trigger.dev/core": patch
---
Fixed batch otel flushing
+5
View File
@@ -0,0 +1,5 @@
---
"trigger.dev": patch
---
Remove the env var check during deploy (too many false negatives)
+5
View File
@@ -0,0 +1,5 @@
---
"trigger.dev": patch
---
Better handle issues with resolving dependency versions during deploy
+5
View File
@@ -0,0 +1,5 @@
---
"trigger.dev": patch
---
Fix permissions inside node_modules
+7
View File
@@ -0,0 +1,7 @@
---
"@trigger.dev/sdk": patch
"trigger.dev": patch
"@trigger.dev/core": patch
---
v3: Environment variable management API and SDK, along with resolveEnvVars CLI hook
+9
View File
@@ -0,0 +1,9 @@
---
"@trigger.dev/sdk": patch
---
Make msw a normal dependency (for now) to fix Module Not Found error in Next.js.
It turns out that webpack will "hoist" dynamically imported modules and attempt to resolve them at build time, even though it's an optional peer dep:
https://x.com/maverickdotdev/status/1782465214308319404
+6
View File
@@ -0,0 +1,6 @@
---
"@trigger.dev/sdk": patch
"@trigger.dev/core": patch
---
Add runs.retrieve management API method to get info about a run by run ID
+5
View File
@@ -0,0 +1,5 @@
---
"@trigger.dev/react": patch
---
Fix for shared queryKey between useRunDetails and useRunStatuses
+5
View File
@@ -0,0 +1,5 @@
---
"@trigger.dev/core": patch
---
Support triggering tasks with non-URL friendly characters in the ID
+5
View File
@@ -0,0 +1,5 @@
---
trigger.dev: patch
---
Fix TypeScript inclusion in tsconfig.json for `cli-v3 init`
+7
View File
@@ -0,0 +1,7 @@
---
"trigger.dev": patch
"@trigger.dev/core": patch
"@trigger.dev/sdk": patch
---
v3: Usage tracking
+5
View File
@@ -0,0 +1,5 @@
---
"@trigger.dev/sdk": patch
---
Fix for calling trigger and passing a custom queue
+5
View File
@@ -0,0 +1,5 @@
---
"@trigger.dev/core": patch
---
Retry 429, 500, and connection error API requests to the trigger.dev server
+5
View File
@@ -0,0 +1,5 @@
---
"@trigger.dev/sdk": patch
---
Export queue from the SDK
+7
View File
@@ -0,0 +1,7 @@
---
"@trigger.dev/core-apps": patch
"trigger.dev": patch
"@trigger.dev/core": patch
---
Display errors for runs and deployments
+5
View File
@@ -0,0 +1,5 @@
---
"trigger.dev": patch
---
v3: fix digest extraction
+5
View File
@@ -0,0 +1,5 @@
---
"@trigger.dev/core": patch
---
Increased the timeout when canceling a checkpoint to 31s (to match the timeout on the server)
+5
View File
@@ -0,0 +1,5 @@
---
"trigger.dev": patch
---
Correctly handle self-hosted deploy command errors
+6
View File
@@ -0,0 +1,6 @@
---
"trigger.dev": patch
"@trigger.dev/core": patch
---
Fix issues with consecutive waits
+5
View File
@@ -0,0 +1,5 @@
---
"trigger.dev": patch
---
Adding some additional telemetry during deploy to help debug issues
+5
View File
@@ -0,0 +1,5 @@
---
"trigger.dev": patch
---
Default to retrying enabled in dev when running init
+5
View File
@@ -0,0 +1,5 @@
---
"@trigger.dev/core": patch
---
Add the "log" level back in as an alias to "info"
+5
View File
@@ -0,0 +1,5 @@
---
"trigger.dev": patch
---
Improve error messages during dev/deploy and handle deploy image build issues
+83
View File
@@ -0,0 +1,83 @@
---
"trigger.dev": patch
"@trigger.dev/core": patch
---
Adds support for `emitDecoratorMetadata: true` and `experimentalDecorators: true` in your tsconfig using the [`@anatine/esbuild-decorators`](https://github.com/anatine/esbuildnx/tree/main/packages/esbuild-decorators) package. This allows you to use libraries like TypeORM:
```ts orm/index.ts
import "reflect-metadata";
import { DataSource } from "typeorm";
import { Entity, Column, PrimaryColumn } from "typeorm";
@Entity()
export class Photo {
@PrimaryColumn()
id!: number;
@Column()
name!: string;
@Column()
description!: string;
@Column()
filename!: string;
@Column()
views!: number;
@Column()
isPublished!: boolean;
}
export const AppDataSource = new DataSource({
type: "postgres",
host: "localhost",
port: 5432,
username: "postgres",
password: "postgres",
database: "v3-catalog",
entities: [Photo],
synchronize: true,
logging: false,
});
```
And then in your trigger.config.ts file you can initialize the datasource using the new `init` option:
```ts trigger.config.ts
import type { TriggerConfig } from "@trigger.dev/sdk/v3";
import { AppDataSource } from "@/trigger/orm";
export const config: TriggerConfig = {
// ... other options here
init: async (payload, { ctx }) => {
await AppDataSource.initialize();
},
};
```
Now you are ready to use this in your tasks:
```ts
import { task } from "@trigger.dev/sdk/v3";
import { AppDataSource, Photo } from "./orm";
export const taskThatUsesDecorators = task({
id: "taskThatUsesDecorators",
run: async (payload: { message: string }) => {
console.log("Creating a photo...");
const photo = new Photo();
photo.id = 2;
photo.name = "Me and Bears";
photo.description = "I am near polar bears";
photo.filename = "photo-with-bears.jpg";
photo.views = 1;
photo.isPublished = true;
await AppDataSource.manager.save(photo);
},
});
```
+5
View File
@@ -0,0 +1,5 @@
---
"@trigger.dev/core": patch
---
v3: Remove aggressive otel flush timeouts in dev/prod
+5
View File
@@ -0,0 +1,5 @@
---
"trigger.dev": patch
---
Fixes an issue that was treating v2 trigger directories as v3
+5
View File
@@ -0,0 +1,5 @@
---
"@trigger.dev/core": patch
---
Fix issue when using SDK in non-node environments by scoping the stream import with node:
+6
View File
@@ -0,0 +1,6 @@
---
"@trigger.dev/sdk": patch
"@trigger.dev/core": patch
---
Added replayRun function to the SDK
+6
View File
@@ -0,0 +1,6 @@
---
"trigger.dev": patch
"@trigger.dev/core": patch
---
Better handle uncaught exceptions
+6
View File
@@ -0,0 +1,6 @@
---
"@trigger.dev/sdk": patch
"@trigger.dev/core": patch
---
Improve the SDK function types and expose a new APIError instead of the APIResult type
+5
View File
@@ -0,0 +1,5 @@
---
"trigger.dev": patch
---
Added a Node.js runtime check for the CLI
+6
View File
@@ -0,0 +1,6 @@
---
"@trigger.dev/sdk": patch
"@trigger.dev/core": patch
---
Add configure function to be able to configure the SDK manually
+5
View File
@@ -0,0 +1,5 @@
---
"trigger.dev": patch
---
Output stderr logs on dev worker failure
+5
View File
@@ -0,0 +1,5 @@
---
"trigger.dev": patch
---
Fix post start hooks
+5
View File
@@ -0,0 +1,5 @@
---
"trigger.dev": patch
---
Fix dev CLI output when not printing update messages
+5
View File
@@ -0,0 +1,5 @@
---
"trigger.dev": patch
---
Use the dashboard url instead of the API url for the View logs link
+105 -1
View File
@@ -44,15 +44,119 @@
"@trigger.dev/yalt": "2.3.18"
},
"changesets": [
"afraid-sheep-joke",
"angry-eagles-trade",
"beige-pears-explode",
"beige-pens-dance",
"big-tomatoes-deliver",
"blue-pumas-whisper",
"breezy-gorillas-mate",
"brown-spies-burn",
"chilled-hornets-move",
"clean-pianos-listen",
"clever-apes-collect",
"clever-carrots-travel",
"clever-donkeys-hunt",
"cool-comics-burn",
"cool-glasses-bake",
"cuddly-feet-approve",
"dry-walls-check",
"dull-mangos-press",
"eight-pumas-float",
"eleven-paws-join",
"famous-boats-tease",
"few-students-share",
"five-toes-destroy",
"funny-swans-destroy",
"gorgeous-gorillas-compete",
"green-bags-wink",
"hot-buckets-behave",
"hot-fishes-retire",
"hot-wasps-sin",
"itchy-chairs-itch",
"khaki-apricots-design",
"khaki-poems-lay",
"late-icons-lie",
"late-steaks-behave",
"lazy-files-lay",
"lemon-jobs-repair",
"light-bulldogs-press",
"light-dragons-complain",
"little-crabs-cross",
"long-fireants-search",
"loud-actors-remember",
"lovely-drinks-flash",
"many-ligers-pump",
"mighty-camels-joke",
"mighty-flowers-train",
"mighty-parrots-sin",
"nasty-jars-pump",
"new-pants-beg",
"new-rivers-tell",
"nice-bulldogs-turn",
"ninety-pets-travel",
"odd-poets-own",
"pink-pumas-rhyme",
"plenty-ducks-beam",
"polite-ducks-switch",
"polite-rockets-matter",
"poor-flowers-cross",
"purple-garlics-shop",
"rare-lamps-promise",
"rare-roses-float",
"real-planets-stare",
"rich-kangaroos-unite",
"rotten-beers-refuse",
"rotten-dryers-exercise",
"rude-toys-compare",
"selfish-ducks-sort",
"serious-hats-rest",
"shaggy-spoons-taste",
"sharp-emus-compare",
"sharp-zebras-serve",
"shiny-coats-cry",
"silly-buses-obey",
"silly-suits-switch",
"silver-doors-juggle",
"six-ligers-exist",
"sixty-insects-watch",
"slow-buses-own",
"slow-kiwis-hide",
"slow-sloths-retire",
"smart-needles-move",
"smart-olives-eat",
"sour-pugs-teach",
"spicy-lamps-smoke",
"spicy-terms-bow",
"strange-ghosts-matter",
"strange-sheep-pull",
"strong-lemons-add",
"strong-owls-know",
"strong-phones-smoke",
"stupid-adults-sniff",
"stupid-bulldogs-applaud",
"sweet-ducks-remember",
"sweet-lizards-press",
"tall-bees-wave"
"swift-dragons-peel",
"tall-bees-wave",
"tame-guests-know",
"tender-moose-tell",
"tender-oranges-rhyme",
"tender-turkeys-compete",
"thin-parents-heal",
"thirty-islands-kiss",
"tidy-balloons-suffer",
"tidy-dryers-sleep",
"tidy-tomatoes-explain",
"tiny-doors-type",
"tiny-elephants-scream",
"tricky-bulldogs-heal",
"tricky-keys-attack",
"tricky-ladybugs-unite",
"two-pumas-wait",
"violet-clocks-notice",
"warm-olives-provide",
"warm-planes-taste",
"young-snails-sell"
]
}
+5
View File
@@ -0,0 +1,5 @@
---
"trigger.dev": patch
---
Add typescript as a dependency so the esbuild-decorator will work even when running in npx
+6
View File
@@ -0,0 +1,6 @@
---
"trigger.dev": patch
"@trigger.dev/core": patch
---
Add a postInstall option to allow running scripts after dependencies have been installed in deployed images
+5
View File
@@ -0,0 +1,5 @@
---
"trigger.dev": patch
---
Add openssl to prod worker image and allow passing auth token via env var for deploy
+5
View File
@@ -0,0 +1,5 @@
---
"trigger.dev": patch
---
Fix a bug where revoking the CLI token would prevent you from ever logging in again with the CLI.
+5
View File
@@ -0,0 +1,5 @@
---
"trigger.dev": patch
---
Add git to prod worker image which fixes private package installs
+7
View File
@@ -0,0 +1,7 @@
---
"@trigger.dev/sdk": patch
"trigger.dev": patch
"@trigger.dev/core": patch
---
Adding task with a triggerSource of schedule
+7
View File
@@ -0,0 +1,7 @@
---
"@trigger.dev/core-apps": patch
"trigger.dev": patch
"@trigger.dev/core": patch
---
Capture and display stderr on index failures
+6
View File
@@ -0,0 +1,6 @@
---
"@trigger.dev/sdk": patch
"trigger.dev": patch
---
Lock SDK and CLI deps on exact core version
+5
View File
@@ -0,0 +1,5 @@
---
"trigger.dev": patch
---
v3: [prod] force flush timeout should be 1s
+56
View File
@@ -0,0 +1,56 @@
---
"@trigger.dev/sdk": patch
"@trigger.dev/core": patch
---
Updates the `trigger`, `batchTrigger` and their `*AndWait` variants to use the first parameter for the payload/items, and the second parameter for options.
Before:
```ts
await yourTask.trigger({ payload: { foo: "bar" }, options: { idempotencyKey: "key_1234" } });
await yourTask.triggerAndWait({ payload: { foo: "bar" }, options: { idempotencyKey: "key_1234" } });
await yourTask.batchTrigger({ items: [{ payload: { foo: "bar" } }, { payload: { foo: "baz" } }] });
await yourTask.batchTriggerAndWait({ items: [{ payload: { foo: "bar" } }, { payload: { foo: "baz" } }] });
```
After:
```ts
await yourTask.trigger({ foo: "bar" }, { idempotencyKey: "key_1234" });
await yourTask.triggerAndWait({ foo: "bar" }, { idempotencyKey: "key_1234" });
await yourTask.batchTrigger([{ payload: { foo: "bar" } }, { payload: { foo: "baz" } }]);
await yourTask.batchTriggerAndWait([{ payload: { foo: "bar" } }, { payload: { foo: "baz" } }]);
```
We've also changed the API of the `triggerAndWait` result. Before, if the subtask that was triggered finished with an error, we would automatically "rethrow" the error in the parent task.
Now instead we're returning a `TaskRunResult` object that allows you to discriminate between successful and failed runs in the subtask:
Before:
```ts
try {
const result = await yourTask.triggerAndWait({ foo: "bar" });
// result is the output of your task
console.log("result", result);
} catch (error) {
// handle subtask errors here
}
```
After:
```ts
const result = await yourTask.triggerAndWait({ foo: "bar" });
if (result.ok) {
console.log(`Run ${result.id} succeeded with output`, result.output);
} else {
console.log(`Run ${result.id} failed with error`, result.error);
}
```
+13
View File
@@ -0,0 +1,13 @@
---
"@trigger.dev/sdk": patch
"trigger.dev": patch
"@trigger.dev/core": patch
---
When using idempotency keys, triggerAndWait and batchTriggerAndWait will still work even if the existing runs have already been completed (or even partially completed, in the case of batchTriggerAndWait)
- TaskRunExecutionResult.id is now the run friendlyId, not the attempt friendlyId
- A single TaskRun can now have many batchItems, in the case of batchTriggerAndWait while using idempotency keys
- A runs idempotencyKey is now added to the ctx as well as the TaskEvent and displayed in the span view
- When resolving batchTriggerAndWait, the runtimes no longer reject promises, leading to an error in the parent task
+5
View File
@@ -0,0 +1,5 @@
---
"trigger.dev": patch
---
Update trigger.dev CLI for new batch otel support
+5
View File
@@ -0,0 +1,5 @@
---
"trigger.dev": patch
---
Add additional logging around cleaning up dev workers, and always kill them after 5 seconds if they haven't already exited
+5
View File
@@ -0,0 +1,5 @@
---
"@trigger.dev/core": patch
---
Add callback to checkpoint created message
+6
View File
@@ -0,0 +1,6 @@
---
"@trigger.dev/otlp-importer": patch
"trigger.dev": patch
---
Fix package builds and CLI commands on Windows
+5
View File
@@ -0,0 +1,5 @@
---
"@trigger.dev/core": patch
---
Make deduplicationKey required when creating/updating a schedule
+5
View File
@@ -0,0 +1,5 @@
---
"@trigger.dev/core": patch
---
Improve the display of non-object return types in the run trace viewer
+5
View File
@@ -0,0 +1,5 @@
---
"trigger.dev": patch
---
Set the deploy timeout to 3mins from 1min
+8
View File
@@ -0,0 +1,8 @@
---
"@trigger.dev/sdk": patch
"trigger.dev": patch
"@trigger.dev/core": patch
"@trigger.dev/cli": patch
---
Move to our global system from AsyncLocalStorage for the current task context storage
+6
View File
@@ -0,0 +1,6 @@
---
"trigger.dev": patch
"@trigger.dev/core": patch
---
v3: vercel edge runtime support
+7
View File
@@ -0,0 +1,7 @@
---
"@trigger.dev/core-apps": patch
"@trigger.dev/core": patch
---
- Fix uncaught provider exception
- Remove unused provider messages
+5
View File
@@ -0,0 +1,5 @@
---
"@trigger.dev/sdk": patch
---
Remove unimplemented batchOptions
+5
View File
@@ -0,0 +1,5 @@
---
"trigger.dev": patch
---
Fix CLI logout and add list-profiles command
+6
View File
@@ -0,0 +1,6 @@
---
"trigger.dev": patch
"@trigger.dev/core": patch
---
v3: fix otel flushing causing CLEANUP ack timeout errors by always setting a forceFlushTimeoutMillis value
+5
View File
@@ -0,0 +1,5 @@
---
"trigger.dev": patch
---
Fixing an issue with bundling @trigger.dev/core/v3 in dev when using pnpm
+9
View File
@@ -0,0 +1,9 @@
---
"@trigger.dev/core-apps": patch
"trigger.dev": patch
---
- Fix init command SDK pinning
- Show --api-url / -a flag where needed
- CLI now also respects `TRIGGER_TELEMETRY_DISABLED`
- Dedicated docker checkpoint test function
+6
View File
@@ -0,0 +1,6 @@
---
"@trigger.dev/sdk": patch
"@trigger.dev/core": patch
---
v2: Better handle recovering from platform communication errors by auto-yielding back to the platform in case of temporary API failures
+5
View File
@@ -0,0 +1,5 @@
---
"@trigger.dev/core": patch
---
Make optional schedule object fields nullish
+6
View File
@@ -0,0 +1,6 @@
---
"@trigger.dev/sdk": patch
"@trigger.dev/core": patch
---
When a v2 run hits the rate limit, reschedule with the reset date
+5
View File
@@ -0,0 +1,5 @@
---
"trigger.dev": patch
---
fix: allow command login to read api url from cli args
+6
View File
@@ -0,0 +1,6 @@
---
"@trigger.dev/sdk": patch
"@trigger.dev/core": patch
---
Management SDK overhaul and adding the runs.list API
+5
View File
@@ -0,0 +1,5 @@
---
"@trigger.dev/sdk": patch
---
v3: Adding SDK functions for triggering tasks in a typesafe way, without importing task file
+5
View File
@@ -0,0 +1,5 @@
---
"@trigger.dev/core": patch
---
Added JSDocs to the schedule SDK types
+5
View File
@@ -0,0 +1,5 @@
---
"trigger.dev": patch
---
Changed the binary name from trigger.dev to triggerdev to fix a Windows issue
+5
View File
@@ -0,0 +1,5 @@
---
"trigger.dev": patch
---
Use locked package versions when resolving dependencies in deployed workers
+5
View File
@@ -0,0 +1,5 @@
---
"trigger.dev": patch
---
Stop swallowing deployment errors and display them better
+6
View File
@@ -0,0 +1,6 @@
---
"@trigger.dev/sdk": patch
"@trigger.dev/core": patch
---
Added timezone support to schedules
+5
View File
@@ -0,0 +1,5 @@
---
"trigger.dev": patch
---
Ensure @trigger.dev/sdk and @trigger.dev/core are always in the list of deployed dependencies
+5
View File
@@ -0,0 +1,5 @@
---
"trigger.dev": patch
---
Fix for typo in v3 CLI login command
+5
View File
@@ -0,0 +1,5 @@
---
"trigger.dev": patch
---
Init command was failing on Windows because of bad template paths
+10
View File
@@ -0,0 +1,10 @@
---
"trigger.dev": patch
"@trigger.dev/core": patch
---
Fixes an issue that caused failed tasks when resuming after calling `triggerAndWait` or `batchTriggerAndWait` in prod/staging (this doesn't effect dev).
The version of Node.js we use for deployed workers (latest 20) would crash with an out-of-memory error when the checkpoint was restored. This crash does not happen on Node 18x or Node21x, so we've decided to upgrade the worker version to Node.js21x, to mitigate this issue.
You'll need to re-deploy to production to fix the issue.
+5
View File
@@ -0,0 +1,5 @@
---
"@trigger.dev/core": patch
---
Fix 3rd party otel propagation from breaking our Task Events data from being properly correlated to the correct trace
+6
View File
@@ -0,0 +1,6 @@
---
"@trigger.dev/sdk": patch
"@trigger.dev/core": patch
---
Added cancelRun to the SDK
+7
View File
@@ -0,0 +1,7 @@
---
"trigger.dev": patch
"@trigger.dev/core": patch
---
- Add graceful exit for prod workers
- Prevent overflow in long waits
+7
View File
@@ -0,0 +1,7 @@
---
"@trigger.dev/sdk": patch
"trigger.dev": patch
"@trigger.dev/core": patch
---
Added a new global - Task Catalog - to better handle task metadata
+14
View File
@@ -0,0 +1,14 @@
---
"trigger.dev": patch
"@trigger.dev/core": patch
---
- Clear paused states before retry
- Detect and handle unrecoverable worker errors
- Remove checkpoints after successful push
- Permanently switch to DO hosted busybox image
- Fix IPC timeout issue, or at least handle it more gracefully
- Handle checkpoint failures
- Basic chaos monkey for checkpoint testing
- Stack traces are back in the dashboard
- Display final errors on root span

Some files were not shown because too many files have changed in this diff Show More