Commit Graph

5252 Commits

Author SHA1 Message Date
Matt Aitken 4495fa9b95 Added ElectricSQL to docker-compose, available on 3060 2024-08-29 11:30:53 +01:00
Eugen Istoc 7c1339c108 Update upgrading-packages.mdx (#1273)
I believe the name of the executable in the `bin` dir is called `triggerdev` instead of `trigger.dev`
2024-08-28 10:50:52 +01:00
nicktrn 4cfb52ef65 Remove packageManager override from docs package.json 2024-08-28 10:39:34 +01:00
Matt Aitken f7d32b83b1 Free up concurrency when using triggerAndWait. Improved errors on runs (#1272)
* Add an error to the final attempt if there isn’t one

* Improved the checkpointResumer test task

* Using triggerAndWait or batchTriggerAndWait frees up concurrency

Normally it frees up just env and org concurrency. If it’s a recursive task then it will free up the run concurrency too (e.g. a task calling itself).

* Removed the filepath and export name from attempt spans

* Run inspector: only show an error if the run is in a finished state
2024-08-25 20:15:45 +01:00
Matt Aitken 0979a521bd Updated docs to 0.0.0-prerelease-20240825150620 2024-08-25 16:12:55 +01:00
Eric Allam e30beb779f Add support + docs for custom esbuild plugins (#1270)
* Add support + docs for custom esbuild plugins

* Fixed the missing subpath export

* Add support for specifying the prisma client generator

* Add `—env-file` support to dev and deploy
2024-08-25 16:04:26 +01:00
Matt Aitken 99a63d4cd5 When enqueuing clear concurrency for that run (#1271)
* Better logging if a background worker isn’t created

* When enqueuing a message into the queue clear the concurrency
2024-08-25 15:56:40 +01:00
Matt Aitken 6744617879 Added @ splat route 2024-08-23 18:28:28 +01:00
Matt Aitken d62979914f Updated the new build system docs with the prerelease package 2024-08-23 14:26:56 +01:00
Matt Aitken fc02e95f2a Removed another old changeset 2024-08-23 14:20:48 +01:00
Matt Aitken 8688822723 Another fix for changeset with old package (react) 2024-08-23 14:20:36 +01:00
Matt Aitken 0d829f9530 Fixed changesets for packages no longer in the repo 2024-08-23 14:20:00 +01:00
James Ritchie 6496539c01 Usage doesn’t show as a red bar if you’re beyond the plan 2024-08-23 14:04:57 +01:00
Matt Aitken 79a5c36d6e build pavkage: weird whitespace change 2024-08-23 14:04:46 +01:00
Eric Allam 2f0d65a490 Fix build typecheck 2024-08-23 13:58:36 +01:00
Eric Allam 92d98e5499 Build before running typecheck now 2024-08-23 13:46:18 +01:00
Matt Aitken a9810b0fd8 Stop heartbeats when creating andWait checkpoints, fix for concurrency tracker (#1268)
* RESUME messages were missing some data that is needed for the concurrency tracker

* Cancel heartbeats when checkpoints are created for triggerAndWait and batchTriggerAndWait

* Added some logs when canceling heartbeats

* Improved the nested dependencies test task
2024-08-23 13:10:39 +01:00
Eric Allam f9ec66c562 v3: new build system (#1265)
* upgrade @opentelemetry packages to the latest versions

* remove v2 only packages, will be moved to a dedicated repo

* remove more v2 code and run pnpm install

* use the npm yalt package in the webapp

* convert @trigger.dev/core to tshy

* Switch from jest to vitest in @trigger.dev/core

* Fixed core test

* move core-backend code into core subpath export

* convert @trigger.dev/sdk to tshy

* Removed hono

* move core-apps to core/v3/apps, remove core-apps, start converting cli-v3

* Fix up some of the commands

* cli now building and loadable

* using package-json-from-dist to get package version now in core and cli

* dev command WIP

* cleaned up some repetition and structure of the entry point stuff

* bringing back the background worker stuff

* Indexing of the v3 catalog

* getting closer to executing dev runs...

* centralize dev logging using event emitter

* Move indexing to it’s own entry point, simplify code

* dev runs working

* Get instrumentation to work with openai

* debugging achieved internally

* provide worker files as part of the worker creation on the server

* support for cjs and esm javascript

* Fixed timeout

* worker manifest now has the config path

* auto-upgrade config to non-deprecated alternatives

* Adding package preview release

* deployment WIP

* improve the syncEnvVars output and adapt resolveEnvVars

* WIP bun runtime

* WIP bun support

* seed tasks with the machine preset if listed in the config

* deploy run executions WIP, extracted TaskRunProcess into 1 place

* deployed tasks running and executing 🎉

* support for waits and better flushing & process cleanup

* Fixed the heartbeating

* Better warning messages

* Improve and unify the indexing between dev and deploy

* Support for external deps that need node-gyp to build

* build extensions can now install custom packages and run instructions in the image. Also prisma extension now works and also works with multiple schema files

* Add back in the main/types/module to sdk

* dev no longer is Ink/React, grace period for disconnections in dev

* Fix the changeset config

* More changeset fixes

* Remove config packages

* More changeset fixes

* Fixed typescript issues (needed to revert back to zod 3.22.3

* Fix pr_checks workflow

* Remove the prepare script

* Fixed tests and package versions

* Remove cli test script

* Remove packages from tailwind watch paths

* Add repo to public packages

* Just commit the generated files and do the building at dev time

* Try and get pkg.pr.new working

* Try again

* Fix emitDecoratorMetadata importing named export from typescript

* config file backwards compat with export const config

* Fixed issue where import errors weren’t coming through

* p-retry is a prod dep

* typescript needs to be a prod dependency for emitDecoratorMetadata

* Add better debug logging to help track down import-in-the-middle bug

* An external is only considered resolvable if it resolves to the same path as the collected external

* Fix runtime checks to allow >=18.20

* Move extensions to a new build package

* Fixed building packages in dockerfile

* Remove the e2e test from publish workflow for now

* Don’t treat pkg.pr.new versions has needing upgrading

* making sure config handleError works, and discovered path aliases don’t work in config files

* Strip empty string env vars so they accidentally override real values

* Couple of things

* Update version to use preview instead of beta

* Hopefully fix re-attempts with >30s delay

* Match socket emit messages to current latest in main

* Initial guide

* Go back to beta

* Go back to the preview, and update guide to use pr preview tags

* Go back to beta

---------

Co-authored-by: Matt Aitken <matt@mattaitken.com>
2024-08-23 13:10:15 +01:00
James Ritchie 2cd8658190 If an org has been deleted, show a ☠️ icon on the admin page 2024-08-22 13:15:00 +01:00
James Ritchie e5895424db Added a link to a Next.js example in the docs 2024-08-21 15:58:18 +01:00
Matt Aitken a724a6fa01 Added Task index parentId 2024-08-21 15:54:23 +01:00
Matt Aitken ba7bbdbbdb Dashboard query improvements (#1266)
* Remove run count from JobPresenter

* Remove job/integration counts from org/projects query

* Removed v2 counts from ProjectPresenter query

* JobRun index idx_jobrun_organizationId_createdAt

* JobRun index idx_jobrun_versionId

* BatchTaskRunItem index taskRunAttemptId

* BatchTaskRunItem index taskRunId
2024-08-21 15:49:18 +01:00
Matt Aitken 7746302b51 Run failed alerts (#1262)
* Add new ProjectAlertType ‘TASK_RUN’ and then migrate channels to it

* WIP new alerts for run failures

* Consolidate failed run status into taskStatus.ts

* Added the read replica into BaseService, could be useful

* Send task run alerts code

* Revert "Added the read replica into BaseService, could be useful"

This reverts commit cc2348a40254ea3f9e11c0feaf303ecd8794433a.

* Allow adding task run alerts in the UI

* Use task run alert not attempt alert…

* Use the primary
2024-08-21 14:53:48 +01:00
Matt Aitken 6558e5f8eb Fix for permanently frozen tasks when at max concurrency
Heartbeats now free up concurrency for paused/frozen runs, then they don’t get crashed but get acked by the sharedQueueConsumer
2024-08-21 11:37:02 +01:00
Matt Aitken 9f07df8abc Don’t nack in requeueTaskRun if it’s WAITING_TO_RESUME or PAUSED 2024-08-20 19:00:29 +01:00
Matt Aitken 8ad599f90e Try set the checkpointEventId when resuming batches and dependency 2024-08-20 16:09:43 +01:00
Matt Aitken 78dac99d72 Release 3.0.0-beta.55
🚀 Publish Trigger.dev Docker / e2e (push) Failing after 4s
🚀 Publish Trigger.dev Docker / typecheck (push) Failing after 2s
🚀 Publish Trigger.dev Docker / units (push) Failing after 19s
🚀 Publish Trigger.dev Docker / publish (push) Has been skipped
🚀 Publish Trigger.dev Docker / publish-infra (push) Has been skipped
@trigger.dev/airtable@3.0.0-beta.55 @trigger.dev/astro@3.0.0-beta.55 @trigger.dev/cli@3.0.0-beta.55 @trigger.dev/core@3.0.0-beta.55 @trigger.dev/hono@3.0.0-beta.55 @trigger.dev/linear@3.0.0-beta.55 @trigger.dev/nestjs@3.0.0-beta.55 @trigger.dev/nextjs@3.0.0-beta.55 @trigger.dev/otlp-importer@3.0.0-beta.55 @trigger.dev/react@3.0.0-beta.55 @trigger.dev/replicate@3.0.0-beta.55 @trigger.dev/sdk@3.0.0-beta.55 @trigger.dev/shopify@3.0.0-beta.55 @trigger.dev/slack@3.0.0-beta.55 @trigger.dev/stripe@3.0.0-beta.55 @trigger.dev/supabase@3.0.0-beta.55 @trigger.dev/sveltekit@3.0.0-beta.55 @trigger.dev/testing@3.0.0-beta.55 @trigger.dev/typeform@3.0.0-beta.55 @trigger.dev/yalt@3.0.0-beta.55 v.docker.3.0.0-beta.55 v.docker.3.0.0-beta.56 @trigger.dev/github@3.0.0-beta.55 @trigger.dev/plain@3.0.0-beta.55 @trigger.dev/eslint-plugin@3.0.0-beta.55 @trigger.dev/resend@3.0.0-beta.55 @trigger.dev/openai@3.0.0-beta.55 @trigger.dev/integration-kit@3.0.0-beta.55 @trigger.dev/sendgrid@3.0.0-beta.55 @trigger.dev/remix@3.0.0-beta.55 @trigger.dev/core-backend@3.0.0-beta.55 trigger.dev@3.0.0-beta.55 @trigger.dev/express@3.0.0-beta.55
2024-08-20 15:50:14 +01:00
James Ritchie d6786002b2 Full Next.js guide (#1259)
* tasks no longer inside a group in the side menu (and added “cron”)

* Delay using a timezone

* Added React Not Defined error to the troubleshooting page

* Improved the React common problem

* Link to v2 docs

* New Development section and entry in Common Problems

* Concurrently running the terminal

* Fixed the .env weirdness

* Added section on creating PATs for Github actions

* Improved the Machine spec and limits page

* Quick start steps now have nice images

* Added a diagram for the lifecycle functions

* Added note about onFailure

* CRON -> cron/Cron

* WIP adding more steps to the next.js guide

* WIP next.js

* WIP adding tabbed steps for pages/app router

* References to Infisical links to their homepage so it’s clearer

* WIP updating the nextjs guide

* WIP nextjs guide

* More nextjs guide steps

* More copy

* Added rate limit trouble shooting

* Removed old prisma error title

* Added secret key step

* Added a note for logging in using a specified domain if self hosting

* typo

* App router docs copy

* Deploy copy update

* Added a favicon.png to fix a docs build error

* Removed unused snippet

* Server actions now inside a tab

* Server actions + restructured the triggering section

* Added troubleshooting snippet for react event handlers

* Added a new troubleshooting snippet for ESM

* Updated old replaying image to reflect the new UI

* Removed references to reattempting

* Updated replaying from the run page

* Added a bulk replay section

* Updated the writing tasks intro page

* Removed edge runtime code for now

* Added edge runtime – it seems to just work!

* import type
2024-08-20 14:50:39 +01:00
github-actions[bot] 9d529e9f17 chore: Update version for release (beta) (#1264)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-08-20 14:46:19 +01:00
Matt Aitken d48eb9bac8 Fix: was showing deleted team member dev environments (#1261)
* Env vars: don’t show deleted dev environments

* Concurrency limits: don’t show deleted dev environments

* Create/edit schedule: don’t show deleted dev environments
2024-08-20 14:03:24 +01:00
Matt Aitken 0591db5f2b Fix for short runs inside (batch)triggerAndWait (#1263)
* Test for checkpoints

* Make sourceTaskAttemptId optional on resumeBatchRun

* Removed all completions/executions logic from the shared queue consumer

* Removed the sourceTaskAttemptId from ResumeBatchRunService

* Revert "Removed all completions/executions logic from the shared queue consumer"

This reverts commit d35398d50463c81a1975bb5d5bcfca66a24b8ede.

* WIP on triggerAndWait…

* Fixed triggerAndWait continuing when a checkpoint completes

* Remove the ResumeAttempt code that fails attempts (was protecting against infinite restores)

* Removed messageBody.data.completedAttemptIds.length === 0 commented out code

* Don’t ack if there’s no batchRun

* Added the marqs?.replaceMessage back in but NOT when there’s no checkpoint. More logging

This is a fix for when some attempts fail

* Improvement to the test task that now randomly fails attempts

* When a checkpoint happens, only continue the attempt if it’s in the correct state

* Changeset for rollback in branch

* Set keepRunAlive to false when the dependent task isn’t finished

* Changeset manual version (to get inline with the hotfix branch)

* Changeset: Fixes for continuing after waits

* Latest lockfile (after manual changeset version)
2024-08-20 13:57:08 +01:00
nicktrn a64064965d Prevent runs that fail fast from never sending a heartbeat 2024-08-19 13:17:33 +01:00
Matt Aitken 41ec450bee Revert "Remove the RequeueTaskRunService"
This reverts commit 3421f5ec0b.
2024-08-15 20:43:47 +01:00
Matt Aitken 9a9bdc20c9 Revert "Disable upserting task queues when triggering a task"
This reverts commit 6652edf7a7.
2024-08-15 20:42:50 +01:00
Matt Aitken 6652edf7a7 Disable upserting task queues when triggering a task 2024-08-15 19:34:22 +01:00
Matt Aitken 369bc38560 Instead of upserting taskQueues, do a findFirst then create/update 2024-08-15 18:48:50 +01:00
Matt Aitken 27f7132509 Don’t do the auto-increment inside a transaction, this means triggerTask isn’t inside one 2024-08-15 17:44:10 +01:00
Matt Aitken b6fc68aaae Don’t use Graphile for invocation usage reporting 2024-08-15 17:22:52 +01:00
Matt Aitken 3421f5ec0b Remove the RequeueTaskRunService 2024-08-15 17:20:40 +01:00
Matt Aitken 15097a3a67 Use the replica for the v2 run and task presenters 2024-08-15 15:34:57 +01:00
Matt Aitken 1b7bb90852 Only show the first 1000 task on the v2 run page 2024-08-15 15:26:12 +01:00
Matt Aitken 7e68c02636 Don’t reload v2 run page if more than 100 tasks 2024-08-15 15:25:59 +01:00
nicktrn 01e01376f8 Fix attempt retry mechanics for new workers 2024-08-15 12:13:13 +01:00
James Ritchie db9163e775 Added a note to use batchTrigger() instead of trigger() 2024-08-14 13:25:08 +01:00
nicktrn e9bdd0c7e5 Release 3.0.0-beta.53 @trigger.dev/airtable@3.0.0-beta.53 @trigger.dev/astro@3.0.0-beta.53 @trigger.dev/core-backend@3.0.0-beta.53 @trigger.dev/core@3.0.0-beta.53 @trigger.dev/eslint-plugin@3.0.0-beta.53 @trigger.dev/express@3.0.0-beta.53 @trigger.dev/github@3.0.0-beta.53 @trigger.dev/hono@3.0.0-beta.53 @trigger.dev/integration-kit@3.0.0-beta.53 @trigger.dev/linear@3.0.0-beta.53 @trigger.dev/nestjs@3.0.0-beta.53 @trigger.dev/nextjs@3.0.0-beta.53 @trigger.dev/openai@3.0.0-beta.53 @trigger.dev/otlp-importer@3.0.0-beta.53 @trigger.dev/plain@3.0.0-beta.53 @trigger.dev/react@3.0.0-beta.53 @trigger.dev/remix@3.0.0-beta.53 @trigger.dev/sendgrid@3.0.0-beta.53 @trigger.dev/slack@3.0.0-beta.53 @trigger.dev/stripe@3.0.0-beta.53 @trigger.dev/supabase@3.0.0-beta.53 @trigger.dev/sveltekit@3.0.0-beta.53 @trigger.dev/typeform@3.0.0-beta.53 @trigger.dev/yalt@3.0.0-beta.53 trigger.dev@3.0.0-beta.53 @trigger.dev/replicate@3.0.0-beta.53 @trigger.dev/cli@3.0.0-beta.53 @trigger.dev/shopify@3.0.0-beta.53 @trigger.dev/resend@3.0.0-beta.53 @trigger.dev/sdk@3.0.0-beta.53 @trigger.dev/testing@3.0.0-beta.53 2024-08-14 11:40:38 +01:00
github-actions[bot] 7cb977ff2c chore: Update version for release (beta) (#1256)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-08-14 11:39:44 +01:00
James Ritchie 2940b07430 Added a description panel to the Concurrency admin page 2024-08-14 11:33:15 +01:00
James Ritchie c46e3c97b5 Fix for schedule page not scrolling 2024-08-14 11:23:16 +01:00
nicktrn 16cb1c46ac Fix restored container names 2024-08-13 18:10:09 +01:00
Matt Aitken 2b0d68a0b6 Include the old message data when replacing, so we get the projectId etc. 2024-08-13 17:09:29 +01:00