Commit Graph

4839 Commits

Author SHA1 Message Date
Eric Allam eb60126284 Fixed batched otel in dev and made prod configurable to be batched as well 2024-04-02 12:58:56 +01:00
Eric Allam d876c358d2 Update the default env var concurrency limit and set them when orgs and envs are created 2024-04-02 11:55:00 +01:00
Eric Allam 6e9c8ab555 Fix pnpm lock file @trigger.dev/airtable@3.0.0-beta.4 @trigger.dev/astro@3.0.0-beta.4 @trigger.dev/cli@3.0.0-beta.4 @trigger.dev/core-backend@3.0.0-beta.4 @trigger.dev/core@3.0.0-beta.4 @trigger.dev/eslint-plugin@3.0.0-beta.4 @trigger.dev/express@3.0.0-beta.4 @trigger.dev/github@3.0.0-beta.4 @trigger.dev/hono@3.0.0-beta.4 @trigger.dev/integration-kit@3.0.0-beta.4 @trigger.dev/linear@3.0.0-beta.4 @trigger.dev/nestjs@3.0.0-beta.4 @trigger.dev/nextjs@3.0.0-beta.4 @trigger.dev/openai@3.0.0-beta.4 @trigger.dev/otlp-importer@3.0.0-beta.4 @trigger.dev/plain@3.0.0-beta.4 @trigger.dev/react@3.0.0-beta.4 @trigger.dev/remix@3.0.0-beta.4 @trigger.dev/replicate@3.0.0-beta.4 @trigger.dev/resend@3.0.0-beta.4 @trigger.dev/sdk@3.0.0-beta.4 @trigger.dev/sendgrid@3.0.0-beta.4 @trigger.dev/shopify@3.0.0-beta.4 @trigger.dev/slack@3.0.0-beta.4 @trigger.dev/stripe@3.0.0-beta.4 @trigger.dev/supabase@3.0.0-beta.4 @trigger.dev/sveltekit@3.0.0-beta.4 @trigger.dev/testing@3.0.0-beta.4 @trigger.dev/typeform@3.0.0-beta.4 @trigger.dev/yalt@3.0.0-beta.4 2024-04-02 11:13:31 +01:00
github-actions[bot] 9a3eb289cd chore: Update version for release (beta) (#992)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-04-02 11:12:56 +01:00
nicktrn 51315fc3c8 v3: shared queue pool (#994)
* configurable retry delay checkpoint threshold

* shared queue consumer pool

* add more attributes to marqs spans

* inject trace context into queued messages
2024-04-02 10:22:52 +01:00
Eric Allam 4047f00562 Allow each service to define their own name 2024-04-01 22:01:36 +01:00
Matt Aitken 545f85b44e Fix for shared queryKey between useRunDetails and useRunStatuses 2024-04-01 13:20:39 +01:00
Eric Allam abe202e7b7 Fix pnpm lock file @trigger.dev/airtable@3.0.0-beta.3 @trigger.dev/astro@3.0.0-beta.3 @trigger.dev/cli@3.0.0-beta.3 @trigger.dev/core-backend@3.0.0-beta.3 @trigger.dev/core@3.0.0-beta.3 @trigger.dev/eslint-plugin@3.0.0-beta.3 @trigger.dev/express@3.0.0-beta.3 @trigger.dev/github@3.0.0-beta.3 @trigger.dev/hono@3.0.0-beta.3 @trigger.dev/integration-kit@3.0.0-beta.3 @trigger.dev/linear@3.0.0-beta.3 @trigger.dev/nestjs@3.0.0-beta.3 @trigger.dev/nextjs@3.0.0-beta.3 @trigger.dev/openai@3.0.0-beta.3 @trigger.dev/otlp-importer@3.0.0-beta.3 @trigger.dev/plain@3.0.0-beta.3 @trigger.dev/react@3.0.0-beta.3 @trigger.dev/remix@3.0.0-beta.3 @trigger.dev/replicate@3.0.0-beta.3 @trigger.dev/resend@3.0.0-beta.3 @trigger.dev/sdk@3.0.0-beta.3 @trigger.dev/sendgrid@3.0.0-beta.3 @trigger.dev/shopify@3.0.0-beta.3 @trigger.dev/slack@3.0.0-beta.3 @trigger.dev/stripe@3.0.0-beta.3 @trigger.dev/supabase@3.0.0-beta.3 @trigger.dev/sveltekit@3.0.0-beta.3 @trigger.dev/testing@3.0.0-beta.3 @trigger.dev/typeform@3.0.0-beta.3 @trigger.dev/yalt@3.0.0-beta.3 trigger.dev@3.0.0-beta.4 2024-04-01 10:54:29 +01:00
github-actions[bot] ae27fd83af chore: Update version for release (beta) (#987)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-04-01 10:53:45 +01:00
Eric Allam c702d6a9ca v3: better task metadata errors (#991)
* v3: better handle task metadata parse errors, and display nicely formatted errors (dev, deploy, UI)

* Add changeset
2024-03-31 21:47:05 +01:00
Eric Allam 9af2570da6 Retry 429, 500, and connection error API requests to the trigger.dev server 2024-03-31 16:06:12 +01:00
Eric Allam a946797d95 v3: Upgrades to MarQS (#989)
* Upgrades to MarQS

- It’s “queue choosing” algorithm is now MUCH better (pretty much didn’t work before) and should be performant even when we have a bunch of prod queues
- There are now concurrency limits at the environment and organization level, as well as the task/queue level. So if any of them are at capacity the message won’t be dequeued. This means we can have an org wide concurrency limit, as well as different limits for dev/prod/staging
- I’ve added an admin API that can be used to update the org/env concurrency limits

* More MarQS upgrades

- Extract the queue priortity choosing strategy into an interface
- Implement a much better weighted average strategy
- “Slide the window” of queue candidates if the parent queue sends all at-capacity queues (so we won’t get stuck attempting to choose the same 12 full queues)
- Added some unit tests for the priority stuff
- Added some ideas for expanding the priority choosing strategy with more dynamic features

* Fixes an issue with the shared queue consumer relying on the queue name to get the env id
2024-03-31 15:58:00 +01:00
Eric Allam 8c4df326cc v3: Better dev/deploy errors and handle image build errors (#988)
* Better error output when an ESM only package is required

* Add better errors for npm package errors, and bring deploy errors to the dev CLI

* Handle depot build errors and write out a temporary file to view the build logs

* Add changeset
2024-03-29 12:46:11 +00:00
Eric Allam 11b997d2bf Make env var value input fields passwords 2024-03-29 09:05:53 +00:00
Eric Allam 8694e573f5 Fix CLI logout and add list-profiles command 2024-03-28 22:18:15 +00:00
Eric Allam b271742dca v3: implement configurable log levels via config file and env var (#985)
* v3: implement configurable log levels via config file and TRIGGER_LOG_LEVEL

Also, test runs automatically set the TRIGGER_LOG_LEVEL to debug

* Fix type error and changeset

* Added a Node.js runtime check for the CLI dev command
2024-03-28 22:00:38 +00:00
Eric Allam 6f9f25481e Update pnpm lock file @trigger.dev/airtable@3.0.0-beta.2 @trigger.dev/astro@3.0.0-beta.2 @trigger.dev/cli@3.0.0-beta.2 @trigger.dev/core-backend@3.0.0-beta.2 @trigger.dev/core@3.0.0-beta.2 @trigger.dev/eslint-plugin@3.0.0-beta.2 @trigger.dev/express@3.0.0-beta.2 @trigger.dev/github@3.0.0-beta.2 @trigger.dev/hono@3.0.0-beta.2 @trigger.dev/integration-kit@3.0.0-beta.2 @trigger.dev/linear@3.0.0-beta.2 @trigger.dev/nestjs@3.0.0-beta.2 @trigger.dev/nextjs@3.0.0-beta.2 @trigger.dev/openai@3.0.0-beta.2 @trigger.dev/otlp-importer@3.0.0-beta.2 @trigger.dev/plain@3.0.0-beta.2 @trigger.dev/react@3.0.0-beta.2 @trigger.dev/remix@3.0.0-beta.2 @trigger.dev/replicate@3.0.0-beta.2 @trigger.dev/resend@3.0.0-beta.2 @trigger.dev/sdk@3.0.0-beta.2 @trigger.dev/sendgrid@3.0.0-beta.2 @trigger.dev/shopify@3.0.0-beta.2 @trigger.dev/slack@3.0.0-beta.2 @trigger.dev/stripe@3.0.0-beta.2 @trigger.dev/supabase@3.0.0-beta.2 @trigger.dev/sveltekit@3.0.0-beta.2 @trigger.dev/testing@3.0.0-beta.2 @trigger.dev/typeform@3.0.0-beta.2 @trigger.dev/yalt@3.0.0-beta.2 trigger.dev@3.0.0-beta.3 2024-03-28 19:55:14 +00:00
github-actions[bot] 67aaffb6fc chore: Update version for release (beta) (#984)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-03-28 19:54:21 +00:00
Eric Allam e3cf456c69 v3: fix string and non-standard outputs (#983)
* Fixed string outputs missing and incorrectly formatted

* Handle non-serializable outputs like a function

* Add changeset
2024-03-28 19:51:29 +00:00
Matt Aitken bf7827e7b8 Fix for the test latest runs showing other environments 2024-03-28 18:14:47 +00:00
Eric Allam bc020a3ffe Get the internal telemetry ready for test/prod 2024-03-28 18:12:01 +00:00
Eric Allam b361afbfe4 Update pnpm lock file @trigger.dev/airtable@3.0.0-beta.1 @trigger.dev/astro@3.0.0-beta.1 @trigger.dev/core-backend@3.0.0-beta.1 @trigger.dev/core@3.0.0-beta.1 @trigger.dev/eslint-plugin@3.0.0-beta.1 @trigger.dev/express@3.0.0-beta.1 @trigger.dev/github@3.0.0-beta.1 @trigger.dev/hono@3.0.0-beta.1 @trigger.dev/integration-kit@3.0.0-beta.1 @trigger.dev/linear@3.0.0-beta.1 @trigger.dev/nestjs@3.0.0-beta.1 @trigger.dev/nextjs@3.0.0-beta.1 @trigger.dev/openai@3.0.0-beta.1 @trigger.dev/otlp-importer@3.0.0-beta.1 @trigger.dev/plain@3.0.0-beta.1 @trigger.dev/react@3.0.0-beta.1 @trigger.dev/remix@3.0.0-beta.1 @trigger.dev/replicate@3.0.0-beta.1 @trigger.dev/resend@3.0.0-beta.1 @trigger.dev/sdk@3.0.0-beta.1 @trigger.dev/sendgrid@3.0.0-beta.1 @trigger.dev/shopify@3.0.0-beta.1 @trigger.dev/slack@3.0.0-beta.1 @trigger.dev/stripe@3.0.0-beta.1 @trigger.dev/supabase@3.0.0-beta.1 @trigger.dev/sveltekit@3.0.0-beta.1 @trigger.dev/testing@3.0.0-beta.1 @trigger.dev/typeform@3.0.0-beta.1 @trigger.dev/yalt@3.0.0-beta.1 trigger.dev@3.0.0-beta.2 @trigger.dev/cli@3.0.0-beta.1 2024-03-28 16:29:17 +00:00
github-actions[bot] a3d809740d chore: Update version for release (beta) (#979)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-03-28 16:28:41 +00:00
Eric Allam f93eae300e v3: superjson dynamic import and deploy fixes (#982)
* v3: Dynamically import superjson and fix some bundling issues

* Added changeset

* Better error handling in the registry proxy and catch uncaught exceptions and unhandled promise rejections instead of crashing the server

* Await the prettyPrintPackage
2024-03-28 16:22:00 +00:00
Matt Aitken a2365e406d Fixed v3 docs link in the sidebar 2024-03-28 14:46:47 +00:00
Eric Allam 42d319c2d1 Remove all the v3-catalog code for triggering in code 2024-03-28 14:33:01 +00:00
nicktrn b66d5525ef v3: machine config (#978)
* add and use machine config

* assign tasks to worker nodes only

* add secure flag to zod connection

* changeset

* add pre stop hook

* don't use secure connection by default

* pass more identifiers to provider and apply labels
2024-03-28 14:21:49 +00:00
Eric Allam 719c0a0b94 v3: better clock management for spans/logs in CRIU envs (#980)
* durable clock WIP

* Fixed incorrect span timings around checkpoints by implementing a precise wall clock that resets after restores

* Add changeset
2024-03-28 14:20:44 +00:00
Matt Aitken f1c768a255 Docs for API keys, fixed links to docs from app (#981)
* Add missing API keys docs and link from the web app to them. Renamed Server API keys to Secret keys

* Correct link to env var docs

* Duplicate image fixed on environment variables docs
2024-03-28 14:00:42 +00:00
Matt Aitken d9c9e80bc4 Changed "Worker" to "Version" in the CLI dev command key 2024-03-28 12:13:58 +00:00
Matt Aitken d39932ebf7 Changed the throttle so the function fires at the end of the period, not immediately when first called 2024-03-28 12:05:48 +00:00
Matt Aitken 9bcb8cb42a Ignore the DEBUG env var when deploying 2024-03-28 11:28:34 +00:00
nicktrn 2374f8e8ac fix docs for shopify scopes 2024-03-28 09:57:04 +00:00
github-actions[bot] a22b5869e4 chore: Update version for release (beta) (#977)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
trigger.dev@3.0.0-beta.1
2024-03-27 19:43:19 +00:00
Eric Allam f1571cbfab Fixed an issue where the trigger.dev package was not being built before publishing to npm 2024-03-27 19:39:18 +00:00
Eric Allam 222dfed4d2 Fixed pnpm lock file @trigger.dev/airtable@3.0.0-beta.0 @trigger.dev/astro@3.0.0-beta.0 @trigger.dev/cli@3.0.0-beta.0 @trigger.dev/core-backend@3.0.0-beta.0 @trigger.dev/core@3.0.0-beta.0 @trigger.dev/eslint-plugin@3.0.0-beta.0 @trigger.dev/express@3.0.0-beta.0 @trigger.dev/hono@3.0.0-beta.0 @trigger.dev/integration-kit@3.0.0-beta.0 @trigger.dev/linear@3.0.0-beta.0 @trigger.dev/nestjs@3.0.0-beta.0 @trigger.dev/nextjs@3.0.0-beta.0 @trigger.dev/openai@3.0.0-beta.0 @trigger.dev/otlp-importer@3.0.0-beta.0 @trigger.dev/plain@3.0.0-beta.0 @trigger.dev/react@3.0.0-beta.0 @trigger.dev/remix@3.0.0-beta.0 @trigger.dev/replicate@3.0.0-beta.0 @trigger.dev/resend@3.0.0-beta.0 @trigger.dev/sdk@3.0.0-beta.0 @trigger.dev/sendgrid@3.0.0-beta.0 @trigger.dev/shopify@3.0.0-beta.0 @trigger.dev/slack@3.0.0-beta.0 @trigger.dev/stripe@3.0.0-beta.0 @trigger.dev/supabase@3.0.0-beta.0 @trigger.dev/sveltekit@3.0.0-beta.0 @trigger.dev/testing@3.0.0-beta.0 @trigger.dev/typeform@3.0.0-beta.0 @trigger.dev/yalt@3.0.0-beta.0 trigger.dev@3.0.0-beta.0 @trigger.dev/github@3.0.0-beta.0 2024-03-27 18:56:10 +00:00
github-actions[bot] 89c0e68fb5 chore: Update version for release (beta) (#975)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-03-27 18:54:28 +00:00
Eric Allam 5b745dc1a4 Improved the dev CLI output 2024-03-27 18:36:52 +00:00
Eric Allam 2dea8dec35 Implement simple 7 day log retention and blank states (#976) 2024-03-27 17:22:16 +00:00
Matt Aitken 252f5b3967 Fix for janky scrolling on the run page 2024-03-27 16:55:03 +00:00
Eric Allam 014d233339 Couple changes to better test the output/payload offloading 2024-03-27 16:27:45 +00:00
Eric Allam e398bdf859 Only use 16vcpu on e2e 2024-03-27 16:27:28 +00:00
Matt Aitken 426316aa3c There was an error about a div being in a paragraph 2024-03-27 16:23:11 +00:00
Matt Aitken 4f9b8d6721 Task that logs a lot 2024-03-27 16:22:29 +00:00
Matt Aitken a345ad2d57 Fix for the large JSON generator 2024-03-27 15:49:46 +00:00
Matt Aitken e2b67400ea TRIGGER_API_URL defaults to https://api.trigger.dev 2024-03-27 15:18:40 +00:00
Eric Allam c4dc2849af Remove changesets from the pre.json (accidentially added) 2024-03-27 15:17:42 +00:00
Eric Allam a8263dec86 Include core-apps in trigger.dev typecheck tsconfig file trigger.dev@2.3.18 2024-03-27 15:01:02 +00:00
Eric Allam 53054d0146 Enter into changesets “beta” tag for packages 2024-03-27 14:55:14 +00:00
Eric Allam 74236ae7c1 Fix changeset workflow (thought I’d already fixed this?) 2024-03-27 14:47:42 +00:00