Commit Graph

4769 Commits

Author SHA1 Message Date
Eric Allam bc020a3ffe Get the internal telemetry ready for test/prod 2024-03-28 18:12:01 +00:00
Eric Allam b361afbfe4 Update pnpm lock file @trigger.dev/airtable@3.0.0-beta.1 @trigger.dev/astro@3.0.0-beta.1 @trigger.dev/core-backend@3.0.0-beta.1 @trigger.dev/core@3.0.0-beta.1 @trigger.dev/eslint-plugin@3.0.0-beta.1 @trigger.dev/express@3.0.0-beta.1 @trigger.dev/github@3.0.0-beta.1 @trigger.dev/hono@3.0.0-beta.1 @trigger.dev/integration-kit@3.0.0-beta.1 @trigger.dev/linear@3.0.0-beta.1 @trigger.dev/nestjs@3.0.0-beta.1 @trigger.dev/nextjs@3.0.0-beta.1 @trigger.dev/openai@3.0.0-beta.1 @trigger.dev/otlp-importer@3.0.0-beta.1 @trigger.dev/plain@3.0.0-beta.1 @trigger.dev/react@3.0.0-beta.1 @trigger.dev/remix@3.0.0-beta.1 @trigger.dev/replicate@3.0.0-beta.1 @trigger.dev/resend@3.0.0-beta.1 @trigger.dev/sdk@3.0.0-beta.1 @trigger.dev/sendgrid@3.0.0-beta.1 @trigger.dev/shopify@3.0.0-beta.1 @trigger.dev/slack@3.0.0-beta.1 @trigger.dev/stripe@3.0.0-beta.1 @trigger.dev/supabase@3.0.0-beta.1 @trigger.dev/sveltekit@3.0.0-beta.1 @trigger.dev/testing@3.0.0-beta.1 @trigger.dev/typeform@3.0.0-beta.1 @trigger.dev/yalt@3.0.0-beta.1 trigger.dev@3.0.0-beta.2 @trigger.dev/cli@3.0.0-beta.1 2024-03-28 16:29:17 +00:00
github-actions[bot] a3d809740d chore: Update version for release (beta) (#979)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-03-28 16:28:41 +00:00
Eric Allam f93eae300e v3: superjson dynamic import and deploy fixes (#982)
* v3: Dynamically import superjson and fix some bundling issues

* Added changeset

* Better error handling in the registry proxy and catch uncaught exceptions and unhandled promise rejections instead of crashing the server

* Await the prettyPrintPackage
2024-03-28 16:22:00 +00:00
Matt Aitken a2365e406d Fixed v3 docs link in the sidebar 2024-03-28 14:46:47 +00:00
Eric Allam 42d319c2d1 Remove all the v3-catalog code for triggering in code 2024-03-28 14:33:01 +00:00
nicktrn b66d5525ef v3: machine config (#978)
* add and use machine config

* assign tasks to worker nodes only

* add secure flag to zod connection

* changeset

* add pre stop hook

* don't use secure connection by default

* pass more identifiers to provider and apply labels
2024-03-28 14:21:49 +00:00
Eric Allam 719c0a0b94 v3: better clock management for spans/logs in CRIU envs (#980)
* durable clock WIP

* Fixed incorrect span timings around checkpoints by implementing a precise wall clock that resets after restores

* Add changeset
2024-03-28 14:20:44 +00:00
Matt Aitken f1c768a255 Docs for API keys, fixed links to docs from app (#981)
* Add missing API keys docs and link from the web app to them. Renamed Server API keys to Secret keys

* Correct link to env var docs

* Duplicate image fixed on environment variables docs
2024-03-28 14:00:42 +00:00
Matt Aitken d9c9e80bc4 Changed "Worker" to "Version" in the CLI dev command key 2024-03-28 12:13:58 +00:00
Matt Aitken d39932ebf7 Changed the throttle so the function fires at the end of the period, not immediately when first called 2024-03-28 12:05:48 +00:00
Matt Aitken 9bcb8cb42a Ignore the DEBUG env var when deploying 2024-03-28 11:28:34 +00:00
nicktrn 2374f8e8ac fix docs for shopify scopes 2024-03-28 09:57:04 +00:00
github-actions[bot] a22b5869e4 chore: Update version for release (beta) (#977)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
trigger.dev@3.0.0-beta.1
2024-03-27 19:43:19 +00:00
Eric Allam f1571cbfab Fixed an issue where the trigger.dev package was not being built before publishing to npm 2024-03-27 19:39:18 +00:00
Eric Allam 222dfed4d2 Fixed pnpm lock file @trigger.dev/airtable@3.0.0-beta.0 @trigger.dev/astro@3.0.0-beta.0 @trigger.dev/cli@3.0.0-beta.0 @trigger.dev/core-backend@3.0.0-beta.0 @trigger.dev/core@3.0.0-beta.0 @trigger.dev/eslint-plugin@3.0.0-beta.0 @trigger.dev/express@3.0.0-beta.0 @trigger.dev/hono@3.0.0-beta.0 @trigger.dev/integration-kit@3.0.0-beta.0 @trigger.dev/linear@3.0.0-beta.0 @trigger.dev/nestjs@3.0.0-beta.0 @trigger.dev/nextjs@3.0.0-beta.0 @trigger.dev/openai@3.0.0-beta.0 @trigger.dev/otlp-importer@3.0.0-beta.0 @trigger.dev/plain@3.0.0-beta.0 @trigger.dev/react@3.0.0-beta.0 @trigger.dev/remix@3.0.0-beta.0 @trigger.dev/replicate@3.0.0-beta.0 @trigger.dev/resend@3.0.0-beta.0 @trigger.dev/sdk@3.0.0-beta.0 @trigger.dev/sendgrid@3.0.0-beta.0 @trigger.dev/shopify@3.0.0-beta.0 @trigger.dev/slack@3.0.0-beta.0 @trigger.dev/stripe@3.0.0-beta.0 @trigger.dev/supabase@3.0.0-beta.0 @trigger.dev/sveltekit@3.0.0-beta.0 @trigger.dev/testing@3.0.0-beta.0 @trigger.dev/typeform@3.0.0-beta.0 @trigger.dev/yalt@3.0.0-beta.0 trigger.dev@3.0.0-beta.0 @trigger.dev/github@3.0.0-beta.0 2024-03-27 18:56:10 +00:00
github-actions[bot] 89c0e68fb5 chore: Update version for release (beta) (#975)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-03-27 18:54:28 +00:00
Eric Allam 5b745dc1a4 Improved the dev CLI output 2024-03-27 18:36:52 +00:00
Eric Allam 2dea8dec35 Implement simple 7 day log retention and blank states (#976) 2024-03-27 17:22:16 +00:00
Matt Aitken 252f5b3967 Fix for janky scrolling on the run page 2024-03-27 16:55:03 +00:00
Eric Allam 014d233339 Couple changes to better test the output/payload offloading 2024-03-27 16:27:45 +00:00
Eric Allam e398bdf859 Only use 16vcpu on e2e 2024-03-27 16:27:28 +00:00
Matt Aitken 426316aa3c There was an error about a div being in a paragraph 2024-03-27 16:23:11 +00:00
Matt Aitken 4f9b8d6721 Task that logs a lot 2024-03-27 16:22:29 +00:00
Matt Aitken a345ad2d57 Fix for the large JSON generator 2024-03-27 15:49:46 +00:00
Matt Aitken e2b67400ea TRIGGER_API_URL defaults to https://api.trigger.dev 2024-03-27 15:18:40 +00:00
Eric Allam c4dc2849af Remove changesets from the pre.json (accidentially added) 2024-03-27 15:17:42 +00:00
Eric Allam a8263dec86 Include core-apps in trigger.dev typecheck tsconfig file trigger.dev@2.3.18 2024-03-27 15:01:02 +00:00
Eric Allam 53054d0146 Enter into changesets “beta” tag for packages 2024-03-27 14:55:14 +00:00
Eric Allam 74236ae7c1 Fix changeset workflow (thought I’d already fixed this?) 2024-03-27 14:47:42 +00:00
Eric Allam 055d468b38 Setup socket.io redis adapter and fix deployment builds for superjson error 2024-03-27 14:42:10 +00:00
Matt Aitken fc473105e3 Better error messages if you haven’t set the API URL or SECRET for the v3 client 2024-03-27 14:08:13 +00:00
Matt Aitken 3a35d60091 Change the npm tag from "latest" to "beta" in docs/cli/onboarding (#974)
* The v3 init command defaults to the “@v3” tag

* Change it to beta

* Updated the docs and in-app docs

* Updated GitHub action
2024-03-27 13:39:53 +00:00
Eric Allam f11c0f991b Log out the headers and url when no key is found 2024-03-27 13:30:47 +00:00
Eric Allam 1cf181fd02 Control api rate limiter logging through env vars 2024-03-27 12:42:05 +00:00
Eric Allam 412d0149d5 Fixed release workflow 2024-03-27 12:37:23 +00:00
Eric Allam d614c8d2ca More CI fixes for node 20/pnpm 8 2024-03-27 12:33:24 +00:00
Eric Allam 4329d1130a Attempt to fix typecheck 2024-03-27 12:30:54 +00:00
Eric Allam 17f9fc8273 Upgrade to pnpm 8.15.5 to fix ERR_INVALID_THIS issue
https://github.com/pnpm/pnpm/issues/6424
2024-03-27 12:27:19 +00:00
James Ritchie db611bebba Text truncation fixes (#973)
* Span titles and code blocks truncate when window is small

* Tree view toggle arrow doesn’t have red hover state if row is an error
2024-03-27 12:16:20 +00:00
Eric Allam 215b60c742 v3: superjson output support (#971)
* superjson output support

* Better support for superjson in the task events, limiting output attributes

* Offload large outputs to object store (r2)

* Finishing up the offloading of large outputs/payloads to an object store
2024-03-27 11:57:22 +00:00
Matt Aitken 9c89ab40aa Throttle the live streaming of the run to 1 second updates (#972) 2024-03-26 14:20:10 +00:00
Matt Aitken b342e06583 Fix for scrolling with long run traces 2024-03-26 11:30:05 +00:00
Eric Allam 4418de515b Whitelist api endpoints that don’t use an authorization endpoint from the rate limiter 2024-03-25 21:18:05 +00:00
Eric Allam 0d2a71c5cf handle when traces/logs don’t have traceIds/spanIds 2024-03-25 21:10:31 +00:00
Eric Allam 9e02b45808 Fixed export 2024-03-25 17:24:23 +00:00
Eric Allam 395abe1b92 v3: Various fixes to support papermark use-case (#970)
* Adding a changeset for v3

* Add a version field to @trigger.dev/core-apps package.json

* Build trigger.dev when doing a prerelease

* bundle @trigger.dev/core-apps with trigger.dev cli

* Fix the init command config template

* Don’t use * for the @trigger.dev/core dep version specifier

* strip workspace: from the package version before installing it

* Added dependenciesToBundle config option to bundle ESM only packages

* Added logging around resolving dependency paths

* Try again

* Resolve dependencies based on the project dir first

* flip the bundled default

* Adding some logs around dev task completion notifications

* Adding some additional logs

* Add more logs

* Write out the log using process.stdout

* Store pending completion notifications and resume them when awaited (fixes race condition)

* Cleanup some of the logs

* Copy over the postinstall step from the projects package.json

* Add support for including additional files when deploying (e.g. prisma schema)

* Don’t run scripts when resolving deps

* copy all the files just in case anything is needed in postinstall

* Remove duplicate option

* Use the tag when outputting the dev command

* Remove the postinstall script

* add the trigger dir to the config if the default is not chosen

* Remove the “hud” display in the dev command

* trigger file names with dashes now work

* Better file watching in dev

* Much better duplicate ID experience now

* Much better “Project not found” error

* Export the handleError function types from sdk

* Add support for configuring instrumentation

* Upgrade and unify @opentelemetry/* packages (and remove storybook from the webapp)

* Fix typescript error in react package

* Upgrade react types in webapp

* Allow span icons to be determined based on the span name (e.g. prisma:)

* Ignore built-in env vars when checking for env vars, and allow continuing the deployment even if missing env vars were detected

* Improve the retry.fetch default behavior and option structure

* Fixed typescript errors with packages/email react types

* Update the retry.fetch docs
2024-03-25 17:02:49 +00:00
Matt Aitken b35eebb666 Rate limit API requests and changed SQS reading speed (#969)
* Trying to use the @upstash/ratelimit package with ioredis…

* WIP using the redis package instead

* Revert the action back

* Removed redis

* Started refactoring

* SQS setting for the poll interval. Set the default queue reading to be slower

* API rate limiter as Express middleware

* Organise imports

* Fixed spelling mistake “limitter”

* No authorization header response is problem+json
2024-03-25 16:39:20 +00:00
nicktrn 9ecf07731a v3: small features and fixes for prod infra (#968)
* add checkpoint restore events

* fix retries.enabledInDev

* fix tsconfig paths

* fix provider build and dev

* update kubernetes provider and fix builds again

* upgrade prod base to node 20

* update infra publish workflow

* rethrow k8s errors after logging

* shorten index container names

* fix error type assertion

* rename type assertion

* remove resource limits for now

* add missing run id on create

* add push to deploy command for self-hosting

* checkpointing fixes

* update coordinator image

* ensure valid registry login

* delete checkpoint archive after successful push

* log options on error

* structured logs for socket connections

* fix structured log merge

* exit process after checkpointing

* update restore pull secret name

* append shortcode to restore names

* log handler payload

* disable post start lifecycle hook

* pass in coordinator host via volume

* replace dapi with taskinfo

* add missing restore label

* don't restart restored containers

* remove init container from create

* atomic post-completion checkpoints

* switch to run id for container names

* improve wait accuracy

* measure basic checkpoint perf

* always log disconnect reason

* use system clock to end wait spans

* checkpoint readiness and cancel signals

* restore from checkpoint events and fix statuses

* remove attempt id env var

* restore dependencies from events

* reconnect wip

* lifecycle hooks are back

* fix hooks and improve reconnect

* make docker send postStart hook

* only checkpoint for retry if large delay

* fix a few more resume issues

* lifecycle hook fixes

* skip connection handler when waiting for post start hook

---------

Co-authored-by: Eric Allam <eric@trigger.dev>
2024-03-25 16:28:56 +00:00
Matt Aitken b890a64046 The in-progress stripes are much more subtle 2024-03-25 16:20:51 +00:00