Eric Allam
f9ec66c562
* upgrade @opentelemetry packages to the latest versions * remove v2 only packages, will be moved to a dedicated repo * remove more v2 code and run pnpm install * use the npm yalt package in the webapp * convert @trigger.dev/core to tshy * Switch from jest to vitest in @trigger.dev/core * Fixed core test * move core-backend code into core subpath export * convert @trigger.dev/sdk to tshy * Removed hono * move core-apps to core/v3/apps, remove core-apps, start converting cli-v3 * Fix up some of the commands * cli now building and loadable * using package-json-from-dist to get package version now in core and cli * dev command WIP * cleaned up some repetition and structure of the entry point stuff * bringing back the background worker stuff * Indexing of the v3 catalog * getting closer to executing dev runs... * centralize dev logging using event emitter * Move indexing to it’s own entry point, simplify code * dev runs working * Get instrumentation to work with openai * debugging achieved internally * provide worker files as part of the worker creation on the server * support for cjs and esm javascript * Fixed timeout * worker manifest now has the config path * auto-upgrade config to non-deprecated alternatives * Adding package preview release * deployment WIP * improve the syncEnvVars output and adapt resolveEnvVars * WIP bun runtime * WIP bun support * seed tasks with the machine preset if listed in the config * deploy run executions WIP, extracted TaskRunProcess into 1 place * deployed tasks running and executing 🎉 * support for waits and better flushing & process cleanup * Fixed the heartbeating * Better warning messages * Improve and unify the indexing between dev and deploy * Support for external deps that need node-gyp to build * build extensions can now install custom packages and run instructions in the image. Also prisma extension now works and also works with multiple schema files * Add back in the main/types/module to sdk * dev no longer is Ink/React, grace period for disconnections in dev * Fix the changeset config * More changeset fixes * Remove config packages * More changeset fixes * Fixed typescript issues (needed to revert back to zod 3.22.3 * Fix pr_checks workflow * Remove the prepare script * Fixed tests and package versions * Remove cli test script * Remove packages from tailwind watch paths * Add repo to public packages * Just commit the generated files and do the building at dev time * Try and get pkg.pr.new working * Try again * Fix emitDecoratorMetadata importing named export from typescript * config file backwards compat with export const config * Fixed issue where import errors weren’t coming through * p-retry is a prod dep * typescript needs to be a prod dependency for emitDecoratorMetadata * Add better debug logging to help track down import-in-the-middle bug * An external is only considered resolvable if it resolves to the same path as the collected external * Fix runtime checks to allow >=18.20 * Move extensions to a new build package * Fixed building packages in dockerfile * Remove the e2e test from publish workflow for now * Don’t treat pkg.pr.new versions has needing upgrading * making sure config handleError works, and discovered path aliases don’t work in config files * Strip empty string env vars so they accidentally override real values * Couple of things * Update version to use preview instead of beta * Hopefully fix re-attempts with >30s delay * Match socket emit messages to current latest in main * Initial guide * Go back to beta * Go back to the preview, and update guide to use pr preview tags * Go back to beta --------- Co-authored-by: Matt Aitken <matt@mattaitken.com>
Trigger.dev proxy
This is an optional module that can be used to proxy and queue requests to the Trigger.dev API.
Why?
The Trigger.dev API is designed to be fast and reliable. However, if you have a lot of traffic, you may want to use this proxy to queue requests to the API. It intercepts some requests to the API and adds them to an AWS SQS queue, then the webapp can be setup to process the queue.
Current features
- Intercepts
sendEventrequests and adds them to an AWS SQS queue. The webapp then reads from the queue and creates the events.
Setup
Create an AWS SQS queue
In AWS you should create a new AWS SQS queue with appropriate security settings. You will need the queue URL for the next step.
Environment variables
Cloudflare secrets
Locally you should copy the .dev.var.example file to .dev.var and fill in the values.
When deploying you should use wrangler (the Cloudflare CLI tool) to set secrets. Make sure you set the correct --env ("staging" or "prod")
wrangler secret put REWRITE_HOSTNAME --env staging
wrangler secret put AWS_SQS_ACCESS_KEY_ID --env staging
wrangler secret put AWS_SQS_SECRET_ACCESS_KEY --env staging
wrangler secret put AWS_SQS_QUEUE_URL --env staging
wrangler secret put AWS_SQS_REGION --env staging
You need to set your API CNAME entry to be proxied by Cloudflare. You can do this in the Cloudflare dashboard.
Webapp
These env vars also need setting in the webapp.
AWS_SQS_REGION
AWS_SQS_ACCESS_KEY_ID
AWS_SQS_SECRET_ACCESS_KEY
AWS_SQS_QUEUE_URL
AWS_SQS_BATCH_SIZE
Deployment
Staging:
npx wrangler@latest deploy --route "<your-api-subdomain>/*" --env staging
Prod:
npx wrangler@latest deploy --route "<your-api-subdomain>/*" --env prod
Development
Set the environment variables as described above.
pnpm installpnpm run dev --filter proxy