nicktrn
14c2bdf89b
* only checkpoint retries with delays greater than threshold * rename checkpoint threshold env var * log task monitor ignores * crash runs with unbounded attempts * fix retry check in shared queue consumer * add missing stop for env var sync spinner * prod entry point refactor * missing awaits * more verbose prod flush and exit logs * reduce checkpoint support logs * heartbeat while checkpointing between retries * dynamic coordinator config * measure lazy attempt creation time in prod * simplify delay threshold * heartbeat clarifications * crash run if it doesn't reach checkpointable state * require dynamic config threshold * fix retry prep, await previous worker kill * unify wait mechanics * fix prod worker without tasks error * ensure worker is ready to be checkpointed for dependency waits * improve worker attempt creation logging * prevent crashes caused by failed socket schema parsing * fix dynamic imports in v3 catalog * clarify attempt retry mechanics * move backoff helper to core-apps * remove core-apps barrel file * add backoff execute with callback * deprecate non-lazy attempt messages * update socket.io-client to v4.7.5 * fix socket.io types for emits with timeout * retry all the things * remove todo * fix retry restores * improve index failure logs * retry incomplete dependency waits * fix checkpoint in-progress detection * prevent losing messages during reconnect * checkpoint when greater or equal to threshold * improve handling of duration wait edge cases * add ready for lazy attempt replay * retry attempt completion * allow failing runs with unfriendly run id * fix min max jitter * cancel checkpoints on run failure * improve attempt creation errors * prevent crashing run on failed cleanup * handle at-least-once execute lazy attempt delivery * log exit code on prepare for retry * fix timeout promise * mark some things * chaos monkey superpowers * refactor checkpointer * set chaos monkey defaults * less chaos * fix backoff * handle uncaught entry point exceptions * only replay rpcs on true reconnects * allow resume unless final run status * add changeset * small fixes
Trigger.dev proxy
This is an optional module that can be used to proxy and queue requests to the Trigger.dev API.
Why?
The Trigger.dev API is designed to be fast and reliable. However, if you have a lot of traffic, you may want to use this proxy to queue requests to the API. It intercepts some requests to the API and adds them to an AWS SQS queue, then the webapp can be setup to process the queue.
Current features
- Intercepts
sendEventrequests and adds them to an AWS SQS queue. The webapp then reads from the queue and creates the events.
Setup
Create an AWS SQS queue
In AWS you should create a new AWS SQS queue with appropriate security settings. You will need the queue URL for the next step.
Environment variables
Cloudflare secrets
Locally you should copy the .dev.var.example file to .dev.var and fill in the values.
When deploying you should use wrangler (the Cloudflare CLI tool) to set secrets. Make sure you set the correct --env ("staging" or "prod")
wrangler secret put REWRITE_HOSTNAME --env staging
wrangler secret put AWS_SQS_ACCESS_KEY_ID --env staging
wrangler secret put AWS_SQS_SECRET_ACCESS_KEY --env staging
wrangler secret put AWS_SQS_QUEUE_URL --env staging
wrangler secret put AWS_SQS_REGION --env staging
You need to set your API CNAME entry to be proxied by Cloudflare. You can do this in the Cloudflare dashboard.
Webapp
These env vars also need setting in the webapp.
AWS_SQS_REGION
AWS_SQS_ACCESS_KEY_ID
AWS_SQS_SECRET_ACCESS_KEY
AWS_SQS_QUEUE_URL
AWS_SQS_BATCH_SIZE
Deployment
Staging:
npx wrangler@latest deploy --route "<your-api-subdomain>/*" --env staging
Prod:
npx wrangler@latest deploy --route "<your-api-subdomain>/*" --env prod
Development
Set the environment variables as described above.
pnpm installpnpm run dev --filter proxy