@trigger.dev/react-hooks@3.2.1
3 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Eric Allam
|
23b43be952 |
feat/realtime-streams (#1470)
* WIP realtime streams * Handle realtime with large payloads or outputs #1451 * feat: optimize Redis stream handling with batching Add STREAM_ORIGIN to environment schema. Improve performance in RealtimeStreams by using TextDecoderStream for simpler text decoding and implementing batching of XADD commands for Redis streams. Limit stream size using MAXLEN option. Update environment variable repository with new variable type. Adjust import statements for Redis key and value types. * 🔧 chore: add dev dependencies for bundle analysis * add metadata tests and a few more utilties * Add stream tests and improve streaming * Added AI tool tasks, descriptions to tasks * Use the config file path to determine the workingDir, then the package.json path * Remove stream test files * useTaskTrigger react hook that allows triggering a task from the client * Add streaming support for the realtime react hooks * Add ability to stream results after useTaskTrigger * Improve the stream throttling * Use the runId as the ID key to bust the cache after triggering * Upgrade to to the latest electric sql client and server * Make realtime server backwards compat with 3.1.2 release * Pass the runId into useRealtimeRun * Fix scopes when specifiying reading all runs * WIP @trigger.dev/rsc package * Various fixes and accepted recommendations by CodeRabbit * Regenerate pnpm lock file * A couple tweaks to rsc and give up on rendering react in tasks for now * Add changeset * Remove triggerRequest from the useEffect deps * Improve realtime & frontend authentication errors * Fixed authorization tests * Remove unnecessary log * Add metadata.stream limits and improve the metadata streams structure * Streams can now have up to 2500 entries * Various coderabbit fixes * additional react-hooks jsdocs |
||
|
Eric Allam
|
2d8a41b18b |
feat: realtime (#1402)
* Denormalize run tags, increase character limit to 128 * WIP realtime subscribing to runs * extracted the stream stuff into core, made it more reusable * WIP tags * Remove tags for now because it’s not support in electric * Support async iterables, readable stream, and callback style subscription styles * Remove tags streaming endpoint * Add realtime rate limits and scope them to the /realtime path * WIP rate limt per org * Introduce per org rate limits * WIP JWT auth * Move migrations into new internal db package * Resolve pnpm lock file * Authenticating to the realtime API with JWTs are working * realtime in the client * Created react-hooks package and starting to move stuff in there * Improve types for hooks * schema tasks * Added useBatch hook * build uploadthing/fal demo and change how run metadata is synced to the server * tweaks * WIL realtime concurrency tracking * Implement test for realtime client using testcontainers also updated electric to latest version * Allow customizing the expiration time of the automatic JWT created after triggering a task * Add support for subscribing to run tags * Improve auth types and API * finalize the realtime API * Fixed some example stuff * Allow up to 10 run tags * Remove core from docker-provider tsconfig paths to prevent it from being typechecked * do the same for the kubernetes provider * Fixing some typecheck errors * Fix webapp type errors * Update @trigger.dev/platform to 1.0.13 * Fix attw error * Remove from/to in subscribeToRuns query params * Add tests for the rate limit middleware and add custom JWT rate limits * turn off webapp test parallelism * Finish renaming jwt -> publicAccessToken and automatically give the JWT read access to the tags when using trigger * Add changeset * Attempt to fix unit tests in CI * Skip running the auth rate limit middleware tests for now * Try a beefier machine * Try and run webapp tests separately * Setup env vars * Make sliding window test more reliabile |
||
|
Matt Aitken
|
f2babbf637 |
Internal packages (testcontainers, redis-worker and zod-worker) (#1392)
* Some notes on the new run engine * lockfile with setup for the run engine * Documenting where TaskRun is currently mutated, to try figure out the shape of the new system * Added notes about how triggering currently works * Details about when triggering happens * Lots of notes about waitpoints * Started scaffolding the RunEngine * Sketch of Prisma waitpoint schema while it’s fresh in my mind * Got Prisma working with testcontainers * Use beforeEach/afterEach * Simple Prisma and Redis test * Return Redis options instead of a client * Simplified things * A very simple FIFO pull-based queue to check the tests working properly * Use vitest extend * Separate redis, postgres and combined tests for faster testing * Some fixes and test improvements * Pass a logger into the queue * A queue processor that processes items from the given queue as fast as it can * Test for retrying an item that wasn’t processed * First draft of waitpoints in the Prisma schema * Remove the custom logger from the test * Added a completedAt to Waitpoint * Notes on the flow for an execution starting * Added redlock, moved some files around * Starting point for the TaskRunExecutionSnapshot table * Added relationships to TaskRunExecutionSnapshot * Change some tsconfig * Moved some things around * Added some packages * WIP on the RunQueue * Fix for some imports * Key producer with some tests * Removed the nv type from the keys… it’s not useful to do global queries * Passing unit tests for all the public key producer functions * Some basic tests passing for the RunQueue * Simple enqueue test working * Enqueue and dequeue for dev is working * Don’t log everything during the tests * Enqueuing/dequeuing from the shared queue is working * Tests for getting a shared queue * The key producer sharedQueue can now be named, to allow multiple separate queues * The key producer uses the name of the queue as the input * Extra info in the Prisma schema * Dequeuing a message gets the payload and sets the task concurrency all in one Lua script * Adding more keys so we can read the concurrency from the queue * Setting the concurrency with dequeue and enquque is working * Improved the tests and fixed some bugs * Acking is resetting the concurrencies * Check the key has been removed after acking * Nacking is working * Changed the package to CommonJS + Node10 so it works with Redlock * Moved the database, otel and emails packages to be in internal-packages * Moved some Prisma code to the database package * Started using the RunEngine for triggering * Progress on run engine triggering, first waitpoint code * Create a delay waitpoint * Moved ZodWorker to an internal package so it can be used in the run engine as well as the webapp * Web app now uses the zod worker package * Added parseNaturalLanguageDuration to core/apps * internal-packages/zod-worker in the lockfile * Pass in the master queue, remove old rebalance workers code * Add masterQueue to TaskRun * Fixed the tests * Moved waitpoint code into the run engine, also the zod worker * Completing waitpoints * An experiment to create a new test container with environment * More changes to triggering * Started testing triggering * Test for a run getting triggered and being enqueued * Removed dequeueMessageInEnv * Update dev queue tests to use the shared queue function * Schema changes for TaskRunExecutionSnapshot * First execution snapshot when the run is created. Dequeue run function added to the engine * Separate internal package for testcontainers so they can be used elsewhere * Remove the simple queue and testcontainers from the run-engine. They’re going to be separate * Fix for the wrong path to the Prisma schem,a * Added the testcontainers package to the run-engine * redis-worker package, just a copy of the simple queue for now * The queue now uses Lua to enqueue dequeue * The queue now has a catalog and an invisible period after dequeuing * Added a visibility timeout and acking, with tests * Added more Redis connection logging, deleted todos * Visibility timeouts are now defined on the catalog and can be overridden when enqueuing * Dequeue multiple items at once * Test for dequeuing multiple items * Export some types to be used elsewhere * Partial refactor of the processor * First stab at a worker with concurrency and NodeWorkers * Don’t have a default visibility timeout in the queue * Worker setup and processing items in a simple test * Process jobs in parallel with retrying * Get the attempt when dequeuing * Workers do exponential backoff * Moved todos * DLQ functionality * DLQ tests * Same cluster for all keys in the same queue * Added DLQ tests * Whitespace * Redis pubsub to redrive from the worker * Fixed database paths * Fix for path to zod-worker * Fixes for typecheck errors, mostly with TS versions and module resolution * Redlock required a patch * Moved the new DB migrations to the new database package folder * Remove the run-engine package * Remove the RunEngine prisma schema changes * Delete triggerTaskV2 * Remove zodworker test script (no tests) * Update test-containers readme * Generate the client first * Use a specific version of the prisma package * Generate the prisma client before running the unit tests |