5 Commits

Author SHA1 Message Date
Eric Allam 536d9fa217 feat(realtime): Realtime streams v2 (#2632) 2025-11-11 14:54:00 +00:00
Eric Allam 0f66023b54 v4: metadata collapsing now preserves operation order for correctness (#2115)
* v4: metadata collapsing now preserves operation order for correctness

* Add changeset
2025-05-28 14:52:06 +01:00
Eric Allam b74c000fb6 Improve metadata flushing efficiency by collapsing operations (fix #2104) (#2106)
* Improve metadata flushing efficiency by collapsing operations (fix #2104)

* Add changeset

* A couple of style tweaks
2025-05-27 14:08:15 +01:00
Eric Allam 34f8bd588e Add ability to update parent and root run metadata from children (#1563)
* Increase the number of active streams from 2 to 5

* WIP parent metadata updates

* Fix noop metadata manager

* Implement run metadata updates from ancestor tasks

* Add changeset

* Add ability to stream into parent and root task runs

* Don't duplicate exporting run metadata types

* Add ability to fetch streams through metadata

* Couple of fixes from CodeRabbit

* Fix metadata tests

* Make sure streams are subscribed in the "background"

* Move the stream subscription stuff to the API client, expose it through `runs.fetchStream`

* Fixed run stream tests
2025-01-08 11:29:24 +00:00
Eric Allam 23b43be952 feat/realtime-streams (#1470)
* WIP realtime streams

* Handle realtime with large payloads or outputs #1451

* feat: optimize Redis stream handling with batching

Add STREAM_ORIGIN to environment schema. Improve performance in
RealtimeStreams by using TextDecoderStream for simpler text
decoding and implementing batching of XADD commands for Redis
streams. Limit stream size using MAXLEN option. Update
environment variable repository with new variable type. Adjust
import statements for Redis key and value types.

* 🔧 chore: add dev dependencies for bundle analysis

* add metadata tests and a few more utilties

* Add stream tests and improve streaming

* Added AI tool tasks, descriptions to tasks

* Use the config file path to determine the workingDir, then the package.json path

* Remove stream test files

* useTaskTrigger react hook that allows triggering a task from the client

* Add streaming support for the realtime react hooks

* Add ability to stream results after useTaskTrigger

* Improve the stream throttling

* Use the runId as the ID key to bust the cache after triggering

* Upgrade to to the latest electric sql client and server

* Make realtime server backwards compat with 3.1.2 release

* Pass the runId into useRealtimeRun

* Fix scopes when specifiying reading all runs

* WIP @trigger.dev/rsc package

* Various fixes and accepted recommendations by CodeRabbit

* Regenerate pnpm lock file

* A couple tweaks to rsc and give up on rendering react in tasks for now

* Add changeset

* Remove triggerRequest from the useEffect deps

* Improve realtime & frontend authentication errors

* Fixed authorization tests

* Remove unnecessary log

* Add metadata.stream limits and improve the metadata streams structure

* Streams can now have up to 2500 entries

* Various coderabbit fixes

* additional react-hooks jsdocs
2024-11-19 13:14:25 +00:00