* run engine v1: orgs are no longer considered for concurrency
* Add reserve concurrency concept to allow waiting to resume parent tasks to release concurrency at the env level for child tasks to use (or else there is a deadlock). WIP recursive tasks
* child tasks inherit the queue timestamp from their parent tasks to prioritize completing child tasks based on when their parent started
* handle reserve concurrency with recursive deadlocks
* Finish docs update for concurrency
* Some fixes from badge conflict resolution
* WIP priority queues
* Implement MarQS priority queues
* Fix the migrations
* WIP with webhook SDK function and types
* JSDocs added to the schema
* Webhooks are working
* Expanded the alert docs
* Remove duplicate export of waitUntil.js
* Use uncrypto
* Don’t rate limit webhooks
* Create slow-olives-fix.md
* Move the task run heartbeats to RedisWorker
* Move alerts to redis worker, improving redis worker
* Fix typecheck errors
* Use single threaded tests for redis worker
* Enable/disable the redis workers independently
* Remove preview release from PR checks
* engine v1 fairness improvements (env based queue selection)
* Add ability to bias env selection by concurrency limit and capacity
* Add some randomization to the list of queues within each evn
* Configure biases using env vars
* Typo
* revert v3 catalog project ref change
* Fix v2 marqs
* WIP langsmith & AI SDK integration
* Add changeset
* Add exporter support to deployed tasks
* Better support for external exporters and group exporters and instrumentations under the telemetry config property
* Missing changes
* Increase the number of active streams from 2 to 5
* WIP parent metadata updates
* Fix noop metadata manager
* Implement run metadata updates from ancestor tasks
* Add changeset
* Add ability to stream into parent and root task runs
* Don't duplicate exporting run metadata types
* Add ability to fetch streams through metadata
* Couple of fixes from CodeRabbit
* Fix metadata tests
* Make sure streams are subscribed in the "background"
* Move the stream subscription stuff to the API client, expose it through `runs.fetchStream`
* Fixed run stream tests
* Fix streaming splits in realtime streams v2
* Add changeset
* Skip all flaky tests 😡
* Improve the way we stream from tasks to the server
* Improve the v1 realtime streams (Redis)
* Turn on the relay realtime stream service
* Improved the relay realtime cleanup
* Fixed consuming realtime runs w/streams after the run is already finished
* Remove some logs
* Update changeset
* Fixed runStream tests
* Realtime streams now powered by electric, and fix the streaming update duplicate issues by converting the electric Shape materialized view into a ReadableStream of changes
* Ensure realtime subscription stops when runs are finished, and add an onComplete handle to use realtime hooks
* Fix tests
* WIP realtime streams
* Handle realtime with large payloads or outputs #1451
* feat: optimize Redis stream handling with batching
Add STREAM_ORIGIN to environment schema. Improve performance in
RealtimeStreams by using TextDecoderStream for simpler text
decoding and implementing batching of XADD commands for Redis
streams. Limit stream size using MAXLEN option. Update
environment variable repository with new variable type. Adjust
import statements for Redis key and value types.
* 🔧 chore: add dev dependencies for bundle analysis
* add metadata tests and a few more utilties
* Add stream tests and improve streaming
* Added AI tool tasks, descriptions to tasks
* Use the config file path to determine the workingDir, then the package.json path
* Remove stream test files
* useTaskTrigger react hook that allows triggering a task from the client
* Add streaming support for the realtime react hooks
* Add ability to stream results after useTaskTrigger
* Improve the stream throttling
* Use the runId as the ID key to bust the cache after triggering
* Upgrade to to the latest electric sql client and server
* Make realtime server backwards compat with 3.1.2 release
* Pass the runId into useRealtimeRun
* Fix scopes when specifiying reading all runs
* WIP @trigger.dev/rsc package
* Various fixes and accepted recommendations by CodeRabbit
* Regenerate pnpm lock file
* A couple tweaks to rsc and give up on rendering react in tasks for now
* Add changeset
* Remove triggerRequest from the useEffect deps
* Improve realtime & frontend authentication errors
* Fixed authorization tests
* Remove unnecessary log
* Add metadata.stream limits and improve the metadata streams structure
* Streams can now have up to 2500 entries
* Various coderabbit fixes
* additional react-hooks jsdocs
* Updated styling
* Updated the main Table component styles to the new design
* Run page table layout style updates
* Updated row styling and new scrolling behaviour
* Fixed bulk action bar no appearing
* Improved table component styles
* Pagination arrows now chevrons
* Added link to Runs docs page
* Small popover tweak
* Moved the extra Test button out of Tables and into the RunsTable
* WIP refactoring the sticky table cell
* Support for the existing way we have menus in the sticky table cells
* Created a specific Docs style button
* Improved dialog trigger button styles
* Improved docs button styles
* Updated Docs buttons to the new style in the tooltips
* Adding a Test button on hover
* Removed the page padding and made table scroll
* Fixed table scrolling
* cleaned up imports
* WIP adding a test button that links to the test page
* added a docs button to the header
* Much nicer display bar for showing how many schedules you have (and it works with the updated table)
* Fixed the button types in the schedules info panel
* Fixed button icon colour
* Removed the pagination as it’s not being used
* Delete modal button position updated
* Fixed padding on recent payloads panel
* Fixed button icon colours
* Better style for active table row
* Table scrolling now behaves properly
* environment buttons now have a hover state
* Bigger Run test bar to match the Run page
* Small layout fixes to the schedule side panel
* Test page now has link to docs
* fixed text alignment with table headers
* API keys page now has full width tables
* Fix typo and modal layout
* Table uses new sticky button type
* Tidy imports
* Env var table now full width
* Form buttons have a border top
* Updated button styles
* Updated blank state message style to be consistent
* Removed unnecessary form button border
* Alerts page tables now full width
* Allowing the Detail cell to be styled
* Updated the page structure to be the same as the scheduled page – with better progress indicator
* Concurrency limits page now full width
* Fixed function name label extending too far
* Improved the button sizes on the replay modal
* Added a minimal style to the info panels
* Make sure you can always see the info and unlock staging info when the table scrolls
* Info panel style updated to match the others
* Removed the duplicate header
* Usage page tables now full width
* Personal access token page using full width tables
* Small CSS fixes
* Deployments page table now full width
* Deploy page scrolls properly and pagination stays on screen if table is scrollable
* TableCellMenu using the correct popover
* Added missing button variant
* Button group is now named so it doesn’t cause conflicts
* Added docs link to the Run page
* Typo
* Added a link to the test page if you have no runs
* Make the table sticky cell buttons more obvious
* Env var page now using the new popover menu
* Improved the menu items
* Copy tweak
* Made the icons bright on hover
* Updated the cancel run dialog
* Improved the API keys sticky menu behaviour
* Alerts table menu using the new popover
* Correct sized buttons in the Env Var page table menu
* Fixed weird Dialog text styling
* Schedules page blank state using correct docs button style
* Copy tweak
* Menu buttons styled correctly
* Added close buttons to the deploy dialogs
* Made the Dialog footer match the Form footer style
* Fixed feedback form buttons
* Fixed cancel run dialog
* Test page uses the taskIdentifier in the path
* Test task button working
* Added Spline component for 404 logo to remove spline logo
* Removed todo comment
* Added a docs button to the PAT page
---------
Co-authored-by: Matt Aitken <matt@mattaitken.com>
* feat: exit if docker buildx can't be found for self-hosted builds
* chore: tweak debug log message
* chore: improve error message based on rabbit's suggestion
* refactor finalize run service
* refactor complete attempt service
* remove separate graceful exit handling
* refactor task status helpers
* clearly separate statuses in prisma schema
* all non-final statuses should be failable
* new import payload error code
* store default retry config if none set on task
* failed run service now respects retries
* fix merged task retry config indexing
* some errors should never be retried
* finalize run service takes care of acks now
* execution payload helper now with single object arg
* internal error code enum export
* unify failed and crashed run retries
* Prevent uncaught socket ack exceptions (#1415)
* catch all the remaining socket acks that could possibly throw
* wrap the remaining handlers in try catch
* New onboarding question (#1404)
* Updated “Twitter” to be “X (Twitter)”
* added Textarea to storybook
* Updated textarea styling to match input field
* WIP adding new text field to org creation page
* Added description to field
* Submit feedback to Plain when an org signs up
* Formatting improvement
* type improvement
* removed userId
* Moved submitting to Plain into its own file
* Change orgName with name
* use sendToPlain function for the help & feedback email form
* use name not orgName
* import cleanup
* Downgrading plan form uses sendToPlain
* Get the userId from requireUser only
* Added whitespace-pre-wrap to the message property on the run page
* use requireUserId
* Removed old Plain submit code
* Added a new Context page for the docs (#1416)
* Added a new context page with task context properties
* Removed code comments
* Added more crosslinks
* Fix updating many environment variables at once (#1413)
* Move code example to the side menu
* New docs example for creating a HN email summary
* doc: add instructions to create new reference project and run it locally (#1417)
* doc: add instructions to create new reference project and run it locally
* doc: Add instruction for running tunnel
* minor language improvement
* Fix several restore and resume bugs (#1418)
* try to correct resume messages with missing checkpoint
* prevent creating checkpoints for outdated task waits
* prevent creating checkpoints for outdated batch waits
* use heartbeats to check for and clean up any leftover containers
* lint
* improve exec logging
* improve resume attempt logs
* fix for resuming parents of canceled child runs
* separate SIGTERM from maybe OOM errors
* pretty errors can have magic dashboard links
* prevent uncancellable checkpoints
* simplify task run error code enum export
* grab the last, not the first child run
* Revert "prevent creating checkpoints for outdated batch waits"
This reverts commit f2b5c2ac42.
* Revert "grab the last, not the first child run"
This reverts commit 89ec5c8bfd.
* Revert "prevent creating checkpoints for outdated task waits"
This reverts commit 11066b4e74.
* more logs for resume message handling
* add magic error link comment
* add changeset
* chore: Update version for release (#1410)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
* Release 3.0.13
* capture ffmpeg oom errors
* respect maxAttempts=1 when failing before first attempt creation
* request worker exit on fatal errors
* fix error code merge
* add new error code to should retry
* pretty segfault errors
* pretty internal errors for attempt spans
* decrease oom false positives
* fix timeline event color for failed runs
* auto-retry packet import and export
* add sdk version check and complete event while completing attempt
* all internal errors become crashes by default
* use pretty error helpers exclusively
* error to debug log
* zodfetch fixes
* rename import payload to task input error
* fix true non-zero exit error display
* fix retry config parsing
* correctly mark crashes as crashed
* add changeset
* remove non-zero exit comment
* pretend we don't support default default retry configs yet
---------
Co-authored-by: James Ritchie <james@trigger.dev>
Co-authored-by: shubham yadav <126192924+yadavshubham01@users.noreply.github.com>
Co-authored-by: Tarun Pratap Singh <101409098+Wackyator@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
* Denormalize run tags, increase character limit to 128
* WIP realtime subscribing to runs
* extracted the stream stuff into core, made it more reusable
* WIP tags
* Remove tags for now because it’s not support in electric
* Support async iterables, readable stream, and callback style subscription styles
* Remove tags streaming endpoint
* Add realtime rate limits and scope them to the /realtime path
* WIP rate limt per org
* Introduce per org rate limits
* WIP JWT auth
* Move migrations into new internal db package
* Resolve pnpm lock file
* Authenticating to the realtime API with JWTs are working
* realtime in the client
* Created react-hooks package and starting to move stuff in there
* Improve types for hooks
* schema tasks
* Added useBatch hook
* build uploadthing/fal demo and change how run metadata is synced to the server
* tweaks
* WIL realtime concurrency tracking
* Implement test for realtime client using testcontainers
also updated electric to latest version
* Allow customizing the expiration time of the automatic JWT created after triggering a task
* Add support for subscribing to run tags
* Improve auth types and API
* finalize the realtime API
* Fixed some example stuff
* Allow up to 10 run tags
* Remove core from docker-provider tsconfig paths to prevent it from being typechecked
* do the same for the kubernetes provider
* Fixing some typecheck errors
* Fix webapp type errors
* Update @trigger.dev/platform to 1.0.13
* Fix attw error
* Remove from/to in subscribeToRuns query params
* Add tests for the rate limit middleware and add custom JWT rate limits
* turn off webapp test parallelism
* Finish renaming jwt -> publicAccessToken and automatically give the JWT read access to the tags when using trigger
* Add changeset
* Attempt to fix unit tests in CI
* Skip running the auth rate limit middleware tests for now
* Try a beefier machine
* Try and run webapp tests separately
* Setup env vars
* Make sliding window test more reliabile
Eric Allam