* Add createdAt filter to realtime subscribing with tags
* Filter realtime colums and expose ability to skip some columns
* Add sharding support for electric
* Use unkey cache for the created at filter caching
* Remove 2 unused indexes on TaskRun
* Run list now filters by a single runtime environment
* Remove project ID indexes
* Use clickhouse in task list aggregation queries instead of pg (keep pg for self-hosters)
* WIP clickhouse powered runs list
stuff
* Improve the query to get the latest tasks for the task list presenter
* Update the usage task list to use clickhouse
* Implement next runs list powered by clickhouse
* Add new index for TaskRun for the runs list, by environment ID
* Add runTags gin index
* Handle possibly malicious inputs
* Ignore claude settings
* Better handling not finding an environment on the schedule page
* Use ms since epoch in test, not seconds
* Remove unused function
* Fix test
* Use an env var for the realtime maximum createdAt filter duration (defaults to 1 day)
* Fixed the query builder to correct the group by / order by order
* Make sure runs.list still works
* Create small-birds-arrive.md
* Delete v2 Stripe routes
* Delete v2 billing/usage pages
* Delete v2 integration pages
* Delete v2 project pages
* Deleted a load of components and services
* Deleted a load more components, presenters and services
* Deleted another 100 files or so…
* Removed old v2 paths
* Removed named icons from form titles
* Removed more string icons
* Delete NamedIcon
* Fixed some type errors
* Delete endpointApi
* Removed v2 from core/sdk
* Post merge fixes
* added explicit return types
* using the new sdk export without v3
* Delete old v2 file
* Added explicit return types because TS was complaining…
* Don’t export RuntimeEnvironmentType from two core files. Was causing TS issue
* Fix for removal of NamedIcon in new route
* Removed strange eslintrc rule
* Use the new redis client
---------
Co-authored-by: James Ritchie <james@trigger.dev>
* Initial TaskRunConcurrencyTracker implementation
* MARQS calls a subscriber to events
* When enqueuing add the extra required metadata
* Track concurrency per environment for tasks too
* Admin page for global concurrency
* Use the new concurrency tracker on the tasks page
* Useful performance test task
* getAllTaskIdentifiers()
* New page for concurrency
* BackgroundWorkerTask index for quick lookup of task identifiers
* Added a way to get concurrency for environments
* Added upgrade/request more concurrency button
* Queued task column working
* Use defer and suspense
* Added queue column to the concurrency environments table
* Some comments added for clarity
* Fixed bad log message
* Sidemenu: move lower and rename to “Concurrency limits”
* Only show the environments, not tasks. Renamed to “Concurrency limits”
* Add OpenAI support for 4.16.0
* Add support for background polling and use that in OpenAI integration to power assistants
* Much improved OpenAI docs
* Added backgroundPoll docs
* Implements waitForEvent and added docs for more built in tasks
* Add sendEvent API referenc
* Write the task libray
* Add changeset and warning for waitForEvent
* Implement manually invokable jobs through the invokeTrigger
Also implemented a job run notification system, that will POST details of a run on completion. This combines with the task callbackUrl system to implement the invokeAndWait
* Document the invoke trigger
* batch invoke and wait
* background fetch timeouts
* Use @whatwg-node/fetch instead of the polyfilled fetch
* Fix some outdated dependencies in webapp
* Improved subtask error propogation messages
* Document the OpenAI changes and the batch invoke stuff
* Fix dequeuing jobs
* Don’t retry the OpenAI completion background task
* Added OpenAI changesets
* Use the new ResumeTaskService in ProcessCallbackTimeout as well
* Auto-yield run execution to help prevent duplicate task executions
* Add auto-yield config to endpoints
* Refactor run execution with buffer and limits
Introduced constants RUN_CHUNK_EXECUTION_BUFFER and MAX_RUN_CHUNK_EXECUTION_LIMIT. Adjusted PerformRunExecutionV2Service to use the new constants to fine-tune execution timings and buffers.
* Add endpoint probing functionality
Added new `RESPONSE_TIMEOUT_STATUS_CODES` in `consts.ts` to manage timeout responses. Additional functions `detectResponseIsTimeout(response: Response)` was added in `endpoint.server.ts` to detect if a response was a timeout based on the status codes from `RESPONSE_TIMEOUT_STATUS_CODES`.
Update actions to use new endpoint probing endpoint service. This allows for the early probing of endpoints to determine if they're up and running.
A new class `ProbeEndpointService` was created in `probeEndpoint.server.ts` which makes HTTP requests to a given endpoint and updates its properties based on the result.
Finally, `detectResponseIsTimeout(response)` is used in `performRunExecutionV2.server.ts` for marking the execution as succeeded when facing a timeout.
* Refactored probe method in EndpointApi class
The probe method of the EndpointApi class has been refactored to remove the error handling part and it now takes a timeout sent from the client directly. The corresponding changes were also made in the ProbeEndpointService and TriggerClient objects to reflect the alterations in the probe method.
The error handling related to the timeout has been removed and the responsibility of handling the timeout has been shifted to the client. Thus, the probe method has been greatly simplified. The `probeEndpoint.server.ts` file was also changed to accommodate the change in behavior of the probe result.
In the `triggerClient.ts` the timeout for probe is now read from the incoming request object. For backward compatibility, if no timeout is provided in the request, the default value of 15 minutes is used.
* Remove performRunExecution v1 enqueue function
* Better document limits and add docs on increasing function timeouts
* Upgrade webapp docker container to use 18.18.2
* force clients to yield when a run is executing in a gracefully shutting down worker
* Renamed task `key` to `cacheKey` and added more task documentation
* Index the `@trigger.dev/sdk` version on Endpoints
* Improves the perform of run resuming
When runs resume, we try and make sure that tasks that have already been completed are cached and reused. Worst case scenario the client needs to hit the API server once for a non-cached task that is indeed completed on the server, but this can get pretty expensive when there are a larger number of tasks.
This commit does 2 different things to help:
- noop tasks are no longer “cached” using the cachedTasks strategy, instead their idempotency keys are shoved into a bloom filter and the client tests for their inclusion in the bloom filter before running them (since they don’t have any concept of output, this works)
- Additional cached tasks are lazy loaded when a task is run. This allows us to progressively fetch additional tasks to be cached on the client, which will cut down on cache misses by a decent amount
* Create warm-carrots-float.md
* Make io.yield backwards compat with older platform versions
* Better support old clients connecting to server versions that support lazy loading cached tasks
* Fixed type errors when settings headers with unknown value
* Better yield not support error message
* Rename _version to _serverVersion to be more clear
* Setup project-wide prettier
* Remove old workspace file
* Remove old debugging directives
* New top-level .prettierignore
* Updated Prettier config settings
* Contrubuting guide: Fix for some bad code blocks
* Added more ignores
* Improved the format script command
* printWidth set to 100
* Formatted entire repo (pnpm run format)
* Changeset should ignore the example projects
* Removed note in Contributing instructions about not adding a changeset for internal
* Renamed @trigger.dev/internal to @trigger.dev/core. Set sdk and internal to be ES2020, so we don’t get errors about private identifiers
* Env vars in nextjs-example use square bracket syntax to avoid Turbo Repo errors
* Set the tsconfigs back for core and sdk
* @examples/nextjs compile error with undefined tasks
* Set the example projects to use ES2015 to avoid private modifier complaints
* package-tester example, which will use built packages
* package-tester package.json
* Created a readme for the package-tester
* Added all the packages to package-tester
* Create .env.local.example and readme instructions
* Added name to the package.json
* Upgraded @types/react and @types/react-dom everywhrre, so we can use server actions
* Upgraded @types/react and @types/react-dom everywhrre, so we can use server actions
* Reworked the react package build, so it generates separate files
* The SDK no longer bundles core
* package-tester setup with a server action and react hooks
* Added new SDK methods with logging
* Working tsup settings for client and server
* Explicit react hooks return types
* Added all the hooks for testing
* Added OpenAI step to the job to check types are still ok in integrations
* Get rid of rogue Changeset ignores
* Changeset: @trigger.dev/core is now a separate package
* Exited prerelease mode, added a changeset
* Changed @next to @latest
* Deleted seed.js, this shouldn’t be committed
* Ignore seed.js
* The webapp was importing @trigger.dev/core with a folder path rather package name…
* More webapp imports instead of @trigger.dev/core were a folder path
* Accidentally edited Stripe internal API url in the comment
* Added “sideEffects”: false so @trigger.dev/core is tree shaken by Remix
Eric Allam