Commit Graph

1651 Commits

Author SHA1 Message Date
nicktrn 7591b6e619 Fix v4 restore race condition (#1870) 2025-04-02 18:05:43 +01:00
Eric Allam 8e5d5cf690 fix: only pass engine version header when inside a task (#1871)
This will allow migrating to the v4 SDK in your application backend before deploying v4 to trigger, and everything will just work
2025-04-02 17:40:07 +01:00
nicktrn dcacc3a9e2 Fixes for v4 waits and restores (#1868)
* remove dummy metrics from heartbeat

* fix heartbeat timeouts..

* reset clock when resolving waitpoints

* optionally set metadata url when scheduling run

* don't use global vitest as it's outdated

* don't run test files in parallel

* handle special graceful shutdown code

* exit codes for success and failure can now be set and overridden

* ensure immediate cleanup in tests

* update lockfile after conflict
2025-04-02 15:27:01 +01:00
Eric Allam 3e341701ae v4 engine version selection logic updated (#1866) 2025-04-02 14:04:01 +01:00
Eric Allam 51bb3ee464 v4 ai improvements (#1863)
* AI SDK sql generator and approval example WIP

* Create some nice spans and extract the tools out so we can get the type of the streams

* Enable AI SDK telemetry

* Adding ai.tool and removing toolTask, 3rd party telemetry spans now wil create partials, better ai SDK telemetry icons

* Created a separate d3-chat example, split out from the openai-agents example

* Fixed the tool options being passed to metadata in `ai.tool`

* Add a link to the run

* Slightly improved design

* Add a crawler task using crawl4ai

* Use a tool to get the userId

* Couple of tweaks

* Adding markdown rendering to assistant messages and added an e2b based chart rendering task

* Backup to anthropic

* Add changeset
2025-04-02 12:00:02 +01:00
Himanshu Sharma 28121249bb fix: prevent enqueuing into closed ReadableStream (#1781)
* fix: prevent enqueuing into closed ReadableStream

* Create smart-coins-hammer.md

---------

Co-authored-by: Eric Allam <eallam@icloud.com>
2025-04-01 11:52:41 +01:00
nicktrn bed834b589 Misc v4 checkpoint fixes (#1859)
* logs for optional services

* print env vars on startup in debug mode

* routes need to explicitly ask to keep connection alive

* log indicators for now

* make workload api listen host configurable

* expose supervisor metrics and make more configurable

* configurable pull secrets, no defaults

* remove restore route

* run controller to handle queued executing

* fix v3 deploys in v4 project

* update admin worker route

* only start pod cleaner et al in k8s mode

* set new worker group as default if none yet

* make image ref optional

* checkpoint image ref is optional for output as well

* export feature flag const

* one last image ref type fix

* make runner intervals configurable

* ability to set arbitrary env vars on new runners

* set default runtime back to node 21

* move all runner env vars to the same section
2025-04-01 10:08:15 +01:00
Eric Allam f320e14f38 cli: add dev lock file to prevent 2 dev processes running at the same time in the same dir (#1854)
* cli: add dev lock file to prevent 2 dev processes running at the same time in the same dir

* Make sure the .trigger dir exists before creating the dev.lock file
2025-03-31 16:27:11 +01:00
Matt Aitken 2b34dbe4b0 Fix for TaskRunErrorCodes.MAX_DURATION_EXCEEDED error saying ms when it’s seconds (#1848) 2025-03-31 14:48:03 +01:00
nicktrn 394f1decd3 Misc v4 fixes and improved logging (#1831)
* logs for optional services

* print env vars on startup in debug mode

* routes need to explicitly ask to keep connection alive

* log indicators for now

* make workload api listen host configurable

* expose supervisor metrics and make more configurable

* configurable pull secrets, no defaults

* remove restore route

* run controller to handle queued executing

* fix v3 deploys in v4 project

* update admin worker route

* only start pod cleaner et al in k8s mode

* set new worker group as default if none yet
2025-03-31 14:06:17 +01:00
Eric Allam c963dcda57 fix: use default retry settings when no catchError handlers are defined (#1852) 2025-03-31 13:57:02 +01:00
Eric Allam f4ef008e2c feat: Python otel support, enriching spans, OpenAI Agents SDK example (#1839)
* D3 demo WIP

* Agent working and completing token successfully

* Sending OpenAI Agent SDK spans through to the platform now works

* A couple of perf tweaks for enriching events
2025-03-28 13:01:59 +00:00
Matt Aitken 174484fb32 Added default time period to Runs, Waitpoint and Batch list pages. Improved dev presence (#1832)
* New time period filter (permanently displayed)

* Batches, and fix for blank state

* Waitpoint token filtering

* Tags query: remove things we’re not using

* Run tag and waitpoint tags use startsWith for faster search

* Fix for run page on wrong env. Added schedule last triggered column

* Removed the Redis pubsub, just use the presence key

* Improve the dev presence responsiveness

* The CLI presence connection recovers when the webapp is restarted

* Dev schedules are now working for v4

* Refactored to make the dev presence stuff

* Got rid of stupid extra /dev, added a connecting state with icon

* Remove unused Redis client
2025-03-28 09:32:42 +00:00
Eric Allam 589f7c3bcc v4: Fix HandleErrorFunction type (#1829)
* v4: Fix HandleErrorFunction type

* Only show the catchError span if there are any catchError hooks
2025-03-28 09:15:33 +00:00
Eric Allam 0e82698548 v4: Waitpoint PAT completion, fix dequeuing without retry config, fixed metadata (#1826)
* WIP

* Completing waitpoints using public access tokens

* fixed example

* adding agent-loops reference, adding useWaitToken react hook, adding a way to throw an error to output from the run, adding an icon option to logger.trace

* Fixed metadata system

* Check if the waitpoint is COMPLETED already in the complete endpoint and return true
2025-03-27 10:04:12 +00:00
James Ritchie be02439d1d Icons: updates and new icons for the side menu, run page and general improvements to the way they work (#1825)
* New format for the functions instead of “onWait() task”

* Make the icon sizes match the hero icons

* Updating icons for the new task hooks

* Adds new icons to the RunIcon.tsx component

* Improves the size of the menu icons

* Fixes the missaligned environment dropdown and dev connected button

* Changes the button colours so they are all unique

* More icon updates

* Adds new variables for main page icons

* Improves icons sizes

* Using new color variables in the side menu and blank states

* Adds preview environment color variable

* Align the env icon in the menu

* Updates the env icons to be the correct size and padding to match hero icons

* Label uses new colors, removes unused cases and new env icons

* Lower case env label

* Removes unused code

* Use full env title

* Organize imports

* More variants of the runs icon to work at smaller sizes

* small padding adjustment

* reformat init function span

* Fix last init formatting

* New colour for settings

* Organize switch statement for icons nicely

* Use new color variable

* Update icon svgs

* Renamed lifecycle hook icon to Function icon

* Renamed function icon file name
2025-03-26 19:12:12 +00:00
Matt Aitken 00586ffaaf Waitpoint tokens page, wait.listTokens() and wait.retrieveToken() (#1824)
* Added waitpoints/tokens to the sidebar

* Added indexes to the Waitpoint time for filtering

* Begun work on `WaitpointTokenListPresenter`, the pag is a copy of the Queues page for now

* MVP of waitpoint token page

* Added status

* Expiry of timeout/ttl

* Improvements to the waitpoint table

* Improved columns and icon

* Changes from the RunTag copy on hover branch

* Fix for nested button error

* Added waitpoint tags to the DB/table

* Applied Eric’s task run tag fix (it’s live on prod in the legacy run engine branch)

* Added tags to waitpoints

* Removed todos that have been done

* Added token support for releaseConcurrency. Also added a ton of JSDocs

* Added releaseConcurrency to the API token endpoint…

* WIP on waitpoint page filters

* Fix for tags filtering

* Waitpoint filters working

* Fix for badly named function

* WaitpointPresenter used from SpanPresenter

* Waitpoint detail panel WIP

* Fix for server client hydration issue with CodeBlock

* Selected waitpoint panel

* Added a blank state

* Added waitpoint docs link

* Fix for animated number going past the target

* Fix for the queue list pagination and upgrade status

* Engine version error for waitpoint token list

* RunTag component doesn’t get squished and hover behaviour is nicer

* Associating runs with waitpoints

* Added triggered icon

* Link directly to the waitpoint

* Fix for TS error on waitpoint retrieve

* Added CopyableText component, used for waitpoint id in the table

* Removed the confetti 🎊

* Deleted some old images

* Moved some schemas/types to core. Use `id` instead of `friendlyId`

* Added wait.listTokens() function. Made some changes to the types to make it nicer

* WIP wait.retrieveToken()

* wait.retrieveToken working

* Added data to retrieve token

* Separate ApiWaitpointPresenter completely

* Added completed time to the waitpoint detail panel

* Fix for the Avatar component having SSR issues. Specify the size in rems and removed the useLayoutEffect

* Fix for applied idempotency key filter dropdown showing the id field

* Use parentheses to make sure the token list query respects idempotency key correctly

* Use the proper logger, and have a decent message with info to track the bug down

* Pass the org title into the Avatar

* Better error when failing to creating a manual waitpoint after X attempts
2025-03-26 18:05:26 +00:00
nicktrn 8c3542cfd6 Small package release tweaks (#1821)
* set core version for redis-worker

* add empty changelog
2025-03-26 10:39:11 +00:00
nicktrn 5f48b9c7f9 Fix worker deployment engine version (#1820)
* enable re2 cli flag by default

* background worker engine determines deployment type
2025-03-26 10:22:13 +00:00
Eric Allam 6d5d10a45a Support other init file names 2025-03-25 16:33:12 +00:00
Eric Allam 1def8d06c2 Use allSettled so onWait/onResume errors don't break anything 2025-03-25 16:32:17 +00:00
Eric Allam 268c9f6898 implement onWait/onResume 2025-03-25 15:55:42 +00:00
Eric Allam 3ca5985277 Hooks now all use the new types, and adding some spans 2025-03-25 15:55:42 +00:00
Eric Allam df9cad6667 Get it all working 2025-03-25 15:55:42 +00:00
Eric Allam ba520fcdd8 Make sure and register all the config hooks 2025-03-25 15:55:42 +00:00
Eric Allam 23ae77afff handle max duration timeout errors better 2025-03-25 15:55:42 +00:00
Eric Allam 30705fd59b implement cleanup 2025-03-25 15:55:42 +00:00
Eric Allam d0a5c16c63 Add cleanup hook 2025-03-25 15:55:42 +00:00
Eric Allam 6ba2b0476b more tests 2025-03-25 15:55:42 +00:00
Eric Allam 5edbb77dfe use tryCatch in TaskExecutor 2025-03-25 15:55:42 +00:00
Eric Allam 7f804f381c middleware working and tests passing 2025-03-25 15:55:42 +00:00
Eric Allam 6ab74c47d6 implement and test handleError (now catchError) 2025-03-25 15:55:42 +00:00
Eric Allam f84ddcd6ea test oncomplete 2025-03-25 15:55:42 +00:00
Eric Allam 87824a8ce3 dry up some stuff 2025-03-25 15:55:42 +00:00
Eric Allam 91225da813 test on failure 2025-03-25 15:55:42 +00:00
Eric Allam 2262a42632 more tests and fixes 2025-03-25 15:55:42 +00:00
Eric Allam ef871b0434 starting test executor 2025-03-25 15:55:42 +00:00
Eric Allam 4a8d6932c8 a couple tweaks 2025-03-25 15:55:42 +00:00
Eric Allam 307309f4f7 implement onComplete 2025-03-25 15:55:42 +00:00
Eric Allam b974d8297d use new onFailure 2025-03-25 15:55:42 +00:00
Eric Allam 116788912b use new onStart 2025-03-25 15:55:42 +00:00
Eric Allam 2e6d0d8783 share the hook registration code 2025-03-25 15:55:42 +00:00
Eric Allam 2d16d664ef Add middleware hooks 2025-03-25 15:55:42 +00:00
Eric Allam 6001dfa7de Add middleware types 2025-03-25 15:55:42 +00:00
Eric Allam dc6e659e94 new locals API 2025-03-25 15:55:42 +00:00
Eric Allam e34e520057 more hooks 2025-03-25 15:55:42 +00:00
Eric Allam 597b7baaaa adding imports 2025-03-25 15:55:42 +00:00
Eric Allam e77e8d48d6 handleError 2025-03-25 15:55:42 +00:00
Eric Allam aaf2ed8a20 onWait and onResume 2025-03-25 15:55:42 +00:00
Eric Allam e816ba4382 onComplete 2025-03-25 15:55:42 +00:00