Commit Graph

3735 Commits

Author SHA1 Message Date
Matt Aitken 9dc666851f Delete the old SSE util and updated the test endpoint to use the new one 2025-04-09 12:24:00 +01:00
Matt Aitken 3ce6144339 Added the Attempt span error back (and properties) (#1900) 2025-04-08 22:45:11 +01:00
Matt Aitken 3cbbece473 Run completed at fix and engine API rate limit whitelist (#1899)
* Use completedAt instead of updatedAt

* Disable verbose SSE logs

* Engine rate limiter: whitelist engine worker-actions

* Show the compute column in the runs table

* The compute column should be invocation + compute cost
2025-04-08 20:55:35 +01:00
Matt Aitken 79d6044dd0 Added engine worker immediatePollIntervalMs and limit env vars (#1898) 2025-04-08 15:39:44 +01:00
nicktrn 72d98778eb Fix node options and OOM retries (#1897)
* don't hardcode max old space size in deployment images

* flags: treat underscores as hyphens

* append attempt number to runner name if >1

* improve retry spans for oom errors
2025-04-08 13:07:45 +01:00
Matt Aitken a51b9b9f98 Fix: custom queue releaseConcurrencyOnWaitpoint. (#1896)
* Make sure releaseConcurrencyOnWaitpoint is set on taskQueue

* Allow task queue to have releaseConcurrencyOnWaitpoint (required to fix custom queues)

* Test for releaseConcurrencyOnWaitpoint

* Added a warning if you use the same queue twice with different settings

* Improved the error some more
2025-04-08 12:32:22 +01:00
James Ritchie b322cf1eda Fix for long file names wrapping instead of truncating on the RunTimeline (#1891) 2025-04-07 16:36:22 +01:00
James Ritchie 86e56144c0 Adds selected state for waitpoint table rows (#1892) 2025-04-07 16:35:58 +01:00
Matt Aitken 32d2992802 Security update of nanoid from 3.3.4 to 3.3.8 (#1887) 2025-04-07 13:56:37 +01:00
nicktrn 58567bfe44 Add v4 timeline metrics to prod runs (#1884)
* set correct worker name on create

* dev: kill run process on failed cancel attempt

* add missing prod timeline events

* add more debug logs after api call failures

* import events are always visible

* set dequeued at on new runners

* skip fork event test

* fix timeline test for import event
2025-04-04 18:39:13 +01:00
Matt Aitken 195c5d758b Update the path for the subcription portal (#1885) 2025-04-04 17:15:07 +01:00
Matt Aitken 36159be544 Warm start UI (#1882)
* Better trace icon

* Better Waitpoint token icon

* Fix for bad jsx

* Warm and cold start icons

* Tooltips now use a <Portal> so they appear on top

* Warm start components

* Added warm start markers to the Run page and inspector

* Fix for getting the correct value from the metadata

* Better trace icon, with fallback to a passed in one

* Removed unused isWarmStart function
2025-04-04 14:20:28 +01:00
nicktrn 7644d92efa Fix restored runner ID and various small enhancements (#1880)
* make worker nodetype configurable

* make max dequeue count configurable from supervisor

* deterministic runner ids

* update runner id on restore

* disable resource monitor

* explicit run controller type imports

* enhance debug logs
2025-04-04 12:35:23 +01:00
Eric Allam e6efb3c61b realtime: allow realtime streams to have it's own redis env vars (#1876) 2025-04-04 12:30:29 +01:00
Eric Allam 7b9ab1781a re2: add ability to selectively disable release concurrency queue consumers via env var (#1881)
* run engine: add ability to selectively disable release concurrency queue consumers via env var

Also added some additional logging

* actually pass the disableConsumers option through to the release concurrency system

* Fixed CLI e2e failing tests
2025-04-04 12:28:39 +01:00
Eric Allam 25c1abac8e v4: eagerly fork child processes before warm start initiates (#1879)
* v4: eagerly fork child process before warm start

Also fixes an issue where the attempt span events weren't coming through in the partial spans

* Making flushing a safer operation

* Add changeset

* Improve attempt metrics when in a warm start, and pass the STYLE_VARIANT for a warm or cold start on attempt spans
2025-04-04 11:07:36 +01:00
Matt Aitken 5bdbf47fb3 Run page improvements (#1874)
* When selecting spans, don’t add to the browser history

* Only non-debug events can extend the total duration of the run timeline
2025-04-02 19:51:09 +01:00
Eric Allam cb6d697e6d chore: improve blank state copy (#1872) 2025-04-02 17:44:18 +01:00
nicktrn dcacc3a9e2 Fixes for v4 waits and restores (#1868)
* remove dummy metrics from heartbeat

* fix heartbeat timeouts..

* reset clock when resolving waitpoints

* optionally set metadata url when scheduling run

* don't use global vitest as it's outdated

* don't run test files in parallel

* handle special graceful shutdown code

* exit codes for success and failure can now be set and overridden

* ensure immediate cleanup in tests

* update lockfile after conflict
2025-04-02 15:27:01 +01:00
Eric Allam 3e341701ae v4 engine version selection logic updated (#1866) 2025-04-02 14:04:01 +01:00
Eric Allam 51bb3ee464 v4 ai improvements (#1863)
* AI SDK sql generator and approval example WIP

* Create some nice spans and extract the tools out so we can get the type of the streams

* Enable AI SDK telemetry

* Adding ai.tool and removing toolTask, 3rd party telemetry spans now wil create partials, better ai SDK telemetry icons

* Created a separate d3-chat example, split out from the openai-agents example

* Fixed the tool options being passed to metadata in `ai.tool`

* Add a link to the run

* Slightly improved design

* Add a crawler task using crawl4ai

* Use a tool to get the userId

* Couple of tweaks

* Adding markdown rendering to assistant messages and added an e2b based chart rendering task

* Backup to anthropic

* Add changeset
2025-04-02 12:00:02 +01:00
Eric Allam e539c7c941 v4: allow disabling of the run engine worker via the RUN_ENGINE_WORKER_ENABLED env var (#1865) 2025-04-02 11:39:55 +01:00
Matt Aitken 7ce8bd1acd Queues page: fix for upgrade state, and upgrade docs link (#1862) 2025-04-01 19:20:23 +01:00
James Ritchie 49c067d755 Fixes the avatar squishing if the project name is too long (#1861) 2025-04-01 17:39:10 +01:00
James Ritchie 551b7947f2 New waitpoint token docs page + updates the URL and description for the waitpoint blank state in the app (#1860)
* Delete old wait-for pages & create new wait-for-token page

* Link to new wait-for-token page and update description

* Adds an upgrade to v4 coming soon docs page

* Removes unncessary property from Note component
2025-04-01 17:15:59 +01:00
James Ritchie a5b9151e3a Adds a "dev connected" banner to the top of the Run and Runs list pages (#1855)
* Adds DevPresenceBanner to the Run page

* Adds DevPresenceBanner to the Run list page

* Makes the DevConnection component reusable and moves components to the DevPresence.tsx file

* SideMenu: only show dev presence when project engine === V2

* WIP on disconnected banner on v3 and v4

* v3 dev connection working. Made it slightly red

* Only show the disconnected banner on v3 if the run is not completed

---------

Co-authored-by: Matt Aitken <matt@mattaitken.com>
2025-04-01 14:34:01 +01:00
James Ritchie 81941c8910 Fixes a docs link and tweaks some wording for the triggering task blank state (#1856) 2025-04-01 11:33:07 +01:00
nicktrn bed834b589 Misc v4 checkpoint fixes (#1859)
* logs for optional services

* print env vars on startup in debug mode

* routes need to explicitly ask to keep connection alive

* log indicators for now

* make workload api listen host configurable

* expose supervisor metrics and make more configurable

* configurable pull secrets, no defaults

* remove restore route

* run controller to handle queued executing

* fix v3 deploys in v4 project

* update admin worker route

* only start pod cleaner et al in k8s mode

* set new worker group as default if none yet

* make image ref optional

* checkpoint image ref is optional for output as well

* export feature flag const

* one last image ref type fix

* make runner intervals configurable

* ability to set arbitrary env vars on new runners

* set default runtime back to node 21

* move all runner env vars to the same section
2025-04-01 10:08:15 +01:00
Eric Allam 9daa3977fa fix: safe upserting of task queues during worker indexing (#1858) 2025-03-31 16:28:08 +01:00
James Ritchie 4d5080d097 Fixes missing truncation on the project name text (#1857) 2025-03-31 14:49:37 +01:00
nicktrn 394f1decd3 Misc v4 fixes and improved logging (#1831)
* logs for optional services

* print env vars on startup in debug mode

* routes need to explicitly ask to keep connection alive

* log indicators for now

* make workload api listen host configurable

* expose supervisor metrics and make more configurable

* configurable pull secrets, no defaults

* remove restore route

* run controller to handle queued executing

* fix v3 deploys in v4 project

* update admin worker route

* only start pod cleaner et al in k8s mode

* set new worker group as default if none yet
2025-03-31 14:06:17 +01:00
Eric Allam c963dcda57 fix: use default retry settings when no catchError handlers are defined (#1852) 2025-03-31 13:57:02 +01:00
Eric Allam 0e77747d45 fix: runs.retrieve now uses the output stored on the TaskRun table instead of deprecated attempts table (#1853)
* fix: runs.retrieve now uses the output stored on the TaskRun table instead of deprecated attempts table

* Fixed typecheck error
2025-03-31 13:54:46 +01:00
Eric Allam 8977d2546c fix: custom queues defined on tasks on v3 now get correctly created (#1846) 2025-03-28 17:56:26 +00:00
Eric Allam af45e8397a fix: ensure task list and test action use correct dev env (#1845)
* Ensure the task list uses the dev environment of the logged in user

* Ensure the test task action uses the correct dev environment

* Fixed issue with missing case statement scope, wrapping in block
2025-03-28 17:55:05 +00:00
James Ritchie b29f3ffb2d Updates the onboarding copy now we have multiple example projects (#1842) 2025-03-28 15:56:26 +00:00
James Ritchie f3efdad797 "Dev connecting" icon update + text color fix for the purple login buttons (#1835)
* Updates the large login buttons to the new purple style

* Updates the dev checking connection icon to include 3 dots on the screen

* Fixes svg property names
2025-03-28 15:56:00 +00:00
James Ritchie 182f5bc706 Fixes the disabled state for the Switch component (#1841) 2025-03-28 15:55:03 +00:00
James Ritchie 94c574bfef Fixes svg icon property for React (#1837) 2025-03-28 15:54:39 +00:00
Matt Aitken 5640467ce5 Task queues weren’t setting concurrencyLimit or releaseConcurrencyOnWaitpoint (#1844) 2025-03-28 15:53:56 +00:00
James Ritchie 73b41c28bf Support solo buttons in table row (#1840)
* Removed old condition that handled old v2 tables

* Adds support for a single button to be shown on hover in a table cell

* Makes the “hiddenButtons” support a single button instead of adding a new property

* Format the docs link for Schedules like the other for consistency
2025-03-28 15:53:15 +00:00
Eric Allam f4ef008e2c feat: Python otel support, enriching spans, OpenAI Agents SDK example (#1839)
* D3 demo WIP

* Agent working and completing token successfully

* Sending OpenAI Agent SDK spans through to the platform now works

* A couple of perf tweaks for enriching events
2025-03-28 13:01:59 +00:00
James Ritchie 49c43a128b Adds a python logo icon (#1834) 2025-03-28 11:04:16 +00:00
Matt Aitken 174484fb32 Added default time period to Runs, Waitpoint and Batch list pages. Improved dev presence (#1832)
* New time period filter (permanently displayed)

* Batches, and fix for blank state

* Waitpoint token filtering

* Tags query: remove things we’re not using

* Run tag and waitpoint tags use startsWith for faster search

* Fix for run page on wrong env. Added schedule last triggered column

* Removed the Redis pubsub, just use the presence key

* Improve the dev presence responsiveness

* The CLI presence connection recovers when the webapp is restarted

* Dev schedules are now working for v4

* Refactored to make the dev presence stuff

* Got rid of stupid extra /dev, added a connecting state with icon

* Remove unused Redis client
2025-03-28 09:32:42 +00:00
James Ritchie ff6359caf2 Buttons: Updates the primary button style to fit the UI style better (#1828)
* Move to indigo primary button to better fit in with the app UI style

* Updates Pause queue button to secondary

* Updates some blank state CTA buttons to primary
2025-03-27 13:25:34 +00:00
Eric Allam 0e82698548 v4: Waitpoint PAT completion, fix dequeuing without retry config, fixed metadata (#1826)
* WIP

* Completing waitpoints using public access tokens

* fixed example

* adding agent-loops reference, adding useWaitToken react hook, adding a way to throw an error to output from the run, adding an icon option to logger.trace

* Fixed metadata system

* Check if the waitpoint is COMPLETED already in the complete endpoint and return true
2025-03-27 10:04:12 +00:00
James Ritchie be02439d1d Icons: updates and new icons for the side menu, run page and general improvements to the way they work (#1825)
* New format for the functions instead of “onWait() task”

* Make the icon sizes match the hero icons

* Updating icons for the new task hooks

* Adds new icons to the RunIcon.tsx component

* Improves the size of the menu icons

* Fixes the missaligned environment dropdown and dev connected button

* Changes the button colours so they are all unique

* More icon updates

* Adds new variables for main page icons

* Improves icons sizes

* Using new color variables in the side menu and blank states

* Adds preview environment color variable

* Align the env icon in the menu

* Updates the env icons to be the correct size and padding to match hero icons

* Label uses new colors, removes unused cases and new env icons

* Lower case env label

* Removes unused code

* Use full env title

* Organize imports

* More variants of the runs icon to work at smaller sizes

* small padding adjustment

* reformat init function span

* Fix last init formatting

* New colour for settings

* Organize switch statement for icons nicely

* Use new color variable

* Update icon svgs

* Renamed lifecycle hook icon to Function icon

* Renamed function icon file name
2025-03-26 19:12:12 +00:00
Matt Aitken 00586ffaaf Waitpoint tokens page, wait.listTokens() and wait.retrieveToken() (#1824)
* Added waitpoints/tokens to the sidebar

* Added indexes to the Waitpoint time for filtering

* Begun work on `WaitpointTokenListPresenter`, the pag is a copy of the Queues page for now

* MVP of waitpoint token page

* Added status

* Expiry of timeout/ttl

* Improvements to the waitpoint table

* Improved columns and icon

* Changes from the RunTag copy on hover branch

* Fix for nested button error

* Added waitpoint tags to the DB/table

* Applied Eric’s task run tag fix (it’s live on prod in the legacy run engine branch)

* Added tags to waitpoints

* Removed todos that have been done

* Added token support for releaseConcurrency. Also added a ton of JSDocs

* Added releaseConcurrency to the API token endpoint…

* WIP on waitpoint page filters

* Fix for tags filtering

* Waitpoint filters working

* Fix for badly named function

* WaitpointPresenter used from SpanPresenter

* Waitpoint detail panel WIP

* Fix for server client hydration issue with CodeBlock

* Selected waitpoint panel

* Added a blank state

* Added waitpoint docs link

* Fix for animated number going past the target

* Fix for the queue list pagination and upgrade status

* Engine version error for waitpoint token list

* RunTag component doesn’t get squished and hover behaviour is nicer

* Associating runs with waitpoints

* Added triggered icon

* Link directly to the waitpoint

* Fix for TS error on waitpoint retrieve

* Added CopyableText component, used for waitpoint id in the table

* Removed the confetti 🎊

* Deleted some old images

* Moved some schemas/types to core. Use `id` instead of `friendlyId`

* Added wait.listTokens() function. Made some changes to the types to make it nicer

* WIP wait.retrieveToken()

* wait.retrieveToken working

* Added data to retrieve token

* Separate ApiWaitpointPresenter completely

* Added completed time to the waitpoint detail panel

* Fix for the Avatar component having SSR issues. Specify the size in rems and removed the useLayoutEffect

* Fix for applied idempotency key filter dropdown showing the id field

* Use parentheses to make sure the token list query respects idempotency key correctly

* Use the proper logger, and have a decent message with info to track the bug down

* Pass the org title into the Avatar

* Better error when failing to creating a manual waitpoint after X attempts
2025-03-26 18:05:26 +00:00
Eric Allam d6fd7f559e Run engine: Using root queue timestamp to prioritize completing runs (#1818)
* Improve the new run engine Trigger/Batch trigger service class names

* centralize queue timestamp logic in EnqueueSystem, adding queueTimestamp support and propagation

* Fixing the create tags in the new run engine trigger task service
2025-03-26 10:25:49 +00:00
nicktrn 4fe1d49f94 Add v4 pod lifecycle handlers (#1819)
* patch k8s client to allow field selector on informers

* add pod cleaner and tests

* add failed pod handler and tests

* remove supervisor test script for now
2025-03-26 10:22:24 +00:00