Commit Graph

4101 Commits

Author SHA1 Message Date
Eric Allam 6a379e4e91 Fix 3rd party otel propagation from breaking our Task Events data from being properly correlated to the correct trace 2024-05-16 12:07:21 +01:00
Eric Allam c0b815c363 Check for an existing restore event before trying to restore a checkpoint 2024-05-16 09:27:44 +01:00
Eric Allam b77ece1567 Log when disabling an endpoint 2024-05-15 21:51:08 +01:00
Eric Allam a73dc80177 Failing jobs shouldn’t log as an error 2024-05-15 21:49:52 +01:00
Eric Allam 2496917a7a v2: Graphile auto-cleanup and auto-endpoint disabling (#1103)
* Auto-cleanup failed graphile jobs instead of keeping them around

* Disable endpoint after a period of sequential indexing failures

* Remove log
2024-05-15 21:15:06 +01:00
Eric Allam 6ede68648a Remove alerts side menu (again) 2024-05-15 17:25:41 +01:00
Eric Allam f24b5b770d Protect against doWork getting called mulitple times per consumer 2024-05-15 17:01:03 +01:00
Matt Aitken b39f79f0ab Use an env var for the Graphile cleanup ttl, set the default to 3 days (was 7) 2024-05-14 19:07:50 +01:00
Matt Aitken 5fdb43ecd1 Bring alerts side menu item back 2024-05-12 18:05:01 +01:00
Eric Allam 2e02743170 Resume rate limiting run executions, but reschedule throttled jobs 10 seconds into the future instead of their original run_at, which will help with database load 2024-05-10 15:44:20 -04:00
nicktrn 1642fd7baf perf: migrate to graphile worker v0.16.6 (#1097)
* migrate to graphile worker v0.16.6

* remove stale docs link

* fix jobs cleanup query
2024-05-10 18:07:16 +01:00
Matt Aitken c6449126b1 Don’t start runs if they have runs disabled 2024-05-10 16:11:06 +01:00
Matt Aitken 1ccf38e440 Return a 404 so the webhook provider doesn’t retry 2024-05-10 16:00:42 +01:00
Matt Aitken 29b773cd36 Prevent ingesting events if the org has runs disabled 2024-05-10 15:59:04 +01:00
Matt Aitken 73fb7cfeb3 Prevent HTTP source requests if the org has runs disabled 2024-05-10 15:58:51 +01:00
nicktrn 5431638926 Revert "Revert "Revert "Revert "disable advisory locks""""
This reverts commit 933a14b44c.
2024-05-10 15:09:35 +01:00
Eric Allam 933a14b44c Revert "Revert "Revert "disable advisory locks"""
This reverts commit 4ff07f8a94.
2024-05-10 09:35:53 -04:00
Eric Allam 4ff07f8a94 Revert "Revert "disable advisory locks""
This reverts commit af9957c085.
2024-05-10 09:30:32 -04:00
Eric Allam 0afdb4cc7c Help out the DB 2024-05-10 09:15:20 -04:00
Eric Allam af9957c085 Revert "disable advisory locks"
This reverts commit 892fd9f212.
2024-05-10 09:12:33 -04:00
nicktrn 892fd9f212 disable advisory locks 2024-05-10 10:21:48 +01:00
James Ritchie 6baf9e5294 Dismiss the dev user tooltip in the task page when your mouse leaves the target 2024-05-09 17:05:17 +01:00
James Ritchie a823421324 Small style improvements to the emails 2024-05-09 16:49:45 +01:00
Eric Allam 0e919f56f2 Alerts v1 (#1065)
* Alerts v1

* Encrypt alert webhook secrets and allow them to be generated by the server

* Alert v1 UI

* Remove unnecessary emails

* Move to using `@react-email/components`

* WIP slack alerts

* More slack alerts WIP

* Update pnpm lock after rebase

* Finish implementing Slack alerts

* Use a more error like emoji

* New secondary variant for the segmented control

* Added a simple checkbox style variant to storybook

* Style tweak to the segmented control

* UI improvements to the alert modal

* Use searchable Select for alerts. Changed default variant for SegmentedControl

* Secondary button now using secondary colour

* segmented control style tweak

* Improved the channel column in the alerts table

* Updated logo-mono.png

* Updated email styles

* Email templates updated to new styles

* Don’t log the decrypted secret

* await enqueing the deployment alert when an index fails

* await enqueing the timeout alert

---------

Co-authored-by: James Ritchie <james@jamesritchie.co.uk>
Co-authored-by: Matt Aitken <matt@mattaitken.com>
2024-05-09 12:12:00 +01:00
Matt Aitken f90960fc13 Better display when there are lots of dev environment (#1093)
* Created an EnvironmentLabels component that puts extra dev environments behind a +

* Added new environment labels components to schedules pages
2024-05-09 10:32:38 +01:00
nicktrn 2306217697 v3: automatic pod cleanup (#1092)
* add automatic pod cleaner

* fix task monitor namespace override
2024-05-09 10:16:00 +01:00
Eric Allam 8d34c63c2d Fix issue when concurrency limit is set to 0 on a queue 2024-05-08 10:19:06 +01:00
Matt Aitken 7d6430dc46 New filtered dropdown and run filters (#1089)
* Extract out useFilterTasks

* WIP on Combobox

* Revert "WIP on Combobox"

This reverts commit f0a9b2e57c67f24d5e0876630e17a54252ceb86b.

* Added ariakit

* Started adding a Listbox component

* Listbox WIP

* More WIP on the list box

* More work on the Listbox

* WIP on filterable Listbox using headless… but it’s going to be a nightmare to make it good. Wrong approach

* WIP using Ariakit

* Multiple selection working

* WIP in reworking it so search is built in and it’s easier to use

* Nicer API for listbox

* Allow sections to be rendered with filtering

* More progress on the styling of the menu

* The empty state

* Removed focus outline

* Allow filtering based on the section title

* Render the shortcut correctly

* Allow passing static children to Select and render a separator if a section has no title

* Static example and added subtle focus

* Allow static data with no items array passed in

* Allow shortcuts on select items

* Only how heading if true

* Tooltip for keyboard shortcut

* useShortcutKeys fixes

- Can be passed an undefined definition (so we render the same number of hooks always).
- Fix for enabledOnInputElements not being used…

* Shortcut keys working with lists

* Don’t render shortcut numbers higher than 0

* More improvements

* WIP on a Link row

* Nicer API for the children function

* Much nicer shortcut key support

* Separated SelectTrigger out

* Heading separated out

* More styled elements created

* Removed unused element

* Reordered statuses

* Added search context

* Fixes for default values

* Removed tab elements

* Combo box

* Simplified it more

* Started work on Filter menu

* Experiments with filter menu

* Filter menu proof of concept working

* Nice hook for using a search param

* The escape key goes back to the previous menu

* Added filter page to storybook

* Added other variants

* Renamed Select file to OldSelect, implemented Feedback form

* Added placeholder support

* Added an optional dropdown icon (defaults to a chevron down)

* Feedback new Select

* Feedback just use static items

* New project page

* Renamed storybook pages

* Lots of progress on the new run filters

* Added new tasks filters

* Show the applied filters

* Added period filtering

* Fix for escape resetting the period value to the previous value when escape is pressed

* Show the Clear button sooner

* Removed unused code

* Reworked useSearchParams so multiple values can be set at once…

* Renamed file: OldSelect -> SImpleSelect

* Fix for new Select not working in forms because props weren’t being passed through

* New Select on the schedules page

* Better layout when there are lots of run filters

* Many style improvements to run filters

* Status filter allows you to select as well

* Rolled out the menus for the other files

* Minor improvements to schedule page and page padding

* Latest lockfile
2024-05-07 22:55:31 +01:00
Eric Allam 6d9dfbc75d Add configure function to be able to configure the SDK manually
And move the ApiClient configuration away from AsyncLocalStorage and use our globals system instead
2024-05-06 12:12:30 +01:00
Eric Allam 203e002087 Add runs.retrieve management API method to get info about a run by run ID 2024-05-04 22:24:42 +01:00
Eric Allam 9d6c424236 Add deploy admin debug tooltips 2024-05-03 11:22:51 +01:00
nicktrn bce398b892 increase indexing specs 2024-05-03 09:36:16 +01:00
Eric Allam ae99807047 Allow admin tooltips while impersonating 2024-05-02 16:39:01 +01:00
Eric Allam ff6e2461d5 Add admin tooltips to make support/debugging easier for admin users 2024-05-02 14:12:16 +01:00
Eric Allam 3ed016bc2b Fixed SharedQueueConsumer internal otel logging 2024-05-02 12:31:55 +01:00
Eric Allam 68dd3804a7 Improve the fair queue selection strategy by paginating through all the queues in the parent queue 2024-05-02 11:56:35 +01:00
Eric Allam 95f0c7b780 increase queue selection count to 36 2024-05-01 22:41:50 +01:00
Eric Allam 339c39d739 Fix type issues and unit tests 2024-05-01 22:23:38 +01:00
Eric Allam bc0ca01319 Fix fair queue priority strategy off-by one error 2024-05-01 22:13:52 +01:00
Eric Allam 9e5382951b Improve the display of non-object return types in the run trace viewer 2024-05-01 13:52:05 +01:00
nicktrn 62700245a3 v3: fix consecutive waits (#1073)
* fix spacing for delete hints

* don't try to resume deps on wait resume

* sending duration wait resumes is not an error anymore

* set correct status with new wait resume flow

* cancel checkpoint schema v2

* don't mix messages and schemas

* prevent unintended case fallthrough in tree view

* completely switch to platform-led duration wait resumes

* prevent infinite restores

* some entries for the catalog

* add pg to additional packages

* add checkpoint safe timeout

* prevent duplicate spans after restore

* wait for post start

* add sdk version to deploy tab

* fail on impossible checkpoint scenarios

* remove debug logs
2024-05-01 10:33:30 +01:00
Matt Aitken 0f0a6884e8 Environment variables pasting uses dotenv (#1075)
* Add dotenv package to the webapp frontend, required some polyfills

* Use dotenv to parse the pasted env vars. Make the panel wider on larger screens
2024-04-30 14:23:49 +01:00
Matt Aitken 7ff8f0ebab Task and run page improvements (#1076)
* TaskListPresenter: if there are no tasks then don’t do stats queries

* RunListPresenter, use BasePresenter and the read replica

* Added populate script

* Simplified the Runs list query, added live timer

* Added TaskRun indexes for the RunList

* Status can’t be null now we’re using the TaskRun status

* Use defer so the page loads and shows a spinner

* Improved the loading style

* Get rid of latest run info from the tasks table super slow

* Fix for the activity graph tooltip getting clipped

* Added a code comment crediting the GitHub issue with the portal fix

* Add search to the tasks list

* Padding

* Fix for the schedules columns not being UTC

* Remove unused function
2024-04-30 14:19:50 +01:00
Matt Aitken 68455c796a Fixes/run filtered keyboard nav (#1074)
* Ensure each switch condition returns a state

* Fix for scrolling when filtered

* Fix for up/down navigation when filtered
2024-04-29 18:52:46 +01:00
Eric Allam b0a2c42e0e Add some additional logging around nacking messages 2024-04-29 17:53:38 +01:00
Matt Aitken ed8d24fd3d Run page performance improvements (#1072)
* lotsOfLogs task now outputs much larger logs

* Removed tree view collapse/expand animation

* JSDocs for useDebounce

* WIP moving filtering into the state

* Reworked the reducer to do the filtering
2024-04-29 17:05:26 +01:00
nicktrn d0ef36260a stop trying to pull init image when already present 2024-04-29 15:33:46 +01:00
Matt Aitken 4fdb7f8288 Bulk insert environment variables (#1069)
* Fix for #1066. Correct environment username if dev

* WIP changing the form

* WIP on the form

* WIP on repository

* Adding environment variables en masse is working

* WIP on pasting

* useList hook with reducer

* Got bulk insert working with pasting… it was a pain

* Allow overwriting of values

* Set a max height on the new env var form
2024-04-28 18:31:31 +01:00
Matt Aitken 37b9b056c4 Upload payload packets outside of the db transaction 2024-04-28 18:18:33 +01:00
Matt Aitken 801c86bf73 Added search to the test tasks list 2024-04-28 16:10:46 +01:00