Commit Graph

3127 Commits

Author SHA1 Message Date
nicktrn efd970a901 v3: correctly handle triggering tasks prior to deploy (#1019)
* add waiting for deploy status

* sort tasks filter

* marqs: optional timestamp on enqueue

* add new status to consts

* detect and execute tasks waiting for deploy

* display runs list even without any deploys

* ensure we only execute after the run creation tx has completed

* ensure correct execution order after waiting for deploy
2024-04-12 11:03:22 +01:00
Eric Allam 89a5e9f7da Move gc to it’s own admin endpoint 2024-04-11 16:24:40 +01:00
James Ritchie 404931e224 Made the message when you’re in a developer preview project punchier 2024-04-11 14:34:45 +01:00
Eric Allam 57bf98307a Run gc before taking heapsnapshot 2024-04-11 13:53:14 +01:00
Eric Allam 7551adfeb6 Add ability to disable tracing to test if it’s the cause of the memory leak 2024-04-11 13:14:02 +01:00
Eric Allam b580d53b59 Better identifiy heap snapshots 2024-04-11 11:13:36 +01:00
Eric Allam d81e21d2ec Remove admin config endpoint 2024-04-11 10:44:26 +01:00
Matt Aitken 826a64fe6f If your dev run hasn’t started, display an info panel saying you need to run the CLI 2024-04-11 10:36:44 +01:00
Eric Allam d1849b0ea9 Adding admin config endpoint (temp) 2024-04-11 10:30:45 +01:00
Eric Allam 3aa634d2b8 Add a metric for number of event repo subscribers and project subscribers 2024-04-11 10:30:37 +01:00
Matt Aitken 5b18298fef Multiple machine CLI support (#1022)
* If a cli PAT exists, associate the authorization code with it

* Improved the error messages
2024-04-10 17:59:21 +01:00
nicktrn 7a9bd18ba2 v3: stop swallowing deployment errors and display them better (#1020)
* exit worker with code 111 for handled errors

* task monitor will ignore previously handled errors

* fix logging of index errors

* deployment error component

* changeset
2024-04-10 17:58:51 +01:00
Eric Allam a5aed0d139 Fix memory leak by calling punsubscribe instead of unsubscribe
Also adding a /metrics endpoint with a few custom metrics and the built-in prom-client ones
2024-04-09 17:42:28 +01:00
Matt Aitken f854cb90eb Replay\Cancel a run (v3) (#1006)
* WIP on replaying a task from the run page

* Don’t pass the existing runs idempotency key, it will cause the replay to always return the original run

* Replay from the run list

* Don’t use fetchers in the replay/cancel dialogs

* API endpoint for replaying a run

* REST API docs (mostly coming soon) but added replay run

* replayRun function added to the SDK

* Cancel run added to the SDK

* v3-catalog file to test canceling and replaying

* Changed the SDK to be runs.replay and runs.cancel

* Removed comment

* Latest lockfile
2024-04-08 19:08:44 +01:00
nicktrn f2894c177a switch to busybox and only call post start hook for restore (#1005) 2024-04-05 15:47:58 +01:00
nicktrn 1207efbbad v3: prod image upgrade and fixes (#1003)
🚢 Publish Infra Images / build (coordinator) (push) Has been cancelled
🚢 Publish Infra Images / build (kubernetes-provider) (push) Has been cancelled
* fix shutdown after final attempt

* fix workdir permissions

* correctly handle self-hosted deploy errors

* switch to node:20-bookworm-slim

* add binaries catalog file

* changeset

* skip recommended packages

* post start hook retries

* comment and improve lifecycle retry
2024-04-05 13:09:07 +01:00
nicktrn 7ea8532cce v3: crash observability (#1002)
* indexing resource limits

* refactor and storage limits

* handle and display worker crashes

* enable prod cancellation

* fix reconnect delay

* improve crash messages

* log and display crash events

* changeset
2024-04-05 13:00:20 +01:00
Eric Allam 6642228f26 Fixing the dev & shared queue consumer telemetry so we can see what’s going on 2024-04-04 17:54:33 +01:00
James Ritchie 1239a3ceb9 V3 project labels (#998)
* Added a side menu label to indicate your project version

* Added a link to request access to v3

* Toggle password visiblility of the env var inputs
2024-04-04 09:53:59 +01:00
James Ritchie a707446989 Fixed Triggers pages not scrolling (#999) 2024-04-04 09:18:18 +01:00
James Ritchie 29b69160e1 Fix for run page not scrolling the full runs content 2024-04-02 19:02:35 +01:00
Eric Allam ae9efe3d8b Added an error property to the problem JSON response 2024-04-02 17:46:40 +01:00
Eric Allam 3feb5ffb5f whitelist more api paths in the rate limiter 2024-04-02 17:29:53 +01:00
Eric Allam 7fb482de64 Allow OPTIONS requests in the rate limiter 2024-04-02 17:08:43 +01:00
nicktrn 28837f39b3 Prevent crash on upgrade errors 2024-04-02 13:58:40 +01:00
Eric Allam eb60126284 Fixed batched otel in dev and made prod configurable to be batched as well 2024-04-02 12:58:56 +01:00
Eric Allam d876c358d2 Update the default env var concurrency limit and set them when orgs and envs are created 2024-04-02 11:55:00 +01:00
nicktrn 51315fc3c8 v3: shared queue pool (#994)
* configurable retry delay checkpoint threshold

* shared queue consumer pool

* add more attributes to marqs spans

* inject trace context into queued messages
2024-04-02 10:22:52 +01:00
Eric Allam 4047f00562 Allow each service to define their own name 2024-04-01 22:01:36 +01:00
Eric Allam c702d6a9ca v3: better task metadata errors (#991)
* v3: better handle task metadata parse errors, and display nicely formatted errors (dev, deploy, UI)

* Add changeset
2024-03-31 21:47:05 +01:00
Eric Allam a946797d95 v3: Upgrades to MarQS (#989)
* Upgrades to MarQS

- It’s “queue choosing” algorithm is now MUCH better (pretty much didn’t work before) and should be performant even when we have a bunch of prod queues
- There are now concurrency limits at the environment and organization level, as well as the task/queue level. So if any of them are at capacity the message won’t be dequeued. This means we can have an org wide concurrency limit, as well as different limits for dev/prod/staging
- I’ve added an admin API that can be used to update the org/env concurrency limits

* More MarQS upgrades

- Extract the queue priortity choosing strategy into an interface
- Implement a much better weighted average strategy
- “Slide the window” of queue candidates if the parent queue sends all at-capacity queues (so we won’t get stuck attempting to choose the same 12 full queues)
- Added some unit tests for the priority stuff
- Added some ideas for expanding the priority choosing strategy with more dynamic features

* Fixes an issue with the shared queue consumer relying on the queue name to get the env id
2024-03-31 15:58:00 +01:00
Eric Allam 11b997d2bf Make env var value input fields passwords 2024-03-29 09:05:53 +00:00
Eric Allam e3cf456c69 v3: fix string and non-standard outputs (#983)
* Fixed string outputs missing and incorrectly formatted

* Handle non-serializable outputs like a function

* Add changeset
2024-03-28 19:51:29 +00:00
Matt Aitken bf7827e7b8 Fix for the test latest runs showing other environments 2024-03-28 18:14:47 +00:00
Eric Allam bc020a3ffe Get the internal telemetry ready for test/prod 2024-03-28 18:12:01 +00:00
Eric Allam f93eae300e v3: superjson dynamic import and deploy fixes (#982)
* v3: Dynamically import superjson and fix some bundling issues

* Added changeset

* Better error handling in the registry proxy and catch uncaught exceptions and unhandled promise rejections instead of crashing the server

* Await the prettyPrintPackage
2024-03-28 16:22:00 +00:00
Matt Aitken a2365e406d Fixed v3 docs link in the sidebar 2024-03-28 14:46:47 +00:00
nicktrn b66d5525ef v3: machine config (#978)
* add and use machine config

* assign tasks to worker nodes only

* add secure flag to zod connection

* changeset

* add pre stop hook

* don't use secure connection by default

* pass more identifiers to provider and apply labels
2024-03-28 14:21:49 +00:00
Eric Allam 719c0a0b94 v3: better clock management for spans/logs in CRIU envs (#980)
* durable clock WIP

* Fixed incorrect span timings around checkpoints by implementing a precise wall clock that resets after restores

* Add changeset
2024-03-28 14:20:44 +00:00
Matt Aitken f1c768a255 Docs for API keys, fixed links to docs from app (#981)
* Add missing API keys docs and link from the web app to them. Renamed Server API keys to Secret keys

* Correct link to env var docs

* Duplicate image fixed on environment variables docs
2024-03-28 14:00:42 +00:00
Matt Aitken d39932ebf7 Changed the throttle so the function fires at the end of the period, not immediately when first called 2024-03-28 12:05:48 +00:00
Eric Allam 2dea8dec35 Implement simple 7 day log retention and blank states (#976) 2024-03-27 17:22:16 +00:00
Matt Aitken 252f5b3967 Fix for janky scrolling on the run page 2024-03-27 16:55:03 +00:00
Matt Aitken 426316aa3c There was an error about a div being in a paragraph 2024-03-27 16:23:11 +00:00
Eric Allam 74236ae7c1 Fix changeset workflow (thought I’d already fixed this?) 2024-03-27 14:47:42 +00:00
Eric Allam 055d468b38 Setup socket.io redis adapter and fix deployment builds for superjson error 2024-03-27 14:42:10 +00:00
Matt Aitken 3a35d60091 Change the npm tag from "latest" to "beta" in docs/cli/onboarding (#974)
* The v3 init command defaults to the “@v3” tag

* Change it to beta

* Updated the docs and in-app docs

* Updated GitHub action
2024-03-27 13:39:53 +00:00
Eric Allam f11c0f991b Log out the headers and url when no key is found 2024-03-27 13:30:47 +00:00
Eric Allam 1cf181fd02 Control api rate limiter logging through env vars 2024-03-27 12:42:05 +00:00
James Ritchie db611bebba Text truncation fixes (#973)
* Span titles and code blocks truncate when window is small

* Tree view toggle arrow doesn’t have red hover state if row is an error
2024-03-27 12:16:20 +00:00