* Created a v3-catalog test script for queues
* SDK: Fix for calling trigger and passing a custom queue
* Support custom queue in TriggerTaskService
* Improved the script in the catalog so it’s clearer what’s going on
* Remove the concurrencyLimit from a queue if the limit is null
* Fix for the test code… stupid
* Improved the existing runs API
* WIP next runs API
* Improve the returned ApiPromise to add ability to return response
* More WIP
* WI{
* Added offset/limit pagination stuff like the cursor one, and converted all API methods to use ApiPromise
* More run API stuff
- Adding schedule output from the retrieveRun endpoint
- Ability to filter by schedule and isTest
* Remove env from retrieve run in openAPI
* prefer duplication over merge
* WIP docs
* Use spread to DRY up some run API schemas
* Finish the overview docs
* Adding changeset
* Fixed typecheck errors
* Typo fix
* Re-export zodfetch from core so the v3 CLI can use it
* Fixed type errors
---------
Co-authored-by: Matt Aitken <matt@mattaitken.com>
* Easier to create a rate limiter, use it in the ApiRateLimiter. Upgraded the Upstash package
* Always prefix any rate limiter in Redis with “ratelimit:”
* By default log when the rate limit is hit
* Added rate limiting to IngestSendEvent
* Log out the EventRecord id
* Increase events.deliverScheduled attempts
* INGEST_EVENT_RATE_LIMIT_MAX is optional
* Removed old API rate limit code
* IngestSendEvent rate limiter is optional. Moved outside of the DB transaction
* Log a message out when the rate limiter is created
* Return undefined if the rate limit has been crossed
* Test page: don’t show deleted dev environments
* Function to filter out environments that are dev and have no orgMember or orgMemberId
* Don’t show tasks from deleted members (or the associated environment labels)
* Show “Dev: Deleted” if the user has been removed.
* Only show your environment in the env vars table
* Renamed function
* Use the displayableEnvironment function
* WIP worker TaskRunAttempt creation
* Handling failing task runs that cannot create an attempt for whatever reason
* Move the visibility queue stuff into a graphile job
* Fixed task runs with unsanitized queue names
* “Borrow” the code from alerts PR to get self hosted deployments working
* Add an admin API endpoint to get info about the shared marqs queue
* Allow admins to view any project metrics
* start adding lazy attempts to prod
* lazy attempt creation for prod workers
* resurrect prod stack traces
* add exception event to failed run spans
* simplify dependency resumes
* fix typecheck
* fix merge
* fresh process for all attempts
* always try sigterm first
* stop heartbeat timeout on non-inplace replace message
* add missing ack on checkpoint creation service failure
* bypass dequeue for retries with running worker
* respect retry delays
* crash runs with invalid run status for execution
* remove debug logs
* fix nack message
* fix version locking
* fresh attempt processes in dev and prod
* improve handling of ipc timeouts
* consider checkpoint failures on cancellation
* add basic chaos monkey to checkpointer
* changeset
* control forced checkpoint simulation via env var
* fix merge
* kill old attempt processes before checkpointing
* detailed perf logging for checkpointing
* add coordinator otlp endpoint example
* improve prod run cancellation
* rename supports lazy attempts migration
* fix graceful exit
* fix retry mechanics
* clear paused state before retry
* remove checkpoint image after push
* crash worker on unrecoverable errors
* refactor unrecoverable error emit
* switch to do hosted busybox image
* increase wait for duration ipc timeout
* add changeset for misc fixes
* fix merge
* fix retry delay span runId
* fix dev retries
* improve prod worker logging
* log checkpoint sizes
* add lazy attempts catalog entries
* Fixed merge issue: use zodFetch, not wrapZodFetch
* Revert "Fixed merge issue: use zodFetch, not wrapZodFetch"
This reverts commit d137e4e1fe.
* importEnvVars uses wrapZodFetch now
* add backwards compat for retries without checkpoints
* handle more cases of unrecoverable runs
* don't kill the child process if it shouldn't be killed
---------
Co-authored-by: nicktrn <55853254+nicktrn@users.noreply.github.com>
Co-authored-by: Matt Aitken <matt@mattaitken.com>
* Updated worker types
* Cloudflare rate limiter applied to sendEvent/sendEvents
* Latest wrangler
* Updated to the latest compatibility_date and nest the unsafe bindings properly
* Added some types from a Discord members
* Better logging and added a reset header so the SDK can use it from inside the run function
* Set staging proxy rate limit to 100/60s
* WIP env var management API
* Add import env var API endpoint
* Adding docs and support for using both API keys and PATs when interacting with the env var endpoints
* WIP envvar SDK
* Uploading env vars in a variety of formats now works
* Finish env var endpoints and add resolveEnvVars hook
* Add changeset
* Added traceId index to TaskEvent
* Get less data and get the trace from the run
* Use the read replica in the EventRepository
* Use the regular client to get the run because the redirect from test can be very fast
* Switch to new TaskRunNumberCounter that uses taskIdentifier + environmentId
* Drop old TaskRunCounter table (not used anymore)
* Revert "Drop old TaskRunCounter table (not used anymore)"
This reverts commit d7311a3d03cd8020df02105da155ceec4fa24b1b.
* Fix: API rate limit error has the correct seconds until reset
* When a v2 run hits the rate limit, reschedule using the reset timestamp
* Still throw AutoYieldRateLimitErrors
* Reschedule runs from the rate limit
* The stress test timeout should be inside the task
* If the rate limit error is thrown, don’t retry the API request
* Added v2Enabled and hasRequestedV3 columns to Organization
* Don’t create a project when you create an org
* Form for requesting v3 access
* Reworked the new project form with the different version states. Refined copy on early access
* If the project isn’t in the org then redirect to the new project page
* Better message for existing users
* Tidy imports
* If it’s not the managed cloud then allow them to create v2 projects
* WIP on multi-select
* WIP on simple checkbox
* CheckboxWIthLabel and Checkbox
* Multi-selection of runs across pages is working
* Fix for selection on seconds page
* Focus the run filter on page load
* Don’t focus the checkbox
* BulkActionBar now shows/hides and has buttons
* Some state to stop escape clearing the selection when the modals are open
* Delete unused formData util
* Improvements to the page
* Created the replay resource action. It doesn’t do anything useful yet.
* Database schema created for BulkActionGroup/BulkActionItem
* The BulkActionService is creating the right data, now we need to process it
* WIP on bulk processing
* Added failed state and made the sourceRun required
* Bulk replaying is working
* WIP on bulk action filtering
* Fixed bulk filters displaying
* Filtering by batch is working
* Some fixes for the bulk id filtering
* Style tweaks
* Load the extra info in parallel
* Bulk canceling working
* Get the most recent 20 bulk actions to display in the filter menu
* Even if the run isn’t cancelable add it to the final list
* Maximum of 250 runs can be bulk actioned
* Don’t let them select more than the maximum (250 currently)
* Separate each bulk item action into it’s own separate graphile job to increase resiliency
---------
Co-authored-by: Eric Allam <eallam@icloud.com>
* Fix issues with posting alerts to public slack channels
* Use the actual values in the new environmentTypes column to display the environment type labels in the alerts list
* Implement environment alert options
Matt Aitken