* Remove the old log truncation
* Added TaskRUn logsDeletedAt column
* Accurate timestamps for the run inspector
* EnsureProperty type when you want to make a single property not nullable
* No logs and upgrade messages working
* Button can be autofocused
* Replay dialog code editor is autofocused
* Fix for wrapping of span duration
* v3: cancel subtasks when parent task runs are cancelled
* v3: recover from server rate limiting errors in a more reliable way
- Changing from sliding window to token bucket in the API rate limiter, to help smooth out traffic
- Adding spans to the API Client core & SDK functions
- Added waiting spans when retrying in the API Client
- Retrying in the API Client now respects the x-ratelimit-reset
- Retrying ApiError’s in tasks now respects the x-ratelimit-reset
- Added AbortTaskRunError that when thrown will stop retries
- Added idempotency keys SDK functions and automatically injecting the run ID when inside a task
- Added the ability to configure ApiRequestOptions (retries only for now) globally and on specific calls
- Implement the maxAttempts TaskRunOption (it wasn’t doing anything before)
* Adding some docs about the request options
* Fix type error
* Remove context propagation through graphile jobs
* Remove logger
* only select a subset of task run columns
* limit columns selected in batchTrigger as well
* added idempotency doc
* allow scoped idempotency keys, and fixed an issue with the unique index on BatchTaskRun and TaskRun
* Removed old cancel task run children code
* WIP
* Handle tasks that have failed but are being auto yielded
* Limit trace view to 25k event records, add a download run logs button
Also added two new indexes to TaskEvent:
```
/// Used on eventRepository.getTraceSummary()
@@index([traceId, startTime])
// Used for getting all logs for a run
@@index([runId])
```
* perf improvements on eventRepository.getSpan()
* v2: Add a 5 minute timeout for run execution requests in dev
* v3: Include presigned urls for downloading large payloads and outputs when using runs.retrieve
* v3: better handle large task payloads and outputs
* Change to 512KB
* v2: paginate trigger schedules endpoint
* v3: add 3MB limit on batch and single payloads
* Update task payload and output limits
* Switch to read replica: getEvent API endpoint
* Switch to read replica: v2 run list presenter
* Switch to read replica: Job presenter
* Switch to read replica: Job list presenter
* Switch to read replica: billing client
* Switch to read replica: OrgUsagePresenter
* Switch to read replica: OrgBillingPlanPresenter
* Switch to read replica: ScheduleListPresenter
* Switch to read replica: EventRepository taskEvent.findMany
* WIP worker TaskRunAttempt creation
* Handling failing task runs that cannot create an attempt for whatever reason
* Move the visibility queue stuff into a graphile job
* Fixed task runs with unsanitized queue names
* “Borrow” the code from alerts PR to get self hosted deployments working
* Add an admin API endpoint to get info about the shared marqs queue
* Allow admins to view any project metrics
* start adding lazy attempts to prod
* lazy attempt creation for prod workers
* resurrect prod stack traces
* add exception event to failed run spans
* simplify dependency resumes
* fix typecheck
* fix merge
* fresh process for all attempts
* always try sigterm first
* stop heartbeat timeout on non-inplace replace message
* add missing ack on checkpoint creation service failure
* bypass dequeue for retries with running worker
* respect retry delays
* crash runs with invalid run status for execution
* remove debug logs
* fix nack message
* fix version locking
* fresh attempt processes in dev and prod
* improve handling of ipc timeouts
* consider checkpoint failures on cancellation
* add basic chaos monkey to checkpointer
* changeset
* control forced checkpoint simulation via env var
* fix merge
* kill old attempt processes before checkpointing
* detailed perf logging for checkpointing
* add coordinator otlp endpoint example
* improve prod run cancellation
* rename supports lazy attempts migration
* fix graceful exit
* fix retry mechanics
* clear paused state before retry
* remove checkpoint image after push
* crash worker on unrecoverable errors
* refactor unrecoverable error emit
* switch to do hosted busybox image
* increase wait for duration ipc timeout
* add changeset for misc fixes
* fix merge
* fix retry delay span runId
* fix dev retries
* improve prod worker logging
* log checkpoint sizes
* add lazy attempts catalog entries
* Fixed merge issue: use zodFetch, not wrapZodFetch
* Revert "Fixed merge issue: use zodFetch, not wrapZodFetch"
This reverts commit d137e4e1fe.
* importEnvVars uses wrapZodFetch now
* add backwards compat for retries without checkpoints
* handle more cases of unrecoverable runs
* don't kill the child process if it shouldn't be killed
---------
Co-authored-by: nicktrn <55853254+nicktrn@users.noreply.github.com>
Co-authored-by: Matt Aitken <matt@mattaitken.com>
* Added traceId index to TaskEvent
* Get less data and get the trace from the run
* Use the read replica in the EventRepository
* Use the regular client to get the run because the redirect from test can be very fast
* Fixes various issues with triggerAndWait and batchTriggerAndWait
When using idempotency keys, triggerAndWait and batchTriggerAndWait will still work even if the existing runs have already been completed (or even partially completed, in the case of batchTriggerAndWait)
- TaskRunExecutionResult.id is now the run friendlyId, not the attempt friendlyId
- A single TaskRun can now have many batchItems, in the case of batchTriggerAndWait while using idempotency keys
- A run’s idempotencyKey is now added to the ctx as well as the TaskEvent and displayed in the span view
- When resolving batchTriggerAndWait, the runtimes no longer reject promises, leading to an error in the parent task
* Remove the default queue concurrency limit as we now have env and org concurrency limits
* Use the run friendlyId in the completion result id
* Added some error logging
* Database schema and migrations for schedules
* Added schedules to the side menu
* The pagination can optionally hide the page numbers for a compactive mode
* Filters for the schedule page
* Added triggerSource (“STANDARD”, “SCHEDULED”) to BackgroundWorkerTask
* Added the ability to disabled a LinkButton
* Started work on the schedule page
* Environment buttons
* The new schedule form styles
* Added a cxouple of extra fields
* Allow a checkbox to have a rich label
* Added cronstrue package to the webapp
* WIP creating tasks using the form
* Improved the form styling
* Creating schedules is working in the UI
* Minor improvements
* Basic schedule table is displaying
* Creating tasks with triggerSource = scheduled. Refactored how task metadata is stored and accessed to be cleaner
* Resource route
* Fixes in the form
* Added a gap between the environments
* WIP on OpenAI generating CRON expressions
* AI generated CRON expressions is working
* Fix for the CRON field being uneditable after an AI generation
* Improvements
* Table padding
* useThrottle now behaves correctly
* Added filtering to the schedules list
* Improved the layout and fixed CRON search
* Fixed pagination for the schedules list. Just use a regular Prisma query
* Page size of 20
* Added links to the schedule rows
* Get rid of the Last run column for now
* Implement triggered scheduled tasks
Also implemented superjson payloads and dev runtime environment “presence” with RuntimeEnvironmentSession
* Move CronPattern and CreateSchedule into a common client-accessible file
* Latest UI changes
* Fix for creating a task schedule with a blank dedup key
* Refactor the human to cron stuff into a separate file and use json_object OpenAI response format
* Fix for trying to use a hook on the server-side…
* A couple of fixes to the new schedule form
* WIP on viewing a scheduled run
* Make the filters all optional
* Use the RunListPresenter from the schedule presenter
* Display a table of runs… the wrong runs but still
* Runs from the schedule
* Deleting schedules from the UI
* Tidied imports and fixed name of options object
* Disabling a schedule
* Editing schedules
* Tidied imports
* Added icons to the task list, needs some design love
* Added a tooltip for CRON pattersn
* Show the last run in the schedules table
* Some tweaks
* Added a placeholder to the CRON AI field
* Improved the trigger source icon
* Dim out disabled schedules
* Scheduled tasks have the correct icon in a run
* Added the task source icon to the test task list
* Added the date field component to storybook
* Style improvements to the date field
* Implement Task Schedule API
* added a medium sized variant to the date field
* Fixed replay run for superjson payload types
Also now linking from the replaying run to the original run (using span links)
Also added a project metrics prometheus endpoint to detect the state of the queues
* WIP on allowing different forms for testing
* If you pass a string to prettyPrintPacket which is json or superjson, safeParse it first
* Test page, deal json and superjson. Started splitting UI for schedules
* WIP on schedule form
* Removed the Label from the DateField
* WIP on test schedule form
* Fix for the runs page showing the wrong message when there are no runs from time filtering
* Removed labels from the DateField
* Fixes for the form
* Test runs are working for schedules
* Fix for nextScheduledTimestamps in triggerScheduledTask
* Implement idempotency key support and fix issue with cancelled runs causing concurrency usage
* Added API documentation for the schedule API
* Remove log
* Removed console log from runs page
* Transform the recent runs test data on the server
* Fix for hydration mismatch
* Current date as the default for the test form
* Recent payloads working
* Delete schedule modal
* Deal with empty strings from the form
* Set the initial value for the scheduled test form
* Add option to print console logs in the dev CLI locally (issue #1014)
* Export queue from the SDK
* Fix for schedules list when you have no schedule tasks
* Blank states improved
* Make task schedules more generic, to support additional schedule generators in the future
* Removed log from maqrs
* Removed “v3/schedules” export from the SDK
---------
Co-authored-by: Eric Allam <eallam@icloud.com>
Co-authored-by: James Ritchie <james@jamesritchie.co.uk>
* v3: Dynamically import superjson and fix some bundling issues
* Added changeset
* Better error handling in the registry proxy and catch uncaught exceptions and unhandled promise rejections instead of crashing the server
* Await the prettyPrintPackage
* superjson output support
* Better support for superjson in the task events, limiting output attributes
* Offload large outputs to object store (r2)
* Finishing up the offloading of large outputs/payloads to an object store
* Store startTime in nanoseconds since epoch, ensure logs using nanosecond precision for their timestamp (so they are not out of order)
* export task events immediately in dev
* The vertical lines in the run tree view are now brighter
* Blank state when you click a task with no runs
* Canceling from the runs table
* Fix for styles coming through incorrectly to properties
* Fix for the OTEL output column having a $output prefix if it’s an object
* Tasks get the SHOW_ACTIONS property
* Override properties with the passed in ones in eventRepository
* Link from deployments to environment variables page, no need to link to docs
* “Show parent items” keeps the selected item selected
* LinkButton can accept onMouseDown, onMouseEnter and onMouseLeave
* “Show parent items” keeps any items focused and has a keyboard shortcut
* Action bar with focus button
* Canceling is working
* Nicer layout and button loading and disabled state
* US english spelling “Cancelling” -> “Canceling”
* Initial work creating the Redis publishing and subscribing
* Live view is working
* Show the live reloading status on the page
* Fixed some style issues on the run page with James
* Fix for invalid JSX attributes in the ExitIcon svg
* Some sexy shit
* The end line is now correctly coloured
* millisecondsToNanoseconds exported from core/v3 and imported correctly
* Fix for e2e tests, we changed the h1 to an h2
* Tidied imports
* Removed unused hook
* Show runs as cancelled when killing the dev CLI
* Fixed error when waiting for a task
* Deprecated workers will no longer be able to run unlocked task runs
* Fixed selecting the latest background worker version in test list
* Don’t propogate trace context when triggering an async task
Also add batch id to the context and the task events
* Support for batch triggers
* Various trace fixes and improvements
* Moved trace into the logger, fixed span cardinality issues with waits and triggers
* Add support for attaching a debugger in dev with —debugger flag
* WIP on saving the resized state in a cookie
* Progress on remembering the panel width
* The resizable panel now remembers even when closed
* Improvements to the span panel
* Hide integrations in the v3 side menu
* Added “UTC” to the times
* WIP changing how styles work
* Reference unflattenAttributes from the package
* Unflatten now correctly unflattens arrays
* Unflatten typecheck fix
* Improvements to the spans and UI for them
* Added the “events” to a span, including special styling for errors
* Unflatten doesn’t work on an array of Attributes
* Added ProjectVersion to projects, defaults to V2
* Moved v3Enabled to the existing feature flags
* Update to using the v3Enabled feature flag
* Allow people to choose v3 when creating a new project
* Added version to the ProjectPresenter
* v2 project and v3 project redirecting
* .env.example for the v3-catalog
* Added sdkVersion and cliVersion columns to the BackgroundWorker table
* Added additional classes for environments
* First draft of the Tasks table
* Link to the task page
* V3 side menu project items
* Moved the ListPagination component into the components folder
* Moved the TaskListPresenter to a v3 folder
* Bare bones task page with tabs
* Task page, we’re going to delete this though
* Bare bones runs table working
* Rejigged the columns
* The run table status
* Environment and status filtering working
* Added time filters
* Cursor and direction is working… I think it’s tricky to know for sure
* Added TaskRun numbers
* Selecting a task now links to the runs page with filters turned on
* Added support for the enqueued status
* Reworked the task table
* Link to the task and environment runs
* The run page is rendering a tree of the events
* Change the task table column to “Created at"
* WIP on making the run tree view look good
* Improvements to the run tree
* Removed the janky scroll bar flash
* Change the title of the task logs to just the task ID
* Live timer when a span is running
* Added “Show parent items” link
* Fixed the bug jumping to parent items
* Style improvements to the run
* Added the resizable handle to the run page
* Moved formatDuration to core/v3
* Wait for now has nice log messages
* The run detail panel is working with just a title for now
* Navigating to a span and persisting between page reloads is working
* The run detail timeline
* Wrap log dates in a paragraph
* Latest span detail view
* Scrolling in the right-hand panel
* Tweak default resizable layout for the run page
* Attempted improvements to stop a recursive issue on TreeView
* TreeView useReducer WIP
* Changed the change callback
* Add changes to the state
* Use the new onSelectedIdChanged in the app
* Don’t update the state from the outside
* Filtering fix, although navigation doesn’t work nicely
* Filtering working when changing the content and clearing it
* Don’t allow the same selectedId callback to be called twice…
* useDebounce hook
* Switch from using defer to just typedjson for now
* Latest attempt at span navigation
* Removed log
* Turn off filtering for now, it’s causing the Links to break somehow
* Fix for page height issue
* Added (Developer Preview) to the v3 project select box
* Revert "Filtering working when changing the content and clearing it"
This reverts commit 20d9fbe36ded619e9d3eb0b23d4ed0568a88a615.
# Conflicts:
# apps/webapp/app/components/primitives/TreeView/TreeView.tsx
* Switch to using the old filtering
* Added useThrottle hook
* Instantly close the panel when deselecting a node, use debounce when navigating to a span
* Better spacing in the right hand panel
* Comment out the v3 side menu pages that don’t exist yet
* Fix for typecheck fail
* Clear the statuses too
* Introducing Modular Asynchronous Reliable Queueing System (MarQS). Works in dev
* Convert MarQS to using lua and dealing with concurrency
* Simplified the timeout queue and current concurrency is now a set instead of a flat value (to support idempotency)
* Implement task heartbeating and reconnect the background workers CLI when the websocket connection reconnects
* Start adding internal telemetry support for the server
* Get env vars to work in dev and implement prisma tracing in webapp
* Cleanup telemetry and implement it in the consumer
* Implement dequeuing a message from a parent shared queue
* Implement a custom logger exporter instead of using console log exporter
* Use node instead of shell for generating protocol buffer code
* Propogate trace context into debug logs, and allow turning off logger exporter through env vars
* Switch to using baselime for internal otel data
* Make orgMember optional to fix type issues
* Provide the CLI dev env vars through the CLI, don’t build dotenv into facade
* Removed the logger import
* Address Matt’s comments
* Addressing more of Matt’s comments
* Handle sending an execution after a websocket connection closes
* Remove auth from the env attributes to prevent obfuscation
Matt Aitken