* v3: Dynamically import superjson and fix some bundling issues
* Added changeset
* Better error handling in the registry proxy and catch uncaught exceptions and unhandled promise rejections instead of crashing the server
* Await the prettyPrintPackage
* add and use machine config
* assign tasks to worker nodes only
* add secure flag to zod connection
* changeset
* add pre stop hook
* don't use secure connection by default
* pass more identifiers to provider and apply labels
* superjson output support
* Better support for superjson in the task events, limiting output attributes
* Offload large outputs to object store (r2)
* Finishing up the offloading of large outputs/payloads to an object store
* Adding a changeset for v3
* Add a version field to @trigger.dev/core-apps package.json
* Build trigger.dev when doing a prerelease
* bundle @trigger.dev/core-apps with trigger.dev cli
* Fix the init command config template
* Don’t use * for the @trigger.dev/core dep version specifier
* strip workspace: from the package version before installing it
* Added dependenciesToBundle config option to bundle ESM only packages
* Added logging around resolving dependency paths
* Try again
* Resolve dependencies based on the project dir first
* flip the bundled default
* Adding some logs around dev task completion notifications
* Adding some additional logs
* Add more logs
* Write out the log using process.stdout
* Store pending completion notifications and resume them when awaited (fixes race condition)
* Cleanup some of the logs
* Copy over the postinstall step from the projects package.json
* Add support for including additional files when deploying (e.g. prisma schema)
* Don’t run scripts when resolving deps
* copy all the files just in case anything is needed in postinstall
* Remove duplicate option
* Use the tag when outputting the dev command
* Remove the postinstall script
* add the trigger dir to the config if the default is not chosen
* Remove the “hud” display in the dev command
* trigger file names with dashes now work
* Better file watching in dev
* Much better duplicate ID experience now
* Much better “Project not found” error
* Export the handleError function types from sdk
* Add support for configuring instrumentation
* Upgrade and unify @opentelemetry/* packages (and remove storybook from the webapp)
* Fix typescript error in react package
* Upgrade react types in webapp
* Allow span icons to be determined based on the span name (e.g. prisma:)
* Ignore built-in env vars when checking for env vars, and allow continuing the deployment even if missing env vars were detected
* Improve the retry.fetch default behavior and option structure
* Fixed typescript errors with packages/email react types
* Update the retry.fetch docs
* add checkpoint restore events
* fix retries.enabledInDev
* fix tsconfig paths
* fix provider build and dev
* update kubernetes provider and fix builds again
* upgrade prod base to node 20
* update infra publish workflow
* rethrow k8s errors after logging
* shorten index container names
* fix error type assertion
* rename type assertion
* remove resource limits for now
* add missing run id on create
* add push to deploy command for self-hosting
* checkpointing fixes
* update coordinator image
* ensure valid registry login
* delete checkpoint archive after successful push
* log options on error
* structured logs for socket connections
* fix structured log merge
* exit process after checkpointing
* update restore pull secret name
* append shortcode to restore names
* log handler payload
* disable post start lifecycle hook
* pass in coordinator host via volume
* replace dapi with taskinfo
* add missing restore label
* don't restart restored containers
* remove init container from create
* atomic post-completion checkpoints
* switch to run id for container names
* improve wait accuracy
* measure basic checkpoint perf
* always log disconnect reason
* use system clock to end wait spans
* checkpoint readiness and cancel signals
* restore from checkpoint events and fix statuses
* remove attempt id env var
* restore dependencies from events
* reconnect wip
* lifecycle hooks are back
* fix hooks and improve reconnect
* make docker send postStart hook
* only checkpoint for retry if large delay
* fix a few more resume issues
* lifecycle hook fixes
* skip connection handler when waiting for post start hook
---------
Co-authored-by: Eric Allam <eric@trigger.dev>
* A bunch of changes to the config file:
- No more default export, the config is exported via the `config` named export
- `handleError` is moved to a separate named export
- Added the ability to include additional packages (e.g. wrangler) that aren’t included by default (think CLI usage)
- More reliably install packages in dev
- The config is now built first instead of directly imported, meaning we can now support typescript config files (`trigger.config.ts`)
* Update to trigger.config.ts in `trigger.dev init`
- Time out deployments to prevent never-ending building/deploying
- Use the CURRENT worker deployment when determining which version to execute
- Fix some of the deployment error handling logic
* Store startTime in nanoseconds since epoch, ensure logs using nanosecond precision for their timestamp (so they are not out of order)
* export task events immediately in dev
* fix basic resume mechanics
* fix dependency resume
* fix deploy page sort
* remove task list console log
* checkpoint restore between attempts
* add checkpoint requirement checks to docker provider
* switch to raw execa for long statements
* catch and report pre-connection index errors
* graceful worker shutdown after completion
* fix final completion
* use same task process for each attempt
* update retry catalog entry
* add immediate return task
---------
Co-authored-by: Eric Allam <eric@trigger.dev>
* Adding telemetry to the deploy command
* Add ability to opt-out of telemetry in the CLI
* Remove log
* Unifying some telemetry stuff and adding it to the login command
* init command
* Remove extra trigger dirs
* Couple tweaks to the init CLI
* Don’t use npm init -y to create the package.json file
* Add the ability to handle errors at the task and project level
* Cancel in progress runs when disconnecting the dev CLI (in between attempts)
* unify the task executor across dev & prod
* zod ipc connection with acks
* pass sender through to ipc handlers
* bits and bobs
* add host networking back in
* disable verbose logs
* restore after tasks and batches.. almost
* restore and resume all the things
* fix for systems without checkpoint support
* Get deployment indexing errors to work with the new ZodIpc stuff
---------
Co-authored-by: Eric Allam <eallam@icloud.com>
* Add support for deploying to staging and prod
* Abort deployment if environment variables are missing
* Actually do typechecking
* Remove the build command and add a —self-hosted option to the deploy command
* WIP proxy/deploy
* WIP, registry proxy in express working
* A couple of notes, preparing for indexing
* Move the changes to prod-worker into the new file
* Deploy command working with indexing and runs (docker provider only for now)
* Removed ts-expect-error directive
* Fixed build command
* add zod socket
* start using zod socket
* use zod socket for shared queue
* use shared provider shell
* update k8s provider task ops
* set otlp endpoint
* update docker actions and fix builds
* remove unused types
* update injected secret key env var name
* complete socket.io types to schemas migration
* update env example to new v3 key var
* fix coordinator build
* set task run label
* ensure attempt id is always set
* increase prod worker timeout
* set otlp endpoint for on runs
* pass otlp env var to prod worker
* auto-remove completed index jobs
* WIP no bundling
* Convert dev CLI to use an unbundled build process to support otel instrumentation
* Fix pnpm lock file
* A couple of fixes to get typechecking to work
* Show runs as cancelled when killing the dev CLI
* Fixed error when waiting for a task
* Deprecated workers will no longer be able to run unlocked task runs
* Fixed selecting the latest background worker version in test list
* Don’t propogate trace context when triggering an async task
Also add batch id to the context and the task events
* WIP
* retry.fetch w/intercepting to test
* Fixed the incorrect renaming of limit to maxAttempts in v2
* Fixed source map support in dev
* Cleanup background worker files once they close
* Support for batch triggers
* Various trace fixes and improvements
* Moved trace into the logger, fixed span cardinality issues with waits and triggers
* Add support for attaching a debugger in dev with —debugger flag
* WIP on saving the resized state in a cookie
* Progress on remembering the panel width
* The resizable panel now remembers even when closed
* Improvements to the span panel
* Hide integrations in the v3 side menu
* Added “UTC” to the times
* WIP changing how styles work
* Reference unflattenAttributes from the package
* Unflatten now correctly unflattens arrays
* Unflatten typecheck fix
* Improvements to the spans and UI for them
* Added the “events” to a span, including special styling for errors
* Unflatten doesn’t work on an array of Attributes
* Added ProjectVersion to projects, defaults to V2
* Moved v3Enabled to the existing feature flags
* Update to using the v3Enabled feature flag
* Allow people to choose v3 when creating a new project
* Added version to the ProjectPresenter
* v2 project and v3 project redirecting
* .env.example for the v3-catalog
* Added sdkVersion and cliVersion columns to the BackgroundWorker table
* Added additional classes for environments
* First draft of the Tasks table
* Link to the task page
* V3 side menu project items
* Moved the ListPagination component into the components folder
* Moved the TaskListPresenter to a v3 folder
* Bare bones task page with tabs
* Task page, we’re going to delete this though
* Bare bones runs table working
* Rejigged the columns
* The run table status
* Environment and status filtering working
* Added time filters
* Cursor and direction is working… I think it’s tricky to know for sure
* Added TaskRun numbers
* Selecting a task now links to the runs page with filters turned on
* Added support for the enqueued status
* Reworked the task table
* Link to the task and environment runs
* The run page is rendering a tree of the events
* Change the task table column to “Created at"
* WIP on making the run tree view look good
* Improvements to the run tree
* Removed the janky scroll bar flash
* Change the title of the task logs to just the task ID
* Live timer when a span is running
* Added “Show parent items” link
* Fixed the bug jumping to parent items
* Style improvements to the run
* Added the resizable handle to the run page
* Moved formatDuration to core/v3
* Wait for now has nice log messages
* The run detail panel is working with just a title for now
* Navigating to a span and persisting between page reloads is working
* The run detail timeline
* Wrap log dates in a paragraph
* Latest span detail view
* Scrolling in the right-hand panel
* Tweak default resizable layout for the run page
* Attempted improvements to stop a recursive issue on TreeView
* TreeView useReducer WIP
* Changed the change callback
* Add changes to the state
* Use the new onSelectedIdChanged in the app
* Don’t update the state from the outside
* Filtering fix, although navigation doesn’t work nicely
* Filtering working when changing the content and clearing it
* Don’t allow the same selectedId callback to be called twice…
* useDebounce hook
* Switch from using defer to just typedjson for now
* Latest attempt at span navigation
* Removed log
* Turn off filtering for now, it’s causing the Links to break somehow
* Fix for page height issue
* Added (Developer Preview) to the v3 project select box
* Revert "Filtering working when changing the content and clearing it"
This reverts commit 20d9fbe36ded619e9d3eb0b23d4ed0568a88a615.
# Conflicts:
# apps/webapp/app/components/primitives/TreeView/TreeView.tsx
* Switch to using the old filtering
* Added useThrottle hook
* Instantly close the panel when deselecting a node, use debounce when navigating to a span
* Better spacing in the right hand panel
* Comment out the v3 side menu pages that don’t exist yet
* Fix for typecheck fail
* Clear the statuses too
* Introducing Modular Asynchronous Reliable Queueing System (MarQS). Works in dev
* Convert MarQS to using lua and dealing with concurrency
* Simplified the timeout queue and current concurrency is now a set instead of a flat value (to support idempotency)
* Implement task heartbeating and reconnect the background workers CLI when the websocket connection reconnects
* Start adding internal telemetry support for the server
* Get env vars to work in dev and implement prisma tracing in webapp
* Cleanup telemetry and implement it in the consumer
* Implement dequeuing a message from a parent shared queue
* Implement a custom logger exporter instead of using console log exporter
* Use node instead of shell for generating protocol buffer code
* Propogate trace context into debug logs, and allow turning off logger exporter through env vars
* Switch to using baselime for internal otel data
* Make orgMember optional to fix type issues
* Provide the CLI dev env vars through the CLI, don’t build dotenv into facade
* Removed the logger import
* Address Matt’s comments
* Addressing more of Matt’s comments
* Handle sending an execution after a websocket connection closes
* Remove auth from the env attributes to prevent obfuscation
github-actions[bot]