* fix spacing for delete hints
* don't try to resume deps on wait resume
* sending duration wait resumes is not an error anymore
* set correct status with new wait resume flow
* cancel checkpoint schema v2
* don't mix messages and schemas
* prevent unintended case fallthrough in tree view
* completely switch to platform-led duration wait resumes
* prevent infinite restores
* some entries for the catalog
* add pg to additional packages
* add checkpoint safe timeout
* prevent duplicate spans after restore
* wait for post start
* add sdk version to deploy tab
* fail on impossible checkpoint scenarios
* remove debug logs
* lotsOfLogs task now outputs much larger logs
* Removed tree view collapse/expand animation
* JSDocs for useDebounce
* WIP moving filtering into the state
* Reworked the reducer to do the filtering
* Adds support for `emitDecoratorMetadata: true` and `experimentalDecorators: true` in your tsconfig
* Implement task.onSuccess/onFailure and config.onSuccess/onFailure
* Added onStart and more docs for lifecycle functions
* Use onStart instead of init for TypeORM
* Fixes an issue that was treating v2 trigger directories as v3
* Make msw a normal dependency (for now) to fix Module Not Found error in Next.js.
* Extract out all the zod* stuff from core so the SDK does not import it
* Add a changeset
* Fixing typecheck errors in the webapp
* Export the Task and TaskOptions types
* Extract additional exports from core/v3 that aren’t used in the SDK
* Move to our global system from AsyncLocalStorage for the current task context storage
* Update the esbuild core bundling plugin for the new core v3 exports
* Fix v3 CLI telemetry
* Add support for tasks located in subdirectories inside trigger dirs
* Remove the env var check during deploy (too many false negatives)
* Fixes various issues with triggerAndWait and batchTriggerAndWait
When using idempotency keys, triggerAndWait and batchTriggerAndWait will still work even if the existing runs have already been completed (or even partially completed, in the case of batchTriggerAndWait)
- TaskRunExecutionResult.id is now the run friendlyId, not the attempt friendlyId
- A single TaskRun can now have many batchItems, in the case of batchTriggerAndWait while using idempotency keys
- A run’s idempotencyKey is now added to the ctx as well as the TaskEvent and displayed in the span view
- When resolving batchTriggerAndWait, the runtimes no longer reject promises, leading to an error in the parent task
* Remove the default queue concurrency limit as we now have env and org concurrency limits
* Use the run friendlyId in the completion result id
* Added some error logging
* graceful exit with timeout
* handle and display graceful timeout errors
* fix for very long waits
* changeset
* increase termination grace period to an hour
* fix dev dependecy resumes
* add catalog entry with multiple trigger waits
* update docker provider lifecycle command
* switch to x for clearing run filters
* Revert "fix dev dependecy resumes"
This reverts commit b4061f2ae71f21889adcf061928e2e77c436f0ed.
* fix dependency resumes, properly this time
* add catalog entry for dependency waits in loops
* advice in docs re parallel dependency waits
* fix link from v3 to v2 docs
* move lifecycle command logging to debug only
* Removed batchOptions from the trigger options and the docs
---------
Co-authored-by: Matt Aitken <matt@mattaitken.com>
* Improve the SDK function types and expose a new APIError instead of the APIResult type
* Skip triggering scheduled tasks if the task isn’t found in the current deployment
Also fixes an issue when editing the environments of a schedule
* Database schema and migrations for schedules
* Added schedules to the side menu
* The pagination can optionally hide the page numbers for a compactive mode
* Filters for the schedule page
* Added triggerSource (“STANDARD”, “SCHEDULED”) to BackgroundWorkerTask
* Added the ability to disabled a LinkButton
* Started work on the schedule page
* Environment buttons
* The new schedule form styles
* Added a cxouple of extra fields
* Allow a checkbox to have a rich label
* Added cronstrue package to the webapp
* WIP creating tasks using the form
* Improved the form styling
* Creating schedules is working in the UI
* Minor improvements
* Basic schedule table is displaying
* Creating tasks with triggerSource = scheduled. Refactored how task metadata is stored and accessed to be cleaner
* Resource route
* Fixes in the form
* Added a gap between the environments
* WIP on OpenAI generating CRON expressions
* AI generated CRON expressions is working
* Fix for the CRON field being uneditable after an AI generation
* Improvements
* Table padding
* useThrottle now behaves correctly
* Added filtering to the schedules list
* Improved the layout and fixed CRON search
* Fixed pagination for the schedules list. Just use a regular Prisma query
* Page size of 20
* Added links to the schedule rows
* Get rid of the Last run column for now
* Implement triggered scheduled tasks
Also implemented superjson payloads and dev runtime environment “presence” with RuntimeEnvironmentSession
* Move CronPattern and CreateSchedule into a common client-accessible file
* Latest UI changes
* Fix for creating a task schedule with a blank dedup key
* Refactor the human to cron stuff into a separate file and use json_object OpenAI response format
* Fix for trying to use a hook on the server-side…
* A couple of fixes to the new schedule form
* WIP on viewing a scheduled run
* Make the filters all optional
* Use the RunListPresenter from the schedule presenter
* Display a table of runs… the wrong runs but still
* Runs from the schedule
* Deleting schedules from the UI
* Tidied imports and fixed name of options object
* Disabling a schedule
* Editing schedules
* Tidied imports
* Added icons to the task list, needs some design love
* Added a tooltip for CRON pattersn
* Show the last run in the schedules table
* Some tweaks
* Added a placeholder to the CRON AI field
* Improved the trigger source icon
* Dim out disabled schedules
* Scheduled tasks have the correct icon in a run
* Added the task source icon to the test task list
* Added the date field component to storybook
* Style improvements to the date field
* Implement Task Schedule API
* added a medium sized variant to the date field
* Fixed replay run for superjson payload types
Also now linking from the replaying run to the original run (using span links)
Also added a project metrics prometheus endpoint to detect the state of the queues
* WIP on allowing different forms for testing
* If you pass a string to prettyPrintPacket which is json or superjson, safeParse it first
* Test page, deal json and superjson. Started splitting UI for schedules
* WIP on schedule form
* Removed the Label from the DateField
* WIP on test schedule form
* Fix for the runs page showing the wrong message when there are no runs from time filtering
* Removed labels from the DateField
* Fixes for the form
* Test runs are working for schedules
* Fix for nextScheduledTimestamps in triggerScheduledTask
* Implement idempotency key support and fix issue with cancelled runs causing concurrency usage
* Added API documentation for the schedule API
* Remove log
* Removed console log from runs page
* Transform the recent runs test data on the server
* Fix for hydration mismatch
* Current date as the default for the test form
* Recent payloads working
* Delete schedule modal
* Deal with empty strings from the form
* Set the initial value for the scheduled test form
* Add option to print console logs in the dev CLI locally (issue #1014)
* Export queue from the SDK
* Fix for schedules list when you have no schedule tasks
* Blank states improved
* Make task schedules more generic, to support additional schedule generators in the future
* Removed log from maqrs
* Removed “v3/schedules” export from the SDK
---------
Co-authored-by: Eric Allam <eallam@icloud.com>
Co-authored-by: James Ritchie <james@jamesritchie.co.uk>
* WIP on replaying a task from the run page
* Don’t pass the existing runs idempotency key, it will cause the replay to always return the original run
* Replay from the run list
* Don’t use fetchers in the replay/cancel dialogs
* API endpoint for replaying a run
* REST API docs (mostly coming soon) but added replay run
* replayRun function added to the SDK
* Cancel run added to the SDK
* v3-catalog file to test canceling and replaying
* Changed the SDK to be runs.replay and runs.cancel
* Removed comment
* Latest lockfile
* WIP fixing pnpm nested install strat
* Add changeset
* Pass (and log) cwd during background worker initialization
* Log out the env
* manually add the .pnpm/node_modules to NODE_PATH if in a pnpm repo
* Remove supports color dep
* Upgrades to MarQS
- It’s “queue choosing” algorithm is now MUCH better (pretty much didn’t work before) and should be performant even when we have a bunch of prod queues
- There are now concurrency limits at the environment and organization level, as well as the task/queue level. So if any of them are at capacity the message won’t be dequeued. This means we can have an org wide concurrency limit, as well as different limits for dev/prod/staging
- I’ve added an admin API that can be used to update the org/env concurrency limits
* More MarQS upgrades
- Extract the queue priortity choosing strategy into an interface
- Implement a much better weighted average strategy
- “Slide the window” of queue candidates if the parent queue sends all at-capacity queues (so we won’t get stuck attempting to choose the same 12 full queues)
- Added some unit tests for the priority stuff
- Added some ideas for expanding the priority choosing strategy with more dynamic features
* Fixes an issue with the shared queue consumer relying on the queue name to get the env id
* v3: implement configurable log levels via config file and TRIGGER_LOG_LEVEL
Also, test runs automatically set the TRIGGER_LOG_LEVEL to debug
* Fix type error and changeset
* Added a Node.js runtime check for the CLI dev command
* v3: Dynamically import superjson and fix some bundling issues
* Added changeset
* Better error handling in the registry proxy and catch uncaught exceptions and unhandled promise rejections instead of crashing the server
* Await the prettyPrintPackage
* superjson output support
* Better support for superjson in the task events, limiting output attributes
* Offload large outputs to object store (r2)
* Finishing up the offloading of large outputs/payloads to an object store
* Adding a changeset for v3
* Add a version field to @trigger.dev/core-apps package.json
* Build trigger.dev when doing a prerelease
* bundle @trigger.dev/core-apps with trigger.dev cli
* Fix the init command config template
* Don’t use * for the @trigger.dev/core dep version specifier
* strip workspace: from the package version before installing it
* Added dependenciesToBundle config option to bundle ESM only packages
* Added logging around resolving dependency paths
* Try again
* Resolve dependencies based on the project dir first
* flip the bundled default
* Adding some logs around dev task completion notifications
* Adding some additional logs
* Add more logs
* Write out the log using process.stdout
* Store pending completion notifications and resume them when awaited (fixes race condition)
* Cleanup some of the logs
* Copy over the postinstall step from the projects package.json
* Add support for including additional files when deploying (e.g. prisma schema)
* Don’t run scripts when resolving deps
* copy all the files just in case anything is needed in postinstall
* Remove duplicate option
* Use the tag when outputting the dev command
* Remove the postinstall script
* add the trigger dir to the config if the default is not chosen
* Remove the “hud” display in the dev command
* trigger file names with dashes now work
* Better file watching in dev
* Much better duplicate ID experience now
* Much better “Project not found” error
* Export the handleError function types from sdk
* Add support for configuring instrumentation
* Upgrade and unify @opentelemetry/* packages (and remove storybook from the webapp)
* Fix typescript error in react package
* Upgrade react types in webapp
* Allow span icons to be determined based on the span name (e.g. prisma:)
* Ignore built-in env vars when checking for env vars, and allow continuing the deployment even if missing env vars were detected
* Improve the retry.fetch default behavior and option structure
* Fixed typescript errors with packages/email react types
* Update the retry.fetch docs
* add checkpoint restore events
* fix retries.enabledInDev
* fix tsconfig paths
* fix provider build and dev
* update kubernetes provider and fix builds again
* upgrade prod base to node 20
* update infra publish workflow
* rethrow k8s errors after logging
* shorten index container names
* fix error type assertion
* rename type assertion
* remove resource limits for now
* add missing run id on create
* add push to deploy command for self-hosting
* checkpointing fixes
* update coordinator image
* ensure valid registry login
* delete checkpoint archive after successful push
* log options on error
* structured logs for socket connections
* fix structured log merge
* exit process after checkpointing
* update restore pull secret name
* append shortcode to restore names
* log handler payload
* disable post start lifecycle hook
* pass in coordinator host via volume
* replace dapi with taskinfo
* add missing restore label
* don't restart restored containers
* remove init container from create
* atomic post-completion checkpoints
* switch to run id for container names
* improve wait accuracy
* measure basic checkpoint perf
* always log disconnect reason
* use system clock to end wait spans
* checkpoint readiness and cancel signals
* restore from checkpoint events and fix statuses
* remove attempt id env var
* restore dependencies from events
* reconnect wip
* lifecycle hooks are back
* fix hooks and improve reconnect
* make docker send postStart hook
* only checkpoint for retry if large delay
* fix a few more resume issues
* lifecycle hook fixes
* skip connection handler when waiting for post start hook
---------
Co-authored-by: Eric Allam <eric@trigger.dev>
* A bunch of changes to the config file:
- No more default export, the config is exported via the `config` named export
- `handleError` is moved to a separate named export
- Added the ability to include additional packages (e.g. wrangler) that aren’t included by default (think CLI usage)
- More reliably install packages in dev
- The config is now built first instead of directly imported, meaning we can now support typescript config files (`trigger.config.ts`)
* Update to trigger.config.ts in `trigger.dev init`
* Store startTime in nanoseconds since epoch, ensure logs using nanosecond precision for their timestamp (so they are not out of order)
* export task events immediately in dev
* fix basic resume mechanics
* fix dependency resume
* fix deploy page sort
* remove task list console log
* checkpoint restore between attempts
* add checkpoint requirement checks to docker provider
* switch to raw execa for long statements
* catch and report pre-connection index errors
* graceful worker shutdown after completion
* fix final completion
* use same task process for each attempt
* update retry catalog entry
* add immediate return task
---------
Co-authored-by: Eric Allam <eric@trigger.dev>
* Adding telemetry to the deploy command
* Add ability to opt-out of telemetry in the CLI
* Remove log
* Unifying some telemetry stuff and adding it to the login command
* init command
* Remove extra trigger dirs
* Couple tweaks to the init CLI
* Don’t use npm init -y to create the package.json file
* Add the ability to handle errors at the task and project level
* Cancel in progress runs when disconnecting the dev CLI (in between attempts)
* unify the task executor across dev & prod
* WIP proxy/deploy
* WIP, registry proxy in express working
* A couple of notes, preparing for indexing
* Move the changes to prod-worker into the new file
* Deploy command working with indexing and runs (docker provider only for now)
* Removed ts-expect-error directive
* Fixed build command
* add zod socket
* start using zod socket
* use zod socket for shared queue
* use shared provider shell
* update k8s provider task ops
* set otlp endpoint
* update docker actions and fix builds
* remove unused types
* update injected secret key env var name
* complete socket.io types to schemas migration
* update env example to new v3 key var
* fix coordinator build
* set task run label
* ensure attempt id is always set
* increase prod worker timeout
* set otlp endpoint for on runs
* pass otlp env var to prod worker
* auto-remove completed index jobs
* Added the new colours to the Tailwind config
* Fixed Tsailwind compile errors
* WIP Geist working in the app
* WIP Storybook
* Storybook is working again… ouch
* Use Geist mono for CodeMirror
* Some find/replace class changes
* Resizable colours improved
* WIP on primary button
* Minor style improvements
* Improve the side menu header styling
* Improved v2 breadcrumbs
* Account side menu tweaks
* The page now works with the NavBar and UpgradePrompt
* Removed unused import
* Refactored the buttons to be dry-er
* Removed the breadcrumbs
* WIP on nav bar
* WIP updating the button styles
* Fix for the old test page
* Fix for type errors
* Removed console.log
* Colour change in the resizable handle
* New loading bar divider
* Removed the design plugin
* Creating our own storybook pages
* Added some “storybook” pages
* Added DetailCell storybook
* WIP new animation for the loading page
* Added the new loading divider
* Fix for the time frame filter having a green border
* Style improvements to the new test page
* Small style improvements
* Improvements to the list pagination components
* Nav bar back button improved
* Improvements to the run page
* Lots of v2 style improvements
* Help looks better
* Improvements to the regenerate API key modal
* More general improvements
* Added section titles to our storybook
* Added form stories
* Fix for missing key prop
* Loading bar is working
* Fixed button submitting state
* Fix for the test button icon
* Code bg improved
* Improved oauth buttons
* Integration page header fix
* Various improvements, mainly changing button variants
* Segmented control style updates
* Added an outline of 3px for the segment control
* Fixes for the billing pages
* More pricing improvements
* Added dialog to storybook
* Typography now in storybook
* Free usage bar added to storybook
* Added inline clode to storybook
* Separated menu types into a sub section in storybook
* NamedIcon now in storybook
* Added page header to storybook
* Added pricing callout to storybook
* Added radio groups to storybook
* Added segmented controllers to storybook
* Added shortcuts to storybook
* Added switches to storybook
* Added toast to storybook
* Added tooltips to storybook
* Added tree view to storybook
* Added usage charts to storybook
* Tarted up the environment variables page
* Fix for the FreePlanUsage panel bg colours
* Added new badge variants
* Added the <TaskPath /> to the tasks table
* Span header improvements
* Started work on the timeline
* Started work on the timeline
* Fix for the run page header
* Timeline tick marks are working
* Timeline working
* Got the durations showing on hover
* Button improvements
* Experimenting with how the duration should be shown
* Lots of refinements to the timeline
* Much nicer API for the Timeline
* The switch is now purple to match the design, with inverted thumb when on
* Removed old stuff
* Export the props
* Added the span with duration back in
* Simplified more and fixed the span durations
* Put in some decent dummy data
* Rendering defaults
* Wip with the mouse cursor
* Improvements to the story
* Added Slider primitive
* Integrated the icons into the slider
* Lots of layout improvements
* Cleaned up imports
* “Root” indicator
* Animated background on partials
* Pull the styles through to the spans and log dots
* Fixed some bugs in the TreeView when nodes get created or destroyed
* Fix for incorrect parentTask reference
* Added run streaming endpoint for live refreshing
* Added the tabs to Storybook
* Fix for status font size mismatch
* Fix for the misaligned run filter status
* Set a minimum duration on the timeline
* The TreeView now re-calculates if the number of nodes has changed
* Partials now render to the end
* Improved the durations gutter
* The duration is calculated using the current time if there are partial events
* Added the environment label to storybook
* Improved the environment label style
* Removed bottom margin from badge
* Button style improvements
* Callouts style fits the new design better
* Got the scrollbars to appear on the tree/timeline
* Added scroll support
* Synced scrolling working
* Close button and exit icon for the detail panel
* secure field shows start of env
* Clipboard improvements
* Latest lockfile
* Fix for the durations wrapping
---------
Co-authored-by: James Ritchie <james@jamesritchie.co.uk>
* WIP no bundling
* Convert dev CLI to use an unbundled build process to support otel instrumentation
* Fix pnpm lock file
* A couple of fixes to get typechecking to work
* Show runs as cancelled when killing the dev CLI
* Fixed error when waiting for a task
* Deprecated workers will no longer be able to run unlocked task runs
* Fixed selecting the latest background worker version in test list
* Don’t propogate trace context when triggering an async task
Also add batch id to the context and the task events
* WIP
* retry.fetch w/intercepting to test
* Fixed the incorrect renaming of limit to maxAttempts in v2
* Fixed source map support in dev
* Cleanup background worker files once they close
Eric Allam