* Fix realtime safari bug because of missing ReadableStream async iterable support
Limit to only safari
* Fix missing TIMED_OUT run status
* When coercing realtime date strings, make sure they are set to UTC
* Added new side menu section and updated intro
* Updated the fal ai realtime project
* Copy update
* Added example project
* Swapped walkthrough and repo
* Restore original fal task page
* Added links back to the intro
* Updated login page logos
* Adds an isSelected state to the Table
* Toast style now matches the design
* Adds a space between the upgrade panel and the list of users
* Fix streaming splits in realtime streams v2
* Add changeset
* Skip all flaky tests 😡
* Improve the way we stream from tasks to the server
* Improve the v1 realtime streams (Redis)
* Turn on the relay realtime stream service
* Improved the relay realtime cleanup
* Fixed consuming realtime runs w/streams after the run is already finished
* Remove some logs
* Update changeset
* Fixed runStream tests
* Upgrading from Hobby to Pro displays a dialog to confirm
* Improved the logic for upgrading to pro
* Added the spinner component to storybook
* Improved the dark style spinner
* Realtime streams now powered by electric, and fix the streaming update duplicate issues by converting the electric Shape materialized view into a ReadableStream of changes
* Ensure realtime subscription stops when runs are finished, and add an onComplete handle to use realtime hooks
* Fix tests
* Mutliple streams can be now consumed simultaneously
* Update prerelease script
* Add changeset
* Make it core
* Handle API error responses when streaming
* JWT scopes for tags and batches can now access runs that have the tag or are in the batch
- useTaskTrigger can now submit options
- auto-generated batch trigger public access tokens no longer need each individual run ID scope
* Add changeset
* Added task scopes to work like tags and batches
Also removed scopes for tags when auto-generating a public access token as that could be dangerous.
* WIP adding a side help panel to the tasks page
* Optionally display the shortcut before the trailing icon in the button
* Updated the close icon
* WIP adding a new side help panel
* WIP adding content to the side help panel
* WIP new side help panel content
* Removed images as not needed any more
* Added content to the side help menu
* Help panel open/closed state stored as cookie
* Removed the icons from the docs and examples links
* Update Infisical instructions to new SDK
Current documentation is using old Infisical SDK implementation.
New SDK has a different name, and authenticates on a method, not in the class constructor.
* Fix import and client secret env
* WIP batch trigger v2
* Fix for the DateField being one month out… getUTCMonth() is zero indexed 🤦♂️
* Added a custom date range filter
* Deal with closing the custom date range
* Child runs filter
* Fix for the clear button untoggling the child runs
* WIP batchTriggerV2
* Finished removing rate limit from the webapp
* Added an index TaskRun to make useRealtimeBatch performant
* Renamed the period filter labels to be “Last X mins”
* Denormalize background worker columns into TaskRun
* Use the runTags column on TaskRun
* Add TaskRun ("projectId", "id" DESC) index
* Improved the v2 batch trigger endpoint to process items in parallel and also added a threshold, below which the processing of items is async
* Added a runId filter, and WIP for batchId filter
* WIP triggerAll
* Add new batch methods for triggering multiple different tasks in a single batch
* Disabled switch styling
* Batch filtering, force child runs to show if filtering by batch/run
* Added schedule ID filtering
* Force child runs to show when filtering by scheduleId, for consistency
* realtime: allow setting enabled: false on useApiClient
* Batches page
* Always complete batches, not only batchTriggerAndWait in deployed tasks
* Add batch.retrieve and allow filtering by batch in runs.list
* Renamed pending to “In progress”
* Tidied up the table a bit
* Deal with old batches: “Legacy batch”
* Added the Batch to the run inspector
* Fixed the migration that created the new idempotency key index on BatchTaskRun
* Fixed the name of the idempotencyKeyExpiresAt option and now default idempotency key TTL is 30 days, not 24 hours
* Timezone fix: wrong month in Usage page dropdown
* The DateField now defaults to local time, but can be overriden to use utc with an option
* Don’t allow the task icon to get squished
* BatchFilters removed unused imports
* In the batch filtering, use `id` instead of `batchId` in the URL
* BatchFilters: we don’t need a child tasks hidden input field
* Creates some common filter components/functions
* Fix for batchVersion check when filtering by batch status
* Add additional logging around telemetry and more attributes for trigger spans
* Show clear button for specific id filters
* Batch list: only allow environments that are part of this project
* Unnecessary optional chain
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
* Add JSDocs
---------
Co-authored-by: Matt Aitken <matt@mattaitken.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
* Stop using Response errors when validating API Keys, instead introduce a new "Result" type that has success and failure conditions. Adding in a way to progressively adopt because this touches everything.
* Make sure authenticateApiRequestWithFailure always returns a result
* Fixed task trigger queue update logic to only update an existing queue if the concurrency limit changes, instead of on every single trigger task call
* Fix bad update from another branch
* WIP realtime streams
* Handle realtime with large payloads or outputs #1451
* feat: optimize Redis stream handling with batching
Add STREAM_ORIGIN to environment schema. Improve performance in
RealtimeStreams by using TextDecoderStream for simpler text
decoding and implementing batching of XADD commands for Redis
streams. Limit stream size using MAXLEN option. Update
environment variable repository with new variable type. Adjust
import statements for Redis key and value types.
* 🔧 chore: add dev dependencies for bundle analysis
* add metadata tests and a few more utilties
* Add stream tests and improve streaming
* Added AI tool tasks, descriptions to tasks
* Use the config file path to determine the workingDir, then the package.json path
* Remove stream test files
* useTaskTrigger react hook that allows triggering a task from the client
* Add streaming support for the realtime react hooks
* Add ability to stream results after useTaskTrigger
* Improve the stream throttling
* Use the runId as the ID key to bust the cache after triggering
* Upgrade to to the latest electric sql client and server
* Make realtime server backwards compat with 3.1.2 release
* Pass the runId into useRealtimeRun
* Fix scopes when specifiying reading all runs
* WIP @trigger.dev/rsc package
* Various fixes and accepted recommendations by CodeRabbit
* Regenerate pnpm lock file
* A couple tweaks to rsc and give up on rendering react in tasks for now
* Add changeset
* Remove triggerRequest from the useEffect deps
* Improve realtime & frontend authentication errors
* Fixed authorization tests
* Remove unnecessary log
* Add metadata.stream limits and improve the metadata streams structure
* Streams can now have up to 2500 entries
* Various coderabbit fixes
* additional react-hooks jsdocs
* Updated styling
* Updated the main Table component styles to the new design
* Run page table layout style updates
* Updated row styling and new scrolling behaviour
* Fixed bulk action bar no appearing
* Improved table component styles
* Pagination arrows now chevrons
* Added link to Runs docs page
* Small popover tweak
* Moved the extra Test button out of Tables and into the RunsTable
* WIP refactoring the sticky table cell
* Support for the existing way we have menus in the sticky table cells
* Created a specific Docs style button
* Improved dialog trigger button styles
* Improved docs button styles
* Updated Docs buttons to the new style in the tooltips
* Adding a Test button on hover
* Removed the page padding and made table scroll
* Fixed table scrolling
* cleaned up imports
* WIP adding a test button that links to the test page
* added a docs button to the header
* Much nicer display bar for showing how many schedules you have (and it works with the updated table)
* Fixed the button types in the schedules info panel
* Fixed button icon colour
* Removed the pagination as it’s not being used
* Delete modal button position updated
* Fixed padding on recent payloads panel
* Fixed button icon colours
* Better style for active table row
* Table scrolling now behaves properly
* environment buttons now have a hover state
* Bigger Run test bar to match the Run page
* Small layout fixes to the schedule side panel
* Test page now has link to docs
* fixed text alignment with table headers
* API keys page now has full width tables
* Fix typo and modal layout
* Table uses new sticky button type
* Tidy imports
* Env var table now full width
* Form buttons have a border top
* Updated button styles
* Updated blank state message style to be consistent
* Removed unnecessary form button border
* Alerts page tables now full width
* Allowing the Detail cell to be styled
* Updated the page structure to be the same as the scheduled page – with better progress indicator
* Concurrency limits page now full width
* Fixed function name label extending too far
* Improved the button sizes on the replay modal
* Added a minimal style to the info panels
* Make sure you can always see the info and unlock staging info when the table scrolls
* Info panel style updated to match the others
* Removed the duplicate header
* Usage page tables now full width
* Personal access token page using full width tables
* Small CSS fixes
* Deployments page table now full width
* Deploy page scrolls properly and pagination stays on screen if table is scrollable
* TableCellMenu using the correct popover
* Added missing button variant
* Button group is now named so it doesn’t cause conflicts
* Added docs link to the Run page
* Typo
* Added a link to the test page if you have no runs
* Make the table sticky cell buttons more obvious
* Env var page now using the new popover menu
* Improved the menu items
* Copy tweak
* Made the icons bright on hover
* Updated the cancel run dialog
* Improved the API keys sticky menu behaviour
* Alerts table menu using the new popover
* Correct sized buttons in the Env Var page table menu
* Fixed weird Dialog text styling
* Schedules page blank state using correct docs button style
* Copy tweak
* Menu buttons styled correctly
* Added close buttons to the deploy dialogs
* Made the Dialog footer match the Form footer style
* Fixed feedback form buttons
* Fixed cancel run dialog
* Test page uses the taskIdentifier in the path
* Test task button working
* Added Spline component for 404 logo to remove spline logo
* Removed todo comment
* Added a docs button to the PAT page
---------
Co-authored-by: Matt Aitken <matt@mattaitken.com>
* feat: exit if docker buildx can't be found for self-hosted builds
* chore: tweak debug log message
* chore: improve error message based on rabbit's suggestion
* Fix parsing of project path for cli init command
* Update development instructions for cli-v3
* Create gentle-mails-sip.md
---------
Co-authored-by: Eric Allam <eallam@icloud.com>
* Fix: Typo on overview.mdx
the import helloWorldTask wasn't called correctly at an example
* Fix: Typo update on overview.mdx
Now the import is called `helloWorld` instead of `helloWorldTask` to be consistent with the filename as requested
* Added sync env vars docs example
* Added links in the config + deploy files
* Updated copy
* Added vercel docs cards and added them to all of the relevant docs pages
* Added sync env vars to the intro page
* Added automatically sync env vars section to nextjs guide
* Added ‘Manually’
* Removed code block and updated title
* updated formatting and added vercelsyncenvvars to the config file
* Added missing comment
* Updated imports to /core
* Added sync env var docs link and note
* Improved the VERCEL_ACCESS_TOKEN note
* Added link to vercel
* Updated docs and improved formatting
* Copy tweaks
* Added space
* Updated deploy docs
* magic links on span event errors
* prevent task monitor from processing errors handled elsewhere
* exclusively use internal error code enum for completion data
* add complete attempt service opts
* reattempts need to go via the queue for task controllers that may have exited
* only infer retry config if completed via crash or system failure
* enhance error before deciding if retriable
* retry on SIGTERM
* enable retry config helper for latest sdk
* don't retry heartbeat timeouts for now
* enable task monitor to update fatal errors
* add missing service
* update retry config since package version
* don't alter completion time when updating existing error
* refactor finalize run service
* refactor complete attempt service
* remove separate graceful exit handling
* refactor task status helpers
* clearly separate statuses in prisma schema
* all non-final statuses should be failable
* new import payload error code
* store default retry config if none set on task
* failed run service now respects retries
* fix merged task retry config indexing
* some errors should never be retried
* finalize run service takes care of acks now
* execution payload helper now with single object arg
* internal error code enum export
* unify failed and crashed run retries
* Prevent uncaught socket ack exceptions (#1415)
* catch all the remaining socket acks that could possibly throw
* wrap the remaining handlers in try catch
* New onboarding question (#1404)
* Updated “Twitter” to be “X (Twitter)”
* added Textarea to storybook
* Updated textarea styling to match input field
* WIP adding new text field to org creation page
* Added description to field
* Submit feedback to Plain when an org signs up
* Formatting improvement
* type improvement
* removed userId
* Moved submitting to Plain into its own file
* Change orgName with name
* use sendToPlain function for the help & feedback email form
* use name not orgName
* import cleanup
* Downgrading plan form uses sendToPlain
* Get the userId from requireUser only
* Added whitespace-pre-wrap to the message property on the run page
* use requireUserId
* Removed old Plain submit code
* Added a new Context page for the docs (#1416)
* Added a new context page with task context properties
* Removed code comments
* Added more crosslinks
* Fix updating many environment variables at once (#1413)
* Move code example to the side menu
* New docs example for creating a HN email summary
* doc: add instructions to create new reference project and run it locally (#1417)
* doc: add instructions to create new reference project and run it locally
* doc: Add instruction for running tunnel
* minor language improvement
* Fix several restore and resume bugs (#1418)
* try to correct resume messages with missing checkpoint
* prevent creating checkpoints for outdated task waits
* prevent creating checkpoints for outdated batch waits
* use heartbeats to check for and clean up any leftover containers
* lint
* improve exec logging
* improve resume attempt logs
* fix for resuming parents of canceled child runs
* separate SIGTERM from maybe OOM errors
* pretty errors can have magic dashboard links
* prevent uncancellable checkpoints
* simplify task run error code enum export
* grab the last, not the first child run
* Revert "prevent creating checkpoints for outdated batch waits"
This reverts commit f2b5c2ac42.
* Revert "grab the last, not the first child run"
This reverts commit 89ec5c8bfd.
* Revert "prevent creating checkpoints for outdated task waits"
This reverts commit 11066b4e74.
* more logs for resume message handling
* add magic error link comment
* add changeset
* chore: Update version for release (#1410)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
* Release 3.0.13
* capture ffmpeg oom errors
* respect maxAttempts=1 when failing before first attempt creation
* request worker exit on fatal errors
* fix error code merge
* add new error code to should retry
* pretty segfault errors
* pretty internal errors for attempt spans
* decrease oom false positives
* fix timeline event color for failed runs
* auto-retry packet import and export
* add sdk version check and complete event while completing attempt
* all internal errors become crashes by default
* use pretty error helpers exclusively
* error to debug log
* zodfetch fixes
* rename import payload to task input error
* fix true non-zero exit error display
* fix retry config parsing
* correctly mark crashes as crashed
* add changeset
* remove non-zero exit comment
* pretend we don't support default default retry configs yet
---------
Co-authored-by: James Ritchie <james@trigger.dev>
Co-authored-by: shubham yadav <126192924+yadavshubham01@users.noreply.github.com>
Co-authored-by: Tarun Pratap Singh <101409098+Wackyator@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
* Denormalize run tags, increase character limit to 128
* WIP realtime subscribing to runs
* extracted the stream stuff into core, made it more reusable
* WIP tags
* Remove tags for now because it’s not support in electric
* Support async iterables, readable stream, and callback style subscription styles
* Remove tags streaming endpoint
* Add realtime rate limits and scope them to the /realtime path
* WIP rate limt per org
* Introduce per org rate limits
* WIP JWT auth
* Move migrations into new internal db package
* Resolve pnpm lock file
* Authenticating to the realtime API with JWTs are working
* realtime in the client
* Created react-hooks package and starting to move stuff in there
* Improve types for hooks
* schema tasks
* Added useBatch hook
* build uploadthing/fal demo and change how run metadata is synced to the server
* tweaks
* WIL realtime concurrency tracking
* Implement test for realtime client using testcontainers
also updated electric to latest version
* Allow customizing the expiration time of the automatic JWT created after triggering a task
* Add support for subscribing to run tags
* Improve auth types and API
* finalize the realtime API
* Fixed some example stuff
* Allow up to 10 run tags
* Remove core from docker-provider tsconfig paths to prevent it from being typechecked
* do the same for the kubernetes provider
* Fixing some typecheck errors
* Fix webapp type errors
* Update @trigger.dev/platform to 1.0.13
* Fix attw error
* Remove from/to in subscribeToRuns query params
* Add tests for the rate limit middleware and add custom JWT rate limits
* turn off webapp test parallelism
* Finish renaming jwt -> publicAccessToken and automatically give the JWT read access to the tags when using trigger
* Add changeset
* Attempt to fix unit tests in CI
* Skip running the auth rate limit middleware tests for now
* Try a beefier machine
* Try and run webapp tests separately
* Setup env vars
* Make sliding window test more reliabile
* try to correct resume messages with missing checkpoint
* prevent creating checkpoints for outdated task waits
* prevent creating checkpoints for outdated batch waits
* use heartbeats to check for and clean up any leftover containers
* lint
* improve exec logging
* improve resume attempt logs
* fix for resuming parents of canceled child runs
* separate SIGTERM from maybe OOM errors
* pretty errors can have magic dashboard links
* prevent uncancellable checkpoints
* simplify task run error code enum export
* grab the last, not the first child run
* Revert "prevent creating checkpoints for outdated batch waits"
This reverts commit f2b5c2ac42.
* Revert "grab the last, not the first child run"
This reverts commit 89ec5c8bfd.
* Revert "prevent creating checkpoints for outdated task waits"
This reverts commit 11066b4e74.
* more logs for resume message handling
* add magic error link comment
* add changeset
* Updated “Twitter” to be “X (Twitter)”
* added Textarea to storybook
* Updated textarea styling to match input field
* WIP adding new text field to org creation page
* Added description to field
* Submit feedback to Plain when an org signs up
* Formatting improvement
* type improvement
* removed userId
* Moved submitting to Plain into its own file
* Change orgName with name
* use sendToPlain function for the help & feedback email form
* use name not orgName
* import cleanup
* Downgrading plan form uses sendToPlain
* Get the userId from requireUser only
* Added whitespace-pre-wrap to the message property on the run page
* use requireUserId
* Removed old Plain submit code
* Added support for the env vars `POD_EPHEMERAL_STORAGE_SIZE_LIMIT` and `POD_EPHEMERAL_STORAGE_SIZE_REQUEST`
* Reverted pnpm lockfile
* Made sure there is no diff between the main lockfile & PR's lockfile
* Not touching the storage for the index containers
* Not touching the storage for the prePull container
* Added a new dropdown help and feedback menu to the side menu
* Added a shortcut to the popover menu
* Removed dev cli connected button for now
* Contact us form uses original Feedback component to prevent broken links
* Improved the messaging when selecting different options in the email form
* buttons style tweak
* SideMenuItem supports the trailingIconClassName
* Adding a consistent focus-visible states
* Removing tooltips for now
* Squashed commit of the following:
commit 7d11123c0a
Author: Eric Goldman <eric@sequin.io>
Date: Mon Sep 30 17:54:06 2024 -0700
Add sequin guide (#1368)
Co-authored-by: James Ritchie <james@trigger.dev>
commit 8da495ac00
Author: nicktrn <55853254+nicktrn@users.noreply.github.com>
Date: Mon Sep 30 13:42:22 2024 +0100
Improve checkpoint reliability and cleanup of temp files (#1367)
* improve cleanup reliability
* improve logging
* bye-bye execa
* fix for trailing newlines
* prettier errors
* trim args and log output by default
* fix archive cleanup
* prevent potential memleak
* more cleanup debug logs
* ignore abort during cleanup
* rename checkpoint dir env var and move to helper
* add global never throw override
* add tmp cleaner
* also clean up checkpoint dir by default
* split by any whitespace, not just tabs
* only create tmp cleaner if paths to clean
commit 69ec68ee31
Author: Eric Allam <eallam@icloud.com>
Date: Sun Sep 29 19:18:39 2024 -0700
Release 3.0.9
commit a6ea8444c9
Author: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Date: Sun Sep 29 19:17:26 2024 -0700
chore: Update version for release (#1366)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
commit 4c1ee3d6ea
Author: Eric Allam <eallam@icloud.com>
Date: Sun Sep 29 19:09:38 2024 -0700
fix: run metadata not working when using npx/pnpm dlx
* More support for custom-focus
* More custom focus styles added
* Support for focus-visible style for the Segmented control
* Fixed table triple dot menu z-index issue
* Improved help menu wording
* When you submit the help form, close the modal
* focus-visible style for radio buttons
* button prop is now optional in the SideMenu component
* focus styling for a text link
* Deleted unused sequin files
* Checkbox component can have its label styles
* Improved the dialog footer
* Handle sending feedback to Slack using Plain
* WIP making the modal conditional
* Moved the Plain form action into the select plan file
* removed comment
* Show a confirmation diaglog if you’re downgrading from Pro to Hobby
* Downgrading to Hobby works
* Use redirectWithErrorMessage instead of throw error
* The cancel form now submits the data correctly
* Modals don’t trigger when you upgrade
* Copy improvements
* Added a tooltip to explain the link to the pricing page
* Squashed commit of the following:
commit 886429b0ed
Author: Matt Aitken <matt@mattaitken.com>
Date: Tue Oct 8 18:36:46 2024 +0100
Removed emails, @trigger.dev/database and @trigger.dev/otlp-importer from changesets config
commit f65157a14f
Author: Matt Aitken <matt@mattaitken.com>
Date: Tue Oct 8 18:31:27 2024 +0100
Lockfile with run-engine removed
commit 3d67bb81e9
Author: Matt Aitken <matt@mattaitken.com>
Date: Tue Oct 8 18:24:31 2024 +0100
Removed run-engine from the webapp package.json/tsconfig
commit d30e9713f4
Author: Matt Aitken <matt@mattaitken.com>
Date: Tue Oct 8 18:06:04 2024 +0100
Dockerfile fix because the database package has been moved
commit f2babbf637
Author: Matt Aitken <matt@mattaitken.com>
Date: Tue Oct 8 09:41:22 2024 -0700
Internal packages (testcontainers, redis-worker and zod-worker) (#1392)
* Some notes on the new run engine
* lockfile with setup for the run engine
* Documenting where TaskRun is currently mutated, to try figure out the shape of the new system
* Added notes about how triggering currently works
* Details about when triggering happens
* Lots of notes about waitpoints
* Started scaffolding the RunEngine
* Sketch of Prisma waitpoint schema while it’s fresh in my mind
* Got Prisma working with testcontainers
* Use beforeEach/afterEach
* Simple Prisma and Redis test
* Return Redis options instead of a client
* Simplified things
* A very simple FIFO pull-based queue to check the tests working properly
* Use vitest extend
* Separate redis, postgres and combined tests for faster testing
* Some fixes and test improvements
* Pass a logger into the queue
* A queue processor that processes items from the given queue as fast as it can
* Test for retrying an item that wasn’t processed
* First draft of waitpoints in the Prisma schema
* Remove the custom logger from the test
* Added a completedAt to Waitpoint
* Notes on the flow for an execution starting
* Added redlock, moved some files around
* Starting point for the TaskRunExecutionSnapshot table
* Added relationships to TaskRunExecutionSnapshot
* Change some tsconfig
* Moved some things around
* Added some packages
* WIP on the RunQueue
* Fix for some imports
* Key producer with some tests
* Removed the nv type from the keys… it’s not useful to do global queries
* Passing unit tests for all the public key producer functions
* Some basic tests passing for the RunQueue
* Simple enqueue test working
* Enqueue and dequeue for dev is working
* Don’t log everything during the tests
* Enqueuing/dequeuing from the shared queue is working
* Tests for getting a shared queue
* The key producer sharedQueue can now be named, to allow multiple separate queues
* The key producer uses the name of the queue as the input
* Extra info in the Prisma schema
* Dequeuing a message gets the payload and sets the task concurrency all in one Lua script
* Adding more keys so we can read the concurrency from the queue
* Setting the concurrency with dequeue and enquque is working
* Improved the tests and fixed some bugs
* Acking is resetting the concurrencies
* Check the key has been removed after acking
* Nacking is working
* Changed the package to CommonJS + Node10 so it works with Redlock
* Moved the database, otel and emails packages to be in internal-packages
* Moved some Prisma code to the database package
* Started using the RunEngine for triggering
* Progress on run engine triggering, first waitpoint code
* Create a delay waitpoint
* Moved ZodWorker to an internal package so it can be used in the run engine as well as the webapp
* Web app now uses the zod worker package
* Added parseNaturalLanguageDuration to core/apps
* internal-packages/zod-worker in the lockfile
* Pass in the master queue, remove old rebalance workers code
* Add masterQueue to TaskRun
* Fixed the tests
* Moved waitpoint code into the run engine, also the zod worker
* Completing waitpoints
* An experiment to create a new test container with environment
* More changes to triggering
* Started testing triggering
* Test for a run getting triggered and being enqueued
* Removed dequeueMessageInEnv
* Update dev queue tests to use the shared queue function
* Schema changes for TaskRunExecutionSnapshot
* First execution snapshot when the run is created. Dequeue run function added to the engine
* Separate internal package for testcontainers so they can be used elsewhere
* Remove the simple queue and testcontainers from the run-engine. They’re going to be separate
* Fix for the wrong path to the Prisma schem,a
* Added the testcontainers package to the run-engine
* redis-worker package, just a copy of the simple queue for now
* The queue now uses Lua to enqueue dequeue
* The queue now has a catalog and an invisible period after dequeuing
* Added a visibility timeout and acking, with tests
* Added more Redis connection logging, deleted todos
* Visibility timeouts are now defined on the catalog and can be overridden when enqueuing
* Dequeue multiple items at once
* Test for dequeuing multiple items
* Export some types to be used elsewhere
* Partial refactor of the processor
* First stab at a worker with concurrency and NodeWorkers
* Don’t have a default visibility timeout in the queue
* Worker setup and processing items in a simple test
* Process jobs in parallel with retrying
* Get the attempt when dequeuing
* Workers do exponential backoff
* Moved todos
* DLQ functionality
* DLQ tests
* Same cluster for all keys in the same queue
* Added DLQ tests
* Whitespace
* Redis pubsub to redrive from the worker
* Fixed database paths
* Fix for path to zod-worker
* Fixes for typecheck errors, mostly with TS versions and module resolution
* Redlock required a patch
* Moved the new DB migrations to the new database package folder
* Remove the run-engine package
* Remove the RunEngine prisma schema changes
* Delete triggerTaskV2
* Remove zodworker test script (no tests)
* Update test-containers readme
* Generate the client first
* Use a specific version of the prisma package
* Generate the prisma client before running the unit tests
commit fc609470c4
Author: Dan <8297864+D-K-P@users.noreply.github.com>
Date: Tue Oct 8 14:36:03 2024 +0100
Supabase database webhook example upgrade (#1386)
* Added overview for guides and examples section and split them all out
* New supabase guide wip
* Updated images and improved docs
* Trimmed the supabase prereqs
* Supabase guide wip
* more updates
* Replaced old database webhook guide
* Created one intro page and removed snippets
* Updated guide sidebar titles
* Code updates
* More improvements
* Updates and added images
* Compressed image
* Updated guides descriptions and edge function basic
* Removed bold
* Updated redirects
* Fixed broken links
* Updated intro
commit 07f82eae54
Author: nicktrn <55853254+nicktrn@users.noreply.github.com>
Date: Tue Oct 8 13:28:54 2024 +0100
Release 3.0.11
commit 13ebfcc5b9
Author: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Date: Tue Oct 8 13:24:38 2024 +0100
chore: Update version for release (#1381)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
commit 2a04d17e1f
Author: nicktrn <55853254+nicktrn@users.noreply.github.com>
Date: Tue Oct 8 09:24:23 2024 +0100
Simplify showLogs expression
commit 002ae4bc0d
Author: nicktrn <55853254+nicktrn@users.noreply.github.com>
Date: Tue Oct 8 09:22:29 2024 +0100
Fix dotenv overrides for dev runs (#1388)
* override dashboard dev env vars with local .env
* add changeset
* add simple task for testing env vars
commit 047cb00417
Author: nicktrn <55853254+nicktrn@users.noreply.github.com>
Date: Tue Oct 8 09:22:05 2024 +0100
Disable schedules for deleted orgs on next tick (#1383)
* disable schedules for deleted orgs
* add debug logs
commit 2c014f725a
Author: James Ritchie <james@trigger.dev>
Date: Sun Oct 6 13:02:00 2024 -0700
Override log retention (#1385)
* set full log retention as admin
* If run.logsDeletedAt is set, don’t bother getting the trace
commit a69e04f03a
Author: nicktrn <55853254+nicktrn@users.noreply.github.com>
Date: Sat Oct 5 14:18:58 2024 +0100
Include push output in logs for self-hosted deploys (#1382)
* include push output in logs
* changeset
commit c5488df8ca
Author: nicktrn <55853254+nicktrn@users.noreply.github.com>
Date: Sat Oct 5 13:12:47 2024 +0100
Fix CLI downgrade check (#1380)
* fix downgrade detection
* remove unused semver package from webapp
* add changeset
commit 1caec27f72
Author: Eric Allam <eric@trigger.dev>
Date: Fri Oct 4 15:33:35 2024 -0700
docs: Max duration (#1379)
* maxDuration docs
* Update the init command to set the maxDuration and include a commented out maxDuration in the config file
commit e14c954e12
Author: Eric Allam <eallam@icloud.com>
Date: Fri Oct 4 15:02:05 2024 -0700
Release 3.0.10
commit 8e61f5d631
Author: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Date: Fri Oct 4 14:59:07 2024 -0700
chore: Update version for release (#1378)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
commit 08db565a72
Author: Eric Allam <eallam@icloud.com>
Date: Thu Oct 3 12:38:25 2024 -0700
improve the timed out description
commit 6d0884254e
Author: Eric Allam <eric@trigger.dev>
Date: Thu Oct 3 12:43:26 2024 -0700
feat: Add maxDuration to tasks (#1377)
* WIP
* Get max duration working on deployed runs
* Actually set the timed out runs to status = TIMED_OUT
* The client status for TIMED_OUT is now MAX_DURATION_EXCEEDED
* New TimedOutIcon
* Added new timedout icon
* Add ability to opt-out of maxDuration with timeout.None
* MAX_DURATION_EXCEEDED -> TIMED_OUT
* changeset
* Improved styling for the status tooltip content
---------
Co-authored-by: James Ritchie <james@trigger.dev>
commit 665ccf88b7
Author: nicktrn <55853254+nicktrn@users.noreply.github.com>
Date: Thu Oct 3 12:33:18 2024 +0100
Update github actions and self-hosting docs
commit 1ff7b861ca
Author: Eric Allam <eric@trigger.dev>
Date: Wed Oct 2 18:26:36 2024 -0700
Add max queue depth limits (#1376)
* Add runs to an env queue, as well as the actual queue
* Add queue size limit guard on triggering tasks
commit c531a9d986
Author: Eric Allam <eric@trigger.dev>
Date: Wed Oct 2 15:30:39 2024 -0700
fix: cleanup ttl expire run graphile jobs (#1373)
* fix: remove ttl expire run graphile jobs when a run is started or completed
* Update expireEnqueuedRun.server.ts
commit 0bf500fa95
Author: Matt Aitken <matt@mattaitken.com>
Date: Wed Oct 2 15:30:16 2024 -0700
Prioritize finishing waited runs (#1375)
* If a tree node is missing, estimate the size as zero
* Task to test prioritizing finishing existing runs after triggerAndWaits
* When requeuing a run with a checkpoint, put it in the queue with the parent run time so it’s correctly prioritized
* The same change but if there’s no checkpoint
* Squashed commit of the following:
commit 886429b0ed
Author: Matt Aitken <matt@mattaitken.com>
Date: Tue Oct 8 18:36:46 2024 +0100
Removed emails, @trigger.dev/database and @trigger.dev/otlp-importer from changesets config
commit f65157a14f
Author: Matt Aitken <matt@mattaitken.com>
Date: Tue Oct 8 18:31:27 2024 +0100
Lockfile with run-engine removed
commit 3d67bb81e9
Author: Matt Aitken <matt@mattaitken.com>
Date: Tue Oct 8 18:24:31 2024 +0100
Removed run-engine from the webapp package.json/tsconfig
commit d30e9713f4
Author: Matt Aitken <matt@mattaitken.com>
Date: Tue Oct 8 18:06:04 2024 +0100
Dockerfile fix because the database package has been moved
commit f2babbf637
Author: Matt Aitken <matt@mattaitken.com>
Date: Tue Oct 8 09:41:22 2024 -0700
Internal packages (testcontainers, redis-worker and zod-worker) (#1392)
* Some notes on the new run engine
* lockfile with setup for the run engine
* Documenting where TaskRun is currently mutated, to try figure out the shape of the new system
* Added notes about how triggering currently works
* Details about when triggering happens
* Lots of notes about waitpoints
* Started scaffolding the RunEngine
* Sketch of Prisma waitpoint schema while it’s fresh in my mind
* Got Prisma working with testcontainers
* Use beforeEach/afterEach
* Simple Prisma and Redis test
* Return Redis options instead of a client
* Simplified things
* A very simple FIFO pull-based queue to check the tests working properly
* Use vitest extend
* Separate redis, postgres and combined tests for faster testing
* Some fixes and test improvements
* Pass a logger into the queue
* A queue processor that processes items from the given queue as fast as it can
* Test for retrying an item that wasn’t processed
* First draft of waitpoints in the Prisma schema
* Remove the custom logger from the test
* Added a completedAt to Waitpoint
* Notes on the flow for an execution starting
* Added redlock, moved some files around
* Starting point for the TaskRunExecutionSnapshot table
* Added relationships to TaskRunExecutionSnapshot
* Change some tsconfig
* Moved some things around
* Added some packages
* WIP on the RunQueue
* Fix for some imports
* Key producer with some tests
* Removed the nv type from the keys… it’s not useful to do global queries
* Passing unit tests for all the public key producer functions
* Some basic tests passing for the RunQueue
* Simple enqueue test working
* Enqueue and dequeue for dev is working
* Don’t log everything during the tests
* Enqueuing/dequeuing from the shared queue is working
* Tests for getting a shared queue
* The key producer sharedQueue can now be named, to allow multiple separate queues
* The key producer uses the name of the queue as the input
* Extra info in the Prisma schema
* Dequeuing a message gets the payload and sets the task concurrency all in one Lua script
* Adding more keys so we can read the concurrency from the queue
* Setting the concurrency with dequeue and enquque is working
* Improved the tests and fixed some bugs
* Acking is resetting the concurrencies
* Check the key has been removed after acking
* Nacking is working
* Changed the package to CommonJS + Node10 so it works with Redlock
* Moved the database, otel and emails packages to be in internal-packages
* Moved some Prisma code to the database package
* Started using the RunEngine for triggering
* Progress on run engine triggering, first waitpoint code
* Create a delay waitpoint
* Moved ZodWorker to an internal package so it can be used in the run engine as well as the webapp
* Web app now uses the zod worker package
* Added parseNaturalLanguageDuration to core/apps
* internal-packages/zod-worker in the lockfile
* Pass in the master queue, remove old rebalance workers code
* Add masterQueue to TaskRun
* Fixed the tests
* Moved waitpoint code into the run engine, also the zod worker
* Completing waitpoints
* An experiment to create a new test container with environment
* More changes to triggering
* Started testing triggering
* Test for a run getting triggered and being enqueued
* Removed dequeueMessageInEnv
* Update dev queue tests to use the shared queue function
* Schema changes for TaskRunExecutionSnapshot
* First execution snapshot when the run is created. Dequeue run function added to the engine
* Separate internal package for testcontainers so they can be used elsewhere
* Remove the simple queue and testcontainers from the run-engine. They’re going to be separate
* Fix for the wrong path to the Prisma schem,a
* Added the testcontainers package to the run-engine
* redis-worker package, just a copy of the simple queue for now
* The queue now uses Lua to enqueue dequeue
* The queue now has a catalog and an invisible period after dequeuing
* Added a visibility timeout and acking, with tests
* Added more Redis connection logging, deleted todos
* Visibility timeouts are now defined on the catalog and can be overridden when enqueuing
* Dequeue multiple items at once
* Test for dequeuing multiple items
* Export some types to be used elsewhere
* Partial refactor of the processor
* First stab at a worker with concurrency and NodeWorkers
* Don’t have a default visibility timeout in the queue
* Worker setup and processing items in a simple test
* Process jobs in parallel with retrying
* Get the attempt when dequeuing
* Workers do exponential backoff
* Moved todos
* DLQ functionality
* DLQ tests
* Same cluster for all keys in the same queue
* Added DLQ tests
* Whitespace
* Redis pubsub to redrive from the worker
* Fixed database paths
* Fix for path to zod-worker
* Fixes for typecheck errors, mostly with TS versions and module resolution
* Redlock required a patch
* Moved the new DB migrations to the new database package folder
* Remove the run-engine package
* Remove the RunEngine prisma schema changes
* Delete triggerTaskV2
* Remove zodworker test script (no tests)
* Update test-containers readme
* Generate the client first
* Use a specific version of the prisma package
* Generate the prisma client before running the unit tests
commit fc609470c4
Author: Dan <8297864+D-K-P@users.noreply.github.com>
Date: Tue Oct 8 14:36:03 2024 +0100
Supabase database webhook example upgrade (#1386)
* Added overview for guides and examples section and split them all out
* New supabase guide wip
* Updated images and improved docs
* Trimmed the supabase prereqs
* Supabase guide wip
* more updates
* Replaced old database webhook guide
* Created one intro page and removed snippets
* Updated guide sidebar titles
* Code updates
* More improvements
* Updates and added images
* Compressed image
* Updated guides descriptions and edge function basic
* Removed bold
* Updated redirects
* Fixed broken links
* Updated intro
commit 07f82eae54
Author: nicktrn <55853254+nicktrn@users.noreply.github.com>
Date: Tue Oct 8 13:28:54 2024 +0100
Release 3.0.11
commit 13ebfcc5b9
Author: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Date: Tue Oct 8 13:24:38 2024 +0100
chore: Update version for release (#1381)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
commit 2a04d17e1f
Author: nicktrn <55853254+nicktrn@users.noreply.github.com>
Date: Tue Oct 8 09:24:23 2024 +0100
Simplify showLogs expression
commit 002ae4bc0d
Author: nicktrn <55853254+nicktrn@users.noreply.github.com>
Date: Tue Oct 8 09:22:29 2024 +0100
Fix dotenv overrides for dev runs (#1388)
* override dashboard dev env vars with local .env
* add changeset
* add simple task for testing env vars
commit 047cb00417
Author: nicktrn <55853254+nicktrn@users.noreply.github.com>
Date: Tue Oct 8 09:22:05 2024 +0100
Disable schedules for deleted orgs on next tick (#1383)
* disable schedules for deleted orgs
* add debug logs
commit 2c014f725a
Author: James Ritchie <james@trigger.dev>
Date: Sun Oct 6 13:02:00 2024 -0700
Override log retention (#1385)
* set full log retention as admin
* If run.logsDeletedAt is set, don’t bother getting the trace
commit a69e04f03a
Author: nicktrn <55853254+nicktrn@users.noreply.github.com>
Date: Sat Oct 5 14:18:58 2024 +0100
Include push output in logs for self-hosted deploys (#1382)
* include push output in logs
* changeset
commit c5488df8ca
Author: nicktrn <55853254+nicktrn@users.noreply.github.com>
Date: Sat Oct 5 13:12:47 2024 +0100
Fix CLI downgrade check (#1380)
* fix downgrade detection
* remove unused semver package from webapp
* add changeset
commit 1caec27f72
Author: Eric Allam <eric@trigger.dev>
Date: Fri Oct 4 15:33:35 2024 -0700
docs: Max duration (#1379)
* maxDuration docs
* Update the init command to set the maxDuration and include a commented out maxDuration in the config file
commit e14c954e12
Author: Eric Allam <eallam@icloud.com>
Date: Fri Oct 4 15:02:05 2024 -0700
Release 3.0.10
commit 8e61f5d631
Author: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Date: Fri Oct 4 14:59:07 2024 -0700
chore: Update version for release (#1378)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
commit 08db565a72
Author: Eric Allam <eallam@icloud.com>
Date: Thu Oct 3 12:38:25 2024 -0700
improve the timed out description
commit 6d0884254e
Author: Eric Allam <eric@trigger.dev>
Date: Thu Oct 3 12:43:26 2024 -0700
feat: Add maxDuration to tasks (#1377)
* WIP
* Get max duration working on deployed runs
* Actually set the timed out runs to status = TIMED_OUT
* The client status for TIMED_OUT is now MAX_DURATION_EXCEEDED
* New TimedOutIcon
* Added new timedout icon
* Add ability to opt-out of maxDuration with timeout.None
* MAX_DURATION_EXCEEDED -> TIMED_OUT
* changeset
* Improved styling for the status tooltip content
---------
Co-authored-by: James Ritchie <james@trigger.dev>
commit 665ccf88b7
Author: nicktrn <55853254+nicktrn@users.noreply.github.com>
Date: Thu Oct 3 12:33:18 2024 +0100
Update github actions and self-hosting docs
commit 1ff7b861ca
Author: Eric Allam <eric@trigger.dev>
Date: Wed Oct 2 18:26:36 2024 -0700
Add max queue depth limits (#1376)
* Add runs to an env queue, as well as the actual queue
* Add queue size limit guard on triggering tasks
commit c531a9d986
Author: Eric Allam <eric@trigger.dev>
Date: Wed Oct 2 15:30:39 2024 -0700
fix: cleanup ttl expire run graphile jobs (#1373)
* fix: remove ttl expire run graphile jobs when a run is started or completed
* Update expireEnqueuedRun.server.ts
commit 0bf500fa95
Author: Matt Aitken <matt@mattaitken.com>
Date: Wed Oct 2 15:30:16 2024 -0700
Prioritize finishing waited runs (#1375)
* If a tree node is missing, estimate the size as zero
* Task to test prioritizing finishing existing runs after triggerAndWaits
* When requeuing a run with a checkpoint, put it in the queue with the parent run time so it’s correctly prioritized
* The same change but if there’s no checkpoint
* Revert "Squashed commit of the following:"
This reverts commit b837b5a558dc5187d89b09135276d16f6856f455.
* Removed console logs
* cleaned up conditionals
* Unlock free plan state
* Fixed subscribe button if you’re already github verified
* Simplified the downgrade reasons logic
* made periodEnd required
---------
Co-authored-by: nicktrn <55853254+nicktrn@users.noreply.github.com>
* Updated intro and moved stripe webhooks into the webhooks section
* Added additional resources links to nextjs and remix
* Updated intro page
* Changed formatting to improve layout
* Removed Supabase card from frameworks
* docker provider shouldn't force simulate by default
* structured logs for all things coordinator
* attributes for handler loggers
* add changeset
* fix tmp cleaner types
* remove unused simple logger import
* fields from args should override existing fields
* improve socket logs
* change log message position for easier scanning
* more logging improvements, more metadata
* Some notes on the new run engine
* lockfile with setup for the run engine
* Documenting where TaskRun is currently mutated, to try figure out the shape of the new system
* Added notes about how triggering currently works
* Details about when triggering happens
* Lots of notes about waitpoints
* Started scaffolding the RunEngine
* Sketch of Prisma waitpoint schema while it’s fresh in my mind
* Got Prisma working with testcontainers
* Use beforeEach/afterEach
* Simple Prisma and Redis test
* Return Redis options instead of a client
* Simplified things
* A very simple FIFO pull-based queue to check the tests working properly
* Use vitest extend
* Separate redis, postgres and combined tests for faster testing
* Some fixes and test improvements
* Pass a logger into the queue
* A queue processor that processes items from the given queue as fast as it can
* Test for retrying an item that wasn’t processed
* First draft of waitpoints in the Prisma schema
* Remove the custom logger from the test
* Added a completedAt to Waitpoint
* Notes on the flow for an execution starting
* Added redlock, moved some files around
* Starting point for the TaskRunExecutionSnapshot table
* Added relationships to TaskRunExecutionSnapshot
* Change some tsconfig
* Moved some things around
* Added some packages
* WIP on the RunQueue
* Fix for some imports
* Key producer with some tests
* Removed the nv type from the keys… it’s not useful to do global queries
* Passing unit tests for all the public key producer functions
* Some basic tests passing for the RunQueue
* Simple enqueue test working
* Enqueue and dequeue for dev is working
* Don’t log everything during the tests
* Enqueuing/dequeuing from the shared queue is working
* Tests for getting a shared queue
* The key producer sharedQueue can now be named, to allow multiple separate queues
* The key producer uses the name of the queue as the input
* Extra info in the Prisma schema
* Dequeuing a message gets the payload and sets the task concurrency all in one Lua script
* Adding more keys so we can read the concurrency from the queue
* Setting the concurrency with dequeue and enquque is working
* Improved the tests and fixed some bugs
* Acking is resetting the concurrencies
* Check the key has been removed after acking
* Nacking is working
* Changed the package to CommonJS + Node10 so it works with Redlock
* Moved the database, otel and emails packages to be in internal-packages
* Moved some Prisma code to the database package
* Started using the RunEngine for triggering
* Progress on run engine triggering, first waitpoint code
* Create a delay waitpoint
* Moved ZodWorker to an internal package so it can be used in the run engine as well as the webapp
* Web app now uses the zod worker package
* Added parseNaturalLanguageDuration to core/apps
* internal-packages/zod-worker in the lockfile
* Pass in the master queue, remove old rebalance workers code
* Add masterQueue to TaskRun
* Fixed the tests
* Moved waitpoint code into the run engine, also the zod worker
* Completing waitpoints
* An experiment to create a new test container with environment
* More changes to triggering
* Started testing triggering
* Test for a run getting triggered and being enqueued
* Removed dequeueMessageInEnv
* Update dev queue tests to use the shared queue function
* Schema changes for TaskRunExecutionSnapshot
* First execution snapshot when the run is created. Dequeue run function added to the engine
* Separate internal package for testcontainers so they can be used elsewhere
* Remove the simple queue and testcontainers from the run-engine. They’re going to be separate
* Fix for the wrong path to the Prisma schem,a
* Added the testcontainers package to the run-engine
* redis-worker package, just a copy of the simple queue for now
* The queue now uses Lua to enqueue dequeue
* The queue now has a catalog and an invisible period after dequeuing
* Added a visibility timeout and acking, with tests
* Added more Redis connection logging, deleted todos
* Visibility timeouts are now defined on the catalog and can be overridden when enqueuing
* Dequeue multiple items at once
* Test for dequeuing multiple items
* Export some types to be used elsewhere
* Partial refactor of the processor
* First stab at a worker with concurrency and NodeWorkers
* Don’t have a default visibility timeout in the queue
* Worker setup and processing items in a simple test
* Process jobs in parallel with retrying
* Get the attempt when dequeuing
* Workers do exponential backoff
* Moved todos
* DLQ functionality
* DLQ tests
* Same cluster for all keys in the same queue
* Added DLQ tests
* Whitespace
* Redis pubsub to redrive from the worker
* Fixed database paths
* Fix for path to zod-worker
* Fixes for typecheck errors, mostly with TS versions and module resolution
* Redlock required a patch
* Moved the new DB migrations to the new database package folder
* Remove the run-engine package
* Remove the RunEngine prisma schema changes
* Delete triggerTaskV2
* Remove zodworker test script (no tests)
* Update test-containers readme
* Generate the client first
* Use a specific version of the prisma package
* Generate the prisma client before running the unit tests
* Added overview for guides and examples section and split them all out
* New supabase guide wip
* Updated images and improved docs
* Trimmed the supabase prereqs
* Supabase guide wip
* more updates
* Replaced old database webhook guide
* Created one intro page and removed snippets
* Updated guide sidebar titles
* Code updates
* More improvements
* Updates and added images
* Compressed image
* Updated guides descriptions and edge function basic
* Removed bold
* Updated redirects
* Fixed broken links
* Updated intro
* WIP
* Get max duration working on deployed runs
* Actually set the timed out runs to status = TIMED_OUT
* The client status for TIMED_OUT is now MAX_DURATION_EXCEEDED
* New TimedOutIcon
* Added new timedout icon
* Add ability to opt-out of maxDuration with timeout.None
* MAX_DURATION_EXCEEDED -> TIMED_OUT
* changeset
* Improved styling for the status tooltip content
---------
Co-authored-by: James Ritchie <james@trigger.dev>
* If a tree node is missing, estimate the size as zero
* Task to test prioritizing finishing existing runs after triggerAndWaits
* When requeuing a run with a checkpoint, put it in the queue with the parent run time so it’s correctly prioritized
* The same change but if there’s no checkpoint
* improve cleanup reliability
* improve logging
* bye-bye execa
* fix for trailing newlines
* prettier errors
* trim args and log output by default
* fix archive cleanup
* prevent potential memleak
* more cleanup debug logs
* ignore abort during cleanup
* rename checkpoint dir env var and move to helper
* add global never throw override
* add tmp cleaner
* also clean up checkpoint dir by default
* split by any whitespace, not just tabs
* only create tmp cleaner if paths to clean
* Removed the inline-code accessory from the logs when calling trigger or batchTrigger from a run
* Removed re2 from the v3 catalog
* Added a Root badge to the runs list
* Keep the side panel open when switching tasks & remove links from detail panel
* Root and parent task styling inspector
* Hide the root badge if the task isn’t the root
* margin between the dev not running message
* improved spacing of items
* Improved Root badge style
* Show a table of triggered runs in the inspector
* Add parentSpanId index to the TaskRun table
* Fix for the run inspector now opening when linked from another run/replay
* Triggered runs table has a max height
* Added a description to the replay run modal and improved the styling slightly
* Only include a bottom border when the triggered run table is more than 4 items
* Improved the triggered runs table borders
* Improved the tables so they can have an optional sticky header
* Added table types to storybook
* Fix for hover states on different backgrounds & runs table select cell
* Added supabase database examples
* Copy updates
* More copy updates
* Added comment about generating the typescript types
* Updated intro table
* Improved title
* Added supabase-docs-cards to go at the bottom of the examples and guides
* merge main
* Link fix
* Fix more links
* Run metadata
* Remove metadata from context, move it to it’s own tab
* More run metadata stuff
- Add metadata to testing
- Make using metadata outside of runs a no-op
- Add docs
* Replaying should copy over the metadata
* transfer final attempt output to the task run
* A couple of minor fixes
* Use the new clientOrThrow() method everywhere
* Cleaned up the update metadata endpoint and added an API doc page for it
* Mirror task run attempt errors and output
* Add safe rootTaskRunId index and a README to @trigger.dev/database
* Associate child runs with the span ID of the span in the parent run that triggered the child run
* Update deprecation notice doc links
* Added supabase storage basic example and updated ffmpeg instructions
* Added stripe webhook example
* Copy tweaks
* Typos
* Updated stripe example to check for a specific event before triggering the task
* Simplified the storage example
* Added examples to intro page
* Added further instructions
* Code improvements
* Final tweaks
* Fix resolving external packages that are ESM only by falling back to mlly resolvePathSync. This will fix mupdf
* when publishing a prerelease and aborting, clear the git stage
* Add task run hierarchical relationships to the database
* Add depth and related runs to the retrieve run API response
* Remove prisma optimize
* restructure the migrations to create the index concurrently
* Delete these tsbuildinfo files
* Fix type error by adding depth to the run list presenter
* Cleanup the task hierarchy, share more code
* Remove some fields from the list run response
* never abort the same controller twice
* prevent uncaught exception when aborting pipe
* abort signal assertions and more logging
* never abort running pipe
* display --profile flag after init if used
* add containerfile debug logs
* log all indexing errors in case of connection issues
* specify dockerfile syntax version
* add network flag for self-hosting
* move all self-hosting tags to latest
* add self-hosting update section
* manual setup subsection
* registry flag implies push
* add changeset
* Added introduction page for guides
* New intro page for examples
* Fixed links
* Updated examples intro to include all of the new ones
* Improved FFmpeg example
* Improved the react pdf example
* Added Supabase overview page
* Updated card-supabase snippet
* Added sharp payload instructions
* Added vercel payload instructions
* Added dall-e payload instructions
* Added openai payload instructions
* Added resend payload instructions
* Made the prompts more consistent
* Minor tweaks and moved bun
* Updated links and added bun logo
---------
Co-authored-by: James Ritchie <james@jamesritchie.co.uk>
* Fixes for CLI update command, and make the hide the "whoami" command output when running in dev
* Fix an issue where a missing tsconfig.json file would throw an error on dev/deploy
* Don’t show latest CLI warning when using a prerelease
* Only print CLI update required message when update is embedded
* Strip out TRIGGER\_ keys when using syncEnvVars, to prevent deploy errors
* Fix for run filtering not working with some special characters (double encoded)
* Add the full dependentTaskAttempt to a ResumeBatchRunService log
* Added RESUME_AFTER_DEPENDENCY_WITH_ACK
* Set the delay to 5s
* If a checkpoint has been created, the coordinator won’t continue the run with RESUME_AFTER_DEPENDENCY_WITH_ACK
* If we’re keeping the run alive then set socket.data.requiresCheckpointResumeWithMessage to undefined
* Log out the data before and after setting socket.data.requiresCheckpointResumeWithMessage
* Attempt to fix false package mismatch warnings
* Add changeset
* Add ability to test update checks in prerelease packages
* Resolve the trigger.dev package based on the package.json dir
* Try this
* Don’t use the version module, just resolve the packageJson
* One more dirname
* Comment
* Remove the version export because we aren’t using it anymore
* Add triggerAndWait().unwrap() to more easily get at the output or throw the subtask error
* Add taskId and runId to SubtaskUnwrapError
* WIP docs update beta -> latest
* trigger.dev init now adds @trigger.dev/build to devDependencies
* How it works doc
* Restructure some docs and update the cli commands
* Config file docs, plus aptGet and ffmpeg extensions
* Update to latest from beta docs
* Add runtime to templates
* Add --runtime option to the init CLI command
* A bunch more doc updates after feedback
* Document triggerAndWait with unwrap and result types
* beta -> latest in the webapp
* CLI update check no longer references beta
* Add major release
* Leave changeset beta, back to normal package release
* Fixed default dirs option in init command
* exclude windows-yarn variation of cli e2e tests because it’s buggy
* Remove cache to try and fix yarn e2e test workflow errors
* Better code snippet to clarify your framework
* New page for react-pdf
* Added 3 more examples
* New react to PDF example
* Alphabeticalise the side menu
* Removed github link and tweaked title
* WIP on using react-window-splitter
* WIP with new resizable panels and SSR
* Use the cookie package
* Resizable storybook page
* Increase indexing memory limit
* Fixed v2 usage meter displaying when on paid plan (#1255)
* Fixed v2 usage meter displaying when on paid plan
* Show the free usage panel only for v3 projects
* Concurrency page and more accurate tracking (#1252)
* Initial TaskRunConcurrencyTracker implementation
* MARQS calls a subscriber to events
* When enqueuing add the extra required metadata
* Track concurrency per environment for tasks too
* Admin page for global concurrency
* Use the new concurrency tracker on the tasks page
* Useful performance test task
* getAllTaskIdentifiers()
* New page for concurrency
* BackgroundWorkerTask index for quick lookup of task identifiers
* Added a way to get concurrency for environments
* Added upgrade/request more concurrency button
* Queued task column working
* Use defer and suspense
* Added queue column to the concurrency environments table
* Some comments added for clarity
* Fixed bad log message
* Sidemenu: move lower and rename to “Concurrency limits”
* Only show the environments, not tasks. Renamed to “Concurrency limits”
* v3: fix unfreezable state crashes for runs with multiple waits (#1253)
* support named capture groups
* write crash errors to attempt.error
* make restored pod names unique per checkpoint
* use last eight characters of checkpoint id instead
* add more chaos monkey env vars
* Ignore unfreezable states
* prevent excessive queue config parsing errors
* handle dependency resume edge case
* better entry point logging
* ignore checkpoint cancellation timeouts
* add missing idempotency keys to wait for dep replays
* remove checkpoints between attempts
* fix retry container names on kubernetes
* add changeset
* fix types
* bring back internal duration timers
* Added more logging to TaskRunConcurrencyTracker and some more try/catches
* Call subscriber.messageDequeued in dequeueMessageInSharedQueue
* Added messageReplaced to concurrency tracking (when freezing)
* Added depenenciesToBundle guide to bundle all packages
* Include the old message data when replacing, so we get the projectId etc.
* Fix restored container names
* Fix for schedule page not scrolling
* Added a description panel to the Concurrency admin page
* chore: Update version for release (beta) (#1256)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
* Release 3.0.0-beta.53
* Added a note to use batchTrigger() instead of trigger()
* The latest react-window-splitter fixes the ESM issues
* Set sensible defaults for the run page
* Deployments page
* Test page
* Schedules page
* Latest version of react-window-splitter (0.2.5)
* Updated to the latest version: react-window-splitter
* Callout if runs don’t start right away now has some top margin
* Small padding fix
* styled the handle focus state
* Added isStaticAtRest prop to resizable panel
* Updated resizable storybook
* Inline code blocks behave nicer when text wraps
* Added ElectricSQL to docker-compose, available on 3060
* Extracted some logic out of the eventRepository for getting a trace. This will be used on the frontend
* Use the new util
* More restructuring ready to use the trace summary from the frontend
* Using ElectricSQL for the run page data
* Min size for resizable panel on test page
* Don’t load the trace in the RunPresenter anymore
* Fix for the resizable panels on the run page
* Added overflow hidden to the panel group
* min size for the test page left hand panel
* Updated to latest window-splitter version
* Removed unused const
* One fix for client-server mismatch
* Slight improvement in the loading state
* Restructured the page so the loading is better
* Improvement to the loading states
* Improved the loading behaviour with the inspector
* WIP on auth, having problems with it
* Upgrade Remix to 2.9.1 (same as PR #1096)
* Switched structure around again so we only call the useTrace hook from the client
* Added auth to the sync
* Overscan more rows in the tree view
* Fix for TS error
* Remove duplicate import
* Revert "Upgrade Remix to 2.9.1 (same as PR #1096)"
This reverts commit e63ee9e924.
* save cookie only when id is used
* Deployment table now scrolls
* removed imports
* A lot of changes to make the inspector live too… WIP
* More major overhauls to get the synced version of the run page working…
* If a span is completed show that
* Set the debounce much lower for selecting the span view
* Load the details run inspector data on demand
* Delete the SpanPresenter
* Use the async payload because it deals with superjson
* Fixed weird merge conflict
* Share some inspector timeline components
* A couple of layout tweaks
* Improved the run inspector loading states
* Fix for paragaph errors
* Fix for focusing on a span
* Undefined typre for useSyncedShape
* ELECTRIC_ORIGIN env var doesn’t have a default, added to the examples
* Updated @electric-sql/react package to the latest
* Fix the timeline duration stretching
* Added some better error handling for the electric sync
* More logging
* Better error when there are bad responses
* Turn off resizable snapshots, there’s a bug
* Added getSpan back
* Added SpanPresenter back
* Updated to the new Electric hooks package
* Made a copy so we have the old run page and the new electric one
* Put the main eventRepository back for now
---------
Co-authored-by: nicktrn <55853254+nicktrn@users.noreply.github.com>
Co-authored-by: James Ritchie <james@trigger.dev>
Co-authored-by: James Ritchie <james@jamesritchie.co.uk>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
* Ignore /packages/cli-v3/src/package.json
* Added more logs when resuming a dependency, added the runId
* A task for reproducing a race condition with checkpoints
* Fix for doing remote image build when not self-hosting
* Set team members, alerts and schedule limits to 100m for self-hosting
* Import fix
* Set the checkpointEventId in marqs when the checkpoint is created for batchTriggerAndWait
This should fix a horrible race condition when at max concurrency
* WIP creating a CLI section
* WIP adding more CLI commands in a new section
* New CLI section with introduction and more CLI commands
* More improvement to the CLI options
* Each CLI command has its own page
* fixed broken links
* Removed outdated copy
* Fixed empty env vars overriding in dev runs
* Don’t import package.json anymore
* fix node10 moduleResolution in @trigger.dev/core
* Support self-hosters pushing to a custom registry when running deploy
* dev: Fixed stuck runs when a child run fails with a process exit
* Make some doc notes about known issues and docker hub private repos
* Fix --project-ref when running deploy
* Fix —config option when deploying
* Fixing the flushing/killing process with the new build system
* Add monorepo-react-email e2e test fixture
* Fix issue with emitDecoratorMetadata and tsconfigs with extends
* Got the emit decorator metadata fixture working
* Fixed typechecking yarn e2e CLI tests in monorepos
* Add remote forced externals system, in case we come across another package that cannot be bundled (spurred on by header-generator)
* Remote externals now powered by JSON Hero to be easier to update
* resolve config source files
* Add a —javascript option to init, defaults to typescript
* Add support for prisma typed sql
* Remove msw and retry.interceptFetch
* Add missing code to the openai retries example
* Don’t generate the v3 catalog prisma client during CI
* Fixed v3-catalog task imports
* Remove interceptor usage in task file
* Only import import-in-the-middle hook if there are instrumented packages
* Fix yarn.lock file
* When resuming a batch, only do marqs operations once
* Made TaskRunDependency clearer in the Prisma schema
* New ResumeDependentParentsService service, use it from checkpoints
* WIP on making resuming more robust
* Turn the declarative schedules off because they make debugging other runs painful
* Resuming batches when there’s an attempt is working
* If there’s no attempt then create one
* Added a log if there are no span events to complete
* If Graphile addJob doesn’t return a row, log and return undefined. No throw
* Pass prisma into the ResumeDependentParentsService
* Removed the todos
* Pass Prisma through to the checkpoint service
* Fix for not checking the batch item correctly
* Fix for when a log flush times out and the process is checkpointed
* Fix for when a log flush times out and the process is checkpointed
* Another test run that does batches with failed subtasks
* Don’t call ResumeTaskRunDependenciesService anymore (we have a new service)
* Only resume if the run is in a final state
* If an attempt doesn’t exist, fix for creating queue with sanitized name
* If DEV then don’t resume using marqs/batches. The CLI manages it
* We don’t need to check the run status again, it’s in the main function now
* Added TaskRunAttempt taskRunId index
* Only allow calling ResumeDependentParentsService with a run ID
* Put the flushing back to what it was
* Separated the triggering docs page into 2 sections
* Added link to GitHub Actions from deploy page
* Improved the batchTriggerAndWait docs for handling errors
* Added docs for using google credential files
* Improved the triggering page copy some more
* Improved the google credentials copy
* Support custom config file names & paths
* Fix entry point paths on windows
* Support custom conditions
Add support for custom conditions (for bundling and running), to support being able to import `ai/rsc` with the “react-server” condition.
- Fixed an issue where symlinking unresolvable externals after rebuilding caused the build to hang
- Fixed an issue with external not working with subpath exports (e.g. “ai” would not match “ai/rsc”)
- Protect better against build extensions breaking builds
* Add changeset
* Fix passing CLI process.env down to the task processes
* Remove unused import
* reviving the e2e CLI tests
* Another attempt at fixing windows
* yet another windows attempt (yawa)
* Output index child stdout and stderr (yawa)
* normalize import paths for windows
* Added some logging (yawa)
* normalize the loader path as well
* Added some logging to figure out why the entry points aren’t being found on windows
* Fix for entry point detection on windows
* Normalize runner import loader path
* Normalize import paths in dev and make sure rewritten build manifest paths are correct on windows as well
* Various cleanup after windows fixes
* Remove the webapp e2e for now
* Add node10 type resolution support for subpath exports (still does not actually import in Node 10 or Browserify/Parcel)
https://github.com/andrewbranch/example-subpath-exports-ts-compat
* init using templates again but downloaded from the repo this time
* Adding init schedule example
* Support for js init
* init now working with js
* Fix issues with links in terminals that don’t support them. Also skip update check of version starts with 0.0.0
* WIP on using react-window-splitter
* WIP with new resizable panels and SSR
* Use the cookie package
* Resizable storybook page
* Increase indexing memory limit
* Fixed v2 usage meter displaying when on paid plan (#1255)
* Fixed v2 usage meter displaying when on paid plan
* Show the free usage panel only for v3 projects
* Concurrency page and more accurate tracking (#1252)
* Initial TaskRunConcurrencyTracker implementation
* MARQS calls a subscriber to events
* When enqueuing add the extra required metadata
* Track concurrency per environment for tasks too
* Admin page for global concurrency
* Use the new concurrency tracker on the tasks page
* Useful performance test task
* getAllTaskIdentifiers()
* New page for concurrency
* BackgroundWorkerTask index for quick lookup of task identifiers
* Added a way to get concurrency for environments
* Added upgrade/request more concurrency button
* Queued task column working
* Use defer and suspense
* Added queue column to the concurrency environments table
* Some comments added for clarity
* Fixed bad log message
* Sidemenu: move lower and rename to “Concurrency limits”
* Only show the environments, not tasks. Renamed to “Concurrency limits”
* v3: fix unfreezable state crashes for runs with multiple waits (#1253)
* support named capture groups
* write crash errors to attempt.error
* make restored pod names unique per checkpoint
* use last eight characters of checkpoint id instead
* add more chaos monkey env vars
* Ignore unfreezable states
* prevent excessive queue config parsing errors
* handle dependency resume edge case
* better entry point logging
* ignore checkpoint cancellation timeouts
* add missing idempotency keys to wait for dep replays
* remove checkpoints between attempts
* fix retry container names on kubernetes
* add changeset
* fix types
* bring back internal duration timers
* Added more logging to TaskRunConcurrencyTracker and some more try/catches
* Call subscriber.messageDequeued in dequeueMessageInSharedQueue
* Added messageReplaced to concurrency tracking (when freezing)
* Added depenenciesToBundle guide to bundle all packages
* Include the old message data when replacing, so we get the projectId etc.
* Fix restored container names
* Fix for schedule page not scrolling
* Added a description panel to the Concurrency admin page
* chore: Update version for release (beta) (#1256)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
* Release 3.0.0-beta.53
* Added a note to use batchTrigger() instead of trigger()
* The latest react-window-splitter fixes the ESM issues
* Set sensible defaults for the run page
* Deployments page
* Test page
* Schedules page
* Latest version of react-window-splitter (0.2.5)
* Updated to the latest version: react-window-splitter
* Callout if runs don’t start right away now has some top margin
* Small padding fix
* styled the handle focus state
* Added isStaticAtRest prop to resizable panel
* Updated resizable storybook
* Inline code blocks behave nicer when text wraps
* Added ElectricSQL to docker-compose, available on 3060
* Extracted some logic out of the eventRepository for getting a trace. This will be used on the frontend
* Use the new util
* More restructuring ready to use the trace summary from the frontend
* Using ElectricSQL for the run page data
* Min size for resizable panel on test page
* Don’t load the trace in the RunPresenter anymore
* Fix for the resizable panels on the run page
* Added overflow hidden to the panel group
* min size for the test page left hand panel
* Updated to latest window-splitter version
* Removed unused const
* One fix for client-server mismatch
* Slight improvement in the loading state
* Restructured the page so the loading is better
* Improvement to the loading states
* Improved the loading behaviour with the inspector
* WIP on auth, having problems with it
* Upgrade Remix to 2.9.1 (same as PR #1096)
* Switched structure around again so we only call the useTrace hook from the client
* Added auth to the sync
* Overscan more rows in the tree view
* Fix for TS error
* Remove duplicate import
* Revert "Upgrade Remix to 2.9.1 (same as PR #1096)"
This reverts commit e63ee9e924.
* save cookie only when id is used
* Deployment table now scrolls
* removed imports
* A lot of changes to make the inspector live too… WIP
* More major overhauls to get the synced version of the run page working…
* If a span is completed show that
* Set the debounce much lower for selecting the span view
* Load the details run inspector data on demand
* Delete the SpanPresenter
* Use the async payload because it deals with superjson
* Fixed weird merge conflict
* Share some inspector timeline components
* A couple of layout tweaks
* Improved the run inspector loading states
* Fix for paragaph errors
* Fix for focusing on a span
* Undefined typre for useSyncedShape
* ELECTRIC_ORIGIN env var doesn’t have a default, added to the examples
* Updated @electric-sql/react package to the latest
* Fix the timeline duration stretching
* Added some better error handling for the electric sync
* More logging
* Better error when there are bad responses
* Turn off resizable snapshots, there’s a bug
---------
Co-authored-by: nicktrn <55853254+nicktrn@users.noreply.github.com>
Co-authored-by: James Ritchie <james@trigger.dev>
Co-authored-by: James Ritchie <james@jamesritchie.co.uk>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
* Add an error to the final attempt if there isn’t one
* Improved the checkpointResumer test task
* Using triggerAndWait or batchTriggerAndWait frees up concurrency
Normally it frees up just env and org concurrency. If it’s a recursive task then it will free up the run concurrency too (e.g. a task calling itself).
* Removed the filepath and export name from attempt spans
* Run inspector: only show an error if the run is in a finished state
* Add support + docs for custom esbuild plugins
* Fixed the missing subpath export
* Add support for specifying the prisma client generator
* Add `—env-file` support to dev and deploy
* RESUME messages were missing some data that is needed for the concurrency tracker
* Cancel heartbeats when checkpoints are created for triggerAndWait and batchTriggerAndWait
* Added some logs when canceling heartbeats
* Improved the nested dependencies test task
* upgrade @opentelemetry packages to the latest versions
* remove v2 only packages, will be moved to a dedicated repo
* remove more v2 code and run pnpm install
* use the npm yalt package in the webapp
* convert @trigger.dev/core to tshy
* Switch from jest to vitest in @trigger.dev/core
* Fixed core test
* move core-backend code into core subpath export
* convert @trigger.dev/sdk to tshy
* Removed hono
* move core-apps to core/v3/apps, remove core-apps, start converting cli-v3
* Fix up some of the commands
* cli now building and loadable
* using package-json-from-dist to get package version now in core and cli
* dev command WIP
* cleaned up some repetition and structure of the entry point stuff
* bringing back the background worker stuff
* Indexing of the v3 catalog
* getting closer to executing dev runs...
* centralize dev logging using event emitter
* Move indexing to it’s own entry point, simplify code
* dev runs working
* Get instrumentation to work with openai
* debugging achieved internally
* provide worker files as part of the worker creation on the server
* support for cjs and esm javascript
* Fixed timeout
* worker manifest now has the config path
* auto-upgrade config to non-deprecated alternatives
* Adding package preview release
* deployment WIP
* improve the syncEnvVars output and adapt resolveEnvVars
* WIP bun runtime
* WIP bun support
* seed tasks with the machine preset if listed in the config
* deploy run executions WIP, extracted TaskRunProcess into 1 place
* deployed tasks running and executing 🎉
* support for waits and better flushing & process cleanup
* Fixed the heartbeating
* Better warning messages
* Improve and unify the indexing between dev and deploy
* Support for external deps that need node-gyp to build
* build extensions can now install custom packages and run instructions in the image. Also prisma extension now works and also works with multiple schema files
* Add back in the main/types/module to sdk
* dev no longer is Ink/React, grace period for disconnections in dev
* Fix the changeset config
* More changeset fixes
* Remove config packages
* More changeset fixes
* Fixed typescript issues (needed to revert back to zod 3.22.3
* Fix pr_checks workflow
* Remove the prepare script
* Fixed tests and package versions
* Remove cli test script
* Remove packages from tailwind watch paths
* Add repo to public packages
* Just commit the generated files and do the building at dev time
* Try and get pkg.pr.new working
* Try again
* Fix emitDecoratorMetadata importing named export from typescript
* config file backwards compat with export const config
* Fixed issue where import errors weren’t coming through
* p-retry is a prod dep
* typescript needs to be a prod dependency for emitDecoratorMetadata
* Add better debug logging to help track down import-in-the-middle bug
* An external is only considered resolvable if it resolves to the same path as the collected external
* Fix runtime checks to allow >=18.20
* Move extensions to a new build package
* Fixed building packages in dockerfile
* Remove the e2e test from publish workflow for now
* Don’t treat pkg.pr.new versions has needing upgrading
* making sure config handleError works, and discovered path aliases don’t work in config files
* Strip empty string env vars so they accidentally override real values
* Couple of things
* Update version to use preview instead of beta
* Hopefully fix re-attempts with >30s delay
* Match socket emit messages to current latest in main
* Initial guide
* Go back to beta
* Go back to the preview, and update guide to use pr preview tags
* Go back to beta
---------
Co-authored-by: Matt Aitken <matt@mattaitken.com>
* Remove run count from JobPresenter
* Remove job/integration counts from org/projects query
* Removed v2 counts from ProjectPresenter query
* JobRun index idx_jobrun_organizationId_createdAt
* JobRun index idx_jobrun_versionId
* BatchTaskRunItem index taskRunAttemptId
* BatchTaskRunItem index taskRunId
* Add new ProjectAlertType ‘TASK_RUN’ and then migrate channels to it
* WIP new alerts for run failures
* Consolidate failed run status into taskStatus.ts
* Added the read replica into BaseService, could be useful
* Send task run alerts code
* Revert "Added the read replica into BaseService, could be useful"
This reverts commit cc2348a40254ea3f9e11c0feaf303ecd8794433a.
* Allow adding task run alerts in the UI
* Use task run alert not attempt alert…
* Use the primary
* tasks no longer inside a group in the side menu (and added “cron”)
* Delay using a timezone
* Added React Not Defined error to the troubleshooting page
* Improved the React common problem
* Link to v2 docs
* New Development section and entry in Common Problems
* Concurrently running the terminal
* Fixed the .env weirdness
* Added section on creating PATs for Github actions
* Improved the Machine spec and limits page
* Quick start steps now have nice images
* Added a diagram for the lifecycle functions
* Added note about onFailure
* CRON -> cron/Cron
* WIP adding more steps to the next.js guide
* WIP next.js
* WIP adding tabbed steps for pages/app router
* References to Infisical links to their homepage so it’s clearer
* WIP updating the nextjs guide
* WIP nextjs guide
* More nextjs guide steps
* More copy
* Added rate limit trouble shooting
* Removed old prisma error title
* Added secret key step
* Added a note for logging in using a specified domain if self hosting
* typo
* App router docs copy
* Deploy copy update
* Added a favicon.png to fix a docs build error
* Removed unused snippet
* Server actions now inside a tab
* Server actions + restructured the triggering section
* Added troubleshooting snippet for react event handlers
* Added a new troubleshooting snippet for ESM
* Updated old replaying image to reflect the new UI
* Removed references to reattempting
* Updated replaying from the run page
* Added a bulk replay section
* Updated the writing tasks intro page
* Removed edge runtime code for now
* Added edge runtime – it seems to just work!
* import type
* Env vars: don’t show deleted dev environments
* Concurrency limits: don’t show deleted dev environments
* Create/edit schedule: don’t show deleted dev environments
* Test for checkpoints
* Make sourceTaskAttemptId optional on resumeBatchRun
* Removed all completions/executions logic from the shared queue consumer
* Removed the sourceTaskAttemptId from ResumeBatchRunService
* Revert "Removed all completions/executions logic from the shared queue consumer"
This reverts commit d35398d50463c81a1975bb5d5bcfca66a24b8ede.
* WIP on triggerAndWait…
* Fixed triggerAndWait continuing when a checkpoint completes
* Remove the ResumeAttempt code that fails attempts (was protecting against infinite restores)
* Removed messageBody.data.completedAttemptIds.length === 0 commented out code
* Don’t ack if there’s no batchRun
* Added the marqs?.replaceMessage back in but NOT when there’s no checkpoint. More logging
This is a fix for when some attempts fail
* Improvement to the test task that now randomly fails attempts
* When a checkpoint happens, only continue the attempt if it’s in the correct state
* Changeset for rollback in branch
* Set keepRunAlive to false when the dependent task isn’t finished
* Changeset manual version (to get inline with the hotfix branch)
* Changeset: Fixes for continuing after waits
* Latest lockfile (after manual changeset version)
* support named capture groups
* write crash errors to attempt.error
* make restored pod names unique per checkpoint
* use last eight characters of checkpoint id instead
* add more chaos monkey env vars
* Ignore unfreezable states
* prevent excessive queue config parsing errors
* handle dependency resume edge case
* better entry point logging
* ignore checkpoint cancellation timeouts
* add missing idempotency keys to wait for dep replays
* remove checkpoints between attempts
* fix retry container names on kubernetes
* add changeset
* fix types
* bring back internal duration timers
* Initial TaskRunConcurrencyTracker implementation
* MARQS calls a subscriber to events
* When enqueuing add the extra required metadata
* Track concurrency per environment for tasks too
* Admin page for global concurrency
* Use the new concurrency tracker on the tasks page
* Useful performance test task
* getAllTaskIdentifiers()
* New page for concurrency
* BackgroundWorkerTask index for quick lookup of task identifiers
* Added a way to get concurrency for environments
* Added upgrade/request more concurrency button
* Queued task column working
* Use defer and suspense
* Added queue column to the concurrency environments table
* Some comments added for clarity
* Fixed bad log message
* Sidemenu: move lower and rename to “Concurrency limits”
* Only show the environments, not tasks. Renamed to “Concurrency limits”
* WIP notes on each location where we’ll use finalize
* Initial FinalizeTaskRunService
* ExpireEnqueuedRunService uses FinalizeTaskRunService
* FailedTaskRunService uses FinalizeTaskRunService
* Allow passing in an include when finalizing the run
* CrashTaskRunService using FinalizeTaskRunService
* Remove comments
* Status is optional
* CancelAttemptService using FinalizeTaskRunService
* Import tidy
* CancelTaskRunService using FinalizeTaskRunService
* Import tidying
* CompleteAttemptService system failure switched to FinalizeTaskRunService
* Added more logging to Finalizing
* CompleteAttemptStatus COMPLETED_SUCCESSFULLY
* CompletedAttempt “SYSTEM_FAILURE”
* CompletedService final pair
* Use satisfies so we can derive types from the groups
* Only allow final states to be used with this service
* BaseService tx support, minor improvements
* tasks no longer inside a group in the side menu (and added “cron”)
* Delay using a timezone
* Added React Not Defined error to the troubleshooting page
* Improved the React common problem
* Link to v2 docs
* New Development section and entry in Common Problems
* Concurrently running the terminal
* Fixed the .env weirdness
* Added section on creating PATs for Github actions
* Improved the Machine spec and limits page
* Quick start steps now have nice images
* Added a diagram for the lifecycle functions
* Added note about onFailure
* CRON -> cron/Cron
* Updated the cli-dev steps for the concurrently package
* Fixed capital letter
* Updated diagram text
* Removed dead page
* Make the tooltip text color grey so it’s readable again the primary color
* Setup and styling for the Guides section
* Better Guides icon
* Improved the prerequisites
* WIP Nextjs guide and new folder structures
* Copy updates
* Useful next steps is now a snippet
* Better icons for the next steps section
* Renamed the “prerequisites” snippet
* Node.js guide
* Added a “Creating a project” guide
* New snippet for prerequisites
* New Remix guide
* Added Remix to the side bar
* Tweaked icons for creating a project page
* Added the hello world step to the onboarding steps
* Better “useful next steps” snippet card links
* Moved prerequisites
* Fixing links to images
* Moved v2 migration page to guides
* Removed dead link
* Attempt fix for redirect
* Fixed redirects
* Getting started section includes link to roadmap
* Removed icon from side menu
* Deleted all v2 pages (excluding updating mint.json)
* Removed v2 pages, redirects and versions from mint.json
* Deleted v2 snippets
* Deleted un-used pages
* Set of more useful coming soon snippets
* All snippets use the updated format
* updated folder “v3/“ with “/pages”
* Moved all main docs files to the route and updated the redirect
* Fixed URLs in the mdx pages to the new route path
* URL goes to the proper pricing page
* Better delayed runs image
* Attempt fix for self hosting page not redirecting
* Updated the v2 project message to include end of life
* Removed the v3 badge. Showing a v2 badge instead
* Removed the v3 badge variant
* Show a badge in the side menu if you’re paying
* Popover menu section header and button state colours now reflects the design UI
* Added a link to the blog in the end-of-life message
* Removed missing variant from badge
* Remove the old log truncation
* Added TaskRUn logsDeletedAt column
* Accurate timestamps for the run inspector
* EnsureProperty type when you want to make a single property not nullable
* No logs and upgrade messages working
* Button can be autofocused
* Replay dialog code editor is autofocused
* Fix for wrapping of span duration
* Added TaskRun completedAt column
* When finalising a run set the completedAt date
* A note to discuss whether we need to set the completedAt to null
* Remove the note in the sharedQueueConsumer
2024-08-02 10:31:38 +01:00
3065 changed files with 105448 additions and 362870 deletions
Tasks should now be much more robust and resilient to reconnects during crucial operations and other failure scenarios.
Task runs now have to signal checkpointable state prior to ALL checkpoints. This ensures flushing always happens.
All important socket.io RPCs will now be retried with backoff. Actions relying on checkpoints will be replayed if we haven't been checkpointed and restored as expected, e.g. after reconnect.
Other changes:
- Fix retry check in shared queue
- Fix env var sync spinner
- Heartbeat between retries
- Fix retry prep
- Fix prod worker no tasks detection
- Fail runs above `MAX_TASK_RUN_ATTEMPTS`
- Additional debug logs in all places
- Prevent crashes due to failed socket schema parsing
Adds support for `emitDecoratorMetadata: true` and `experimentalDecorators: true` in your tsconfig using the [`@anatine/esbuild-decorators`](https://github.com/anatine/esbuildnx/tree/main/packages/esbuild-decorators) package. This allows you to use libraries like TypeORM:
```ts orm/index.ts
import "reflect-metadata";
import { DataSource } from "typeorm";
import { Entity, Column, PrimaryColumn } from "typeorm";
@Entity()
export class Photo {
@PrimaryColumn()
id!: number;
@Column()
name!: string;
@Column()
description!: string;
@Column()
filename!: string;
@Column()
views!: number;
@Column()
isPublished!: boolean;
}
export const AppDataSource = new DataSource({
type: "postgres",
host: "localhost",
port: 5432,
username: "postgres",
password: "postgres",
database: "v3-catalog",
entities: [Photo],
synchronize: true,
logging: false,
});
```
And then in your trigger.config.ts file you can initialize the datasource using the new `init` option:
```ts trigger.config.ts
import type { TriggerConfig } from "@trigger.dev/sdk/v3";
Added config option extraCACerts to ProjectConfig type. This copies the ca file along with additionalFiles and sets NODE_EXTRA_CA_CERTS environment variable in built image as well as running the task.
Updates the `trigger`, `batchTrigger` and their `*AndWait` variants to use the first parameter for the payload/items, and the second parameter for options.
We've also changed the API of the `triggerAndWait` result. Before, if the subtask that was triggered finished with an error, we would automatically "rethrow" the error in the parent task.
Now instead we're returning a `TaskRunResult` object that allows you to discriminate between successful and failed runs in the subtask:
When using idempotency keys, triggerAndWait and batchTriggerAndWait will still work even if the existing runs have already been completed (or even partially completed, in the case of batchTriggerAndWait)
- TaskRunExecutionResult.id is now the run friendlyId, not the attempt friendlyId
- A single TaskRun can now have many batchItems, in the case of batchTriggerAndWait while using idempotency keys
- A run’s idempotencyKey is now added to the ctx as well as the TaskEvent and displayed in the span view
- When resolving batchTriggerAndWait, the runtimes no longer reject promises, leading to an error in the parent task
v3: Copy over more of the project's package.json keys into the deployed package.json (support for custom config like zenstack)
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Eric Allam