* Add the ability to handle errors at the task and project level
* Cancel in progress runs when disconnecting the dev CLI (in between attempts)
* unify the task executor across dev & prod
* WIP proxy/deploy
* WIP, registry proxy in express working
* A couple of notes, preparing for indexing
* Move the changes to prod-worker into the new file
* Deploy command working with indexing and runs (docker provider only for now)
* Removed ts-expect-error directive
* Fixed build command
* add zod socket
* start using zod socket
* use zod socket for shared queue
* use shared provider shell
* update k8s provider task ops
* set otlp endpoint
* update docker actions and fix builds
* remove unused types
* update injected secret key env var name
* complete socket.io types to schemas migration
* update env example to new v3 key var
* fix coordinator build
* set task run label
* ensure attempt id is always set
* increase prod worker timeout
* set otlp endpoint for on runs
* pass otlp env var to prod worker
* auto-remove completed index jobs
* Added the new colours to the Tailwind config
* Fixed Tsailwind compile errors
* WIP Geist working in the app
* WIP Storybook
* Storybook is working again… ouch
* Use Geist mono for CodeMirror
* Some find/replace class changes
* Resizable colours improved
* WIP on primary button
* Minor style improvements
* Improve the side menu header styling
* Improved v2 breadcrumbs
* Account side menu tweaks
* The page now works with the NavBar and UpgradePrompt
* Removed unused import
* Refactored the buttons to be dry-er
* Removed the breadcrumbs
* WIP on nav bar
* WIP updating the button styles
* Fix for the old test page
* Fix for type errors
* Removed console.log
* Colour change in the resizable handle
* New loading bar divider
* Removed the design plugin
* Creating our own storybook pages
* Added some “storybook” pages
* Added DetailCell storybook
* WIP new animation for the loading page
* Added the new loading divider
* Fix for the time frame filter having a green border
* Style improvements to the new test page
* Small style improvements
* Improvements to the list pagination components
* Nav bar back button improved
* Improvements to the run page
* Lots of v2 style improvements
* Help looks better
* Improvements to the regenerate API key modal
* More general improvements
* Added section titles to our storybook
* Added form stories
* Fix for missing key prop
* Loading bar is working
* Fixed button submitting state
* Fix for the test button icon
* Code bg improved
* Improved oauth buttons
* Integration page header fix
* Various improvements, mainly changing button variants
* Segmented control style updates
* Added an outline of 3px for the segment control
* Fixes for the billing pages
* More pricing improvements
* Added dialog to storybook
* Typography now in storybook
* Free usage bar added to storybook
* Added inline clode to storybook
* Separated menu types into a sub section in storybook
* NamedIcon now in storybook
* Added page header to storybook
* Added pricing callout to storybook
* Added radio groups to storybook
* Added segmented controllers to storybook
* Added shortcuts to storybook
* Added switches to storybook
* Added toast to storybook
* Added tooltips to storybook
* Added tree view to storybook
* Added usage charts to storybook
* Tarted up the environment variables page
* Fix for the FreePlanUsage panel bg colours
* Added new badge variants
* Added the <TaskPath /> to the tasks table
* Span header improvements
* Started work on the timeline
* Started work on the timeline
* Fix for the run page header
* Timeline tick marks are working
* Timeline working
* Got the durations showing on hover
* Button improvements
* Experimenting with how the duration should be shown
* Lots of refinements to the timeline
* Much nicer API for the Timeline
* The switch is now purple to match the design, with inverted thumb when on
* Removed old stuff
* Export the props
* Added the span with duration back in
* Simplified more and fixed the span durations
* Put in some decent dummy data
* Rendering defaults
* Wip with the mouse cursor
* Improvements to the story
* Added Slider primitive
* Integrated the icons into the slider
* Lots of layout improvements
* Cleaned up imports
* “Root” indicator
* Animated background on partials
* Pull the styles through to the spans and log dots
* Fixed some bugs in the TreeView when nodes get created or destroyed
* Fix for incorrect parentTask reference
* Added run streaming endpoint for live refreshing
* Added the tabs to Storybook
* Fix for status font size mismatch
* Fix for the misaligned run filter status
* Set a minimum duration on the timeline
* The TreeView now re-calculates if the number of nodes has changed
* Partials now render to the end
* Improved the durations gutter
* The duration is calculated using the current time if there are partial events
* Added the environment label to storybook
* Improved the environment label style
* Removed bottom margin from badge
* Button style improvements
* Callouts style fits the new design better
* Got the scrollbars to appear on the tree/timeline
* Added scroll support
* Synced scrolling working
* Close button and exit icon for the detail panel
* secure field shows start of env
* Clipboard improvements
* Latest lockfile
* Fix for the durations wrapping
---------
Co-authored-by: James Ritchie <james@jamesritchie.co.uk>
* WIP no bundling
* Convert dev CLI to use an unbundled build process to support otel instrumentation
* Fix pnpm lock file
* A couple of fixes to get typechecking to work
* Show runs as cancelled when killing the dev CLI
* Fixed error when waiting for a task
* Deprecated workers will no longer be able to run unlocked task runs
* Fixed selecting the latest background worker version in test list
* Don’t propogate trace context when triggering an async task
Also add batch id to the context and the task events
* WIP
* retry.fetch w/intercepting to test
* Fixed the incorrect renaming of limit to maxAttempts in v2
* Fixed source map support in dev
* Cleanup background worker files once they close
* Support for batch triggers
* Various trace fixes and improvements
* Moved trace into the logger, fixed span cardinality issues with waits and triggers
* Add support for attaching a debugger in dev with —debugger flag
* Added ProjectVersion to projects, defaults to V2
* Moved v3Enabled to the existing feature flags
* Update to using the v3Enabled feature flag
* Allow people to choose v3 when creating a new project
* Added version to the ProjectPresenter
* v2 project and v3 project redirecting
* .env.example for the v3-catalog
* Added sdkVersion and cliVersion columns to the BackgroundWorker table
* Added additional classes for environments
* First draft of the Tasks table
* Link to the task page
* V3 side menu project items
* Moved the ListPagination component into the components folder
* Moved the TaskListPresenter to a v3 folder
* Bare bones task page with tabs
* Task page, we’re going to delete this though
* Bare bones runs table working
* Rejigged the columns
* The run table status
* Environment and status filtering working
* Added time filters
* Cursor and direction is working… I think it’s tricky to know for sure
* Added TaskRun numbers
* Selecting a task now links to the runs page with filters turned on
* Added support for the enqueued status
* Reworked the task table
* Link to the task and environment runs
* The run page is rendering a tree of the events
* Change the task table column to “Created at"
* WIP on making the run tree view look good
* Improvements to the run tree
* Removed the janky scroll bar flash
* Change the title of the task logs to just the task ID
* Live timer when a span is running
* Added “Show parent items” link
* Fixed the bug jumping to parent items
* Style improvements to the run
* Added the resizable handle to the run page
* Moved formatDuration to core/v3
* Wait for now has nice log messages
* The run detail panel is working with just a title for now
* Navigating to a span and persisting between page reloads is working
* The run detail timeline
* Wrap log dates in a paragraph
* Latest span detail view
* Scrolling in the right-hand panel
* Tweak default resizable layout for the run page
* Attempted improvements to stop a recursive issue on TreeView
* TreeView useReducer WIP
* Changed the change callback
* Add changes to the state
* Use the new onSelectedIdChanged in the app
* Don’t update the state from the outside
* Filtering fix, although navigation doesn’t work nicely
* Filtering working when changing the content and clearing it
* Don’t allow the same selectedId callback to be called twice…
* useDebounce hook
* Switch from using defer to just typedjson for now
* Latest attempt at span navigation
* Removed log
* Turn off filtering for now, it’s causing the Links to break somehow
* Fix for page height issue
* Added (Developer Preview) to the v3 project select box
* Revert "Filtering working when changing the content and clearing it"
This reverts commit 20d9fbe36ded619e9d3eb0b23d4ed0568a88a615.
# Conflicts:
# apps/webapp/app/components/primitives/TreeView/TreeView.tsx
* Switch to using the old filtering
* Added useThrottle hook
* Instantly close the panel when deselecting a node, use debounce when navigating to a span
* Better spacing in the right hand panel
* Comment out the v3 side menu pages that don’t exist yet
* Fix for typecheck fail
* Clear the statuses too
* Introducing Modular Asynchronous Reliable Queueing System (MarQS). Works in dev
* Convert MarQS to using lua and dealing with concurrency
* Simplified the timeout queue and current concurrency is now a set instead of a flat value (to support idempotency)
* Implement task heartbeating and reconnect the background workers CLI when the websocket connection reconnects
* Start adding internal telemetry support for the server
* Get env vars to work in dev and implement prisma tracing in webapp
* Cleanup telemetry and implement it in the consumer
* Implement dequeuing a message from a parent shared queue
* Implement a custom logger exporter instead of using console log exporter
* Use node instead of shell for generating protocol buffer code
* Propogate trace context into debug logs, and allow turning off logger exporter through env vars
* Switch to using baselime for internal otel data
* Make orgMember optional to fix type issues
* Provide the CLI dev env vars through the CLI, don’t build dotenv into facade
* Removed the logger import
* Address Matt’s comments
* Addressing more of Matt’s comments
* Handle sending an execution after a websocket connection closes
* Remove auth from the env attributes to prevent obfuscation
* Added a subtask for testing
* Make it easier to run the CLI from the nextjs reference project
* Copies of the run and statuses endpoints, but without simplifying the run statuses
* Use the new v2 endpoints that give the full run statuses
* Removed unused import
* v2 events endpoint with the full run status info
* Changeset
* Initial CLI commit
* Hooked up the CLI so it actually shows stuff…
* Totally reworked the v3 CLI to be based on our existing CLI (Commander)
* WIP on PersonalAccessTokens and AuthorizationCodes
* WIP creating a Personal Access Tokens page. Created a new sidenav for account pages
* Creating tokens is working but the form is broken
* Tokens are created in the UI
* Creating and revoking access tokens from the UI is working
* Improved the create form and copy
* Tokens are a bit shorter and only lowercase
* API endpoint for creating AuthorizationCodes and the web page users hit to create PATs from them
* V3_ENABLED env var and hook that can be used in the UI to show/hide things
* API route to get a PAT (within 10 mins of creating an auth code). Moved some code to core
* Start to build the login command
* Nicer banner when starting the CLI
* Nicer update checking
* Update command style improved
* Removed the template step
* Login command options are now working
* The new CLI is logging in using the Personal Access TOken. But I need to save it still
* Logging in and saving the token is working
* Deal with already being logged in
* Who Am I working with PAT
* Deleted old account side menu header
* Improved the copy on the Auth code page
* Added docs for the login and whoami commands
* Added readme instructions for the update command
* Removed some unused things
* Remove fetchUseProxy for now
* Add OpenAI support for 4.16.0
* Add support for background polling and use that in OpenAI integration to power assistants
* Much improved OpenAI docs
* Added backgroundPoll docs
* Implements waitForEvent and added docs for more built in tasks
* Add sendEvent API referenc
* Write the task libray
* Add changeset and warning for waitForEvent
* Setup basic storybook page
* storybook preview pages don’t have padding
* Added a storybook story for the side menu
* WIP on a new side menu component
* New variant for the side menu button style
* Side menu items can now recieve a badge ‘count’
* Storybook side menu update
* Added a new font size
* you can now set the spacing between an icon and the text
* Moving the error and count badge to the right side of the button
* Added a menu sub item button style
* added sub items to the menu and all the rest of the items
* Proper background class used
* Added a side menu header and main org header
* Removed the no-mobile blocker overlay so you can use the site on a phone
* Better section name
* Created a cusom popover trigger that can take anything as children. Also the popover titles can now take a variant
* Added a popover for the user avatar menu item
* Removed the tool tip
* Arrow Popover can no also go full width
* neater conditiona;
* Removed unnessary div. And error icon can display an error message
* menu item buttons can take a variant
* Added placeholder items for the org dropdown
* add a max height to the org menu popover and handle scrollbars
* Added a border if the menu scrolls when you have lots of project items
* warning badges display tooltips if you give them some error text
* Removed the sticky header so the scrollbar looks better
* Fixed a layout issue from removing the sticky header
* Better spacing between lists
* Popover can take any react component icon and an icon classname
* Side menu headers now have a dropdown by clicking the ellipsis icon
* Side menu items can take an icon colour
* Updated styling for the dropdown Popover menus
* Installed Tabler icons
* Updated some icons to use Tabler
* Started defining the HttpTrigger class
* RequestFilterSchema is part of core
* Use the RequestFilterSchema from core
* EventSpecificationExample now derived from schema
* Need to define the class like EventTrigger
* Catalog reference that uses the httpTrigger
* The logic for the httpTrigger is in the separate file, just call it from the client using defineHttpTrigger
* Basic HttpTrigger
* Import fixes
* Transforming httpTrigger to the format we want
* Initial httpTrigger types working
* Added HTTP generic type
* Added some notes
* Rejigged some things to support verifying payloads
* Use new security function for verifying
* Added truncating text support if side menu names get long
* Renamed some stuff
* Removed test copy
* HttpEndpoin that isn’t a trigger
* Renamed hostname to source
* Created HttpEndpoint as a top-level concept, it produces triggers using onRequest
* Started working on registration of the httpEndpoints
* Removed the new TriggerMetadata for now, http will use static
* Added PayloadType to EventRecord. Default is JSON.
* The payload from an HTTPEndpoint is now a Request
* Initial TriggerHttpEndpoint migrations
* Initial server side work
* Migration to add shortcode to RuntimeEnvironment
* Added TriggerHttpEndpointEnvironment table
* Made the httptrigger catalog example runnable
* Progress creating/updating httpEndpoints
* HTTPEndpointEnvironment updating now works
* Added a note about EventRecord HttpEndpoint relationship
* When creating environments, create a shortcode
* Fix for secret not being retrieved correctly
* Request filtering
* Add DOM types to Core
* Early work dealing with a webhook arriving
* Added some todo notes to discuss tomorrow
* Added relations to the endpoint so we can call the user’s server
* Basic handling of immediate responses
* The immediate responses are now going back to the webhook provider
* Handling is now generating events and responding where relevant. Not yet matching filters and doing verification
* Added source to TriggerHttpEndpointEnvironment
* The event filter is now optional
* The source is passed through from the httpEndpointEnv
* Added optional payloadType when ingesting an event
* Added tabler
* Updated @remix-run/testing usage for remix 2
* Fixed org menu header width and added org and project types
* Added custom icons for Runs and Endpoints
* Updated the menu icons
* storybook side menu shows static project and org data
* Updated side menu
* Plumbed in paths
* Updated eventFilter to deal with undefined/null payloads
* Started work on filtering, but it’s not working
* Strict null
* async filtering it working
* Added environment shortcodes to the seedCloud script
* Don’t pass the secret back, added a verifyPayload function that gets called before runs start
* Fix for skipping triggering runs
* The verify callback is now passed through to the trigger so it can be called
* If verifying fails, have a reason which we can show to the user
* Don’t verify if it’s a test run
* Fixed the log in the example code
* Added loader and action to storybook preview
* Fix for svg attribute name issues in EndpointIcon
* Started work on the side menu
* This logic has moved to a separate file, in a different branch
* Don’t get the environments when getting orgs
* Lots of fixes for the new SideMenu
* Refactored some org/project things to fit the new structure better
* Some progress restructuring the app for the new sidebar
* More changes to data loading
* Deleted some old pages. The redirecting is working if you’re on the root but logged in
* Basic styling for the account page. We need a back button
* Fixes for the integrations pages
* Breadcrumb implemented
* Add a border to the bottom of the breadcrumb
* Feedback is working, not in Storybook
* Removed expandSidebar from the Handle
* Org level breadcrumbs
* Getting the project in the org route now works properly
* orgslug route before using the presenter
* More UI improvements
* Removed commented out line
* Use presenters for org and root
* Removed forceActive
* Split the project selector and user menu out
* The side menu items have correct active states
* Fixed the scrolling
* Put the org name in
* Improved the admin features
* Created the HTTP Endpoints list page
* Work on the HTTP endpoint page
* WIP on the HTTP Endpoint page
* Fix for response filter not showing
* Change page URLs to “http-endpoints” (from httpendpoints)
* Slugify the HTTP endpoint id
* Added todos for EventRecord associations
* Removed the Job/Trigger tab for now
* First draft of the help panels
* Rejigged the HTTP endpoint page so it splits out the deployed and not deployed environments
* Removed irrelevant columns
* Added a link to the deployment guide
* Removed the background image from the frameworks pages
Also…
- reordered the frameworks so supported ones come first in the list
- added logos to 3 framework pages that were missing them
- removed lots of un-used imports
* Fixed an issue where the breadcrumb could be hidden on small screens so broken the pages
The breadcrumb is always visible now
* Renamed ‘Onboarding’ to ‘Framework setup’
* Added a new http-endpoint icon to the side menu
* Side menu now has a nice looking scrollbar
* Scrollbar on the usage and billing page updated to the nice style
* Icon and colour updates to the side bar menu items
* Small copy improvement
* Fixed bug with the height of the org menu dropdown
* Page breadcrumbs now don’t include duplicate path links
* renamed uiBorder to ui-border
* Remove the padding from the small menu item if it’s used as a button
* Made the paragraph text a bit smaller
* EventRecord has relationships with TriggerHttpEndpoint/TriggerHttpEndpointEnvironment
* Improved the presenter used by the side menu. Also added the HTTP Endpoint badge count to the sidebar
* The dropdown org menu has the same badge as the side menu
* Improved the Projects page
* Fixed button style miss-match in the jobs table dropdown
* Made the popover dialogs support Headers and improved the regenerate API key popover
* Made the active status of a job more subtle
* Allow the Job to have a link on its Trigger
* Simplified loader because we ditched the Jobs menu. Also showing no runs help panel now
* Fixed type errors
* Pages that are org level can scroll if the content gets too long
* Fixed a redirect bug so you now redirect to the org page
* Updated the shortcode migration, to add additional letters to the end
* Removed the nav bar from the welcome onboarding page
* Only show the countBadge if the number is > 0
* Jobs icon now has a default colour the same as the nav menu
* Capital I for integration
* Clone the request before calling verify
* Separate the checks out so they can be easily logged
* Make the failure reason optional
* Made the verification error look better, if it’s provided
* Working WhatsApp example
* Can set the immediateResponseFilter to null in an update
* Deal with a null immediateResponseFilter
* Cal.com webhook working
* Header height reduced slightly
* Fixed button padding
* Removed un-used imports
* http endpoint table displays a message if you have no entries in the table
* Removed some duplicate text and swapped primary buttons for secondary style
* All ‘HTTP Endpoints’ now ‘HTTP endpoints’
* You have to use a RequestFilter when using HTTP Trigger
* Synchronous eventDispatcher filtering again. Better content-type check
* Stripe example
* Side menu “Usage & Billing” is just “Usage” if self-hosting
* Fix for filtering not working
* Removed filter log
* Fix for jobs being optional and null
* Removed old nav bar from the invite page
* Deleted old nav bar component
* import cleanup
* NoTask message only displays if the run has status “SUCCESS”
* Added a global page loading indicator
* Added documentation buttons to the most important pages
* Added GitHub
* Use “verifyRequestSignature” helpr
* Removed todo
* Fixed the layout of the Rerun Job popover
* Jobs table has margin bottom so it sits off the bottom of the page
* Small improvement to the search bar layout on the integration page
* HTTP endpoint and Trigger concept docs
* Reference docs and JSDocs for HTTP endpoint
* SDK reference docs for HttpEndpoint
* Added an HttpTrigger SDK reference, which mainly just point you elsewhere
* Improved the copy in the help panels
* Removed unused code
* Improved HTTP endpoint comment about setting the secret
* Delete unused asyncFilter
* Fixed the delete modal not having a title
* Always try get the arrayBuffer for an HTTP request, catch and use undefined as the default
* Add extra method to invokeTrigger
* Use a transaction for the secret store
* verifyRequestSignature docs page and JSDocs
* Added verifyRequestSignature docs page to nav
* Warn if they use the same HttpEndpoint id twice
* Added another job using the same HttpEndpoint
* Renamed the api route to http-endpoints from httpendpoints
* Debug bar for VSCode is now in the commandCenter
---------
Co-authored-by: James Ritchie <james@jamesritchie.co.uk>
* Implement manually invokable jobs through the invokeTrigger
Also implemented a job run notification system, that will POST details of a run on completion. This combines with the task callbackUrl system to implement the invokeAndWait
* Document the invoke trigger
* batch invoke and wait
* background fetch timeouts
* Use @whatwg-node/fetch instead of the polyfilled fetch
* Fix some outdated dependencies in webapp
* Improved subtask error propogation messages
* Document the OpenAI changes and the batch invoke stuff
* Fix dequeuing jobs
* Don’t retry the OpenAI completion background task
* Added OpenAI changesets
* Use the new ResumeTaskService in ProcessCallbackTimeout as well
* Card hover state is now grey
* Added a callout warning if you don’t have any tasks
* Conditionally show the no-task warning at the top of the run
* New job to test the no-task warning message
* Created a new example Job for the CLI that uses Tasks
* Updated the Next.js cli joke job
* Updated the Remix app cli job example
* Updated the Astro app cli job example
* Made sure the example job works in remix
* Added changeset
---------
Co-authored-by: Eric Allam <eric@trigger.dev>
This commit introduces the ability to set custom base URLs, which allows for the use of different APIs such as Perplexity. It also ensures that any OpenAI-related task now includes an idempotency key when making requests. Works for background tasks as well.
* Auto-yield run execution to help prevent duplicate task executions
* Add auto-yield config to endpoints
* Refactor run execution with buffer and limits
Introduced constants RUN_CHUNK_EXECUTION_BUFFER and MAX_RUN_CHUNK_EXECUTION_LIMIT. Adjusted PerformRunExecutionV2Service to use the new constants to fine-tune execution timings and buffers.
* Add endpoint probing functionality
Added new `RESPONSE_TIMEOUT_STATUS_CODES` in `consts.ts` to manage timeout responses. Additional functions `detectResponseIsTimeout(response: Response)` was added in `endpoint.server.ts` to detect if a response was a timeout based on the status codes from `RESPONSE_TIMEOUT_STATUS_CODES`.
Update actions to use new endpoint probing endpoint service. This allows for the early probing of endpoints to determine if they're up and running.
A new class `ProbeEndpointService` was created in `probeEndpoint.server.ts` which makes HTTP requests to a given endpoint and updates its properties based on the result.
Finally, `detectResponseIsTimeout(response)` is used in `performRunExecutionV2.server.ts` for marking the execution as succeeded when facing a timeout.
* Refactored probe method in EndpointApi class
The probe method of the EndpointApi class has been refactored to remove the error handling part and it now takes a timeout sent from the client directly. The corresponding changes were also made in the ProbeEndpointService and TriggerClient objects to reflect the alterations in the probe method.
The error handling related to the timeout has been removed and the responsibility of handling the timeout has been shifted to the client. Thus, the probe method has been greatly simplified. The `probeEndpoint.server.ts` file was also changed to accommodate the change in behavior of the probe result.
In the `triggerClient.ts` the timeout for probe is now read from the incoming request object. For backward compatibility, if no timeout is provided in the request, the default value of 15 minutes is used.
* Remove performRunExecution v1 enqueue function
* Better document limits and add docs on increasing function timeouts
* Upgrade webapp docker container to use 18.18.2
* force clients to yield when a run is executing in a gracefully shutting down worker
* Renamed task `key` to `cacheKey` and added more task documentation
* Index the `@trigger.dev/sdk` version on Endpoints
Eric Allam