* Upgrades to MarQS
- It’s “queue choosing” algorithm is now MUCH better (pretty much didn’t work before) and should be performant even when we have a bunch of prod queues
- There are now concurrency limits at the environment and organization level, as well as the task/queue level. So if any of them are at capacity the message won’t be dequeued. This means we can have an org wide concurrency limit, as well as different limits for dev/prod/staging
- I’ve added an admin API that can be used to update the org/env concurrency limits
* More MarQS upgrades
- Extract the queue priortity choosing strategy into an interface
- Implement a much better weighted average strategy
- “Slide the window” of queue candidates if the parent queue sends all at-capacity queues (so we won’t get stuck attempting to choose the same 12 full queues)
- Added some unit tests for the priority stuff
- Added some ideas for expanding the priority choosing strategy with more dynamic features
* Fixes an issue with the shared queue consumer relying on the queue name to get the env id
* add and use machine config
* assign tasks to worker nodes only
* add secure flag to zod connection
* changeset
* add pre stop hook
* don't use secure connection by default
* pass more identifiers to provider and apply labels
* superjson output support
* Better support for superjson in the task events, limiting output attributes
* Offload large outputs to object store (r2)
* Finishing up the offloading of large outputs/payloads to an object store
* add checkpoint restore events
* fix retries.enabledInDev
* fix tsconfig paths
* fix provider build and dev
* update kubernetes provider and fix builds again
* upgrade prod base to node 20
* update infra publish workflow
* rethrow k8s errors after logging
* shorten index container names
* fix error type assertion
* rename type assertion
* remove resource limits for now
* add missing run id on create
* add push to deploy command for self-hosting
* checkpointing fixes
* update coordinator image
* ensure valid registry login
* delete checkpoint archive after successful push
* log options on error
* structured logs for socket connections
* fix structured log merge
* exit process after checkpointing
* update restore pull secret name
* append shortcode to restore names
* log handler payload
* disable post start lifecycle hook
* pass in coordinator host via volume
* replace dapi with taskinfo
* add missing restore label
* don't restart restored containers
* remove init container from create
* atomic post-completion checkpoints
* switch to run id for container names
* improve wait accuracy
* measure basic checkpoint perf
* always log disconnect reason
* use system clock to end wait spans
* checkpoint readiness and cancel signals
* restore from checkpoint events and fix statuses
* remove attempt id env var
* restore dependencies from events
* reconnect wip
* lifecycle hooks are back
* fix hooks and improve reconnect
* make docker send postStart hook
* only checkpoint for retry if large delay
* fix a few more resume issues
* lifecycle hook fixes
* skip connection handler when waiting for post start hook
---------
Co-authored-by: Eric Allam <eric@trigger.dev>
* v3 projects can only be created if the org has permissions to do it
* Admin page
* Removed unused classes
* Admin org page, with links between user and org pages
* Set v3 enabled or not
- Time out deployments to prevent never-ending building/deploying
- Use the CURRENT worker deployment when determining which version to execute
- Fix some of the deployment error handling logic
* Store startTime in nanoseconds since epoch, ensure logs using nanosecond precision for their timestamp (so they are not out of order)
* export task events immediately in dev
* fix basic resume mechanics
* fix dependency resume
* fix deploy page sort
* remove task list console log
* checkpoint restore between attempts
* add checkpoint requirement checks to docker provider
* switch to raw execa for long statements
* catch and report pre-connection index errors
* graceful worker shutdown after completion
* fix final completion
* use same task process for each attempt
* update retry catalog entry
* add immediate return task
---------
Co-authored-by: Eric Allam <eric@trigger.dev>
* WIP proxy/deploy
* WIP, registry proxy in express working
* A couple of notes, preparing for indexing
* Move the changes to prod-worker into the new file
* Deploy command working with indexing and runs (docker provider only for now)
* Removed ts-expect-error directive
* Fixed build command
* Show runs as cancelled when killing the dev CLI
* Fixed error when waiting for a task
* Deprecated workers will no longer be able to run unlocked task runs
* Fixed selecting the latest background worker version in test list
* Don’t propogate trace context when triggering an async task
Also add batch id to the context and the task events
* Added isTest to TaskRun
* TriggerTaskService now accepts isTest and adds it to TaskRun if set
* Set the cliVersion and sdkVersion when indexing
* Added a test page
* Radio buttons are now circles and accept a ReactNode
* The background of the code editor is now darker
* WIP on the test task page
* Improvements to the test page
* Testing is working
* Fix typo
* Show the real isTest value in the runs table
* Added a key to the outlet so it correctly resets all the state on the sub-route
* New API keys page working
* Added EnvironmentVariable and EnvironmentVariableValue tables
* Environment variables presenter and repository started
* Icon now accepts a ReactNode as well
* The create form is working
* Improvements to validation
* Getting all the environment variables is working
* Editing an env var working
* Fix for deleting a value from an env var
* Separate route for adding new env vars. Works with “Save and add another”
* Editing is working properly, and revealing all values
* Deleting env vars
* Get environment variables with the environment id
* Added th env vars count to the Environment variables page
* Removed the commented out old Env & API keys side menu item
* Fix imports in ClipboardField
* Added a storybook page for all the variants of input fields
* Upgraded Tailwind for the :has selector
* Fix for the edit button not filling the width
* Reworked the input fields so they work with a variable width icon on the left
* Improvements to the new env var modal
* Improve the style of the edit environment variable panel
* Fix for when Input is type=“text”
* Icon now works with FunctionComponents and ComponentTypes properly
* Removed a console log
* Removed unused imports
* Use a const schema for SecretValue
* Added friendlyId to EnvironmentVariable
* Support for batch triggers
* Various trace fixes and improvements
* Moved trace into the logger, fixed span cardinality issues with waits and triggers
* Add support for attaching a debugger in dev with —debugger flag
* Added ProjectVersion to projects, defaults to V2
* Moved v3Enabled to the existing feature flags
* Update to using the v3Enabled feature flag
* Allow people to choose v3 when creating a new project
* Added version to the ProjectPresenter
* v2 project and v3 project redirecting
* .env.example for the v3-catalog
* Added sdkVersion and cliVersion columns to the BackgroundWorker table
* Added additional classes for environments
* First draft of the Tasks table
* Link to the task page
* V3 side menu project items
* Moved the ListPagination component into the components folder
* Moved the TaskListPresenter to a v3 folder
* Bare bones task page with tabs
* Task page, we’re going to delete this though
* Bare bones runs table working
* Rejigged the columns
* The run table status
* Environment and status filtering working
* Added time filters
* Cursor and direction is working… I think it’s tricky to know for sure
* Added TaskRun numbers
* Selecting a task now links to the runs page with filters turned on
* Added support for the enqueued status
* Reworked the task table
* Link to the task and environment runs
* The run page is rendering a tree of the events
* Change the task table column to “Created at"
* WIP on making the run tree view look good
* Improvements to the run tree
* Removed the janky scroll bar flash
* Change the title of the task logs to just the task ID
* Live timer when a span is running
* Added “Show parent items” link
* Fixed the bug jumping to parent items
* Style improvements to the run
* Added the resizable handle to the run page
* Moved formatDuration to core/v3
* Wait for now has nice log messages
* The run detail panel is working with just a title for now
* Navigating to a span and persisting between page reloads is working
* The run detail timeline
* Wrap log dates in a paragraph
* Latest span detail view
* Scrolling in the right-hand panel
* Tweak default resizable layout for the run page
* Attempted improvements to stop a recursive issue on TreeView
* TreeView useReducer WIP
* Changed the change callback
* Add changes to the state
* Use the new onSelectedIdChanged in the app
* Don’t update the state from the outside
* Filtering fix, although navigation doesn’t work nicely
* Filtering working when changing the content and clearing it
* Don’t allow the same selectedId callback to be called twice…
* useDebounce hook
* Switch from using defer to just typedjson for now
* Latest attempt at span navigation
* Removed log
* Turn off filtering for now, it’s causing the Links to break somehow
* Fix for page height issue
* Added (Developer Preview) to the v3 project select box
* Revert "Filtering working when changing the content and clearing it"
This reverts commit 20d9fbe36ded619e9d3eb0b23d4ed0568a88a615.
# Conflicts:
# apps/webapp/app/components/primitives/TreeView/TreeView.tsx
* Switch to using the old filtering
* Added useThrottle hook
* Instantly close the panel when deselecting a node, use debounce when navigating to a span
* Better spacing in the right hand panel
* Comment out the v3 side menu pages that don’t exist yet
* Fix for typecheck fail
* Clear the statuses too
* Introducing Modular Asynchronous Reliable Queueing System (MarQS). Works in dev
* Convert MarQS to using lua and dealing with concurrency
* Simplified the timeout queue and current concurrency is now a set instead of a flat value (to support idempotency)
* Implement task heartbeating and reconnect the background workers CLI when the websocket connection reconnects
* Start adding internal telemetry support for the server
* Get env vars to work in dev and implement prisma tracing in webapp
* Cleanup telemetry and implement it in the consumer
* Implement dequeuing a message from a parent shared queue
* Implement a custom logger exporter instead of using console log exporter
* Use node instead of shell for generating protocol buffer code
* Propogate trace context into debug logs, and allow turning off logger exporter through env vars
* Switch to using baselime for internal otel data
* Make orgMember optional to fix type issues
* Provide the CLI dev env vars through the CLI, don’t build dotenv into facade
* Removed the logger import
* Address Matt’s comments
* Addressing more of Matt’s comments
* Handle sending an execution after a websocket connection closes
* Remove auth from the env attributes to prevent obfuscation
* Added the org settings page to the sidebar
* Added loading states when renaming/deleting projects
* Don’t show deleted orgs in the app
* The actual db migration file
* The Org settings page with the actions working
* Don’t remove org members, just leave them
* Don’t show invites from orgs that are deleted
* Allow disabling IntegrationConnections
* Don’t refresh IntegrationConnections that are disabled
* Set all the integrations as disabled
* Updated the unsubscribe checkbox text
* Unsubscribe route
* Use the magic link secret, not the encryption key. Also make the error message more vague
* Only members of the org or project can rename them
* Don’t throw an error if the connection can’t be refreshed, return undefined instead
* Side menu: selector title changed to project. Changed section headers and made it Slack + Discord
* Added project settings page. Renaming is working
* Added project.deletedAt column
* Deleting projects WIP
* Allow clearing the project from the session
* Don’t load projects that are deleted
* Improved the project selection logic
* Clear the session project id when deleting a project
* Deleting the last project in an org now works. The new project page doesn’t have the sidebar anymore.
* Removed false code comment
* Added the JobRun index back in, using Prisma. Otherwise it tries to remove it on each migration
* Only return environments where the project isn’t deleted. This will block API calls
* Do the project deletedAt check in code, not SQL
* Made it clearer what the code does with a comment and applied the same logic to the public api key
* Optional next/previous with the list pagination
* Defer the loading of the runs table
* Only return the latest run in a separate query, then do a join in code
* Added a composite index to speed up getting the latest run from a job id
* Added Endpoint deletedAt column
* Only show endpoints where they’re not deleted
* Don’t delete Endpoints, set the deletedAt and change their slug name
* Only perform indexing if the endpoint isn’t deleted
* Have a nullable URL for endpoints
* Deal with null URLs throughout the app
* Re-running and retrying behaves properly when there’s no endpoint URL
* Remove console.log
* Better error message when doing a run
* Fix for typo in error when a Job invoke fails
* Can disable runs for an Organization
* Don’t perform runs if runsEnabled === false for an Org
* Exceeded runs messages made into errors, they were very subtle
* Return the runsEnabled property for each org
* More obvious message if you’re over the free run limit
* Added a red border to the free usage panel if you’re over your run limit
* Added missing property to Storybook for the FreePlanUsage component
* Initial CLI commit
* Hooked up the CLI so it actually shows stuff…
* Totally reworked the v3 CLI to be based on our existing CLI (Commander)
* WIP on PersonalAccessTokens and AuthorizationCodes
* WIP creating a Personal Access Tokens page. Created a new sidenav for account pages
* Creating tokens is working but the form is broken
* Tokens are created in the UI
* Creating and revoking access tokens from the UI is working
* Improved the create form and copy
* Tokens are a bit shorter and only lowercase
* API endpoint for creating AuthorizationCodes and the web page users hit to create PATs from them
* V3_ENABLED env var and hook that can be used in the UI to show/hide things
* API route to get a PAT (within 10 mins of creating an auth code). Moved some code to core
* Start to build the login command
* Nicer banner when starting the CLI
* Nicer update checking
* Update command style improved
* Removed the template step
* Login command options are now working
* The new CLI is logging in using the Personal Access TOken. But I need to save it still
* Logging in and saving the token is working
* Deal with already being logged in
* Who Am I working with PAT
* Deleted old account side menu header
* Improved the copy on the Auth code page
* Added docs for the login and whoami commands
* Added readme instructions for the update command
* Removed some unused things
* Remove fetchUseProxy for now
* Add support for the built-in Trigger.dev tunnel for local development
* Remove the test script from yalt
* Add to the README
* Improve the outdated packages warning
* Initial commit - started work on new billing components
* Added a slider component
* Added features to the pricing tiers
* Small pricing tier margin tweaks
* WIP on a concurrecy chart
* Reworked the pricing tiers to include a segmented controller and tooltips
* Renamed the charts storybook page
* Made the way data is added more flexible and added some definition tool tips
* Term definitions are used properly in the tiers
* Callouts can now have an optional CTA on the right hand side
* Alignment fix for the callouts
* organize imports
* Definition tooltip now its own component
* renamed the storybook story
* WIP new volume discount table and usage sliders
* Added pricing calculator sliders
* Fixed alignment of the legend
* Breadcrumb now has an upgrade prompt and button
* New Join our Slack button in the side menu
* New progress meter in the side menu
* Use the highest of 2 values to show progress
* An attempt to fix the step count in the calculator slider
* WIP usage progress bar
* Added the 4 progress bars
* More examples of the usage bar
* Better way to include the percentage in the free plan progress meter
* The usage bar now works with the extra runs over the free limit
* Pricing calculator has better slider logic
* Moved the free plan usage bar into it’s own component and added it to storybook
* Usage bar chart now supports a paying customer option and optional billing limit. Also added more usage examples to storybook
* Added more examples of usage to storybook
* tooltip takes classname
* Format numbers nicely
* Added a tooltip to show the precise numbers in the chart
* small improvements to the billing calculator
* New onboarding choose plan page
* pricing tiers better fill the size of their container
* Removed unused code
* Usage bars animate
* Wording tweak
* Callouts fit the button size better
* Added new routes for the 2 new billing pages
* import cleanup
* Added meta info in the header for bill price, plan type and billing period
* made free a noun
* Added pricing calculator to the plans page
* Fixed some illegal markup when using tooltips
* Added container query support
* Added new concurrency chart to the usage page
* billing now has a green theme
* Simplified the plan summary info int the header
* Fixed padding alignment
* Use a custom lable for the concurrent runs chart
* Removed the Job runs table
* Latest lockfile
* Show a message if you haven’t done runs yet
* Added a layoutId to the pageTabs
* Show a message callout if you’ve exceeeded 10k runs on the free plan
* Added a callout on the plans page if you’re over the runs limit
* Fixed button inside button bug
* Removed the background gradients from the app
* The billing package is importing properly
* Getting the curent plan for an org
* Reading the current plan and usage
* Hooked up the free plan bar
* Render basic billing details
* vol discount table has optional values
* Added a new page to show new subscribers
* Created a new hook for confetti on the subscribed page
* Toast styling updated
* The Invoice and Manage card details links are working
* Meta appEnv data optional
* Data for the plans page
* Format the billing period duration in days
* Switch to 20 icons
* URL for the subscribed page now includes the org path
* New pathBuilder path for the subscribed page
* Plans are upgraded/downgraded successfully
* Fixed badly named paths
* Improved some of the display
* Deal with when the user has canceled so they can re-upgrade
* Subscribing from Stripe is working, and canceling
* Tidied up some bits, latest billing package
* The tiers are now rendering using the real data
* The onboarding screen is hooked up, but not linked to yet
* Onboarding price selection working
* Pricing slider working
* Price estimation working
* Pricing calculator working on the select a plan page
* Button copy change
* Improved the layout of the run calculator marker
* Fix for the period end when you’ve canceled
* Improved the formatting in the calculator
* Pricing table tooltip uses the vol discount pricing table
* Remove the vertical lines from the calculator
* Contact us button in Enterprise tier opens the contact us form
* Added composite index to triggerdotdev_events.run_executions for event_time and organization_id
* Concurrent run chart data
* Improves styling, fix for React error with Enterprise contact button
* Show warning box when you’ve hit the concurrency in the past 30 days
* Lots of work o the usage page
* Improvements to the usage page
* Show 31 days of data, fix for not showing the current date…
* Stripe portal links are generated when the user clicks through
* Better alignment of the reference line and x-axis label
* Readme update
* Fixed pricing button loading buttons
* Show warnings about concurrency and runs on the plans page
* Removed some storybook stories
* Join Slack channel shows if you’re subscribed with instructions
* Added a gap between the runs charts
* Improved the definitions
* Added some margin to the page loading spinner
* A wider, cleaner feedback panel
* Modal backgrounds match the Sheet style
* Fixed menu item text being clipped
* Improved icons for execution time and exclusion count
* Concurrency chart now renders the dates nicely
* Fix for plans data on the plans page
* The healthcheck doesn’t need to do a HEAD request to /
* Better disabled states and fixed the disabled hover state issue
* Improved the segmented controller style
* loading spinner now centered inside the button
* Redirect to the project page when selecting the free plan
* Fix for “Runs” and added real date to upgrade warning
* DeCAPITALIZED some things
---------
Co-authored-by: James Ritchie <james@jamesritchie.co.uk>
* WIP execution concurrency controls implemented via Redis
- Split up resuming a run and executing a run
- Added some new statuses to better show what is going on in a run
- Removed preprocessing runs
* WIP
* Convert to using ZSETs and adding env vars
* Removed unused import
* Improve run number generation using advistory locks, and only on start
* More execution concurrency stuff
* Add support for job concurrency limits and concurrency limit groups
* Create wild-swans-battle.md
* Increase slots refresh timeout to 10s
* Try to fix Redis connection issues
* Don’t be so strict about the APP_ENV
* Add the blank tls option to the normal redis client as well
* Add docs
* Initial commit and WIP new login page
* added wrapper to fix layout
* Improved the layout and style of the page
* Added a new extra-large button style (and added it to storybook)
* Moved the cusom onboarding style tooltip to the Tooltip component
* Populated the bento grid with items
* Imrpoved the quote styling
* Tooltip doesn’t stay if you mouse over it
* Added more content including a list of quote that cycle on page load
* New quote mark positioning
* Added the rest of the content to the tooltips
* Made the bento grid responsive
* Moved the login page layout to a seaparte component
* Login pages now use the new layout component
* Extra large button style tweak
* Match the large input style with the large button style
* Cleaned up imports and made the Ts & Cs text the same for magic links
* Layout fix
* Quotes don’t push content off the screen as much
* Added openai integration to the grid
* Make the quote fit if the screen height is narrower
* Style update to the magic link sent state
* Icon colour updates
* fixed text centering on mobile
* Icons are more responsive
* removed unnecessary w-fulls
* Added 2 onboarding questions
* Revalidate form on submit, not input
* Typo fixes
* Name field focuses by default
* Added conform to the employees question
* Update pnpm lock file
* Implement referral source and company size on managed cloud
* Fixed some type errors
---------
Co-authored-by: James Ritchie <james@jamesritchie.co.uk>
* Add OpenAI support for 4.16.0
* Add support for background polling and use that in OpenAI integration to power assistants
* Much improved OpenAI docs
* Added backgroundPoll docs
* Implements waitForEvent and added docs for more built in tasks
* Add sendEvent API referenc
* Write the task libray
* Add changeset and warning for waitForEvent
* Setup basic storybook page
* storybook preview pages don’t have padding
* Added a storybook story for the side menu
* WIP on a new side menu component
* New variant for the side menu button style
* Side menu items can now recieve a badge ‘count’
* Storybook side menu update
* Added a new font size
* you can now set the spacing between an icon and the text
* Moving the error and count badge to the right side of the button
* Added a menu sub item button style
* added sub items to the menu and all the rest of the items
* Proper background class used
* Added a side menu header and main org header
* Removed the no-mobile blocker overlay so you can use the site on a phone
* Better section name
* Created a cusom popover trigger that can take anything as children. Also the popover titles can now take a variant
* Added a popover for the user avatar menu item
* Removed the tool tip
* Arrow Popover can no also go full width
* neater conditiona;
* Removed unnessary div. And error icon can display an error message
* menu item buttons can take a variant
* Added placeholder items for the org dropdown
* add a max height to the org menu popover and handle scrollbars
* Added a border if the menu scrolls when you have lots of project items
* warning badges display tooltips if you give them some error text
* Removed the sticky header so the scrollbar looks better
* Fixed a layout issue from removing the sticky header
* Better spacing between lists
* Popover can take any react component icon and an icon classname
* Side menu headers now have a dropdown by clicking the ellipsis icon
* Side menu items can take an icon colour
* Updated styling for the dropdown Popover menus
* Installed Tabler icons
* Updated some icons to use Tabler
* Started defining the HttpTrigger class
* RequestFilterSchema is part of core
* Use the RequestFilterSchema from core
* EventSpecificationExample now derived from schema
* Need to define the class like EventTrigger
* Catalog reference that uses the httpTrigger
* The logic for the httpTrigger is in the separate file, just call it from the client using defineHttpTrigger
* Basic HttpTrigger
* Import fixes
* Transforming httpTrigger to the format we want
* Initial httpTrigger types working
* Added HTTP generic type
* Added some notes
* Rejigged some things to support verifying payloads
* Use new security function for verifying
* Added truncating text support if side menu names get long
* Renamed some stuff
* Removed test copy
* HttpEndpoin that isn’t a trigger
* Renamed hostname to source
* Created HttpEndpoint as a top-level concept, it produces triggers using onRequest
* Started working on registration of the httpEndpoints
* Removed the new TriggerMetadata for now, http will use static
* Added PayloadType to EventRecord. Default is JSON.
* The payload from an HTTPEndpoint is now a Request
* Initial TriggerHttpEndpoint migrations
* Initial server side work
* Migration to add shortcode to RuntimeEnvironment
* Added TriggerHttpEndpointEnvironment table
* Made the httptrigger catalog example runnable
* Progress creating/updating httpEndpoints
* HTTPEndpointEnvironment updating now works
* Added a note about EventRecord HttpEndpoint relationship
* When creating environments, create a shortcode
* Fix for secret not being retrieved correctly
* Request filtering
* Add DOM types to Core
* Early work dealing with a webhook arriving
* Added some todo notes to discuss tomorrow
* Added relations to the endpoint so we can call the user’s server
* Basic handling of immediate responses
* The immediate responses are now going back to the webhook provider
* Handling is now generating events and responding where relevant. Not yet matching filters and doing verification
* Added source to TriggerHttpEndpointEnvironment
* The event filter is now optional
* The source is passed through from the httpEndpointEnv
* Added optional payloadType when ingesting an event
* Added tabler
* Updated @remix-run/testing usage for remix 2
* Fixed org menu header width and added org and project types
* Added custom icons for Runs and Endpoints
* Updated the menu icons
* storybook side menu shows static project and org data
* Updated side menu
* Plumbed in paths
* Updated eventFilter to deal with undefined/null payloads
* Started work on filtering, but it’s not working
* Strict null
* async filtering it working
* Added environment shortcodes to the seedCloud script
* Don’t pass the secret back, added a verifyPayload function that gets called before runs start
* Fix for skipping triggering runs
* The verify callback is now passed through to the trigger so it can be called
* If verifying fails, have a reason which we can show to the user
* Don’t verify if it’s a test run
* Fixed the log in the example code
* Added loader and action to storybook preview
* Fix for svg attribute name issues in EndpointIcon
* Started work on the side menu
* This logic has moved to a separate file, in a different branch
* Don’t get the environments when getting orgs
* Lots of fixes for the new SideMenu
* Refactored some org/project things to fit the new structure better
* Some progress restructuring the app for the new sidebar
* More changes to data loading
* Deleted some old pages. The redirecting is working if you’re on the root but logged in
* Basic styling for the account page. We need a back button
* Fixes for the integrations pages
* Breadcrumb implemented
* Add a border to the bottom of the breadcrumb
* Feedback is working, not in Storybook
* Removed expandSidebar from the Handle
* Org level breadcrumbs
* Getting the project in the org route now works properly
* orgslug route before using the presenter
* More UI improvements
* Removed commented out line
* Use presenters for org and root
* Removed forceActive
* Split the project selector and user menu out
* The side menu items have correct active states
* Fixed the scrolling
* Put the org name in
* Improved the admin features
* Created the HTTP Endpoints list page
* Work on the HTTP endpoint page
* WIP on the HTTP Endpoint page
* Fix for response filter not showing
* Change page URLs to “http-endpoints” (from httpendpoints)
* Slugify the HTTP endpoint id
* Added todos for EventRecord associations
* Removed the Job/Trigger tab for now
* First draft of the help panels
* Rejigged the HTTP endpoint page so it splits out the deployed and not deployed environments
* Removed irrelevant columns
* Added a link to the deployment guide
* Removed the background image from the frameworks pages
Also…
- reordered the frameworks so supported ones come first in the list
- added logos to 3 framework pages that were missing them
- removed lots of un-used imports
* Fixed an issue where the breadcrumb could be hidden on small screens so broken the pages
The breadcrumb is always visible now
* Renamed ‘Onboarding’ to ‘Framework setup’
* Added a new http-endpoint icon to the side menu
* Side menu now has a nice looking scrollbar
* Scrollbar on the usage and billing page updated to the nice style
* Icon and colour updates to the side bar menu items
* Small copy improvement
* Fixed bug with the height of the org menu dropdown
* Page breadcrumbs now don’t include duplicate path links
* renamed uiBorder to ui-border
* Remove the padding from the small menu item if it’s used as a button
* Made the paragraph text a bit smaller
* EventRecord has relationships with TriggerHttpEndpoint/TriggerHttpEndpointEnvironment
* Improved the presenter used by the side menu. Also added the HTTP Endpoint badge count to the sidebar
* The dropdown org menu has the same badge as the side menu
* Improved the Projects page
* Fixed button style miss-match in the jobs table dropdown
* Made the popover dialogs support Headers and improved the regenerate API key popover
* Made the active status of a job more subtle
* Allow the Job to have a link on its Trigger
* Simplified loader because we ditched the Jobs menu. Also showing no runs help panel now
* Fixed type errors
* Pages that are org level can scroll if the content gets too long
* Fixed a redirect bug so you now redirect to the org page
* Updated the shortcode migration, to add additional letters to the end
* Removed the nav bar from the welcome onboarding page
* Only show the countBadge if the number is > 0
* Jobs icon now has a default colour the same as the nav menu
* Capital I for integration
* Clone the request before calling verify
* Separate the checks out so they can be easily logged
* Make the failure reason optional
* Made the verification error look better, if it’s provided
* Working WhatsApp example
* Can set the immediateResponseFilter to null in an update
* Deal with a null immediateResponseFilter
* Cal.com webhook working
* Header height reduced slightly
* Fixed button padding
* Removed un-used imports
* http endpoint table displays a message if you have no entries in the table
* Removed some duplicate text and swapped primary buttons for secondary style
* All ‘HTTP Endpoints’ now ‘HTTP endpoints’
* You have to use a RequestFilter when using HTTP Trigger
* Synchronous eventDispatcher filtering again. Better content-type check
* Stripe example
* Side menu “Usage & Billing” is just “Usage” if self-hosting
* Fix for filtering not working
* Removed filter log
* Fix for jobs being optional and null
* Removed old nav bar from the invite page
* Deleted old nav bar component
* import cleanup
* NoTask message only displays if the run has status “SUCCESS”
* Added a global page loading indicator
* Added documentation buttons to the most important pages
* Added GitHub
* Use “verifyRequestSignature” helpr
* Removed todo
* Fixed the layout of the Rerun Job popover
* Jobs table has margin bottom so it sits off the bottom of the page
* Small improvement to the search bar layout on the integration page
* HTTP endpoint and Trigger concept docs
* Reference docs and JSDocs for HTTP endpoint
* SDK reference docs for HttpEndpoint
* Added an HttpTrigger SDK reference, which mainly just point you elsewhere
* Improved the copy in the help panels
* Removed unused code
* Improved HTTP endpoint comment about setting the secret
* Delete unused asyncFilter
* Fixed the delete modal not having a title
* Always try get the arrayBuffer for an HTTP request, catch and use undefined as the default
* Add extra method to invokeTrigger
* Use a transaction for the secret store
* verifyRequestSignature docs page and JSDocs
* Added verifyRequestSignature docs page to nav
* Warn if they use the same HttpEndpoint id twice
* Added another job using the same HttpEndpoint
* Renamed the api route to http-endpoints from httpendpoints
* Debug bar for VSCode is now in the commandCenter
---------
Co-authored-by: James Ritchie <james@jamesritchie.co.uk>
* Implement manually invokable jobs through the invokeTrigger
Also implemented a job run notification system, that will POST details of a run on completion. This combines with the task callbackUrl system to implement the invokeAndWait
* Document the invoke trigger
* batch invoke and wait
* background fetch timeouts
* Use @whatwg-node/fetch instead of the polyfilled fetch
* Fix some outdated dependencies in webapp
* Improved subtask error propogation messages
* Document the OpenAI changes and the batch invoke stuff
* Fix dequeuing jobs
* Don’t retry the OpenAI completion background task
* Added OpenAI changesets
* Use the new ResumeTaskService in ProcessCallbackTimeout as well
* Auto-yield run execution to help prevent duplicate task executions
* Add auto-yield config to endpoints
* Refactor run execution with buffer and limits
Introduced constants RUN_CHUNK_EXECUTION_BUFFER and MAX_RUN_CHUNK_EXECUTION_LIMIT. Adjusted PerformRunExecutionV2Service to use the new constants to fine-tune execution timings and buffers.
* Add endpoint probing functionality
Added new `RESPONSE_TIMEOUT_STATUS_CODES` in `consts.ts` to manage timeout responses. Additional functions `detectResponseIsTimeout(response: Response)` was added in `endpoint.server.ts` to detect if a response was a timeout based on the status codes from `RESPONSE_TIMEOUT_STATUS_CODES`.
Update actions to use new endpoint probing endpoint service. This allows for the early probing of endpoints to determine if they're up and running.
A new class `ProbeEndpointService` was created in `probeEndpoint.server.ts` which makes HTTP requests to a given endpoint and updates its properties based on the result.
Finally, `detectResponseIsTimeout(response)` is used in `performRunExecutionV2.server.ts` for marking the execution as succeeded when facing a timeout.
* Refactored probe method in EndpointApi class
The probe method of the EndpointApi class has been refactored to remove the error handling part and it now takes a timeout sent from the client directly. The corresponding changes were also made in the ProbeEndpointService and TriggerClient objects to reflect the alterations in the probe method.
The error handling related to the timeout has been removed and the responsibility of handling the timeout has been shifted to the client. Thus, the probe method has been greatly simplified. The `probeEndpoint.server.ts` file was also changed to accommodate the change in behavior of the probe result.
In the `triggerClient.ts` the timeout for probe is now read from the incoming request object. For backward compatibility, if no timeout is provided in the request, the default value of 15 minutes is used.
* Remove performRunExecution v1 enqueue function
* Better document limits and add docs on increasing function timeouts
* Upgrade webapp docker container to use 18.18.2
* force clients to yield when a run is executing in a gracefully shutting down worker
* Renamed task `key` to `cacheKey` and added more task documentation
* Index the `@trigger.dev/sdk` version on Endpoints
* Added EndpointIndex status. Default is PENDING, existing rows are SUCCESS
* Made it easier to create a migration SQL file
* Created a job-catalog file for misconfigured Jobs that should error when running the CLI
* EndpointIndex data and state are now optional
* The Environments page now shows the status of the last refresh
* Improved the UI about endpoints
* Added EndpointIndex error column
* WIP making the endpoint indexing more robust
* Indexing errors are now surfaced
* Improved the error show it shows the job id
* Use “performEndpointIndexing” when you create your first endpoint from the UI
* Use “performEndpointIndexing” for the recurring endpoint checker
* Staging is now auto-indexed every 10 mins too
* Use “performEndpointIndexing” for the webhook
* Moved the throttling to a util
* Removed instructional comments
* Created a reusable retry system with exponential backoff
* Use p-retry for retrying with backoff
* The CLI gets indexing results and displays errors
* Improved the indexing error messages and display in the console
* Use a pre so the Indexing error is correctly split over multiple lines
* Use a db transaction for webhook that triggers endpoint indexing
* Tidied up imports
* Support older versions of the server
* Improved the comment on the misconfigured job
* Changeset: When indexing user's jobs errors are now stored and displayed
* feat: add internal flag to job-run table
* add backfill SQL
* Update internal backfill to only update internal jobs
---------
Co-authored-by: Eric Allam <eallam@icloud.com>
nicktrn