* Denormalize run tags, increase character limit to 128
* WIP realtime subscribing to runs
* extracted the stream stuff into core, made it more reusable
* WIP tags
* Remove tags for now because it’s not support in electric
* Support async iterables, readable stream, and callback style subscription styles
* Remove tags streaming endpoint
* Add realtime rate limits and scope them to the /realtime path
* WIP rate limt per org
* Introduce per org rate limits
* WIP JWT auth
* Move migrations into new internal db package
* Resolve pnpm lock file
* Authenticating to the realtime API with JWTs are working
* realtime in the client
* Created react-hooks package and starting to move stuff in there
* Improve types for hooks
* schema tasks
* Added useBatch hook
* build uploadthing/fal demo and change how run metadata is synced to the server
* tweaks
* WIL realtime concurrency tracking
* Implement test for realtime client using testcontainers
also updated electric to latest version
* Allow customizing the expiration time of the automatic JWT created after triggering a task
* Add support for subscribing to run tags
* Improve auth types and API
* finalize the realtime API
* Fixed some example stuff
* Allow up to 10 run tags
* Remove core from docker-provider tsconfig paths to prevent it from being typechecked
* do the same for the kubernetes provider
* Fixing some typecheck errors
* Fix webapp type errors
* Update @trigger.dev/platform to 1.0.13
* Fix attw error
* Remove from/to in subscribeToRuns query params
* Add tests for the rate limit middleware and add custom JWT rate limits
* turn off webapp test parallelism
* Finish renaming jwt -> publicAccessToken and automatically give the JWT read access to the tags when using trigger
* Add changeset
* Attempt to fix unit tests in CI
* Skip running the auth rate limit middleware tests for now
* Try a beefier machine
* Try and run webapp tests separately
* Setup env vars
* Make sliding window test more reliabile
* Fixed empty env vars overriding in dev runs
* Don’t import package.json anymore
* fix node10 moduleResolution in @trigger.dev/core
* Support self-hosters pushing to a custom registry when running deploy
* dev: Fixed stuck runs when a child run fails with a process exit
* Make some doc notes about known issues and docker hub private repos
* Fix --project-ref when running deploy
* Fix —config option when deploying
* Fixing the flushing/killing process with the new build system
* Add monorepo-react-email e2e test fixture
* Fix issue with emitDecoratorMetadata and tsconfigs with extends
* Got the emit decorator metadata fixture working
* Fixed typechecking yarn e2e CLI tests in monorepos
* Add remote forced externals system, in case we come across another package that cannot be bundled (spurred on by header-generator)
* Remote externals now powered by JSON Hero to be easier to update
* resolve config source files
* Add a —javascript option to init, defaults to typescript
* Add support for prisma typed sql
* Remove msw and retry.interceptFetch
* Add missing code to the openai retries example
* Don’t generate the v3 catalog prisma client during CI
* Fixed v3-catalog task imports
* Remove interceptor usage in task file
* Only import import-in-the-middle hook if there are instrumented packages
* Fix yarn.lock file
* Support custom config file names & paths
* Fix entry point paths on windows
* Support custom conditions
Add support for custom conditions (for bundling and running), to support being able to import `ai/rsc` with the “react-server” condition.
- Fixed an issue where symlinking unresolvable externals after rebuilding caused the build to hang
- Fixed an issue with external not working with subpath exports (e.g. “ai” would not match “ai/rsc”)
- Protect better against build extensions breaking builds
* Add changeset
* Fix passing CLI process.env down to the task processes
* Remove unused import
* reviving the e2e CLI tests
* Another attempt at fixing windows
* yet another windows attempt (yawa)
* Output index child stdout and stderr (yawa)
* normalize import paths for windows
* Added some logging (yawa)
* normalize the loader path as well
* Added some logging to figure out why the entry points aren’t being found on windows
* Fix for entry point detection on windows
* Normalize runner import loader path
* Normalize import paths in dev and make sure rewritten build manifest paths are correct on windows as well
* Various cleanup after windows fixes
* Remove the webapp e2e for now
* Add node10 type resolution support for subpath exports (still does not actually import in Node 10 or Browserify/Parcel)
https://github.com/andrewbranch/example-subpath-exports-ts-compat
* init using templates again but downloaded from the repo this time
* Adding init schedule example
* Support for js init
* init now working with js
* Fix issues with links in terminals that don’t support them. Also skip update check of version starts with 0.0.0
* upgrade @opentelemetry packages to the latest versions
* remove v2 only packages, will be moved to a dedicated repo
* remove more v2 code and run pnpm install
* use the npm yalt package in the webapp
* convert @trigger.dev/core to tshy
* Switch from jest to vitest in @trigger.dev/core
* Fixed core test
* move core-backend code into core subpath export
* convert @trigger.dev/sdk to tshy
* Removed hono
* move core-apps to core/v3/apps, remove core-apps, start converting cli-v3
* Fix up some of the commands
* cli now building and loadable
* using package-json-from-dist to get package version now in core and cli
* dev command WIP
* cleaned up some repetition and structure of the entry point stuff
* bringing back the background worker stuff
* Indexing of the v3 catalog
* getting closer to executing dev runs...
* centralize dev logging using event emitter
* Move indexing to it’s own entry point, simplify code
* dev runs working
* Get instrumentation to work with openai
* debugging achieved internally
* provide worker files as part of the worker creation on the server
* support for cjs and esm javascript
* Fixed timeout
* worker manifest now has the config path
* auto-upgrade config to non-deprecated alternatives
* Adding package preview release
* deployment WIP
* improve the syncEnvVars output and adapt resolveEnvVars
* WIP bun runtime
* WIP bun support
* seed tasks with the machine preset if listed in the config
* deploy run executions WIP, extracted TaskRunProcess into 1 place
* deployed tasks running and executing 🎉
* support for waits and better flushing & process cleanup
* Fixed the heartbeating
* Better warning messages
* Improve and unify the indexing between dev and deploy
* Support for external deps that need node-gyp to build
* build extensions can now install custom packages and run instructions in the image. Also prisma extension now works and also works with multiple schema files
* Add back in the main/types/module to sdk
* dev no longer is Ink/React, grace period for disconnections in dev
* Fix the changeset config
* More changeset fixes
* Remove config packages
* More changeset fixes
* Fixed typescript issues (needed to revert back to zod 3.22.3
* Fix pr_checks workflow
* Remove the prepare script
* Fixed tests and package versions
* Remove cli test script
* Remove packages from tailwind watch paths
* Add repo to public packages
* Just commit the generated files and do the building at dev time
* Try and get pkg.pr.new working
* Try again
* Fix emitDecoratorMetadata importing named export from typescript
* config file backwards compat with export const config
* Fixed issue where import errors weren’t coming through
* p-retry is a prod dep
* typescript needs to be a prod dependency for emitDecoratorMetadata
* Add better debug logging to help track down import-in-the-middle bug
* An external is only considered resolvable if it resolves to the same path as the collected external
* Fix runtime checks to allow >=18.20
* Move extensions to a new build package
* Fixed building packages in dockerfile
* Remove the e2e test from publish workflow for now
* Don’t treat pkg.pr.new versions has needing upgrading
* making sure config handleError works, and discovered path aliases don’t work in config files
* Strip empty string env vars so they accidentally override real values
* Couple of things
* Update version to use preview instead of beta
* Hopefully fix re-attempts with >30s delay
* Match socket emit messages to current latest in main
* Initial guide
* Go back to beta
* Go back to the preview, and update guide to use pr preview tags
* Go back to beta
---------
Co-authored-by: Matt Aitken <matt@mattaitken.com>
* WIP env var management API
* Add import env var API endpoint
* Adding docs and support for using both API keys and PATs when interacting with the env var endpoints
* WIP envvar SDK
* Uploading env vars in a variety of formats now works
* Finish env var endpoints and add resolveEnvVars hook
* Add changeset
* Better error output when an ESM only package is required
* Add better errors for npm package errors, and bring deploy errors to the dev CLI
* Handle depot build errors and write out a temporary file to view the build logs
* Add changeset
* A bunch of changes to the config file:
- No more default export, the config is exported via the `config` named export
- `handleError` is moved to a separate named export
- Added the ability to include additional packages (e.g. wrangler) that aren’t included by default (think CLI usage)
- More reliably install packages in dev
- The config is now built first instead of directly imported, meaning we can now support typescript config files (`trigger.config.ts`)
* Update to trigger.config.ts in `trigger.dev init`
* WIP proxy/deploy
* WIP, registry proxy in express working
* A couple of notes, preparing for indexing
* Move the changes to prod-worker into the new file
* Deploy command working with indexing and runs (docker provider only for now)
* Removed ts-expect-error directive
* Fixed build command
* Support for batch triggers
* Various trace fixes and improvements
* Moved trace into the logger, fixed span cardinality issues with waits and triggers
* Add support for attaching a debugger in dev with —debugger flag
* Introducing Modular Asynchronous Reliable Queueing System (MarQS). Works in dev
* Convert MarQS to using lua and dealing with concurrency
* Simplified the timeout queue and current concurrency is now a set instead of a flat value (to support idempotency)
* Implement task heartbeating and reconnect the background workers CLI when the websocket connection reconnects
* Start adding internal telemetry support for the server
* Get env vars to work in dev and implement prisma tracing in webapp
* Cleanup telemetry and implement it in the consumer
* Implement dequeuing a message from a parent shared queue
* Implement a custom logger exporter instead of using console log exporter
* Use node instead of shell for generating protocol buffer code
* Propogate trace context into debug logs, and allow turning off logger exporter through env vars
* Switch to using baselime for internal otel data
* Make orgMember optional to fix type issues
* Provide the CLI dev env vars through the CLI, don’t build dotenv into facade
* Removed the logger import
* Address Matt’s comments
* Addressing more of Matt’s comments
* Handle sending an execution after a websocket connection closes
* Remove auth from the env attributes to prevent obfuscation
* Setup basic storybook page
* storybook preview pages don’t have padding
* Added a storybook story for the side menu
* WIP on a new side menu component
* New variant for the side menu button style
* Side menu items can now recieve a badge ‘count’
* Storybook side menu update
* Added a new font size
* you can now set the spacing between an icon and the text
* Moving the error and count badge to the right side of the button
* Added a menu sub item button style
* added sub items to the menu and all the rest of the items
* Proper background class used
* Added a side menu header and main org header
* Removed the no-mobile blocker overlay so you can use the site on a phone
* Better section name
* Created a cusom popover trigger that can take anything as children. Also the popover titles can now take a variant
* Added a popover for the user avatar menu item
* Removed the tool tip
* Arrow Popover can no also go full width
* neater conditiona;
* Removed unnessary div. And error icon can display an error message
* menu item buttons can take a variant
* Added placeholder items for the org dropdown
* add a max height to the org menu popover and handle scrollbars
* Added a border if the menu scrolls when you have lots of project items
* warning badges display tooltips if you give them some error text
* Removed the sticky header so the scrollbar looks better
* Fixed a layout issue from removing the sticky header
* Better spacing between lists
* Popover can take any react component icon and an icon classname
* Side menu headers now have a dropdown by clicking the ellipsis icon
* Side menu items can take an icon colour
* Updated styling for the dropdown Popover menus
* Installed Tabler icons
* Updated some icons to use Tabler
* Started defining the HttpTrigger class
* RequestFilterSchema is part of core
* Use the RequestFilterSchema from core
* EventSpecificationExample now derived from schema
* Need to define the class like EventTrigger
* Catalog reference that uses the httpTrigger
* The logic for the httpTrigger is in the separate file, just call it from the client using defineHttpTrigger
* Basic HttpTrigger
* Import fixes
* Transforming httpTrigger to the format we want
* Initial httpTrigger types working
* Added HTTP generic type
* Added some notes
* Rejigged some things to support verifying payloads
* Use new security function for verifying
* Added truncating text support if side menu names get long
* Renamed some stuff
* Removed test copy
* HttpEndpoin that isn’t a trigger
* Renamed hostname to source
* Created HttpEndpoint as a top-level concept, it produces triggers using onRequest
* Started working on registration of the httpEndpoints
* Removed the new TriggerMetadata for now, http will use static
* Added PayloadType to EventRecord. Default is JSON.
* The payload from an HTTPEndpoint is now a Request
* Initial TriggerHttpEndpoint migrations
* Initial server side work
* Migration to add shortcode to RuntimeEnvironment
* Added TriggerHttpEndpointEnvironment table
* Made the httptrigger catalog example runnable
* Progress creating/updating httpEndpoints
* HTTPEndpointEnvironment updating now works
* Added a note about EventRecord HttpEndpoint relationship
* When creating environments, create a shortcode
* Fix for secret not being retrieved correctly
* Request filtering
* Add DOM types to Core
* Early work dealing with a webhook arriving
* Added some todo notes to discuss tomorrow
* Added relations to the endpoint so we can call the user’s server
* Basic handling of immediate responses
* The immediate responses are now going back to the webhook provider
* Handling is now generating events and responding where relevant. Not yet matching filters and doing verification
* Added source to TriggerHttpEndpointEnvironment
* The event filter is now optional
* The source is passed through from the httpEndpointEnv
* Added optional payloadType when ingesting an event
* Added tabler
* Updated @remix-run/testing usage for remix 2
* Fixed org menu header width and added org and project types
* Added custom icons for Runs and Endpoints
* Updated the menu icons
* storybook side menu shows static project and org data
* Updated side menu
* Plumbed in paths
* Updated eventFilter to deal with undefined/null payloads
* Started work on filtering, but it’s not working
* Strict null
* async filtering it working
* Added environment shortcodes to the seedCloud script
* Don’t pass the secret back, added a verifyPayload function that gets called before runs start
* Fix for skipping triggering runs
* The verify callback is now passed through to the trigger so it can be called
* If verifying fails, have a reason which we can show to the user
* Don’t verify if it’s a test run
* Fixed the log in the example code
* Added loader and action to storybook preview
* Fix for svg attribute name issues in EndpointIcon
* Started work on the side menu
* This logic has moved to a separate file, in a different branch
* Don’t get the environments when getting orgs
* Lots of fixes for the new SideMenu
* Refactored some org/project things to fit the new structure better
* Some progress restructuring the app for the new sidebar
* More changes to data loading
* Deleted some old pages. The redirecting is working if you’re on the root but logged in
* Basic styling for the account page. We need a back button
* Fixes for the integrations pages
* Breadcrumb implemented
* Add a border to the bottom of the breadcrumb
* Feedback is working, not in Storybook
* Removed expandSidebar from the Handle
* Org level breadcrumbs
* Getting the project in the org route now works properly
* orgslug route before using the presenter
* More UI improvements
* Removed commented out line
* Use presenters for org and root
* Removed forceActive
* Split the project selector and user menu out
* The side menu items have correct active states
* Fixed the scrolling
* Put the org name in
* Improved the admin features
* Created the HTTP Endpoints list page
* Work on the HTTP endpoint page
* WIP on the HTTP Endpoint page
* Fix for response filter not showing
* Change page URLs to “http-endpoints” (from httpendpoints)
* Slugify the HTTP endpoint id
* Added todos for EventRecord associations
* Removed the Job/Trigger tab for now
* First draft of the help panels
* Rejigged the HTTP endpoint page so it splits out the deployed and not deployed environments
* Removed irrelevant columns
* Added a link to the deployment guide
* Removed the background image from the frameworks pages
Also…
- reordered the frameworks so supported ones come first in the list
- added logos to 3 framework pages that were missing them
- removed lots of un-used imports
* Fixed an issue where the breadcrumb could be hidden on small screens so broken the pages
The breadcrumb is always visible now
* Renamed ‘Onboarding’ to ‘Framework setup’
* Added a new http-endpoint icon to the side menu
* Side menu now has a nice looking scrollbar
* Scrollbar on the usage and billing page updated to the nice style
* Icon and colour updates to the side bar menu items
* Small copy improvement
* Fixed bug with the height of the org menu dropdown
* Page breadcrumbs now don’t include duplicate path links
* renamed uiBorder to ui-border
* Remove the padding from the small menu item if it’s used as a button
* Made the paragraph text a bit smaller
* EventRecord has relationships with TriggerHttpEndpoint/TriggerHttpEndpointEnvironment
* Improved the presenter used by the side menu. Also added the HTTP Endpoint badge count to the sidebar
* The dropdown org menu has the same badge as the side menu
* Improved the Projects page
* Fixed button style miss-match in the jobs table dropdown
* Made the popover dialogs support Headers and improved the regenerate API key popover
* Made the active status of a job more subtle
* Allow the Job to have a link on its Trigger
* Simplified loader because we ditched the Jobs menu. Also showing no runs help panel now
* Fixed type errors
* Pages that are org level can scroll if the content gets too long
* Fixed a redirect bug so you now redirect to the org page
* Updated the shortcode migration, to add additional letters to the end
* Removed the nav bar from the welcome onboarding page
* Only show the countBadge if the number is > 0
* Jobs icon now has a default colour the same as the nav menu
* Capital I for integration
* Clone the request before calling verify
* Separate the checks out so they can be easily logged
* Make the failure reason optional
* Made the verification error look better, if it’s provided
* Working WhatsApp example
* Can set the immediateResponseFilter to null in an update
* Deal with a null immediateResponseFilter
* Cal.com webhook working
* Header height reduced slightly
* Fixed button padding
* Removed un-used imports
* http endpoint table displays a message if you have no entries in the table
* Removed some duplicate text and swapped primary buttons for secondary style
* All ‘HTTP Endpoints’ now ‘HTTP endpoints’
* You have to use a RequestFilter when using HTTP Trigger
* Synchronous eventDispatcher filtering again. Better content-type check
* Stripe example
* Side menu “Usage & Billing” is just “Usage” if self-hosting
* Fix for filtering not working
* Removed filter log
* Fix for jobs being optional and null
* Removed old nav bar from the invite page
* Deleted old nav bar component
* import cleanup
* NoTask message only displays if the run has status “SUCCESS”
* Added a global page loading indicator
* Added documentation buttons to the most important pages
* Added GitHub
* Use “verifyRequestSignature” helpr
* Removed todo
* Fixed the layout of the Rerun Job popover
* Jobs table has margin bottom so it sits off the bottom of the page
* Small improvement to the search bar layout on the integration page
* HTTP endpoint and Trigger concept docs
* Reference docs and JSDocs for HTTP endpoint
* SDK reference docs for HttpEndpoint
* Added an HttpTrigger SDK reference, which mainly just point you elsewhere
* Improved the copy in the help panels
* Removed unused code
* Improved HTTP endpoint comment about setting the secret
* Delete unused asyncFilter
* Fixed the delete modal not having a title
* Always try get the arrayBuffer for an HTTP request, catch and use undefined as the default
* Add extra method to invokeTrigger
* Use a transaction for the secret store
* verifyRequestSignature docs page and JSDocs
* Added verifyRequestSignature docs page to nav
* Warn if they use the same HttpEndpoint id twice
* Added another job using the same HttpEndpoint
* Renamed the api route to http-endpoints from httpendpoints
* Debug bar for VSCode is now in the commandCenter
---------
Co-authored-by: James Ritchie <james@jamesritchie.co.uk>
commit c028db229cece2bf7d7acf04e7b1c08fea61798c
Author: Matt Aitken <matt@mattaitken.com>
Date: Fri Oct 13 16:04:54 2023 +0100
Some fixes for getting the endpoint id
commit e31b7ac5f2d451f70e3d42a3fb3e5f718d93485b
Author: Matt Aitken <matt@mattaitken.com>
Date: Fri Oct 13 15:39:18 2023 +0100
Added a link in the instructions if Node isn’t detected
commit 1a59cced775fbeac3e5b94e0c151b490c838d8f8
Author: Matt Aitken <matt@mattaitken.com>
Date: Fri Oct 13 15:39:05 2023 +0100
Fixed some conflict issues
commit 77263ed26bcf7025071ac0faf15fb867fff245d0
Author: Matt Aitken <matt@mattaitken.com>
Date: Fri Oct 13 15:38:56 2023 +0100
Added a changeset
commit 1a7df3f37f0c65d68cb9a9fe7ac50328bbb4d879
Author: GuyBorderless <100129659+guy-borderless@users.noreply.github.com>
Date: Fri Oct 13 17:31:01 2023 +0300
feat: enable deno (#531)
* adding deno reference project and JSRuntime file
* accessing runtime specific functions through JsRuntime
* adding support for both deno.json and deno.jsonc
* no automatic setup for deno projects currently
---------
Co-authored-by: Guy Kedem <guy@propertiz.net>
Co-authored-by: Matt Aitken <matt@mattaitken.com>
* feat: BYO Auth
Define client-side auth resolvers to be able to supply custom authentication credentials for integrations before a run is performed
- Added new defineAuthResolver
- Update all integrations to support the new auth resolvers
- Strip internal symbols from .d.ts in integrations and trigger-sdk
- Added BYO Auth docs
- Update Dynamic Schedule to support associated account IDs
- Create external accounts just-in-time
- Added Account ID field to test job when there are external auth integrations
- Show Account ID on run dashboard
- Added new Run error state called “Unresolved auth”
* Added changeset
* Remove @internal from TriggerIntegration public methods
* Add void to the result union
* DynamicTriggers now work with the new BYO auth system, and added a bunch of docs and docs changes
* Add additional key material for registering dynamic trigger task
* Add new define* instance methods to the overview
* Setup project-wide prettier
* Remove old workspace file
* Remove old debugging directives
* New top-level .prettierignore
* Updated Prettier config settings
* Contrubuting guide: Fix for some bad code blocks
* Added more ignores
* Improved the format script command
* printWidth set to 100
* Formatted entire repo (pnpm run format)
Eric Allam