Commit Graph

464 Commits

Author SHA1 Message Date
github-actions[bot] 1567239718 chore: Update version for release (#771)
* chore: Update version for release

* Release lockfile

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: nicktrn <55853254+nicktrn@users.noreply.github.com>
2023-12-04 14:14:41 +00:00
Eric Allam 5d00fc7cdb Update pnpm lock file for release 2023-11-30 21:54:43 +00:00
Eric Allam 2f755158b4 Updated pnpm lock file 2023-11-29 11:25:27 +00:00
Eric Allam bd4bc51daa Update plain (#762)
* updated the plain SDK

* Updated the action to support the new Threads SDK

* latest lock file

---------

Co-authored-by: James Ritchie <james@jamesritchie.co.uk>
2023-11-29 10:58:06 +00:00
Eric Allam ff540c9e4a Revert "Update plain (#761)"
🚀 Publish Trigger.dev Docker / units (push) Failing after 0s
🚀 Publish Trigger.dev Docker / typecheck (push) Failing after 3s
🚀 Publish Trigger.dev Docker / e2e (push) Failing after 5s
🚀 Publish Trigger.dev Docker / publish (push) Has been skipped
This reverts commit 3f217ff3e0.
2023-11-29 10:51:13 +00:00
Eric Allam 3f217ff3e0 Update plain (#761)
🚀 Publish Trigger.dev Docker / units (push) Failing after 0s
🚀 Publish Trigger.dev Docker / e2e (push) Failing after 0s
🚀 Publish Trigger.dev Docker / typecheck (push) Failing after 5s
🚀 Publish Trigger.dev Docker / publish (push) Has been skipped
* updated the plain SDK

* Updated the action to support the new Threads SDK

---------

Co-authored-by: James Ritchie <james@jamesritchie.co.uk>
2023-11-29 10:49:37 +00:00
Eric Allam 6ebd435e81 Feature: Run execution concurrency limits (#750)
* WIP execution concurrency controls implemented via Redis

- Split up resuming a run and executing a run
- Added some new statuses to better show what is going on in a run
- Removed preprocessing runs

* WIP

* Convert to using ZSETs and adding env vars

* Removed unused import

* Improve run number generation using advistory locks, and only on start

* More execution concurrency stuff

* Add support for job concurrency limits and concurrency limit groups

* Create wild-swans-battle.md

* Increase slots refresh timeout to 10s

* Try to fix Redis connection issues

* Don’t be so strict about the APP_ENV

* Add the blank tls option to the normal redis client as well

* Add docs
2023-11-28 16:21:06 +00:00
Eric Allam dbc2e3f713 Refreshed pnpm lock 2023-11-28 14:53:37 +00:00
nicktrn 096151c014 Fix: Shopify subpath imports, enhance docs (#755)
🚀 Publish Trigger.dev Docker / typecheck (push) Failing after 0s
🚀 Publish Trigger.dev Docker / units (push) Failing after 0s
🚀 Publish Trigger.dev Docker / e2e (push) Failing after 0s
🚀 Publish Trigger.dev Docker / publish (push) Has been skipped
* Fix invalid subpath imports

* Remove @trigger.dev/tsup dep

* Update lockfile

* Small github docs fix

* Pull runtime adaptor import out of integration

* Fix io.store docs examples

* Add sendEvents to task library

* Add KV to task library and small fixes

* Suppress duplicate http endpoint warnings

* Changeset

* Add task returns
2023-11-27 16:03:03 +00:00
Eric Allam d56b2eded5 Features: new login page (#754)
* Initial commit and WIP new login page

* added wrapper to fix layout

* Improved the layout and style of the page

* Added a new extra-large button style (and added it to storybook)

* Moved the cusom onboarding style tooltip to the Tooltip component

* Populated the bento grid with items

* Imrpoved the quote styling

* Tooltip doesn’t stay if you mouse over it

* Added more content including a list of quote that cycle on page load

* New quote mark positioning

* Added the rest of the content to the tooltips

* Made the bento grid responsive

* Moved the login page layout to a seaparte component

* Login pages now use the new layout component

* Extra large button style tweak

* Match the large input style with the large button style

* Cleaned up imports and made the Ts & Cs text the same for magic links

* Layout fix

* Quotes don’t push content off the screen as much

* Added openai integration to the grid

* Make the quote fit if the screen height is narrower

* Style update to the magic link sent state

* Icon colour updates

* fixed text centering on mobile

* Icons are more responsive

* removed unnecessary w-fulls

* Added 2 onboarding questions

* Revalidate form on submit, not input

* Typo fixes

* Name field focuses by default

* Added conform to the employees question

* Update pnpm lock file

* Implement referral source and company size on managed cloud

* Fixed some type errors

---------

Co-authored-by: James Ritchie <james@jamesritchie.co.uk>
2023-11-24 14:08:50 +00:00
nicktrn 067e19fec9 Feature: New Webhook Triggers, KV Store, and Shopify integration (#745)
* Attach webhook to http endpoint

* Create internal webhook handler job

* Fix delivery

* Add webhook trigger

* Index webhooks

* Fix webhook connect

* Webhook activation

* Rename to TriggerWebhook

* Save webhooks to index

* Extend register payload

* Revert attachSource

* Add update webhook helper

* Use crud interface for registration

* Remove filter from params

* Destructure options

* Missing ohash dep

* Chainable trigger filter

* Optional request verification

* Handler call with context

* Update config on register success

* Rename to webhookData

* Add basic key-value store

* Improve kv store

* Webhook trigger UI

* Webhook delivery UI

* Some fixes

* Airtable experiments

* Fix tabs transitions

* Only run register if config differs

* Use subtasks for delivery

* Missing dep

* Error handing and fixes

* Namespace cursor

* Fix hmac verification

* Gut Airtable

* Lockfile

* Template optional api key

* Template event sources and icons

* Template remove filter from params

* Fix models template

* Template fixme

* Fix webhook trigger header

* Actually send params for registration..

* Add back oauthed client

* Verify signature header encoding param

* Fix delivery context

* Webhook create retry with prior delete

* Webhook source type fixes

* Webhook config merge

* Update webhook icon

* Verify webhook task options

* Rename handler to generateEvents

* Send event icons

* Shopify with webhooks

* Shopify job-catalog entry

* Lockfile

* Add webhook migration

* Better icons

* Type fixes

* Make param type optional

* Scope type

* More examples

* More schemas

* Trigger catalog

* Bump versions

* Just a few more events they said

* Simplify session and config

* Typed serializer

* Fix payload type

* Remove unused getters

* Reduce logging output

* Rest resource tasks

* Lockfile

* Bump version

* Remaining triggers

* Fix event types

* Fix import

* Link from webhook trigger to http endpoint

* Use rest resource tasks for crud

* Fix rest save type

* Improve KV types

* Improve attach webhook logging

* Make header schema more lenient

* Fix save type again

* Final error callback

* Shorten keys

* Remove ohash diff

* Disable Airtable webhooks

* Webhook environments

* Fix shopify error import

* Link to integration and http endpoint

* Integration catalog entry

* Disable oauth

* Require api key, simplify client secret

* Move trigger types

* Update catalog example

* Small KV endpoint refactor

* Fix custom migration

* Webhook environment migration

* Improve crud types

* Remove unused schema

* Fix crud context type

* KV limits and HTTP verbs

* KV improvements

* Jobless webhook delivery

* Airtable delivery fixes

* Make deliveries sexy again

* Some docs

* Fix payload types.. again

* Final payload pass

* Schema cleanup

* Refactor handler service

* Remove more stale schemas

* registerJobNamespace import

* Expose KV on TriggerClient

* Remove dummy click handler

* Remove unused components

* Fix delivery pagination

* Enable registration rerun button

* Comment out registration route action handler

* Rename KV as not tied to IO anymore

* Swap api with trigger client

* Disabled fields param

* KV docs

* Changeset

* Shopify docs fixes
2023-11-24 10:22:12 +00:00
Eric Allam 59277731c4 Fix pnpm lock file 2023-11-16 16:32:14 +00:00
Eric Allam 756024da78 Add support for Run Notifications #401 (#738)
* Add support for Run Notifications #401

* Fixed typo and added changeset
2023-11-16 14:19:42 +00:00
Dan 888344797e Added code examples and modified blank state on the integrations & APIs page (#736)
* Added more API icons

* Updated the title

* Added resources.codeexample.ts to pull in example code from github

* Added regex to remove redundant express code from the examples

* Added cal examples

* Code changes onSubmit

* Improved the code loading state

* Removed request integration button

* Improved the Customhelp copy

* Fixed styling issues and removed imports

* Added all the code examples

* Don’t show title if there’s 1 or less examples

* Updated copy for the non example version

* Added example code to the apis without examples, added a focus state

* Fixed titles

* Got rid of the phantom gap

* Made the copy more generic so it works on the integration pages as well

* Removed import

* Added view examples button to the integration pages

* Removed fetch/sdk option from integration sheet

* Apply code example regex on the server

---------

Co-authored-by: Matt Aitken <matt@mattaitken.com>
2023-11-15 17:38:24 +00:00
Eric Allam 14429b8b66 Fix “transaction already closed” error when running tasks (#735) 2023-11-14 23:22:34 +00:00
Matt Aitken 2b48e6d04b Feature: sendEvent can use an AWS SQS queue to increase reliability and throughput (#729)
* Initial commit (by create-cloudflare CLI)

* Changed the prettier rules

* Started writing Readme and got dev working

* Validate the request, parse the event and respond with the correct format

* Use ulidx instead of ulid in core

* Removed some boilerplate and use new function for sendEvent

* Removed old ulid package

* When handling a webhook, it’s not an error if the HTTP endpoint environment isn’t found

* If the env vars aren’t set, log the request and pass it through

* Pass requests through. Setup the env vars

* Flipped the API key detection because we can enforce that it’s the shape of a private API key

* Progress with sendEvent proxy

* Events are being put on the SQS queue

* The SQS event queue is working

* Make sure there’s a timestamp before enqueuing the evnet

* Log the sqs_event, not API key

* Added a bit more detail to the readme

* Improved the Cloudflare logs

* We don’t need the global.window hack for AWS, just globalThis

* Removed commented out wrangler.toml values

* Reworked the proxy to make it easier to add more endpoints

* Use json utility to send Responses

* Start work on proxying bulk events
2023-11-13 15:08:01 +00:00
Eric Allam af57bc208a Update pnpm lock file 2023-11-10 16:54:06 +00:00
Eric Allam 9a7c08c26a Improvements: Fix dangling SSE issue and compression memory leak (#733)
🚀 Publish Trigger.dev Docker / typecheck (push) Failing after 1s
🚀 Publish Trigger.dev Docker / units (push) Failing after 0s
🚀 Publish Trigger.dev Docker / e2e (push) Failing after 0s
🚀 Publish Trigger.dev Docker / publish (push) Has been skipped
* Downgrade to remix-auth-email-link to remove yarn dependency

* Turn off the pg listen service for now

* Add snapshot admin route

* A couple logger fixes

* Add ability to disable compression

* Add ability to disable SSE

* Fix SSE memory leak + DB load issue
2023-11-10 16:02:53 +00:00
Eric Allam cb1825bfaf Add OpenAI support for 4.16.0 (#726)
* Add OpenAI support for 4.16.0

* Add support for background polling and use that in OpenAI integration to power assistants

* Much improved OpenAI docs

* Added backgroundPoll docs

* Implements waitForEvent and added docs for more built in tasks

* Add sendEvent API referenc

* Write the task libray

* Add changeset and warning for waitForEvent
2023-11-09 16:59:58 +00:00
Matt Aitken dc8c8edf04 Latest lockfile 2023-11-07 16:27:31 +00:00
Eric Allam 42185bc36b Upgrade resend integration to use resend-node 2.0.0 (#724)
* Upgrade resend integration to use resend-node 2.0.0, fixes issue with Next.js 14

* Create mighty-dodos-study.md
2023-11-06 18:30:37 +00:00
Matt Aitken 9c6838e5c8 HTTP endpoints, HTTP triggers and new sidebar (#720)
* Setup basic storybook page

* storybook preview pages don’t have padding

* Added a storybook story for the side menu

* WIP on a new side menu component

* New variant for the side menu button style

* Side menu items can now recieve a badge ‘count’

* Storybook side menu update

* Added a new font size

* you can now set the spacing between an icon and the text

* Moving the error and count badge to the right side of the button

* Added a menu sub item button style

* added sub items to the menu and all the rest of the items

* Proper background class used

* Added a side menu header and main org header

* Removed the no-mobile blocker overlay so you can use the site on a phone

* Better section name

* Created a cusom popover trigger that can take anything as children. Also the popover titles can now take a variant

* Added a popover for the user avatar menu item

* Removed the tool tip

* Arrow Popover can no also go full width

* neater conditiona;

* Removed unnessary div. And error icon can display an error message

* menu item buttons can take a variant

* Added placeholder items for the org dropdown

* add a max height to the org menu popover and handle scrollbars

* Added a border if the menu scrolls when you have lots of project items

* warning badges display tooltips if you give them some error text

* Removed the sticky header so the scrollbar looks better

* Fixed a layout issue from removing the sticky header

* Better spacing between lists

* Popover can take any react component icon and an icon classname

* Side menu headers now have a dropdown by clicking the ellipsis icon

* Side menu items can take an icon colour

* Updated styling for the dropdown Popover menus

* Installed Tabler icons

* Updated some icons to use Tabler

* Started defining the HttpTrigger class

* RequestFilterSchema is part of core

* Use the RequestFilterSchema from core

* EventSpecificationExample now derived from schema

* Need to define the class like EventTrigger

* Catalog reference that uses the httpTrigger

* The logic for the httpTrigger is in the separate file, just call it from the client using defineHttpTrigger

* Basic HttpTrigger

* Import fixes

* Transforming httpTrigger to the format we want

* Initial httpTrigger types working

* Added HTTP generic type

* Added some notes

* Rejigged some things to support verifying payloads

* Use new security function for verifying

* Added truncating text support if side menu names get long

* Renamed some stuff

* Removed test copy

* HttpEndpoin that isn’t a trigger

* Renamed hostname to source

* Created HttpEndpoint as a top-level concept, it produces triggers using onRequest

* Started working on registration of the httpEndpoints

* Removed the new TriggerMetadata for now, http will use static

* Added PayloadType to EventRecord. Default is JSON.

* The payload from an HTTPEndpoint is now a Request

* Initial TriggerHttpEndpoint migrations

* Initial server side work

* Migration to add shortcode to RuntimeEnvironment

* Added TriggerHttpEndpointEnvironment table

* Made the httptrigger catalog example runnable

* Progress creating/updating httpEndpoints

* HTTPEndpointEnvironment updating now works

* Added a note about EventRecord HttpEndpoint relationship

* When creating environments, create a shortcode

* Fix for secret not being retrieved correctly

* Request filtering

* Add DOM types to Core

* Early work dealing with a webhook arriving

* Added some todo notes to discuss tomorrow

* Added relations to the endpoint so we can call the user’s server

* Basic handling of immediate responses

* The immediate responses are now going back to the webhook provider

* Handling is now generating events and responding where relevant. Not yet matching filters and doing verification

* Added source to TriggerHttpEndpointEnvironment

* The event filter is now optional

* The source is passed through from the httpEndpointEnv

* Added optional payloadType when ingesting an event

* Added tabler

* Updated @remix-run/testing usage for remix 2

* Fixed org menu header width and added org and project types

* Added custom icons for Runs and Endpoints

* Updated the menu icons

* storybook side menu shows static project and org data

* Updated side menu

* Plumbed in paths

* Updated eventFilter to deal with undefined/null payloads

* Started work on filtering, but it’s not working

* Strict null

* async filtering it working

* Added environment shortcodes to the seedCloud script

* Don’t pass the secret back, added a verifyPayload function that gets called before runs start

* Fix for skipping triggering runs

* The verify callback is now passed through to the trigger so it can be called

* If verifying fails, have a reason which we can show to the user

* Don’t verify if it’s a test run

* Fixed the log in the example code

* Added loader and action to storybook preview

* Fix for svg attribute name issues in EndpointIcon

* Started work on the side menu

* This logic has moved to a separate file, in a different branch

* Don’t get the environments when getting orgs

* Lots of fixes for the new SideMenu

* Refactored some org/project things to fit the new structure better

* Some progress restructuring the app for the new sidebar

* More changes to data loading

* Deleted some old pages. The redirecting is working if you’re on the root but logged in

* Basic styling for the account page. We need a back button

* Fixes for the integrations pages

* Breadcrumb implemented

* Add a border to the bottom of the breadcrumb

* Feedback is working, not in Storybook

* Removed expandSidebar from the Handle

* Org level breadcrumbs

* Getting the project in the org route now works properly

* orgslug route before using the presenter

* More UI improvements

* Removed commented out line

* Use presenters for org and root

* Removed forceActive

* Split the project selector and user menu out

* The side menu items have correct active states

* Fixed the scrolling

* Put the org name in

* Improved the admin features

* Created the HTTP Endpoints list page

* Work on the HTTP endpoint page

* WIP on the HTTP Endpoint page

* Fix for response filter not showing

* Change page URLs to “http-endpoints” (from httpendpoints)

* Slugify the HTTP endpoint id

* Added todos for EventRecord associations

* Removed the Job/Trigger tab for now

* First draft of the help panels

* Rejigged the HTTP endpoint page so it splits out the deployed and not deployed environments

* Removed irrelevant columns

* Added a link to the deployment guide

* Removed the background image from the frameworks pages

Also…
- reordered the frameworks so supported ones come first in the list
- added logos to 3 framework pages that were missing them
- removed lots of un-used imports

* Fixed an issue where the breadcrumb could be hidden on small screens so broken the pages

The breadcrumb is always visible now

* Renamed ‘Onboarding’ to ‘Framework setup’

* Added a new http-endpoint icon to the side menu

* Side menu now has a nice looking scrollbar

* Scrollbar on the usage and billing page updated to the nice style

* Icon and colour updates to the side bar menu items

* Small copy improvement

* Fixed bug with the height of the org menu dropdown

* Page breadcrumbs now don’t include duplicate path links

* renamed uiBorder to ui-border

* Remove the padding from the small menu item if it’s used as a button

* Made the paragraph text a bit smaller

* EventRecord has relationships with TriggerHttpEndpoint/TriggerHttpEndpointEnvironment

* Improved the presenter used by the side menu. Also added the HTTP Endpoint badge count to the sidebar

* The dropdown org menu has the same badge as the side menu

* Improved the Projects page

* Fixed button style miss-match in the jobs table dropdown

* Made the popover dialogs support Headers and improved the regenerate API key popover

* Made the active status of a job more subtle

* Allow the Job to have a link on its Trigger

* Simplified loader because we ditched the Jobs menu. Also showing no runs help panel now

* Fixed type errors

* Pages that are org level can scroll if the content gets too long

* Fixed a redirect bug so you now redirect to the org page

* Updated the shortcode migration, to add additional letters to the end

* Removed the nav bar from the welcome onboarding page

* Only show the countBadge if the number is > 0

* Jobs icon now has a default colour the same as the nav menu

* Capital I for integration

* Clone the request before calling verify

* Separate the checks out so they can be easily logged

* Make the failure reason optional

* Made the verification error look better, if it’s provided

* Working WhatsApp example

* Can set the immediateResponseFilter to null in an update

* Deal with a null immediateResponseFilter

* Cal.com webhook working

* Header height reduced slightly

* Fixed button padding

* Removed un-used imports

* http endpoint table displays a message if you have no entries in the table

* Removed some duplicate text and swapped primary buttons for secondary style

* All ‘HTTP Endpoints’ now ‘HTTP endpoints’

* You have to use a RequestFilter when using HTTP Trigger

* Synchronous eventDispatcher filtering again. Better content-type check

* Stripe example

* Side menu “Usage & Billing” is just “Usage” if self-hosting

* Fix for filtering not working

* Removed filter log

* Fix for jobs being optional and null

* Removed old nav bar from the invite page

* Deleted old nav bar component

* import cleanup

* NoTask message only displays if the run has status “SUCCESS”

* Added a global page loading indicator

* Added documentation buttons to the most important pages

* Added GitHub

* Use “verifyRequestSignature” helpr

* Removed todo

* Fixed the layout of the Rerun Job popover

* Jobs table has margin bottom so it sits off the bottom of the page

* Small improvement to the search bar layout on the integration page

* HTTP endpoint and Trigger concept docs

* Reference docs and JSDocs for HTTP endpoint

* SDK reference docs for HttpEndpoint

* Added an HttpTrigger SDK reference, which mainly just point you elsewhere

* Improved the copy in the help panels

* Removed unused code

* Improved HTTP endpoint comment about setting the secret

* Delete unused asyncFilter

* Fixed the delete modal not having a title

* Always try get the arrayBuffer for an HTTP request, catch and use undefined as the default

* Add extra method to invokeTrigger

* Use a transaction for the secret store

* verifyRequestSignature docs page and JSDocs

* Added verifyRequestSignature docs page to nav

* Warn if they use the same HttpEndpoint id twice

* Added another job using the same HttpEndpoint

* Renamed the api route to http-endpoints from httpendpoints

* Debug bar for VSCode is now in the commandCenter

---------

Co-authored-by: James Ritchie <james@jamesritchie.co.uk>
2023-11-06 15:32:40 +00:00
Eric Allam 14a2f4cc94 Fix lock file 2023-11-03 11:26:16 +00:00
Eric Allam 620b83832b Implement manually invokable jobs through the invokeTrigger (#700)
* Implement manually invokable jobs through the invokeTrigger

Also implemented a job run notification system, that will POST details of a run on completion. This combines with the task callbackUrl system to implement the invokeAndWait

* Document the invoke trigger

* batch invoke and wait

* background fetch timeouts

* Use @whatwg-node/fetch instead of the polyfilled fetch

* Fix some outdated dependencies in webapp

* Improved subtask error propogation messages

* Document the OpenAI changes and the batch invoke stuff

* Fix dequeuing jobs

* Don’t retry the OpenAI completion background task

* Added OpenAI changesets

* Use the new ResumeTaskService in ProcessCallbackTimeout as well
2023-11-03 11:05:00 +00:00
Eric Allam 67b5365d50 Fix pnpm lock 2023-11-01 10:16:52 +00:00
Eric Allam c1710ae72d fix: Buffer import breaks @trigger.dev/react (#714) 2023-10-31 15:28:55 +00:00
nicktrn 2c91c492df Feature: Cross-instance notification system (#689)
* Add notification system

* Graceful worker shutdown on migration notification

* Add notification catalog

* Rename pgListen call to on
2023-10-30 17:08:16 +00:00
Eric Allam 6881de84ab Fix pnpm lock file 2023-10-26 20:04:03 +01:00
Eric Allam 55946fcdb7 Update @trigger.dev/nextjs to support Next.js 14 (#690) 2023-10-26 19:45:19 +01:00
Eric Allam c0f9f10b07 Upgrade to Remix v2.1.0 2023-10-26 15:54:21 +01:00
Eric Allam e80aa2826d @trigger.dev/express: Remove web-fetch dependency and use native node Request/Response 2023-10-26 13:42:20 +01:00
Eric Allam d7c1e98e5e Update pnpm lock 2023-10-24 17:49:39 +01:00
Eric Allam 36a390449a @trigger.dev/openai: Upgrade to openai-node SDK 4.13.0, add support for request options and defaultQuery/defaultHeaders 2023-10-24 17:45:43 +01:00
Eric Allam 408785b8ef Update pnpm lock file 2023-10-24 13:41:35 +01:00
Eric Allam 044d38e390 Improvement: Auto Execution Yielding (#612)
* Auto-yield run execution to help prevent duplicate task executions

* Add auto-yield config to endpoints

* Refactor run execution with buffer and limits

Introduced constants RUN_CHUNK_EXECUTION_BUFFER and MAX_RUN_CHUNK_EXECUTION_LIMIT. Adjusted PerformRunExecutionV2Service to use the new constants to fine-tune execution timings and buffers.

* Add endpoint probing functionality

Added new `RESPONSE_TIMEOUT_STATUS_CODES` in `consts.ts` to manage timeout responses. Additional functions `detectResponseIsTimeout(response: Response)` was added in `endpoint.server.ts` to detect if a response was a timeout based on the status codes from `RESPONSE_TIMEOUT_STATUS_CODES`.

Update actions to use new endpoint probing endpoint service. This allows for the early probing of endpoints to determine if they're up and running.

A new class `ProbeEndpointService` was created in `probeEndpoint.server.ts` which makes HTTP requests to a given endpoint and updates its properties based on the result.

Finally, `detectResponseIsTimeout(response)` is used in `performRunExecutionV2.server.ts` for marking the execution as succeeded when facing a timeout.

* Refactored probe method in EndpointApi class

The probe method of the EndpointApi class has been refactored to remove the error handling part and it now takes a timeout sent from the client directly. The corresponding changes were also made in the ProbeEndpointService and TriggerClient objects to reflect the alterations in the probe method.

The error handling related to the timeout has been removed and the responsibility of handling the timeout has been shifted to the client. Thus, the probe method has been greatly simplified. The `probeEndpoint.server.ts` file was also changed to accommodate the change in behavior of the probe result.

In the `triggerClient.ts` the timeout for probe is now read from the incoming request object. For backward compatibility, if no timeout is provided in the request, the default value of 15 minutes is used.

* Remove performRunExecution v1 enqueue function
* Better document limits and add docs on increasing function timeouts
* Upgrade webapp docker container to use 18.18.2
* force clients to yield when a run is executing in a gracefully shutting down worker
* Renamed task `key` to `cacheKey` and added more task documentation
* Index the `@trigger.dev/sdk` version on Endpoints
2023-10-20 17:44:10 +01:00
Dan 51f2cd6b3f Updated posthog to 1.83.0 (#608) 2023-10-19 16:34:41 +01:00
Eric Allam c8aaea8ad0 Improvements: Gracefully shutdown to prevent locked jobs (#648)
🚀 Publish Trigger.dev Docker / typecheck (push) Failing after 6s
🚀 Publish Trigger.dev Docker / units (push) Failing after 5s
🚀 Publish Trigger.dev Docker / e2e (push) Failing after 6s
🚀 Publish Trigger.dev Docker / publish (push) Has been skipped
* WIP

* Report ECS task info on startup and shutdown

* Fixed lifecycle name

* Re-add terminus

* Require the build dir when http server is disabled

* Remove unnecessary logs

* Implement graceful shutdown in ZodWorker

* Re-order some code

* Increase the keepAliveTimeout to 65 seconds to prevent LB 502 errors
2023-10-19 11:18:19 +01:00
Matt Aitken 39ec34f84f Latest lockfile 2023-10-16 13:16:02 +01:00
Eric Allam ebe78017ef Update pnpm lock file after package release merge 2023-10-13 13:21:52 +01:00
Matt Aitken 50e3d9e43a Indexing errors don't get displayed anywhere (#605)
* Added EndpointIndex status. Default is PENDING, existing rows are SUCCESS

* Made it easier to create a migration SQL file

* Created a job-catalog file for misconfigured Jobs that should error when running the CLI

* EndpointIndex data and state are now optional

* The Environments page now shows the status of the last refresh

* Improved the UI about endpoints

* Added EndpointIndex error column

* WIP making the endpoint indexing more robust

* Indexing errors are now surfaced

* Improved the error show it shows the job id

* Use “performEndpointIndexing” when you create your first endpoint from the UI

* Use “performEndpointIndexing” for the recurring endpoint checker

* Staging is now auto-indexed every 10 mins too

* Use “performEndpointIndexing” for the webhook

* Moved the throttling to a util

* Removed instructional comments

* Created a reusable retry system with exponential backoff

* Use p-retry for retrying with backoff

* The CLI gets indexing results and displays errors

* Improved the indexing error messages and display in the console

* Use a pre so the Indexing error is correctly split over multiple lines

* Use a db transaction for webhook that triggers endpoint indexing

* Tidied up imports

* Support older versions of the server

* Improved the comment on the misconfigured job

* Changeset: When indexing user's jobs errors are now stored and displayed
2023-10-11 17:31:19 +01:00
Arjun S 498f56de3e Feat: Replaicing react hot toast with Sonner toasts (#600)
* Feat: Replaicing react hot toast with  Sonner toasts

* pnpm lock file fix

* Quickfix
2023-10-11 10:52:10 +01:00
Rutam Prita Mishra 975c5f1d3c [TRI-1333] : Upgrade all packages to use Node18 and use fetch instead node-fetch (#581)
* [TRI-1333] : Upgrade all packages to use Node18 and use fetch instead of node-fetch

* Changeset Added

* Update pnpm-lock to compatible pnpm v7.18.1

* Restored pnpm-lock to the last known good point compatible with pnpm v7.18.1

* Update chilled-plants-exercise.md

---------

Co-authored-by: Eric Allam <eallam@icloud.com>
2023-10-10 11:42:02 +01:00
Matt Aitken 0558b2c595 feat: sveltekit adaptor (#575)
* Feat/svelte kit support (#467)

* feat: implemented svelte-kit adapter

* feat: implemented the @trigger.dev/svelte package

* scaffolded a sveltekit example project with trigger.dev

* added the convertToStandardRequest function in the sveltekit package

* example sveltekit project with the @trigger.dev/svelte package

* doc: added the manual setup guide for sveltekit

* updated the web app onboard for sveltekit

* formatted the manual setup guide for sveltekit

* added a link to webapp sveltekit onboarding

* added a wait function to the sveltekit example app job

* typo fix

* removed comments from the @trigger.dev/svelte

- commented out the useEventRunDetails

* updated package.json to use internal packages

* added a .env.example file

* updated the example-svelte-app

* updated the svelte-example

* updated the onboarding page to reflect requested changes

* made the manual setup guide more specific

* created a .env.example file for svelte-example

* added import aliases

* updated the tsconfig and svelte.config file

* updated to use uint8Array

* added missing paths in the tsconfig

---------

Co-authored-by: Matt Aitken <matt@mattaitken.com>

* Moved examples to references

* Tweaked the SveleteKit manual setup guide

* Updated zod, and set the svelte/sveltekit package versions to match other packages

* Added the headers to the response from the SvelteKit api/trigger route

* Changeset for Svelte and SvelteKit packages

* Revert change to rawBody

* Remove core from changeset packages

* Delete svelte .npmrc file

* Build svelte package to build, not build/lib

* Made the snapshot instructions easier to copy

* Delete the svelte package

* Latest lockfile

* Updated the SvelteKit docs

* Attempt to get Svelte to import the package

* Updates for port and host options

* Updated lockfile from main

* Update neat-feet-wait.md

---------

Co-authored-by: Chigala <92148630+Chigala@users.noreply.github.com>
2023-10-06 17:12:52 +01:00
Eric Allam 24d892e322 Merges 573 with fix for Dockerfile (#578)
* update prisma to 5.4.1 (#573)

* Get prisma 5.4.1 to work in the Dockerfile

---------

Co-authored-by: Karthik <kartim316@gmail.com>
2023-10-06 15:10:43 +01:00
Matt Aitken 1f3b423c22 Removed testing dependencies from nestjs example 2023-10-06 13:18:47 +01:00
Matt Aitken 5910339bf4 Latest lockfile 2023-10-06 12:56:56 +01:00
Matt Aitken a847b492bc NestJS framework suport (from @H4ad) (#574)
* feat: added nestjs package adapter

* Latest lockfile

* Updated Astro setup docs: env import

* Astro docs improvements

* Docs: improved the limitations

* Bumped package versions to 2.1.3

* Updated the Astro docs with SSR notes

* test/368/use vitest instead of jest (#470)

* chore: add vitest dependencies

* refactor: replace jest by vitest

* test: disable broken test

* Update pnpm-lock.yaml

* Update pnpm-lock.yaml

---------

Co-authored-by: Matt Aitken <matt@mattaitken.com>

* Created Temp for pull req, bugreport & feature req

* A few teaks to the templates

* Added instructions for how to do Changeset snapshots

* Redirect people to discord to ask a question

* Documentation Update: Added <github_username> instead of triggerdotdev to avoid confusion while cloning the repository (#477)

* Added a link to the homepage from the side menu (#479)

* Added 2 new named icons

* New side menu link to the homepage

* Updated lock file

* Removed un-used import

* Renamed the examples dir to references (true examples are in another repo and this was confusing)

* Add a references README

* Fixes broken pnpm lock file

* Improve the Astro manual setup guide

* Fixed cal.com link

* Upgrade to the latest remix (pre v2)

* React status hooks (#493)

* Added stripInternal to SDK tsconfig

* Statuses can now be set from a run, and are stored in the database

* Added the key to the returned status

* Made the test job have an extra step and only pass in some of the options

* client.getRunStatuses() and the corresponding endpoint

* client.getRun() now includes status info

* Fixed circular dependency schema

* Translate null to undefined

* Added the react package to the nextjs-reference tsconfig

* Removed unused OpenAI integration from nextjs-reference project

* New hooks for getting the statuses

* Disabled most of the nextjs-reference jobs

* Updated the hooks UI

* Updated the endpoints to deal with null statuses values

* The hook is working, with an example

* Changeset: “You can create statuses in your Jobs that can then be read using React hooks”

* Changeset config is back to the old changelog style

* WIP on new React hooks guide

* Guide docs for the new hooks

* Added the status hooks to the React hooks guide

* Removed the links to the status hooks reference for now

* Re-ordered the hooks

* Fix for an error in the docs

* Set a default of a blank array for the GetRunSchema

* Fixed dependency

* Revert "Upgrade to the latest remix (pre v2)"

This reverts commit 4edc7112ab.

* Update introduction.mdx (#498)

Changed https://github.com/triggerdotdev/examples/tree/main/resend (which was a 404) to https://github.com/triggerdotdev/examples/tree/main/resend-email-form

* Use the bell icon for the new status Tasks

* Going exponential with `Linear` (#478)

* Unleash GPT magic

* Clean up after GPT

* All the hooks

* Provisional integration catalog entry

* Sample webhook jobs

* Attachments with alpha warnings

* Remove some verbose logs

* Fix IP restrictions

* Remove tunnel

* Revert "Remove tunnel"

This reverts commit c5b69ce6524e3b40c26b66cdc56e087b8576b8c6.

* Resolve event name clashes

* Remove circular dependency

* Use correct payload uuid

* Schema fixes

* Fix webhook event name

* Start to Linearify catalog entry

* Remove todo

* More catalog updates

* Make OAuth work

* Rename webhook helper

* Schema juggling

* More discrimination

* Add Issue SLA event

* Simplify triggers

* Handle rate limits

* Fix Project schema

* Payload examples

* Improve event props

* Remove redundant source metadata

* One type to rule them all

* Recursive WithoutFunctions type

* Linear output serializer

* Some tasks

* Update catalog entry

* Dynamic usage sample

* Bump version

* Remove tunnel

* More tasks

* Add optional skipRetrying on runTask errors

* Fail fast on user errors

* Entity getter tasks

* Another couple of tasks

* Token to apiKey

* Sort tasks

* Add filtered issue SLA triggers

* Add docs

* Type fixes

* Job catalog examples

* Serialization helper docs

* Add changeset

* Refactor webhooks

* Enhance properties

* Pagination helper and docs

* Clean up imports

* Change misc catalog job

---------

Co-authored-by: Matt Aitken <matt@mattaitken.com>

* Use absolute image paths (#490)

* Update sendevent.mdx

* fix: Fail client-side on invalid Stripe event names (#492)

* Parse event names

* Add changeset

* feat: BYO Auth (#491)

* feat: BYO Auth

Define client-side auth resolvers to be able to supply custom authentication credentials for integrations before a run is performed

- Added new defineAuthResolver
- Update all integrations to support the new auth resolvers
- Strip internal symbols from .d.ts in integrations and trigger-sdk
- Added BYO Auth docs
- Update Dynamic Schedule to support associated account IDs
- Create external accounts just-in-time
- Added Account ID field to test job when there are external auth integrations
- Show Account ID on run dashboard
- Added new Run error state called “Unresolved auth”

* Added changeset

* Remove @internal from TriggerIntegration public methods

* Add void to the result union

* DynamicTriggers now work with the new BYO auth system, and added a bunch of docs and docs changes

* Add additional key material for registering dynamic trigger task

* Add new define* instance methods to the overview

* CLI now supports multiple frameworks (with tests) (#480)

* Early work defining CLI framework support

* WIP moving CLI init logic to the Framework class

* Installing files should now work for Nextjs

* Some fixes

* WIP creating unit tests for Next.js project detection

* Delete old jest config

* Latest lockfile

* Detect use of src directory test

* Tests for detection pages/app directory

* Correct detection of Next.js project

* Renamed test file

* Create install files from template files with replacements. With tests

* Added multiple uses of the same replacement

* Created a test for the install step (it fails right now with JS)

* Removed unused import

* Another test that should pass but currently fails…

* Path alias fixed and now has tests

* New pathAlias function used

* Nextjs page install tests

* Fixed app directory install (with tests)

* Removed e2e CLI test, switched to unit testing strategy instead

* Latest lockfile

* The install files are now actual files that are copied and transformed

* Got the template files working correctly after building

* Next steps are now framework specific

* createFileFromTemplate now works with a path again. Uses mock if specified.

* Renamed apiRoute.js to pagesApiRoute.js

* Simplified pages file generation

* Next.js app API route template

* Next.js App routing support, with common files logic shared

* Dev command now uses framework default values if they exist and aren’t overridden

* Unused import

* pathAlias now works for all frameworks

* Added a test to detect Next from the next.config.js

* WIP on Remix framework support

* Tests for Remix install

* Replaced references to Next.js

* Use a green ✔️ instead of  in the CLI

* Support for multiple hostnames

* Tunneling can now use the hostname and port

* Work on multiple ports

* Improved the error messages. Added some extra pots to Next.js

* Update the Remix templates to have .server in the imports

* Remix updated to use server-runtime instead of node. Node v18+

* Frameworks can specify the watch paths and ignore paths

* Define the watch variables above, so we can easily log them for debugging

* Don’t wait for outdated package checking when running the dev command

* Improved the Remix manual setup guide

* Rewriting docs for quickstart

* Updated the Next.js quickstart

* Remix quick start

* Added a changeset

* Improved the Next.js manual setup

* Tweaked the Linear scopes

* Latest lockfile

* Fix for getPathAlias typecheck failure

* Linear getAll type error (weirdly not in VSCode…) and removed the pagination example that uses the SDK as won’t work with timeouts

* Add BYO auth for oauth options

* Add back in Job.toJSON to fix the testing package

* Removed dynamicTrigger @internal from toJSON

* The CLI now checks for a dev server API key in init and dev commands

* Remix onboarding now uses the CLI init command

* Decouple zod (#500)

Zod Schemas is no longer required for validating/inferring event triggers. We’ve taken inspiration from how domain-functions did it: https://github.com/seasonedcc/domain-functions/pull/114

* chore: Update version for release (#481)

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

* Latest lockfile

* feat: Basic usage dashboard to show run volume (#501)

* New usage dashboard with static data

* Implement org usage dash

* Grab chart data for the last 12 months

* If no org is found just return undefined so a 404 will be shown

* Remove mock data

* Fill in missing months with 0s

---------

Co-authored-by: James Ritchie <james@jamesritchie.co.uk>

* Fixed duplicate end month

* hotfix

* hotfix 2

* Astro CLI support (#506)

* Astro framework CLI support

* Changeset: Added Astro automatic installation

* Fixed the package name – was remix, now astro

* Fixed the export of the example

* Added IPv6 localhost to Astro hostnames

* Updated Astro onboarding to show the CLI init command, instead of manual instructions

* Astro quickstart

* Fix for type in Remix quickstart

* Next.js framework detection allows different config file extensions and “next” devDependency

* Made the dev command port more general so it works with various frameworks

* chore: Update version for release (#508)

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

* Fix for incorrectly named Next.js package in manual setup

* Latest lockfile

* Increased the intervalTrigger max from 1 day to 30 days

* Implement the task output redacting to prevent redacted values from showing in the logs

* Express frameworks docs + CLI (#512)

* Manual setup docs

* Added the onboarding

* The emails package now works with Node > 18

* Added CLi support for Express, including custom init command finished messages

* Need to actually log out the installation complete message…

* Fix for an old Remix reference

* Renamed the page export

* Use resolvedOptions.triggerUrl

* Typo in manual instructions

* chore: Update version for release (#513)

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

* Latest lockfile

* Add the STAGING environment by default

* Fixed sparodically failed run creations

- Use a better way of getting the latest job run number to increment
- Make the CreateRunService transaction more reliable
- Invoke dispatchers in parallel
- No longer swallow prisma errors in $transaction

* Swapped out the Homepage link in the side menu for a link to the Changelog

* Youtube embedded video fits its aspect ratio instead of going full width

* Improved CLI init Next.js middleware detection

* CLI init: adds public key as “TRIGGER_PUBLIC_API_KEY” except for Next which overrides this

* Updated the docs for the React hooks

* chore: Update version for release (#521)

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

* Display framework logos on the onboarding setup pages (#519)

* autofocus the search field on the Job page

* Add some documentation around canceling scheduled events

* Updated outdated lockfile

* Updated docs README.md

* Improves the performance of run resuming (#522)

* Improves the perform of run resuming

When runs resume, we try and make sure that tasks that have already been completed are cached and reused. Worst case scenario the client needs to hit the API server once for a non-cached task that is indeed completed on the server, but this can get pretty expensive when there are a larger number of tasks.

This commit does 2 different things to help:

- noop tasks are no longer “cached” using the cachedTasks strategy, instead their idempotency keys are shoved into a bloom filter and the client tests for their inclusion in the bloom filter before running them (since they don’t have any concept of output, this works)
- Additional cached tasks are lazy loaded when a task is run. This allows us to progressively fetch additional tasks to be cached on the client, which will cut down on cache misses by a decent amount

* Create warm-carrots-float.md

* Make io.yield backwards compat with older platform versions

* Better support old clients connecting to server versions that support lazy loading cached tasks

* Fixed type errors when settings headers with unknown value

* Better yield not support error message

* Rename _version to _serverVersion to be more clear

* Add typed filters to `Linear` getAll helper (#517)

* Fix getAll params type

* Changeset

* Search param types

* Update changeset

* `Replicate` integration and remote callbacks (#507)

* Support tasks with remote callbacks

* Add common integration tsconfig

* Add Replicate integration

* Basic job catalog example

* Integration catalog entry

* Check for callbackUrl during executeTask

* Fix getAll

* Improve JSDoc

* Bump version

* Remove named queue

* Simplify runTask types

* Trust the types

* Fail tasks on timeout

* Callback timeout as param

* Mess with types

* performRunExecutionV1

* Update runTask docs

* Shorten callback task methods

* Fix run method return type

* Image processing jobs

* Replicate docs

* Text output example

* Changeset

* Version bump

* Roll back ugly types

* Remove missing types

* Quicker return when waiting on remote callback

* Remote callback example

* Bump version

* Remove schema parsing

* Only schedule positive callback timeout

* Decrease callback secret length

* Explicit default timeouts

* Import deployments tasks

* JSDoc

* Deployments docs

* Fix runTask examples, mention wrappers

---------

Co-authored-by: Eric Allam <eric@trigger.dev>

* Update pnpm lock file

* Allow blank issues

* chore: Update version for release (#538)

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

* Fix pnpm lock file

* feat: New Test page (#558)

* Improvements to the shortcut keys

* Updated code mirror theme to the new dark mode style

* Removed the old test page help panel copy

* WIP new structure for the test page

* Added some of the side panel options for examples and account ID

* Conditionally hide the side panel components if they’re blank

* Example with selected state working

* Allow selecting the example again to overwrite any edits

* Previous run payloads working

* Select a recent payload if there’s no example

* Created Icon and DetailCell components

* The integrations now use the DetailCell

* Use DetailCell on the integrations page

* Added support for description to DetailCell. With proper variants now

* DateTimeAccurate and formatDateTimeAccurate weren’t displaying correctly.

fractionalSecondDigits isn’t in the types, but is supported for 92.5% of users

* DetailCell now allows label and description to be React components

* Using the DetailCell on the Test page

* Styled the CodeMirror scrollbars to match elsewhere

* Adding copy and clear to the JSONEditor, not working properly yet though

* Removed the copy/clear buttons from the Test route

* Delete the light color theme

* The editor now supports copying/clearing etc

* Made it clear when text is copied

* Changed the learn link to a tertiary button

* Fixed CMD shortcut keys

* Run test button now works with ⌘ Enter

* CodeMirror now allows ⌘Enter to escape the field

* Added JSON linter to the test editor

* Fix for buttons not being aligned correctly

* Copy/clear buttons now aligned

* Made the integration DetailCell text smaller

* The test recent payloads now have colored text to help identify the status

* DetailCell descriptions are dimmer

* Added link to docs for BYOA

* Fix for buttons going full width when they shouldn’t

---------

Co-authored-by: James Ritchie <james@jamesritchie.co.uk>

* chore: update doc title for groups (#539)

* chore: update doc title for groups

* update docs for sdk and integrations

* Add instructions for triggering job runs to the job catalog readme

* Updated the Test docs

* Removed a console.log when a user’s file is changed that CLI dev is listening for

* feat: allow cancelling jobs from trigger-client sdk (#562)

* feat: allow cancelling jobs from trigger-client sdk

* use presenter instead of service for non-mutating logic

* chore: upgrade zod to 3.22.3 (#570)

* chore: upgrade zod to 3.22.3

* add changeset

* Created a changeset with the correct starting package version

* Added headers to NestJS response

* Added some types to the NestJS project

* Added fastify types

* Removed log from CodeMirror

* Added InstallPackages component

* Improved the NestJS onboarding instructions

* Updated the onboarding instructions

* Made all the quickstart framework cards snippets, so the page isn’t a nightmare to edit

* NestJS docs updates

* Added dotenv to the earlier code sample

* Latest lockfile

* Moved the nestjs-example to the reference folder

---------

Co-authored-by: Vinícius Lourenço <contact@viniciusl.com.br>
Co-authored-by: Wesley <100464352+ologbonowiwi@users.noreply.github.com>
Co-authored-by: Vishesh Rawal <92795514+visheshrwl@users.noreply.github.com>
Co-authored-by: Eric Allam <eallam@icloud.com>
Co-authored-by: Aniket Bindhani <aniketbindhani44@gmail.com>
Co-authored-by: James Ritchie <james@trigger.dev>
Co-authored-by: D-K-P <dkp.github@pm.me>
Co-authored-by: Gregory <93215236+gjohnsx@users.noreply.github.com>
Co-authored-by: nicktrn <55853254+nicktrn@users.noreply.github.com>
Co-authored-by: Eric Allam <eric@trigger.dev>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: James Ritchie <james@jamesritchie.co.uk>
Co-authored-by: Hemachandar <132386067+hmacr@users.noreply.github.com>
2023-10-06 12:27:15 +01:00
Hemachandar 9a187f9eda chore: upgrade zod to 3.22.3 (#570)
* chore: upgrade zod to 3.22.3

* add changeset
2023-10-05 17:00:33 +01:00
Matt Aitken 4e35871861 feat: New Test page (#558)
* Improvements to the shortcut keys

* Updated code mirror theme to the new dark mode style

* Removed the old test page help panel copy

* WIP new structure for the test page

* Added some of the side panel options for examples and account ID

* Conditionally hide the side panel components if they’re blank

* Example with selected state working

* Allow selecting the example again to overwrite any edits

* Previous run payloads working

* Select a recent payload if there’s no example

* Created Icon and DetailCell components

* The integrations now use the DetailCell

* Use DetailCell on the integrations page

* Added support for description to DetailCell. With proper variants now

* DateTimeAccurate and formatDateTimeAccurate weren’t displaying correctly.

fractionalSecondDigits isn’t in the types, but is supported for 92.5% of users

* DetailCell now allows label and description to be React components

* Using the DetailCell on the Test page

* Styled the CodeMirror scrollbars to match elsewhere

* Adding copy and clear to the JSONEditor, not working properly yet though

* Removed the copy/clear buttons from the Test route

* Delete the light color theme

* The editor now supports copying/clearing etc

* Made it clear when text is copied

* Changed the learn link to a tertiary button

* Fixed CMD shortcut keys

* Run test button now works with ⌘ Enter

* CodeMirror now allows ⌘Enter to escape the field

* Added JSON linter to the test editor

* Fix for buttons not being aligned correctly

* Copy/clear buttons now aligned

* Made the integration DetailCell text smaller

* The test recent payloads now have colored text to help identify the status

* DetailCell descriptions are dimmer

* Added link to docs for BYOA

* Fix for buttons going full width when they shouldn’t

---------

Co-authored-by: James Ritchie <james@jamesritchie.co.uk>
2023-10-05 10:38:28 +01:00
Eric Allam 90f52de147 Fix pnpm lock file 2023-10-04 14:01:48 +01:00