Commit Graph

27 Commits

Author SHA1 Message Date
Eric Allam f9ec66c562 v3: new build system (#1265)
* upgrade @opentelemetry packages to the latest versions

* remove v2 only packages, will be moved to a dedicated repo

* remove more v2 code and run pnpm install

* use the npm yalt package in the webapp

* convert @trigger.dev/core to tshy

* Switch from jest to vitest in @trigger.dev/core

* Fixed core test

* move core-backend code into core subpath export

* convert @trigger.dev/sdk to tshy

* Removed hono

* move core-apps to core/v3/apps, remove core-apps, start converting cli-v3

* Fix up some of the commands

* cli now building and loadable

* using package-json-from-dist to get package version now in core and cli

* dev command WIP

* cleaned up some repetition and structure of the entry point stuff

* bringing back the background worker stuff

* Indexing of the v3 catalog

* getting closer to executing dev runs...

* centralize dev logging using event emitter

* Move indexing to it’s own entry point, simplify code

* dev runs working

* Get instrumentation to work with openai

* debugging achieved internally

* provide worker files as part of the worker creation on the server

* support for cjs and esm javascript

* Fixed timeout

* worker manifest now has the config path

* auto-upgrade config to non-deprecated alternatives

* Adding package preview release

* deployment WIP

* improve the syncEnvVars output and adapt resolveEnvVars

* WIP bun runtime

* WIP bun support

* seed tasks with the machine preset if listed in the config

* deploy run executions WIP, extracted TaskRunProcess into 1 place

* deployed tasks running and executing 🎉

* support for waits and better flushing & process cleanup

* Fixed the heartbeating

* Better warning messages

* Improve and unify the indexing between dev and deploy

* Support for external deps that need node-gyp to build

* build extensions can now install custom packages and run instructions in the image. Also prisma extension now works and also works with multiple schema files

* Add back in the main/types/module to sdk

* dev no longer is Ink/React, grace period for disconnections in dev

* Fix the changeset config

* More changeset fixes

* Remove config packages

* More changeset fixes

* Fixed typescript issues (needed to revert back to zod 3.22.3

* Fix pr_checks workflow

* Remove the prepare script

* Fixed tests and package versions

* Remove cli test script

* Remove packages from tailwind watch paths

* Add repo to public packages

* Just commit the generated files and do the building at dev time

* Try and get pkg.pr.new working

* Try again

* Fix emitDecoratorMetadata importing named export from typescript

* config file backwards compat with export const config

* Fixed issue where import errors weren’t coming through

* p-retry is a prod dep

* typescript needs to be a prod dependency for emitDecoratorMetadata

* Add better debug logging to help track down import-in-the-middle bug

* An external is only considered resolvable if it resolves to the same path as the collected external

* Fix runtime checks to allow >=18.20

* Move extensions to a new build package

* Fixed building packages in dockerfile

* Remove the e2e test from publish workflow for now

* Don’t treat pkg.pr.new versions has needing upgrading

* making sure config handleError works, and discovered path aliases don’t work in config files

* Strip empty string env vars so they accidentally override real values

* Couple of things

* Update version to use preview instead of beta

* Hopefully fix re-attempts with >30s delay

* Match socket emit messages to current latest in main

* Initial guide

* Go back to beta

* Go back to the preview, and update guide to use pr preview tags

* Go back to beta

---------

Co-authored-by: Matt Aitken <matt@mattaitken.com>
2024-08-23 13:10:15 +01:00
Matt Aitken c405ae7117 Schedule limits and timezone support (#1165)
* Added maximumScheduleInstancesLimit column to Org, default to 20

* Docs on the schedule limits and improved soft-limit communication

* Added limit info to the schedules list page

* Created a task that creates schedules, useful for testing

* Make deduplicationKey required when creating/updating a schedule using the SDK

* New schedule button shows an alert if you’re over the limit

* Added timezone to the form and db

* WIP on the timezone dropdown for the create/edit schedule form

* Use the new filter search for timezones

* Made the timezone dropdown faster by fixing the virtualization

* The preview table is working and added a nice message about daylight savings

* Created a page where you can view the full list of timezones

The URL is included in the error message if you send an invalid time using the SDK

* Creating tasks with the timezone

* Added timezone support the the scheduler and the schedules list

* Added timezone support to more of the schedules UI

* The timezone comes through to scheduled runs with nice JSDocs

* Allow setting the timezone from the SDK

* Always have a timezone on a schedule

* Updated jsdocs

* Updated catalog example

* Changed the column to be a string, not null. Added the timezone across the SDK

* API endpoint for getting the timezones

* Added an SDK function to get the list of timezones

* Added timezones to the docs

* Changeset: Added timezone support to schedules

* Added support for testing timezone

* Tidied up imports

* Imports

* Imports

* Update limits.mdx

* Fixed a couple type issues and use the already exported zodfetch

---------

Co-authored-by: Eric Allam <eallam@icloud.com>
2024-06-14 13:31:27 +01:00
Eric Allam a946797d95 v3: Upgrades to MarQS (#989)
* Upgrades to MarQS

- It’s “queue choosing” algorithm is now MUCH better (pretty much didn’t work before) and should be performant even when we have a bunch of prod queues
- There are now concurrency limits at the environment and organization level, as well as the task/queue level. So if any of them are at capacity the message won’t be dequeued. This means we can have an org wide concurrency limit, as well as different limits for dev/prod/staging
- I’ve added an admin API that can be used to update the org/env concurrency limits

* More MarQS upgrades

- Extract the queue priortity choosing strategy into an interface
- Implement a much better weighted average strategy
- “Slide the window” of queue candidates if the parent queue sends all at-capacity queues (so we won’t get stuck attempting to choose the same 12 full queues)
- Added some unit tests for the priority stuff
- Added some ideas for expanding the priority choosing strategy with more dynamic features

* Fixes an issue with the shared queue consumer relying on the queue name to get the env id
2024-03-31 15:58:00 +01:00
Eric Allam baf3a84cda v3 dev cli command + more (#894) 2024-02-12 14:24:32 +00:00
Eric Allam 812b5afe6f Add support for the built-in Trigger.dev tunnel for local development (#795)
🚀 Publish Trigger.dev Docker / units (push) Failing after 4s
🚀 Publish Trigger.dev Docker / e2e (push) Failing after 3s
🚀 Publish Trigger.dev Docker / typecheck (push) Failing after 13s
🚀 Publish Trigger.dev Docker / publish (push) Has been skipped
* Add support for the built-in Trigger.dev tunnel for local development

* Remove the test script from yalt

* Add to the README

* Improve the outdated packages warning
2023-12-14 15:33:56 +00:00
Eric Allam c1710ae72d fix: Buffer import breaks @trigger.dev/react (#714) 2023-10-31 15:28:55 +00:00
Eric Allam c0f9f10b07 Upgrade to Remix v2.1.0 2023-10-26 15:54:21 +01:00
Matt Aitken 711019032f Added the SDK to webapp tsconfig 2023-09-11 19:20:30 +01:00
Matt Aitken 92233f2e90 Core package and move to @latest (#235)
* Changeset should ignore the example projects

* Removed note in Contributing instructions about not adding a changeset for internal

* Renamed @trigger.dev/internal to @trigger.dev/core. Set sdk and internal to be ES2020, so we don’t get errors about private identifiers

* Env vars in nextjs-example use square bracket syntax to avoid Turbo Repo errors

* Set the tsconfigs back for core and sdk

* @examples/nextjs compile error with undefined tasks

* Set the example projects to use ES2015 to avoid private modifier complaints

* package-tester example, which will use built packages

* package-tester package.json

* Created a readme for the package-tester

* Added all the packages to package-tester

* Create .env.local.example and readme instructions

* Added name to the package.json

* Upgraded @types/react and @types/react-dom everywhrre, so we can use server actions

* Upgraded @types/react and @types/react-dom everywhrre, so we can use server actions

* Reworked the react package build, so it generates separate files

* The SDK no longer bundles core

* package-tester setup with a server action and react hooks

* Added new SDK methods with logging

* Working tsup settings for client and server

* Explicit react hooks return types

* Added all the hooks for testing

* Added OpenAI step to the job to check types are still ok in integrations

* Get rid of rogue Changeset ignores

* Changeset: @trigger.dev/core is now a separate package

* Exited prerelease mode, added a changeset

* Changed @next to @latest

* Deleted seed.js, this shouldn’t be committed

* Ignore seed.js

* The webapp was importing @trigger.dev/core with a folder path rather package name…

* More webapp imports instead of @trigger.dev/core were a folder path

* Accidentally edited Stripe internal API url in the comment

* Added “sideEffects”: false so @trigger.dev/core is tree shaken by Remix
2023-07-31 18:47:34 +01:00
Eric Allam 9f49b7f4c0 Move prisma to an internal database package 2023-06-15 14:18:34 +01:00
Matt Aitken b087b5cdee More progress on oauth and started on the api UI 2023-04-21 16:00:25 -07:00
Eric Allam 2d5c2394c8 Initial commit of the new serverless execution runtime 2023-03-31 14:32:00 +01:00
Matt Aitken a20a26e748 Updated settings for WhatsApp to match other integration packages 2023-01-31 09:53:24 +00:00
Eric Allam acf8bf038f Remove trigger-providers, trigger-integrations, and internal-integrations and replace with integrations-catalog 2023-01-30 15:58:25 +00:00
Eric Allam 3769d12bd8 Breakout shopify into it’s own package 2023-01-30 14:05:23 +00:00
Eric Allam 3759dfc434 Breakout resend into it’s own package 2023-01-30 13:52:56 +00:00
Eric Allam f9208f45a7 Breakout slack into it’s own package 2023-01-30 13:25:55 +00:00
Eric Allam 2d0e766b1a Break out github into it’s own package 2023-01-30 12:57:37 +00:00
Eric Allam 39853e5ce0 Add uptime workflow that runs every minute 2023-01-25 21:55:04 +00:00
Matt Aitken 92ae9bd894 Added tsconfig paths so we’re not linking to /dist folders anywhere 2023-01-21 13:32:14 -08:00
Eric Allam e77f259fb1 Improvements to pulsar topics and structure
- Centralized the pulsar client creation and topics to internal-pulsar
- Cleaned up the pulsar message catalogs and split triggers and run command responses
- Improved the naming of the task queue
- Figured out how to use the pulsar client with the hosted version of Pulsar
2023-01-06 19:36:38 +00:00
Matt Aitken 91bc0764d9 Magic link and welcome (without delay atm) email now sending with Resend 2023-01-04 17:13:33 +00:00
Eric Allam 67612a8e0c Registering webhooks with existing connections (and github only) 2022-12-21 16:56:32 +00:00
Eric Allam d457085f7b Workflow runs, with a workflow step for output 2022-12-20 11:14:03 +00:00
Eric Allam d8c22b61ef Workflows are now being registered in the webapp, along with their trigger 2022-12-15 16:41:08 +00:00
Eric Allam 271db37844 Fully rename to @trigger.dev 2022-12-08 13:12:07 +00:00
Matt Aitken dc2e4c3a87 Initial commit of the mono repo 2022-12-06 12:28:16 +00:00