Commit Graph

25 Commits

Author SHA1 Message Date
Eric Allam 2d8a41b18b feat: realtime (#1402)
* Denormalize run tags, increase character limit to 128

* WIP realtime subscribing to runs

* extracted the stream stuff into core, made it more reusable

* WIP tags

* Remove tags for now because it’s not support in electric

* Support async iterables, readable stream, and callback style subscription styles

* Remove tags streaming endpoint

* Add realtime rate limits and scope them to the /realtime path

* WIP rate limt per org

* Introduce per org rate limits

* WIP JWT auth

* Move migrations into new internal db package

* Resolve pnpm lock file

* Authenticating to the realtime API with JWTs are working

* realtime in the client

* Created react-hooks package and starting to move stuff in there

* Improve types for hooks

* schema tasks

* Added useBatch hook

* build uploadthing/fal demo and change how run metadata is synced to the server

* tweaks

* WIL realtime concurrency tracking

* Implement test for realtime client using testcontainers

also updated electric to latest version

* Allow customizing the expiration time of the automatic JWT created after triggering a task

* Add support for subscribing to run tags

* Improve auth types and API

* finalize the realtime API

* Fixed some example stuff

* Allow up to 10 run tags

* Remove core from docker-provider tsconfig paths to prevent it from being typechecked

* do the same for the kubernetes provider

* Fixing some typecheck errors

* Fix webapp type errors

* Update @trigger.dev/platform to 1.0.13

* Fix attw error

* Remove from/to in subscribeToRuns query params

* Add tests for the rate limit middleware and add custom JWT rate limits

* turn off webapp test parallelism

* Finish renaming jwt -> publicAccessToken and automatically give the JWT read access to the tags when using trigger

* Add changeset

* Attempt to fix unit tests in CI

* Skip running the auth rate limit middleware tests for now

* Try a beefier machine

* Try and run webapp tests separately

* Setup env vars

* Make sliding window test more reliabile
2024-10-21 15:07:08 +01:00
nicktrn b027d69365 Update one last action to node 20 2024-09-23 15:34:49 +01:00
nicktrn c8eae9d37a Comment out changesets mock 2024-09-23 15:19:02 +01:00
nicktrn 8e6eb4f6d8 GitHub workflow improvements (#1306)
* lint and action updates

* add custom image tag action

* remove publish infra bloat

* replace cancel action with native concurrency control

* ignore all docs changes for release trigger

* image action updates

* update publish trigger paths

* simplify semver checks

* make it possible to call publish

* add tests for custom action

* rename publish workflows

* call publish after successful package release

* fix inputs and published package parsing

* clarify image tag action errors

* move test action

* add test events

* add changesets action mock

* add docs checks

* add github action test readme

* fix docs check on push

* update action versions in docs

* cache npm for docs check

* pretend to fix bad docs link

* fix docs link

* Revert "fix docs link"

This reverts commit a7508f5d55.

* Revert "Revert "fix docs link""

This reverts commit ec642af3dc.

* improve caching

* disable automatic publishing after release

* limit docs checks to pushes to main and PRs

* fix broken links

* prevent word splitting and globbing

* use latest checkout action everywhere

* correctly detect prereleases as valid semver

* update to latest cache action
2024-09-23 15:14:34 +01:00
Eric Allam 282df76392 Enable changesets creating a github release 2024-09-17 10:38:58 +01:00
Niels 55b7ffeb72 Don't run release workflow on mdx changes (#1304) 2024-09-15 11:24:35 +01:00
Eric Allam 92d98e5499 Build before running typecheck now 2024-08-23 13:46:18 +01:00
Eric Allam f9ec66c562 v3: new build system (#1265)
* upgrade @opentelemetry packages to the latest versions

* remove v2 only packages, will be moved to a dedicated repo

* remove more v2 code and run pnpm install

* use the npm yalt package in the webapp

* convert @trigger.dev/core to tshy

* Switch from jest to vitest in @trigger.dev/core

* Fixed core test

* move core-backend code into core subpath export

* convert @trigger.dev/sdk to tshy

* Removed hono

* move core-apps to core/v3/apps, remove core-apps, start converting cli-v3

* Fix up some of the commands

* cli now building and loadable

* using package-json-from-dist to get package version now in core and cli

* dev command WIP

* cleaned up some repetition and structure of the entry point stuff

* bringing back the background worker stuff

* Indexing of the v3 catalog

* getting closer to executing dev runs...

* centralize dev logging using event emitter

* Move indexing to it’s own entry point, simplify code

* dev runs working

* Get instrumentation to work with openai

* debugging achieved internally

* provide worker files as part of the worker creation on the server

* support for cjs and esm javascript

* Fixed timeout

* worker manifest now has the config path

* auto-upgrade config to non-deprecated alternatives

* Adding package preview release

* deployment WIP

* improve the syncEnvVars output and adapt resolveEnvVars

* WIP bun runtime

* WIP bun support

* seed tasks with the machine preset if listed in the config

* deploy run executions WIP, extracted TaskRunProcess into 1 place

* deployed tasks running and executing 🎉

* support for waits and better flushing & process cleanup

* Fixed the heartbeating

* Better warning messages

* Improve and unify the indexing between dev and deploy

* Support for external deps that need node-gyp to build

* build extensions can now install custom packages and run instructions in the image. Also prisma extension now works and also works with multiple schema files

* Add back in the main/types/module to sdk

* dev no longer is Ink/React, grace period for disconnections in dev

* Fix the changeset config

* More changeset fixes

* Remove config packages

* More changeset fixes

* Fixed typescript issues (needed to revert back to zod 3.22.3

* Fix pr_checks workflow

* Remove the prepare script

* Fixed tests and package versions

* Remove cli test script

* Remove packages from tailwind watch paths

* Add repo to public packages

* Just commit the generated files and do the building at dev time

* Try and get pkg.pr.new working

* Try again

* Fix emitDecoratorMetadata importing named export from typescript

* config file backwards compat with export const config

* Fixed issue where import errors weren’t coming through

* p-retry is a prod dep

* typescript needs to be a prod dependency for emitDecoratorMetadata

* Add better debug logging to help track down import-in-the-middle bug

* An external is only considered resolvable if it resolves to the same path as the collected external

* Fix runtime checks to allow >=18.20

* Move extensions to a new build package

* Fixed building packages in dockerfile

* Remove the e2e test from publish workflow for now

* Don’t treat pkg.pr.new versions has needing upgrading

* making sure config handleError works, and discovered path aliases don’t work in config files

* Strip empty string env vars so they accidentally override real values

* Couple of things

* Update version to use preview instead of beta

* Hopefully fix re-attempts with >30s delay

* Match socket emit messages to current latest in main

* Initial guide

* Go back to beta

* Go back to the preview, and update guide to use pr preview tags

* Go back to beta

---------

Co-authored-by: Matt Aitken <matt@mattaitken.com>
2024-08-23 13:10:15 +01:00
nicktrn 7aed154854 Update pnpm setup action 2024-07-03 17:06:57 +01:00
nicktrn 1e93ec4216 Revert "change action runner location" (resolved)
This reverts commit cb27b7278a.
2024-04-24 07:45:08 +01:00
nicktrn cb27b7278a change action runner location ⬆️⬆️⬇️⬇️⬅️➡️⬅️➡️🅱️🅰️ 2024-04-23 18:13:04 +01:00
Eric Allam e398bdf859 Only use 16vcpu on e2e 2024-03-27 16:27:28 +00:00
Eric Allam 53054d0146 Enter into changesets “beta” tag for packages 2024-03-27 14:55:14 +00:00
Eric Allam 412d0149d5 Fixed release workflow 2024-03-27 12:37:23 +00:00
Eric Allam affa3f25ee Trying beefier buildjet machines for CI 2024-03-19 11:38:48 +00:00
Matt Aitken d04c8e89f6 Added a “repo-token” to the Install Protoc GitHub action steps
🚀 Publish Trigger.dev Docker / typecheck (push) Failing after 0s
🚀 Publish Trigger.dev Docker / units (push) Failing after 0s
🚀 Publish Trigger.dev Docker / e2e (push) Failing after 19s
🚀 Publish Trigger.dev Docker / publish (push) Has been skipped
2024-02-19 14:15:15 +00:00
Eric Allam baf3a84cda v3 dev cli command + more (#894) 2024-02-12 14:24:32 +00:00
Eric Allam 624072c23b Update release.yml 2023-10-10 15:04:13 +01:00
Eric Allam d37576f814 Update release.yml 2023-10-10 15:03:30 +01:00
Matt Aitken 2636d9091e Release workflow: the changeset folder wasn’t set correctly 2023-09-05 15:14:46 +01:00
Eric Allam 6ad91123f2 Turned off automatic github releases 2023-08-28 10:49:24 +01:00
Eric Allam 3028b6ad9d Try to automatically create github releases when packages are updated 2023-08-28 10:38:01 +01:00
Eric Allam 8883476dcb Specify limited paths that will cause our github workflows to run 2023-08-02 13:27:44 +01:00
Eric Allam de00fa91ae Generate db client before typecheck 2023-06-26 11:53:31 +01:00
Eric Allam 7b1a2f60d0 Adding changesets workflow to automatically version/release package changes 2023-06-26 11:41:40 +01:00