Commit Graph

30 Commits

Author SHA1 Message Date
Chris Arderne 4fde283e76 chore: format and lint webapp also (#4056)
#3977 added formatting and linting everywhere else.

This extends it to the webapp.
2026-06-26 13:02:53 +01:00
Chris Arderne df78ef96d9 feat: multi dev branches (#4023)
Closes this feature request:
[https://triggerdev.featurebase.app/p/isolated-dev-sessions-for-multiple-local-trigger-dev-instances](https://triggerdev.featurebase.app/p/isolated-dev-sessions-for-multiple-local-trigger-dev-instances)

### Feature notes:
- CLI `trigger dev` works as before
- `trigger dev --branch my-branch` to create a new branch and run
against it.
- `trigger dev archive --branch my-branch` to archive (or in webapp).
- New webapp page to manage and archive dev branches, currently feature
flagged.

### Implementation details:
- No changes to data model, no backfill. `isBranchableEnvironment`
column is ignored for dev branches, we use `parentEnvironmentId IS NULL`
instead.
- `x-trigger-branch` overloaded for preview and dev branches
- New `TRIGGER_DEV_BRANCH` env var available locally.
`TRIGGER_PREVIEW_BRANCH` overloaded for child runs.
- Lots of new glue code to sanitise the branch checks.

### Rollout
- Deploy webapp/API changes (all backwards compatible)
- Manual tests on some orgs
- Deploy docs, release CLI, flip feature flag for webapp feature

### NB
- `api.v1.projects.$projectRef.environments.ts` will return
`isBranchableEnvironment: true` for all dev environments.

### Prerequisites
- [x] Typecheck will not pass until we make a new release of
`@trigger.dev/platform` and bump it here
2026-06-26 09:01:37 +01:00
Eric Allam 5052d895b3 feat(webapp,core): add a public HTTP API for errors (#4005)
## Summary

Adds an environment-scoped HTTP API over the Errors feature, mirroring
the runs API. Task-run failures are grouped by a fingerprint into "error
groups," and this exposes everything you can do with them in the
dashboard:

- `GET /api/v1/errors` lists error groups, with
`filter[taskIdentifier]`, `filter[version]`, `filter[status]`
(`unresolved`/`resolved`/`ignored`), `filter[search]`, a time range, and
cursor pagination.
- `GET /api/v1/errors/{errorId}` retrieves a single group (summary,
lifecycle state, affected versions).
- `POST /api/v1/errors/{errorId}/{resolve,ignore,unresolve}` changes its
state.
- `GET /api/v1/runs?filter[error]={errorId}` lists the runs behind a
group.

Request and response schemas are exported from `@trigger.dev/core/v3` so
the SDK can reuse them, and all endpoints are documented in the API
reference (OpenAPI). `errorId` is the `error_<fingerprint>` friendly id.

## Attribution

State changes record who made them. A plain environment API key has no
user, so `resolvedBy`/`ignoredByUserId` stay null. When the caller uses
an environment JWT obtained by exchanging a personal access token or a
delegated user token at `POST /api/v1/projects/:ref/:env/jwt`, that
exchange now stamps an `act` delegation claim, and the write endpoints
read `act.sub` to attribute the change to the acting user. This is the
first endpoint to consume the `act` claim, so two small pieces of
plumbing ride along: the exchange stamps `act` for personal-access-token
subjects too (it was delegated-token-only), and the public-JWT
bearer-auth path surfaces `act.sub` to the handler.

Built on the delegated-token work in #3997.
2026-06-21 09:29:13 +01:00
Matt Aitken e4981d1b11 feat(webapp): consolidate auth path + add comprehensive auth tests (#3499)
## Summary

Consolidates the webapp's authentication and authorization into a small
set of route helpers, replacing the ad-hoc `requireUser` /
`requireUserId` / `authenticatedEnvironmentForAuthentication` calls
scattered across routes. Same security model, but the per-request flow
(authenticate → authorize → load) now lives in one place per route
family.

Introduces a plugin seam (`@trigger.dev/plugins`) that lets the cloud
build install a richer RBAC implementation without touching webapp code.
The OSS fallback keeps the pre-RBAC permissive behaviour intact, so
self-hosted deployments work unchanged.

Adds a comprehensive end-to-end auth test suite that didn't exist before
— 193 `it()` blocks (vitest reports ~199 after `it.each` expansion)
covering API key, PAT and JWT auth across the public API surface, plus
dashboard session auth for admin pages.

## Changes

### Plugin contract — `@trigger.dev/plugins`

`RoleBaseAccessController` interface authoritative for both OSS
(fallback) and cloud (enterprise plugin):
- `authenticateBearer(request, { allowJWT? })` — API-key / public-JWT
auth, returns env + ability
- `authenticateSession(request, { userId, organizationId?, projectId?
})` — dashboard auth, caller resolves `userId` from the session cookie
and passes it in (no `helpers.getSessionUserId` callback — decouples the
plugin host from session-cookie code)
- `authenticatePat(request, { organizationId?, projectId? })` — PAT
auth, returns identity + `lastAccessedAt` so the host can throttle the
per-request update
- `authenticateAuthorize*` variants for the auth-and-check-in-one-call
cases
- `isUsingPlugin(): Promise<boolean>` — capability flag for UI /
branching where plugin-present-ness matters; replaces the
sentinel-string coupling that had `personalAccessToken.server` matching
`"RBAC plugin not installed"` literally

### Dashboard auth (started, partial rollout)

Admin and settings pages migrated to a unified `dashboardLoader` /
`dashboardAction` helper that authenticates the session, runs an
authorization check, and exposes the result to the route. Other
dashboard routes still on the old pattern; remaining migration tracked
in TRI-8730.

Migrated routes:
- `admin.*` (14 admin / back-office / feature-flags / LLM-models /
notifications / orgs / concurrency pages)
- `_app.orgs.$organizationSlug.settings.team`
- `_app.orgs.$organizationSlug.settings.roles`

### API / realtime / engine auth (complete for the migrated families)

71 routes migrated to a unified `apiBuilder` that centralizes Bearer /
PAT / Public-JWT authentication and applies the per-route authorization
check before the handler runs. Includes:
- `api.v1.*` and `api.v2.*` and `api.v3.*` — tasks, runs, batches,
queues, prompts, deployments, query, sessions, waitpoints, packets,
workers, idempotency keys
- `realtime.v1.*` — runs, batches, sessions, streams
- `engine.v1.*` — dev / worker-action protocols

29 routes still on the legacy `authenticateApiRequest*` helpers —
tracked as a post-deploy follow-up in TRI-9228.

Multi-resource auth direction is now explicit at the call site via
`anyResource(...)` (OR) and `everyResource(...)` (AND). Bare arrays no
longer typecheck — fixes a class of bug where a JWT scoped to one
resource could implicitly access others under OR semantics.

PAT auth path consolidated: was three DB queries per request (legacy
`authenticateApiRequestWithPersonalAccessToken` findFirst +
`rbac.authenticatePat` join + `lastAccessedAt` update). Now one query in
the steady state — plugin returns `lastAccessedAt`, host smart-skips the
update via JS-side throttle when fresh.

Side effect: action aliases preserved historic JWT scope semantics where
the new model is stricter (e.g. a `write:tasks` JWT now also satisfies
`trigger` / `batchTrigger` / `update` actions on the same resource —
matched at the auth boundary, not in the route handler).

### Backwards-compat fixes

The strict-match model regressed several real-world JWT shapes. Each
preserved via explicit `anyResource(...)` entries in the route's authz
block:

- **Batch retrieve routes** (`api.v1.batches.$batchId`, `api.v2.*`,
`realtime.v1.batches.*`) accept `read:runs` JWTs again (pre-RBAC
literal-match superScope behaviour)
- **Runs list routes** (`api.v1.runs`, `realtime.v1.runs`) accept
type-level `read:tasks` / `read:tags` on unfiltered queries (matched the
legacy `Object.keys` iteration semantic)
- **PAT/OAT auth shape** normalized through `toAuthenticated` so all
auth methods return the same slim `AuthenticatedEnvironment` (was:
API-key returned the slim shape but PAT/OAT returned raw Prisma
`Decimal` / no `orgMember`)
- **Scope `:` preservation** in resource ids — `read:tags:env:staging`
now correctly identifies the tag id as `env:staging`, not `env`

### Slim `AuthenticatedEnvironment`

Extracted to `@trigger.dev/core/v3/auth/environment` — a structural
shape independent of `@trigger.dev/database`. The plugin contract
returns this; webapp consumers import from there; the cloud plugin
(Drizzle) returns the same shape without Prisma's `Decimal` class
leaking into the public surface. Lets internal-packages (run-engine,
etc.) refer to `AuthenticatedEnvironment` without pulling Prisma in.

### Auth test suite (new — `*.e2e.full.test.ts`)

193 e2e tests run against a real spawned webapp + Postgres (no mocks).
Coverage matrix:

- **API key auth** — read / write / trigger / batchTrigger / deploy
actions across runs, batches, deployments, prompts, queues, query,
sessions, input-streams, waitpoints, tasks, idempotency keys; multi-key
resources (a run carries batch / tag / task identifiers — auth must
accept any matching scope)
- **Personal Access Token auth** — comprehensive matrix: scope match,
scope mismatch, missing scope, expired token, malformed token
- **Public JWT auth** — sub-vs-URL environment resolution, expired JWTs,
signature verification, scope checking, otu (one-time-use) token
semantics, branch-environment signing-key fallback
- **Dashboard session auth** — admin-only pages reject non-admins;
per-action gating
- **Cross-cutting edge cases** — revoked API key grace window, JWT
cross-environment isolation, MissingResource branch behaviour

### Hygiene cleanups

- Deleted dead `app/services/authorization.server.ts` (legacy
`checkAuthorization` + types — no live consumers post-migration) and its
orphaned test
- Dropped the never-populated `scopes` field from
`ApiAuthenticationResultSuccess`
- `scheduleEmail` moved out of `email.server.ts` into its own module —
breaks a `commonWorker → marqs/V1` import chain that was poisoning the
auth test graph
- OSS Roles page shows a deployment-aware empty state ("Roles aren't
available in this self-hosted deployment" vs the plan-upsell copy) via
`rbac.isUsingPlugin()`
- Team action handler: explicit per-intent ability gates
(`manage:billing` for purchase-seats, `manage:members` for set-role +
remove-member with self-leave carve-out)

### Cross-repo coordination

All public-package contract changes paired in `triggerdotdev/cloud#763`
(rbac-packages branch) — the enterprise plugin implements the same
`RoleBaseAccessController` interface against Drizzle.

## Test plan

- [x] `pnpm run typecheck --filter webapp` clean
- [x] `pnpm --filter webapp exec vitest run --config
vitest.e2e.full.config.ts` — 193/193 pass (requires Docker for
testcontainers)
- [x] Spot-check an authed API endpoint with a valid + invalid API key
against a local stack
- [x] Spot-check the migrated admin pages render and gate non-admins

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 17:16:20 +01:00
Matt Aitken cf1c311dea Use the replica for API auth queries (#2932)
When we auth API keys we get the environment, project and org. This is a
very hot path so even though these queries are fast they contribute a
significant percentage of total load.

This moves them to use the read replica instead.
2026-01-23 16:23:51 +00:00
Eric Allam 4347499799 fix(api): Fix preview branch targeting in environment variable API routes (#2697)
This PR fixes the `x-trigger-branch` header support for targeting specific preview branches when managing environment variables. The header was documented but not actually being extracted or used in the environment variable API routes. Additionally, the query logic in `authenticatedEnvironmentForAuthentication` was fundamentally broken—it searched for environments with both `slug: "preview"` (parent environment property) AND a specific `branchName` (child environment property), which no environment could satisfy simultaneously. The fix extracts the branch name using `branchNameFromRequest()` and correctly queries for child branch environments using `type: "PREVIEW"` and the specific `branchName`. This ensures that environment variable operations (create, update, get, list) properly target individual preview branches instead of affecting all preview environments.
2025-11-19 18:01:12 +00:00
Eric Allam abee783d3f chore(logs): remove unnecessary debug logs (#2696) 2025-11-19 13:56:15 +00:00
Eric Allam 6464eeed53 fix(webapp): correctly generate JWT tokens for preview branches after triggering a run (fix #2678) (#2695) 2025-11-19 13:42:12 +00:00
Saadi Myftija e629810e96 fix: dev env connection issue (#2449)
This PR fixes an issue introduced in #2391 where the dev runtime env was not being resolved correctly.
2025-08-28 14:05:10 +01:00
Saadi Myftija 1cc62230ab feat: introduce organization access tokens (#2391)
* Create schema and migration for organization access tokens

* Add helpers for creating and authenticating OATs

* Adapt the auth service to also accept OATs

* Accept OATs in the whoami v2 endpoint

* Enable deployments with the CLI using OATs

* Avoid reading env variables directly in the token utils

* Remove duplicate cli token utils

* Validate ENCRYPTION_KEY length when parsing env vars

* Make token utils a server-only module

* Disallow revoking already revoked OATs

* Simplify generics in authenticateRequest

* Use 32 bytes mock encryption key in the test setup

* Update dummy encryption key values in tests and templates

* Add a column in the OATs table to differentiate between user and system generated

* Simplify args for v3ProjectPath

Co-authored-by: Matt Aitken <matt@mattaitken.com>

* Add index on org id and createdAt

* Avoid storing the encrypted oat token and its obfuscated version in the DB at all

It is a safer approach. Also we do not need to ever read the decrypted token value after creation.

* Fix prisma update condition

* Add token type to the OAT table index

* Accept OATs in the mcp auth flow

* Simplify env auth flow around the /projects endpoints

---------

Co-authored-by: Matt Aitken <matt@mattaitken.com>
2025-08-27 13:23:45 +02:00
Eric Allam 0f3179f493 Provide realtime skipColumns option via untamperable public access tokens (#2201)
* Provide realtime skipColumns option via untamperable public access tokens

* Fixed realtime client tests

* Create red-rings-marry.md
2025-06-26 15:13:26 +01:00
Matt Aitken 068c024477 Preview branches (#2086)
* Initial preview migrations

* Modified the staging endpoint to create preview environments

* Added isBranchableEnvironment to RuntimeEnvironment

* Staging = yellow Preview = orange

* Changed the env sort order

* Set isBranchableEnvironment correctly. Create preview for new projects

* Very basic branch menu

* Creating branches from the dashboard

* Fix for string icons on project delete page

* Don’t show branch API keys

* WIP on the manage branches page

* RuntimeEnvironment added projectId index

* Only create the parentEnvironmentId column if it doesn’t exist already

* Improved the limit wording

* Add search to the branch list

* contains in both places

* Many style improvements

* Branch dropdown and v4 badge

* Arching/unarchive branches working in the dashboard

* Tidied imports

* Change preview slug from `prev` to `preview`

* Use correct color for side menu preview branch icon

* Upsert the branch and use the shortcode as a unique constraint

* Upserting working with nice messages in the dashboard

* Better errors when upserting branches

* Button shortcut, don’t allow event to propagate

* Better duplicate error message

* Filter out archived branches from the env selector

* Archiving/creating tweaked some more

* Add an archived banner to the app, fixes for archived branches and upsells

* Fixed pagination

* Disable editing schedules, pausing queues, testing tasks

* Don’t allow replaying if the env is archived

* When deploying detect the correct environment

* Get the projectClient when there’s a branch

* createGitMeta function, most code from the vercel CLI repo

* Deploy, getting the correct environment client

* Added git column to WorkerDeployment

* Add GitMeta to core schemas

* Create branch when deploying

* WIP on branch support in the API

* Delete old createTaskRunAttempt fn

* apiAuth remove export from internal functions

* Rename env var to “TRIGGER_PREVIEW_BRANCH”

* Add TRIGGER_PREVIEW_BRANCH to resolved env vars for runs

* First preview deploy and run working

* Set the preview branch in the main SDK

* Added git links to the preview branches table

* Better errors when replaying/testing archived branches

* Don’t dequeue archived environments

* Env var resolution with parent environment

* Hello world default machine small-2x to save my memory

* Fix for more env var functions

* Only return non-archived envs

* Switch to controlled state for the checkboxes

* Uncheck everything when PREVIEW is checked

* WIP on branch UI

* Show the preview branch label on the env vars list

* Fix for overriding env vars

* Adding preview branch env vars working

* Progress on new env vars

* Only allow selecting a single branch

* Layout fix when there are errors

* Set the defaultValue so there are some fields

* Conform fix for team invite page

* Archived environments don’t run scheduled tasks

* Added Git data to deployments

* Added git data to the deployment inspector

* Don’t allow upserting schedules when archived

* Deduplicate and blacklist some env vars

* Fix for wrong conform function being used

* Show a better error if all vars were blacklisted

* Added environment variable search (by key and value)

* Improved preview branch icon

* Replay now supports branches

* Schedule page render branches properly

* Show the env icon in bottom-left of the test page

* When editing older schedules (that have multi-env) show preview branches correctly

* Fix for incorrect disallowed branch name character

* Extract and improve the directory verification code

* WIP for CLI preview archive command

* Improved the preview branch action buttons

* Redirect to the project if we don’t find a matching env

* Archiving branch via the CLI working

* Fix for archiving branches

* Public access token test task

* JWTs working are with preview branches

* Add branch and git data to the Run ctx

* Updated GitMeta functions to work in CI

* Added pullRequestState

* Archive when deploying if the PR is closed/merged

* Fix for the changesets guide

* Fix for CLI dev bug introduced

* CLI promote now supports preview branches

* Add PR title. Reordered them and added tooltips

* syncEnvVars working with branches

* Added preview branch support to syncVercelEnvVars()

* Detect the branch from Vercel env var (set during build)

* Allow passing a branch in

* Use process.env.VERCEL_TOKEN as well… this used in Vercel CI

* Temp delete

* Improved regenerate api key modal

* Added Accordion component (with styles)

* Redesigned the API keys page

* Revert "Temp delete"

This reverts commit 177b92cd935a6161456bde65d01294e23ecfd47f.

* Changeset

* Fixed docs link

* The new branch panel closes when a branch is created

* Update apps/webapp/app/services/upsertBranch.server.ts

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>

* Removed findUniques from WorkerGroupTokenService

* Made the parentEnvironmentId migrations safe

* Latest lockfile

* Update packages/cli-v3/src/commands/workers/build.ts

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>

* Move isValidGitBranchName to a separate file

* Move the sanitize fn too

* removeBlacklistedVariables moved to a separate file

* Moved deduplicateVariableArray to a separate file…

* Fix broken sanitizeBranchName import

* Another import fix…

* Improved blacklisted error message

* SImplified migration to use `ADD COLUMN IF NOT EXISTS "parentEnvironmentId" TEXT`

---------

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
2025-05-28 11:02:56 +01:00
Eric Allam f96bf7210b Engine v1 improvements (#1627)
* Efficiency improvements to SharedQueueConsumer and MarQS for run engine v1

* Reliabily resolve task queue for a run and ack runs where we can't find the queue

* engine v1: stop nacking after reaching the max nack count, ack to remove the message

* Handle division by 0 possibility in distributeQueues
2025-01-21 14:24:02 +00:00
Eric Allam 364ea565ed Use one-time use tokens when triggering or batch triggering from the frontend (#1515)
* Use one-time use tokens when triggering or batch triggering from the frontend

* Add changeset
2024-11-29 20:16:37 +00:00
Eric Allam 7e209e0771 Stop using Response errors when validating API Keys (#1498)
* Stop using Response errors when validating API Keys, instead introduce a new "Result" type that has success and failure conditions. Adding in a way to progressively adopt because this touches everything.

* Make sure authenticateApiRequestWithFailure always returns a result
2024-11-25 15:19:05 +00:00
Eric Allam 23b43be952 feat/realtime-streams (#1470)
* WIP realtime streams

* Handle realtime with large payloads or outputs #1451

* feat: optimize Redis stream handling with batching

Add STREAM_ORIGIN to environment schema. Improve performance in
RealtimeStreams by using TextDecoderStream for simpler text
decoding and implementing batching of XADD commands for Redis
streams. Limit stream size using MAXLEN option. Update
environment variable repository with new variable type. Adjust
import statements for Redis key and value types.

* 🔧 chore: add dev dependencies for bundle analysis

* add metadata tests and a few more utilties

* Add stream tests and improve streaming

* Added AI tool tasks, descriptions to tasks

* Use the config file path to determine the workingDir, then the package.json path

* Remove stream test files

* useTaskTrigger react hook that allows triggering a task from the client

* Add streaming support for the realtime react hooks

* Add ability to stream results after useTaskTrigger

* Improve the stream throttling

* Use the runId as the ID key to bust the cache after triggering

* Upgrade to to the latest electric sql client and server

* Make realtime server backwards compat with 3.1.2 release

* Pass the runId into useRealtimeRun

* Fix scopes when specifiying reading all runs

* WIP @trigger.dev/rsc package

* Various fixes and accepted recommendations by CodeRabbit

* Regenerate pnpm lock file

* A couple tweaks to rsc and give up on rendering react in tasks for now

* Add changeset

* Remove triggerRequest from the useEffect deps

* Improve realtime & frontend authentication errors

* Fixed authorization tests

* Remove unnecessary log

* Add metadata.stream limits and improve the metadata streams structure

* Streams can now have up to 2500 entries

* Various coderabbit fixes

* additional react-hooks jsdocs
2024-11-19 13:14:25 +00:00
Eric Allam 2d8a41b18b feat: realtime (#1402)
* Denormalize run tags, increase character limit to 128

* WIP realtime subscribing to runs

* extracted the stream stuff into core, made it more reusable

* WIP tags

* Remove tags for now because it’s not support in electric

* Support async iterables, readable stream, and callback style subscription styles

* Remove tags streaming endpoint

* Add realtime rate limits and scope them to the /realtime path

* WIP rate limt per org

* Introduce per org rate limits

* WIP JWT auth

* Move migrations into new internal db package

* Resolve pnpm lock file

* Authenticating to the realtime API with JWTs are working

* realtime in the client

* Created react-hooks package and starting to move stuff in there

* Improve types for hooks

* schema tasks

* Added useBatch hook

* build uploadthing/fal demo and change how run metadata is synced to the server

* tweaks

* WIL realtime concurrency tracking

* Implement test for realtime client using testcontainers

also updated electric to latest version

* Allow customizing the expiration time of the automatic JWT created after triggering a task

* Add support for subscribing to run tags

* Improve auth types and API

* finalize the realtime API

* Fixed some example stuff

* Allow up to 10 run tags

* Remove core from docker-provider tsconfig paths to prevent it from being typechecked

* do the same for the kubernetes provider

* Fixing some typecheck errors

* Fix webapp type errors

* Update @trigger.dev/platform to 1.0.13

* Fix attw error

* Remove from/to in subscribeToRuns query params

* Add tests for the rate limit middleware and add custom JWT rate limits

* turn off webapp test parallelism

* Finish renaming jwt -> publicAccessToken and automatically give the JWT read access to the tags when using trigger

* Add changeset

* Attempt to fix unit tests in CI

* Skip running the auth rate limit middleware tests for now

* Try a beefier machine

* Try and run webapp tests separately

* Setup env vars

* Make sliding window test more reliabile
2024-10-21 15:07:08 +01:00
Matt Aitken ec8381e2d0 Fix for staging environment variable API endpoints
Ugly but effective fix for the fact we can pass the slug in as “staging” when we use “stg” internally
2024-07-15 16:33:00 +01:00
Eric Allam 85a543d8ec v3: usage tracking (#1163)
* Starting to measure wall time and cpu time in the workers, and reporting that via otel and to completed task run attempts

* Move usage tracking outside of the executor

* WIP prod usage tracking

* WIP

* WIP custom fetch to openmeter

* Create a usage client

* WIP

* WIP

* Implement new machine preset stuff and send usage reports to OpenMeter from webapp

* WIP

* Expose usage info to the client

* Add usage and cost to TaskEvent

* Add ability to globally configure the task machine preset

* Report start run usage

* Change the machine docs to use presets

* setExpirationTime to 24h

* Removed logs

* Update machines.mdx

* Removed console.logs

* Handle revalidating JWT tokens

* Couple tweaks

---------

Co-authored-by: Matt Aitken <matt@mattaitken.com>
2024-06-18 09:40:23 +01:00
Eric Allam 3a1b0c486a v3: env var management API (#1116)
* WIP env var management API

* Add import env var API endpoint

* Adding docs and support for using both API keys and PATs when interacting with the env var endpoints

* WIP envvar SDK

* Uploading env vars in a variety of formats now works

* Finish env var endpoints and add resolveEnvVars hook

* Add changeset
2024-05-23 16:11:25 +01:00
Eric Allam 07f7c65054 v3 queue system (#903)
* Introducing Modular Asynchronous Reliable Queueing System (MarQS). Works in dev

* Convert MarQS to using lua and dealing with concurrency

* Simplified the timeout queue and current concurrency is now a set instead of a flat value (to support idempotency)

* Implement task heartbeating and reconnect the background workers CLI when the websocket connection reconnects

* Start adding internal telemetry support for the server

* Get env vars to work in dev and implement prisma tracing in webapp

* Cleanup telemetry and implement it in the consumer

* Implement dequeuing a message from a parent shared queue

* Implement a custom logger exporter instead of using console log exporter

* Use node instead of shell for generating protocol buffer code

* Propogate trace context into debug logs, and allow turning off logger exporter through env vars

* Switch to using baselime for internal otel data

* Make orgMember optional to fix type issues

* Provide the CLI dev env vars through the CLI, don’t build dotenv into facade

* Removed the logger import

* Address Matt’s comments

* Addressing more of Matt’s comments

* Handle sending an execution after a websocket connection closes

* Remove auth from the env attributes to prevent obfuscation
2024-02-20 12:16:41 +00:00
Matt Aitken 2b48e6d04b Feature: sendEvent can use an AWS SQS queue to increase reliability and throughput (#729)
* Initial commit (by create-cloudflare CLI)

* Changed the prettier rules

* Started writing Readme and got dev working

* Validate the request, parse the event and respond with the correct format

* Use ulidx instead of ulid in core

* Removed some boilerplate and use new function for sendEvent

* Removed old ulid package

* When handling a webhook, it’s not an error if the HTTP endpoint environment isn’t found

* If the env vars aren’t set, log the request and pass it through

* Pass requests through. Setup the env vars

* Flipped the API key detection because we can enforce that it’s the shape of a private API key

* Progress with sendEvent proxy

* Events are being put on the SQS queue

* The SQS event queue is working

* Make sure there’s a timestamp before enqueuing the evnet

* Log the sqs_event, not API key

* Added a bit more detail to the readme

* Improved the Cloudflare logs

* We don’t need the global.window hack for AWS, just globalThis

* Removed commented out wrangler.toml values

* Reworked the proxy to make it easier to add more endpoints

* Use json utility to send Responses

* Start work on proxying bulk events
2023-11-13 15:08:01 +00:00
Matt Aitken 9ff4c0dbd5 Project-wide Prettier setup (#237)
* Setup project-wide prettier

* Remove old workspace file

* Remove old debugging directives

* New top-level .prettierignore

* Updated Prettier config settings

* Contrubuting guide: Fix for some bad code blocks

* Added more ignores

* Improved the format script command

* printWidth set to 100

* Formatted entire repo (pnpm run format)
2023-08-01 10:21:22 +01:00
Matt Aitken 4c79129242 authenticateApiRequest() now returns an object with the apiKey, type and environment 2023-07-26 21:54:20 +01:00
Matt Aitken 3a6c25d42d API authentication updated to work correctly with public and private keys 2023-07-25 17:52:20 +01:00
Matt Aitken 4d2a8728be Getting events is allowed with allowPublicKey 2023-07-23 19:06:18 +01:00
Matt Aitken ae3afdfc7e Allow public key api requests, if the flag is passed 2023-07-23 19:05:30 +01:00
Eric Allam 3e87e832c7 WIP trigger variants 2023-05-04 10:06:06 +01:00
Eric Allam 1bba822e20 Added projects, moved envs to projects, and renamed executions to runs 2023-04-27 13:50:22 +01:00
Eric Allam d8c22b61ef Workflows are now being registered in the webapp, along with their trigger 2022-12-15 16:41:08 +00:00