ts7
5 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Chris Arderne
|
c7861be520 |
chore: activate no-unused-vars and import linters (#4096)
Once this is merged, oxlint is at a pretty sensible baseline. **Enable `no-unused-vars`, `typescript/consistent-type-imports`, and `import/no-duplicates` lint rules** Turns on three previously-disabled oxlint rules across the monorepo and fixes all violations: - **`no-unused-vars`** – enabled as an error with standard ignore patterns: unused function arguments are ignored by default (`args: "none"`), variables/caught errors/destructured array elements prefixed with `_` are allowed, and rest siblings are permitted. - **`typescript/consistent-type-imports`** – enforced as an error; all type-only imports now use the `import type` syntax. - **`import/no-duplicates`** – enforced as an error; duplicate import statements from the same module have been merged. The remaining commits clean up the violations found across the codebase: removing unused variables/imports/type aliases, adding `_` prefixes to intentionally unused bindings, fixing duplicate imports, and converting value imports to `import type` where appropriate. |
||
|
Matt Aitken
|
e4981d1b11 |
feat(webapp): consolidate auth path + add comprehensive auth tests (#3499)
## Summary
Consolidates the webapp's authentication and authorization into a small
set of route helpers, replacing the ad-hoc `requireUser` /
`requireUserId` / `authenticatedEnvironmentForAuthentication` calls
scattered across routes. Same security model, but the per-request flow
(authenticate → authorize → load) now lives in one place per route
family.
Introduces a plugin seam (`@trigger.dev/plugins`) that lets the cloud
build install a richer RBAC implementation without touching webapp code.
The OSS fallback keeps the pre-RBAC permissive behaviour intact, so
self-hosted deployments work unchanged.
Adds a comprehensive end-to-end auth test suite that didn't exist before
— 193 `it()` blocks (vitest reports ~199 after `it.each` expansion)
covering API key, PAT and JWT auth across the public API surface, plus
dashboard session auth for admin pages.
## Changes
### Plugin contract — `@trigger.dev/plugins`
`RoleBaseAccessController` interface authoritative for both OSS
(fallback) and cloud (enterprise plugin):
- `authenticateBearer(request, { allowJWT? })` — API-key / public-JWT
auth, returns env + ability
- `authenticateSession(request, { userId, organizationId?, projectId?
})` — dashboard auth, caller resolves `userId` from the session cookie
and passes it in (no `helpers.getSessionUserId` callback — decouples the
plugin host from session-cookie code)
- `authenticatePat(request, { organizationId?, projectId? })` — PAT
auth, returns identity + `lastAccessedAt` so the host can throttle the
per-request update
- `authenticateAuthorize*` variants for the auth-and-check-in-one-call
cases
- `isUsingPlugin(): Promise<boolean>` — capability flag for UI /
branching where plugin-present-ness matters; replaces the
sentinel-string coupling that had `personalAccessToken.server` matching
`"RBAC plugin not installed"` literally
### Dashboard auth (started, partial rollout)
Admin and settings pages migrated to a unified `dashboardLoader` /
`dashboardAction` helper that authenticates the session, runs an
authorization check, and exposes the result to the route. Other
dashboard routes still on the old pattern; remaining migration tracked
in TRI-8730.
Migrated routes:
- `admin.*` (14 admin / back-office / feature-flags / LLM-models /
notifications / orgs / concurrency pages)
- `_app.orgs.$organizationSlug.settings.team`
- `_app.orgs.$organizationSlug.settings.roles`
### API / realtime / engine auth (complete for the migrated families)
71 routes migrated to a unified `apiBuilder` that centralizes Bearer /
PAT / Public-JWT authentication and applies the per-route authorization
check before the handler runs. Includes:
- `api.v1.*` and `api.v2.*` and `api.v3.*` — tasks, runs, batches,
queues, prompts, deployments, query, sessions, waitpoints, packets,
workers, idempotency keys
- `realtime.v1.*` — runs, batches, sessions, streams
- `engine.v1.*` — dev / worker-action protocols
29 routes still on the legacy `authenticateApiRequest*` helpers —
tracked as a post-deploy follow-up in TRI-9228.
Multi-resource auth direction is now explicit at the call site via
`anyResource(...)` (OR) and `everyResource(...)` (AND). Bare arrays no
longer typecheck — fixes a class of bug where a JWT scoped to one
resource could implicitly access others under OR semantics.
PAT auth path consolidated: was three DB queries per request (legacy
`authenticateApiRequestWithPersonalAccessToken` findFirst +
`rbac.authenticatePat` join + `lastAccessedAt` update). Now one query in
the steady state — plugin returns `lastAccessedAt`, host smart-skips the
update via JS-side throttle when fresh.
Side effect: action aliases preserved historic JWT scope semantics where
the new model is stricter (e.g. a `write:tasks` JWT now also satisfies
`trigger` / `batchTrigger` / `update` actions on the same resource —
matched at the auth boundary, not in the route handler).
### Backwards-compat fixes
The strict-match model regressed several real-world JWT shapes. Each
preserved via explicit `anyResource(...)` entries in the route's authz
block:
- **Batch retrieve routes** (`api.v1.batches.$batchId`, `api.v2.*`,
`realtime.v1.batches.*`) accept `read:runs` JWTs again (pre-RBAC
literal-match superScope behaviour)
- **Runs list routes** (`api.v1.runs`, `realtime.v1.runs`) accept
type-level `read:tasks` / `read:tags` on unfiltered queries (matched the
legacy `Object.keys` iteration semantic)
- **PAT/OAT auth shape** normalized through `toAuthenticated` so all
auth methods return the same slim `AuthenticatedEnvironment` (was:
API-key returned the slim shape but PAT/OAT returned raw Prisma
`Decimal` / no `orgMember`)
- **Scope `:` preservation** in resource ids — `read:tags:env:staging`
now correctly identifies the tag id as `env:staging`, not `env`
### Slim `AuthenticatedEnvironment`
Extracted to `@trigger.dev/core/v3/auth/environment` — a structural
shape independent of `@trigger.dev/database`. The plugin contract
returns this; webapp consumers import from there; the cloud plugin
(Drizzle) returns the same shape without Prisma's `Decimal` class
leaking into the public surface. Lets internal-packages (run-engine,
etc.) refer to `AuthenticatedEnvironment` without pulling Prisma in.
### Auth test suite (new — `*.e2e.full.test.ts`)
193 e2e tests run against a real spawned webapp + Postgres (no mocks).
Coverage matrix:
- **API key auth** — read / write / trigger / batchTrigger / deploy
actions across runs, batches, deployments, prompts, queues, query,
sessions, input-streams, waitpoints, tasks, idempotency keys; multi-key
resources (a run carries batch / tag / task identifiers — auth must
accept any matching scope)
- **Personal Access Token auth** — comprehensive matrix: scope match,
scope mismatch, missing scope, expired token, malformed token
- **Public JWT auth** — sub-vs-URL environment resolution, expired JWTs,
signature verification, scope checking, otu (one-time-use) token
semantics, branch-environment signing-key fallback
- **Dashboard session auth** — admin-only pages reject non-admins;
per-action gating
- **Cross-cutting edge cases** — revoked API key grace window, JWT
cross-environment isolation, MissingResource branch behaviour
### Hygiene cleanups
- Deleted dead `app/services/authorization.server.ts` (legacy
`checkAuthorization` + types — no live consumers post-migration) and its
orphaned test
- Dropped the never-populated `scopes` field from
`ApiAuthenticationResultSuccess`
- `scheduleEmail` moved out of `email.server.ts` into its own module —
breaks a `commonWorker → marqs/V1` import chain that was poisoning the
auth test graph
- OSS Roles page shows a deployment-aware empty state ("Roles aren't
available in this self-hosted deployment" vs the plan-upsell copy) via
`rbac.isUsingPlugin()`
- Team action handler: explicit per-intent ability gates
(`manage:billing` for purchase-seats, `manage:members` for set-role +
remove-member with self-leave carve-out)
### Cross-repo coordination
All public-package contract changes paired in `triggerdotdev/cloud#763`
(rbac-packages branch) — the enterprise plugin implements the same
`RoleBaseAccessController` interface against Drizzle.
## Test plan
- [x] `pnpm run typecheck --filter webapp` clean
- [x] `pnpm --filter webapp exec vitest run --config
vitest.e2e.full.config.ts` — 193/193 pass (requires Docker for
testcontainers)
- [x] Spot-check an authed API endpoint with a valid + invalid API key
against a local stack
- [x] Spot-check the migrated admin pages render and gate non-admins
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
James Ritchie
|
7a7c4b1a82 |
feat(webapp): New limits page (#2885)
<img width="1381" height="1362" alt="CleanShot 2026-01-14 at 13 41 02" src="https://github.com/user-attachments/assets/0537dccf-60c7-4ab7-a0e4-3164eac1e97d" /> --------- Co-authored-by: Matt Aitken <matt@mattaitken.com> |
||
|
Eric Allam
|
8ba7526d51 |
fix(batch): rate limiting by token bucket no longer incorrectly goes negative (#2837)
Also improves the BatchTriggerError when a result of getting rate limited. |
||
|
Eric Allam
|
a999d9ea3f |
feat(engine): Batch trigger reloaded (#2779)
New batch trigger system with larger payloads, streaming ingestion, larger batch sizes, and a fair processing system. This PR introduces a new `FairQueue` abstraction inspired by our own `RunQueue` that enables multi-tenant fair queueing with concurrency limits. The new `BatchQueue` is built on top of the `FairQueue`, and handles processing Batch triggers in a fair manner with per-environment concurrency limits defined per-org. Additionally, there is a global concurrency limit to prevent the BatchQueue system from creating too many runs too quickly, which can cause downstream issues. For this new BatchQueue system we have a completely new batch trigger creation and ingestion system. Previously this was a single endpoint with a single JSON body that defined details about the batch as well as all the items in the batch. We're introducing a two-phase batch trigger ingestion system. In the first phase, the BatchTaskRun record is created (and possibly rate limited). The second phase is another endpoint that accepts an NDJSON body with each line being a single item/run with payload and options. At ingestion time all items are added to a queue, in order, and then processed by the BatchQueue system. ## New batch trigger rate limits This PR implements a new batch trigger specific rate limit, configured on the `Organization.batchRateLimitConfig` column, and defaults using these environment variables: - `BATCH_RATE_LIMIT_REFILL_RATE` defaults to 10 - `BATCH_RATE_LIMIT_REFILL_INTERVAL` the duration interval, defaults to `"10s"` - `BATCH_RATE_LIMIT_MAX` defaults to 1200 This rate limiter is scoped to the environment ID and controls how many runs can be submitted via batch triggers per interval. The SDK handles the retrying side. ## Batch queue concurrency limits The new column `Organization.batchQueueConcurrencyConfig` now defines an org specific `processingConcurrency` value, with a backup of the env var `BATCH_CONCURRENCY_LIMIT_DEFAULT` which defaults to 10. This controls how many batch queue items are processed concurrently per environment. There is also a global rate limit for the batch queue set via the `BATCH_QUEUE_GLOBAL_RATE_LIMIT` which defaults to being disabled. If set, the entire batch queue system won't process more than `BATCH_QUEUE_GLOBAL_RATE_LIMIT` items per second. This allows controlling the maximum number of runs created per second via batch triggers. ## Batch trigger settings - `STREAMING_BATCH_MAX_ITEMS` controls the maximum number of items in a single batch - `STREAMING_BATCH_ITEM_MAXIMUM_SIZE` controls the maximum size of each item in a batch - `BATCH_CONCURRENCY_DEFAULT_CONCURRENCY` controls the default environment concurrency - `BATCH_QUEUE_DRR_QUANTUM` how many credits each environment gets each round for the DRR scheduler - `BATCH_QUEUE_MAX_DEFICIT` the maximum deficit for the DRR scheduler - `BATCH_QUEUE_CONSUMER_COUNT` how many queue consumers to run - `BATCH_QUEUE_CONSUMER_INTERVAL_MS` how frequently they poll for items in the queue ### Configuration Recommendations by Use Case **High-throughput priority (fairness acceptable at 0.98+):** ```env BATCH_QUEUE_DRR_QUANTUM=25 BATCH_QUEUE_MAX_DEFICIT=100 BATCH_QUEUE_CONSUMER_COUNT=10 BATCH_QUEUE_CONSUMER_INTERVAL_MS=50 BATCH_CONCURRENCY_DEFAULT_CONCURRENCY=25 ``` **Strict fairness priority (throughput can be lower):** ```env BATCH_QUEUE_DRR_QUANTUM=5 BATCH_QUEUE_MAX_DEFICIT=25 BATCH_QUEUE_CONSUMER_COUNT=3 BATCH_QUEUE_CONSUMER_INTERVAL_MS=100 BATCH_CONCURRENCY_DEFAULT_CONCURRENCY=5 ``` |