github-featured/triggerdotdev--trigger.dev
2
0
Fork 0
Code Issues Pull Requests Deployments Models Agent Notes Packages Projects Releases Wiki Activity
Files
docs/mcp-readonly-runtime-flag
triggerdotdev--trigger.dev/apps/webapp/app/env.server.ts
T
Daniel Sutton 139eede41c feat(redis-worker): batched pop in MollifierDrainer for fast single-env drains (#3797) 
## Summary

Adds `drainBatchSize` to `MollifierDrainer` (default `1` — preserves
existing behaviour) and wires `TRIGGER_MOLLIFIER_DRAIN_BATCH_SIZE`
through the webapp (default `50`). Each tick the drainer now pops up to
`drainBatchSize` from each chosen env, then dispatches every popped
entry through the shared `concurrency`-bounded `pLimit`. Per-org/per-env
fairness is unchanged — only the in-env pop count grows.

Pre-existing behaviour was one pop per env per tick. For a single-env
burst that single-flighted the drain at the per-tick floor of `pop +
engine.trigger ≈ 50–60 ms`. With buffer entries piling up under a
real-world tenant burst that's tens of minutes of tail latency to fully
materialise — even though PG itself could comfortably sustain the
writes.

## Why this matters — heavy-tail illustration

Scenario: 100 customers in one window — 94 fire 20 triggers each, 5 fire
100, 1 fires 1000. Gate at `THRESHOLD=10/s`, `HOLD_MS=500`. First 10 of
each burst hit PG directly; the rest buffer.

| Customers | Triggers each | PG direct | Buffered each | Total buffered
|
|---|---|---|---|---|
| 94 small  | 20    | 10  | 10  | 940 |
| 5 medium  | 100   | 10  | 90  | 450 |
| 1 heavy   | 1000  | 10  | 990 | 990 |

**With `DRAIN_BATCH_SIZE=50`, `DRAIN_CONCURRENCY=50`, ~50 ms
`engine.trigger`:**

| Tick | Pops | Dispatch waves | Wall-clock |
|---|---|---|---|
| 1 | 94×10 + 5×50 + 1×50 = 1 240 | 25 × 50 ms | ~1 300 ms (94 smalls
done) |
| 2 | 5×40 + 1×50 = 250 | 5 × 50 ms | ~300 ms (5 mediums done) |
| 3–20 | heavy alone, 50/tick | 1 × 50 ms | ~100 ms each |

| Customer class | Buffered fully drained |
|---|---|
| 94 small  | **~1.3 s** |
| 5 medium  | **~1.6 s** |
| 1 heavy   | **~3.4 s** |

**Without batching (one pop per env per tick — current behaviour):**

| Customer class | Buffered fully drained |
|---|---|
| 94 small  | ~500 ms |
| 5 medium  | ~4.5 s |
| 1 heavy   | **~49 s** |

So the heavy single-tenant tail drops from ~49 s to ~3.4 s (~14× faster)
without changing PG load characteristics. Smalls go up slightly in this
scenario (500 ms → 1.3 s) because all 100 envs share one tick's dispatch
queue — that's the trade we accept for the heavy tail; the worst-case
small wait is still inside one tick. PG load is identical either way (50
concurrent inserts at a time, capped by `DRAIN_CONCURRENCY`).

## What changed

**`packages/redis-worker`**
- New `drainBatchSize` option (default 1 — full backward compat).
- `runOnce()` refactored to pop per-env batches in parallel, then
dispatch all popped entries through the existing global `pLimit`.
Mid-batch pop failure aborts only that env's batch and counts as one
failure (same semantic as the old per-env path).
- Removed the now-unused `processOneFromEnv` helper.

**`apps/webapp`**
- `TRIGGER_MOLLIFIER_DRAIN_BATCH_SIZE` env var (default 50, matching
`DRAIN_CONCURRENCY`).
- Wired into `mollifierDrainer.server.ts`.

**Test cloud config** (separate cloud PR):
`TRIGGER_MOLLIFIER_DRAIN_BATCH_SIZE="50"` on the worker service.
Production rollout deferred until we've watched it on test cloud.

## Test plan

- [x] All 25 stub-based drainer tests pass (18 pre-existing + 7 new). 7
new tests under `MollifierDrainer.drainBatchSize`:
  - pops up to `drainBatchSize` across ticks
  - global `concurrency` cap still holds when batch > concurrency
  - mid-batch pop failure isolation
  - multi-env batch fan-out in one tick
- **hierarchical org fairness preserved at `drainBatchSize > 1`**
(load-bearing — guards against future regressions to
per-env-instead-of-per-org rotation)
  - mixed success/failure accounting in a batched tick
  - bounded pops on empty queue (no Lua spam past `drainBatchSize`)
- [x] All pre-existing tests still pass unchanged at default
`drainBatchSize=1` → backward-compat locked.
- [x] `pnpm run build --filter @trigger.dev/redis-worker` clean.
- [x] `pnpm run typecheck --filter webapp` clean.
- [x] `redisTest` block (real Redis via testcontainers) — couldn't run
locally on this branch due to testcontainers runtime discovery; will
validate in CI.
- [ ] Test-cloud smoke after cloud PR lands: fire `burst 50` against a
flagged env and confirm the 50th entry's drain time drops from ~2.5 s to
<200 ms.

## Notes

- Per-tick memory bound: `maxOrgsPerTick × drainBatchSize` entries can
sit in the JS pLimit queue between pop and dispatch. At defaults that's
`500 × 50 = 25 000` × ~5 KB snapshot ≈ ~125 MB worst case per worker —
well within headroom.
- The pre-batch model's strict per-env throughput cap of `1/tick` is
documented as the fairness baseline elsewhere. Org-level fairness is
what callers actually rely on; this change does not weaken that.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-02 14:21:37 +01:00

1698 lines
76 KiB
TypeScript
Raw Permalink Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
import { z } from "zod";
import { MachinePresetName } from "@trigger.dev/core/v3";
import { BoolEnv } from "./utils/boolEnv";
import { isValidDatabaseUrl } from "./utils/db";
import { isValidRegex } from "./utils/regex";
import { isValidDuration } from "./services/realtime/duration.server";
// `z.string()` constrained to a `parseDuration`-parseable string (e.g.
// `7d`, `1h`). Validated at boot so a typo'd duration fails fast.
function durationString() {
return z
.string()
.refine(isValidDuration, "must be a duration like 7d, 30d, 365d, 1h, 1y");
}
// Parses a CSV of machine preset names (e.g. "small-1x,small-2x") into a
// non-empty array of MachinePresetName. Used by COMPUTE_TEMPLATE_MACHINE_PRESETS
// and its _REQUIRED variant. Adds zod issues for empty input or unknown names.
const parseMachinePresetCsv = (
raw: string,
ctx: z.RefinementCtx
): MachinePresetName[] => {
const names = raw
.split(",")
.map((s) => s.trim())
.filter(Boolean);
if (names.length === 0) {
ctx.addIssue({
code: z.ZodIssueCode.custom,
message: "must list at least one machine preset",
});
return z.NEVER;
}
const out: MachinePresetName[] = [];
for (const name of names) {
const parsed = MachinePresetName.safeParse(name);
if (!parsed.success) {
ctx.addIssue({
code: z.ZodIssueCode.custom,
message: `unknown machine preset: "${name}"`,
});
return z.NEVER;
}
out.push(parsed.data);
}
return out;
};
const GithubAppEnvSchema = z.preprocess(
(val) => {
const obj = val as any;
if (!obj || !obj.GITHUB_APP_ENABLED) {
return { ...obj, GITHUB_APP_ENABLED: "0" };
}
return obj;
},
z.discriminatedUnion("GITHUB_APP_ENABLED", [
z.object({
GITHUB_APP_ENABLED: z.literal("1"),
GITHUB_APP_ID: z.string(),
GITHUB_APP_PRIVATE_KEY: z.string(),
GITHUB_APP_WEBHOOK_SECRET: z.string(),
GITHUB_APP_SLUG: z.string(),
}),
z.object({
GITHUB_APP_ENABLED: z.literal("0"),
}),
])
);
// eventually we can make all S2 env vars required once the S2 OSS version is out
const S2EnvSchema = z.preprocess(
(val) => {
const obj = val as any;
if (!obj || !obj.S2_ENABLED) {
return { ...obj, S2_ENABLED: "0" };
}
return obj;
},
z.discriminatedUnion("S2_ENABLED", [
z.object({
S2_ENABLED: z.literal("1"),
S2_ACCESS_TOKEN: z.string(),
S2_DEPLOYMENT_LOGS_BASIN_NAME: z.string(),
S2_DEPLOYMENT_STREAMS_LOCAL: z.string().default("0"),
}),
z.object({
S2_ENABLED: z.literal("0"),
}),
])
);
const EnvironmentSchema = z
.object({
NODE_ENV: z.union([z.literal("development"), z.literal("production"), z.literal("test")]),
DATABASE_URL: z
.string()
.refine(
isValidDatabaseUrl,
"DATABASE_URL is invalid, for details please check the additional output above this message."
),
DATABASE_CONNECTION_LIMIT: z.coerce.number().int().default(10),
DATABASE_POOL_TIMEOUT: z.coerce.number().int().default(60),
DATABASE_CONNECTION_TIMEOUT: z.coerce.number().int().default(20),
DIRECT_URL: z
.string()
.refine(
isValidDatabaseUrl,
"DIRECT_URL is invalid, for details please check the additional output above this message."
),
DATABASE_READ_REPLICA_URL: z.string().optional(),
SESSION_SECRET: z.string(),
MAGIC_LINK_SECRET: z.string(),
ENCRYPTION_KEY: z
.string()
.refine(
(val) => Buffer.from(val, "utf8").length === 32,
"ENCRYPTION_KEY must be exactly 32 bytes"
),
WHITELISTED_EMAILS: z
.string()
.refine(isValidRegex, "WHITELISTED_EMAILS must be a valid regex.")
.optional(),
ADMIN_EMAILS: z.string().refine(isValidRegex, "ADMIN_EMAILS must be a valid regex.").optional(),
REMIX_APP_PORT: z.string().optional(),
LOGIN_ORIGIN: z.string().default("http://localhost:3030"),
LOGIN_RATE_LIMITS_ENABLED: BoolEnv.default(true),
APP_ORIGIN: z.string().default("http://localhost:3030"),
API_ORIGIN: z.string().optional(),
STREAM_ORIGIN: z.string().optional(),
ELECTRIC_ORIGIN: z.string().default("http://localhost:3060"),
// A comma separated list of electric origins to shard into different electric instances by environmentId
// example: "http://localhost:3060,http://localhost:3061,http://localhost:3062"
ELECTRIC_ORIGIN_SHARDS: z.string().optional(),
APP_ENV: z.string().default(process.env.NODE_ENV),
SERVICE_NAME: z.string().default("trigger.dev webapp"),
SENTRY_DSN: z.string().optional(),
POSTHOG_PROJECT_KEY: z.string().default("phc_LFH7kJiGhdIlnO22hTAKgHpaKhpM8gkzWAFvHmf5vfS"),
TRIGGER_TELEMETRY_DISABLED: z.string().optional(),
AUTH_GITHUB_CLIENT_ID: z.string().optional(),
AUTH_GITHUB_CLIENT_SECRET: z.string().optional(),
AUTH_GOOGLE_CLIENT_ID: z.string().optional(),
AUTH_GOOGLE_CLIENT_SECRET: z.string().optional(),
EMAIL_TRANSPORT: z.enum(["resend", "smtp", "aws-ses"]).optional(),
FROM_EMAIL: z.string().optional(),
REPLY_TO_EMAIL: z.string().optional(),
RESEND_API_KEY: z.string().optional(),
SMTP_HOST: z.string().optional(),
SMTP_PORT: z.coerce.number().optional(),
SMTP_SECURE: BoolEnv.optional(),
SMTP_USER: z.string().optional(),
SMTP_PASSWORD: z.string().optional(),
PLAIN_API_KEY: z.string().optional(),
PLAIN_CUSTOMER_CARDS_SECRET: z.string().optional(),
PLAIN_CUSTOMER_CARDS_KEY: z.string().optional(),
PLAIN_CUSTOMER_CARDS_HEADERS: z.string().optional(),
WORKER_SCHEMA: z.string().default("graphile_worker"),
WORKER_CONCURRENCY: z.coerce.number().int().default(10),
WORKER_POLL_INTERVAL: z.coerce.number().int().default(1000),
WORKER_ENABLED: z.string().default("true"),
GRACEFUL_SHUTDOWN_TIMEOUT: z.coerce.number().int().default(60000),
DISABLE_SSE: z.string().optional(),
OPENAI_API_KEY: z.string().optional(),
// Redis options
REDIS_HOST: z.string().optional(),
REDIS_READER_HOST: z.string().optional(),
REDIS_READER_PORT: z.coerce.number().optional(),
REDIS_PORT: z.coerce.number().optional(),
REDIS_USERNAME: z.string().optional(),
REDIS_PASSWORD: z.string().optional(),
REDIS_TLS_DISABLED: z.string().optional(),
RATE_LIMIT_REDIS_HOST: z
.string()
.optional()
.transform((v) => v ?? process.env.REDIS_HOST),
RATE_LIMIT_REDIS_READER_HOST: z
.string()
.optional()
.transform((v) => v ?? process.env.REDIS_READER_HOST),
RATE_LIMIT_REDIS_READER_PORT: z.coerce
.number()
.optional()
.transform(
(v) =>
v ?? (process.env.REDIS_READER_PORT ? parseInt(process.env.REDIS_READER_PORT) : undefined)
),
RATE_LIMIT_REDIS_PORT: z.coerce
.number()
.optional()
.transform(
(v) => v ?? (process.env.REDIS_PORT ? parseInt(process.env.REDIS_PORT) : undefined)
),
RATE_LIMIT_REDIS_USERNAME: z
.string()
.optional()
.transform((v) => v ?? process.env.REDIS_USERNAME),
RATE_LIMIT_REDIS_PASSWORD: z
.string()
.optional()
.transform((v) => v ?? process.env.REDIS_PASSWORD),
RATE_LIMIT_REDIS_TLS_DISABLED: z.string().default(process.env.REDIS_TLS_DISABLED ?? "false"),
RATE_LIMIT_REDIS_CLUSTER_MODE_ENABLED: z.string().default("0"),
CACHE_REDIS_HOST: z
.string()
.optional()
.transform((v) => v ?? process.env.REDIS_HOST),
CACHE_REDIS_READER_HOST: z
.string()
.optional()
.transform((v) => v ?? process.env.REDIS_READER_HOST),
CACHE_REDIS_READER_PORT: z.coerce
.number()
.optional()
.transform(
(v) =>
v ?? (process.env.REDIS_READER_PORT ? parseInt(process.env.REDIS_READER_PORT) : undefined)
),
CACHE_REDIS_PORT: z.coerce
.number()
.optional()
.transform(
(v) => v ?? (process.env.REDIS_PORT ? parseInt(process.env.REDIS_PORT) : undefined)
),
CACHE_REDIS_USERNAME: z
.string()
.optional()
.transform((v) => v ?? process.env.REDIS_USERNAME),
CACHE_REDIS_PASSWORD: z
.string()
.optional()
.transform((v) => v ?? process.env.REDIS_PASSWORD),
CACHE_REDIS_TLS_DISABLED: z.string().default(process.env.REDIS_TLS_DISABLED ?? "false"),
CACHE_REDIS_CLUSTER_MODE_ENABLED: z.string().default("0"),
TASK_META_CACHE_REDIS_HOST: z
.string()
.optional()
.transform((v) => v ?? process.env.REDIS_HOST),
TASK_META_CACHE_REDIS_PORT: z.coerce
.number()
.optional()
.transform(
(v) => v ?? (process.env.REDIS_PORT ? parseInt(process.env.REDIS_PORT) : undefined)
),
TASK_META_CACHE_REDIS_USERNAME: z
.string()
.optional()
.transform((v) => v ?? process.env.REDIS_USERNAME),
TASK_META_CACHE_REDIS_PASSWORD: z
.string()
.optional()
.transform((v) => v ?? process.env.REDIS_PASSWORD),
TASK_META_CACHE_REDIS_TLS_DISABLED: z
.string()
.default(process.env.REDIS_TLS_DISABLED ?? "false"),
TASK_META_CACHE_CURRENT_ENV_TTL_SECONDS: z.coerce.number().default(86400),
TASK_META_CACHE_BY_WORKER_TTL_SECONDS: z.coerce.number().default(2592000),
REALTIME_STREAMS_REDIS_HOST: z
.string()
.optional()
.transform((v) => v ?? process.env.REDIS_HOST),
REALTIME_STREAMS_REDIS_READER_HOST: z
.string()
.optional()
.transform((v) => v ?? process.env.REDIS_READER_HOST),
REALTIME_STREAMS_REDIS_READER_PORT: z.coerce
.number()
.optional()
.transform(
(v) =>
v ?? (process.env.REDIS_READER_PORT ? parseInt(process.env.REDIS_READER_PORT) : undefined)
),
REALTIME_STREAMS_REDIS_PORT: z.coerce
.number()
.optional()
.transform(
(v) => v ?? (process.env.REDIS_PORT ? parseInt(process.env.REDIS_PORT) : undefined)
),
REALTIME_STREAMS_REDIS_USERNAME: z
.string()
.optional()
.transform((v) => v ?? process.env.REDIS_USERNAME),
REALTIME_STREAMS_REDIS_PASSWORD: z
.string()
.optional()
.transform((v) => v ?? process.env.REDIS_PASSWORD),
REALTIME_STREAMS_REDIS_TLS_DISABLED: z
.string()
.default(process.env.REDIS_TLS_DISABLED ?? "false"),
REALTIME_STREAMS_REDIS_CLUSTER_MODE_ENABLED: z.string().default("0"),
REALTIME_STREAMS_INACTIVITY_TIMEOUT_MS: z.coerce.number().int().default(60000), // 1 minute
REALTIME_MAXIMUM_CREATED_AT_FILTER_AGE_IN_MS: z.coerce
.number()
.int()
.default(24 * 60 * 60 * 1000), // 1 day in milliseconds
PUBSUB_REDIS_HOST: z
.string()
.optional()
.transform((v) => v ?? process.env.REDIS_HOST),
PUBSUB_REDIS_READER_HOST: z
.string()
.optional()
.transform((v) => v ?? process.env.REDIS_READER_HOST),
PUBSUB_REDIS_READER_PORT: z.coerce
.number()
.optional()
.transform(
(v) =>
v ?? (process.env.REDIS_READER_PORT ? parseInt(process.env.REDIS_READER_PORT) : undefined)
),
PUBSUB_REDIS_PORT: z.coerce
.number()
.optional()
.transform(
(v) => v ?? (process.env.REDIS_PORT ? parseInt(process.env.REDIS_PORT) : undefined)
),
PUBSUB_REDIS_USERNAME: z
.string()
.optional()
.transform((v) => v ?? process.env.REDIS_USERNAME),
PUBSUB_REDIS_PASSWORD: z
.string()
.optional()
.transform((v) => v ?? process.env.REDIS_PASSWORD),
PUBSUB_REDIS_TLS_DISABLED: z.string().default(process.env.REDIS_TLS_DISABLED ?? "false"),
PUBSUB_REDIS_CLUSTER_MODE_ENABLED: z.string().default("0"),
DEFAULT_ENV_EXECUTION_CONCURRENCY_LIMIT: z.coerce.number().int().default(100),
DEFAULT_ENV_EXECUTION_CONCURRENCY_BURST_FACTOR: z.coerce.number().default(1.0),
DEFAULT_ORG_EXECUTION_CONCURRENCY_LIMIT: z.coerce.number().int().default(300),
DEFAULT_DEV_ENV_EXECUTION_ATTEMPTS: z.coerce.number().int().positive().default(1),
//API Rate limiting
/**
* @example "60s"
* @example "1m"
* @example "1h"
* @example "1d"
* @example "1000ms"
* @example "1000s"
*/
API_RATE_LIMIT_REFILL_INTERVAL: z.string().default("10s"), // refill 250 tokens every 10 seconds
API_RATE_LIMIT_MAX: z.coerce.number().int().default(750), // allow bursts of 750 requests
API_RATE_LIMIT_REFILL_RATE: z.coerce.number().int().default(250), // refix 250 tokens every 10 seconds
API_RATE_LIMIT_REQUEST_LOGS_ENABLED: z.string().default("0"),
API_RATE_LIMIT_REJECTION_LOGS_ENABLED: z.string().default("1"),
API_RATE_LIMIT_LIMITER_LOGS_ENABLED: z.string().default("0"),
API_RATE_LIMIT_JWT_WINDOW: z.string().default("1m"),
API_RATE_LIMIT_JWT_TOKENS: z.coerce.number().int().default(60),
//v3
PROVIDER_SECRET: z.string().default("provider-secret"),
COORDINATOR_SECRET: z.string().default("coordinator-secret"),
DEPOT_TOKEN: z.string().optional(),
DEPOT_ORG_ID: z.string().optional(),
DEPOT_REGION: z.string().default("us-east-1"),
// Deployment registry (v3)
DEPLOY_REGISTRY_HOST: z.string().min(1),
DEPLOY_REGISTRY_USERNAME: z.string().optional(),
DEPLOY_REGISTRY_PASSWORD: z.string().optional(),
DEPLOY_REGISTRY_NAMESPACE: z.string().min(1).default("trigger"),
DEPLOY_REGISTRY_ECR_TAGS: z.string().optional(), // csv, for example: "key1=value1,key2=value2"
DEPLOY_REGISTRY_ECR_ASSUME_ROLE_ARN: z.string().optional(),
DEPLOY_REGISTRY_ECR_ASSUME_ROLE_EXTERNAL_ID: z.string().optional(),
DEPLOY_REGISTRY_ECR_DEFAULT_REPOSITORY_POLICY: z.string().optional(), // raw IAM policy JSON applied to every repo created by the webapp
// Deployment registry (v4) - falls back to v3 registry if not specified
V4_DEPLOY_REGISTRY_HOST: z
.string()
.optional()
.transform((v) => v ?? process.env.DEPLOY_REGISTRY_HOST)
.pipe(z.string().min(1)), // Ensure final type is required string
V4_DEPLOY_REGISTRY_USERNAME: z
.string()
.optional()
.transform((v) => v ?? process.env.DEPLOY_REGISTRY_USERNAME),
V4_DEPLOY_REGISTRY_PASSWORD: z
.string()
.optional()
.transform((v) => v ?? process.env.DEPLOY_REGISTRY_PASSWORD),
V4_DEPLOY_REGISTRY_NAMESPACE: z
.string()
.optional()
.transform((v) => v ?? process.env.DEPLOY_REGISTRY_NAMESPACE)
.pipe(z.string().min(1).default("trigger")), // Ensure final type is required string
V4_DEPLOY_REGISTRY_ECR_TAGS: z
.string()
.optional()
.transform((v) => v ?? process.env.DEPLOY_REGISTRY_ECR_TAGS),
V4_DEPLOY_REGISTRY_ECR_ASSUME_ROLE_ARN: z
.string()
.optional()
.transform((v) => v ?? process.env.DEPLOY_REGISTRY_ECR_ASSUME_ROLE_ARN),
V4_DEPLOY_REGISTRY_ECR_ASSUME_ROLE_EXTERNAL_ID: z
.string()
.optional()
.transform((v) => v ?? process.env.DEPLOY_REGISTRY_ECR_ASSUME_ROLE_EXTERNAL_ID),
V4_DEPLOY_REGISTRY_ECR_DEFAULT_REPOSITORY_POLICY: z
.string()
.optional()
.transform((v) => v ?? process.env.DEPLOY_REGISTRY_ECR_DEFAULT_REPOSITORY_POLICY),
// Compute gateway (template creation during deploy finalize)
COMPUTE_GATEWAY_URL: z.string().optional(),
COMPUTE_GATEWAY_AUTH_TOKEN: z.string().optional(),
COMPUTE_TEMPLATE_SHADOW_ROLLOUT_PCT: z.string().optional(),
// Comma-separated machine preset names to build boot snapshots for on
// deploy (e.g. "small-1x,small-2x,medium-1x"). Default: "small-1x".
COMPUTE_TEMPLATE_MACHINE_PRESETS: z
.string()
.default("small-1x")
.transform(parseMachinePresetCsv),
// Subset of COMPUTE_TEMPLATE_MACHINE_PRESETS that must succeed for a
// required-mode deploy to be considered successful. Failures of presets
// outside this list are logged but don't fail the deploy. Defaults to the
// full COMPUTE_TEMPLATE_MACHINE_PRESETS list when unset (everything required).
COMPUTE_TEMPLATE_MACHINE_PRESETS_REQUIRED: z
.string()
.optional()
.transform((v, ctx) =>
parseMachinePresetCsv(
v ?? process.env.COMPUTE_TEMPLATE_MACHINE_PRESETS ?? "small-1x",
ctx
)
),
DEPLOY_IMAGE_PLATFORM: z.string().default("linux/amd64"),
DEPLOY_TIMEOUT_MS: z.coerce
.number()
.int()
.default(60 * 1000 * 8), // 8 minutes
DEPLOY_QUEUE_TIMEOUT_MS: z.coerce
.number()
.int()
.default(60 * 1000 * 15), // 15 minutes
// When enabled, reject deploys made by v3 CLI versions (i.e. payloads that
// omit the `type` field). v4 CLI versions always send `type` ("MANAGED" or "V1"),
// so they are unaffected. Defaults to off so detection can run in
// log-only mode before enforcement.
DEPRECATE_V3_CLI_DEPLOYS_ENABLED: z.string().default("0"),
OBJECT_STORE_BASE_URL: z.string().optional(),
OBJECT_STORE_BUCKET: z.string().optional(),
OBJECT_STORE_ACCESS_KEY_ID: z.string().optional(),
OBJECT_STORE_SECRET_ACCESS_KEY: z.string().optional(),
OBJECT_STORE_REGION: z.string().optional(),
OBJECT_STORE_SERVICE: z.string().default("s3"),
// Protocol to use for new uploads (e.g., "s3", "r2"). Data without protocol uses default provider above.
// If specified, you must configure the corresponding provider using OBJECT_STORE_{PROTOCOL}_* env vars.
// Example: OBJECT_STORE_DEFAULT_PROTOCOL=s3 requires OBJECT_STORE_S3_BASE_URL, OBJECT_STORE_S3_ACCESS_KEY_ID, etc.
// Enables zero-downtime migration between providers (old data keeps working, new data uses new provider).
OBJECT_STORE_DEFAULT_PROTOCOL: z
.string()
.regex(/^[a-z0-9]+$/)
.optional(),
ARTIFACTS_OBJECT_STORE_BUCKET: z.string().optional(),
ARTIFACTS_OBJECT_STORE_BASE_URL: z.string().optional(),
ARTIFACTS_OBJECT_STORE_ACCESS_KEY_ID: z.string().optional(),
ARTIFACTS_OBJECT_STORE_SECRET_ACCESS_KEY: z.string().optional(),
ARTIFACTS_OBJECT_STORE_REGION: z.string().optional(),
EVENTS_BATCH_SIZE: z.coerce.number().int().default(100),
EVENTS_BATCH_INTERVAL: z.coerce.number().int().default(1000),
EVENTS_DEFAULT_LOG_RETENTION: z.coerce.number().int().default(7),
EVENTS_MIN_CONCURRENCY: z.coerce.number().int().default(1),
EVENTS_MAX_CONCURRENCY: z.coerce.number().int().default(10),
EVENTS_MAX_BATCH_SIZE: z.coerce.number().int().default(500),
EVENTS_MEMORY_PRESSURE_THRESHOLD: z.coerce.number().int().default(5000),
EVENTS_LOAD_SHEDDING_THRESHOLD: z.coerce.number().int().default(100000),
EVENTS_LOAD_SHEDDING_ENABLED: z.string().default("1"),
SHARED_QUEUE_CONSUMER_POOL_SIZE: z.coerce.number().int().default(10),
SHARED_QUEUE_CONSUMER_INTERVAL_MS: z.coerce.number().int().default(100),
SHARED_QUEUE_CONSUMER_NEXT_TICK_INTERVAL_MS: z.coerce.number().int().default(100),
SHARED_QUEUE_CONSUMER_EMIT_RESUME_DEPENDENCY_TIMEOUT_MS: z.coerce.number().int().default(1000),
SHARED_QUEUE_CONSUMER_RESOLVE_PAYLOADS_BATCH_SIZE: z.coerce.number().int().default(25),
MANAGED_WORKER_SECRET: z.string().default("managed-secret"),
// Development OTEL environment variables
DEV_OTEL_EXPORTER_OTLP_ENDPOINT: z.string().optional(),
DEV_OTEL_METRICS_ENDPOINT: z.string().optional(),
// If this is set to 1, then the below variables are used to configure the batch processor for spans and logs
DEV_OTEL_BATCH_PROCESSING_ENABLED: z.string().default("0"),
DEV_OTEL_SPAN_MAX_EXPORT_BATCH_SIZE: z.string().default("64"),
DEV_OTEL_SPAN_SCHEDULED_DELAY_MILLIS: z.string().default("200"),
DEV_OTEL_SPAN_EXPORT_TIMEOUT_MILLIS: z.string().default("30000"),
DEV_OTEL_SPAN_MAX_QUEUE_SIZE: z.string().default("512"),
DEV_OTEL_LOG_MAX_EXPORT_BATCH_SIZE: z.string().default("64"),
DEV_OTEL_LOG_SCHEDULED_DELAY_MILLIS: z.string().default("200"),
DEV_OTEL_LOG_EXPORT_TIMEOUT_MILLIS: z.string().default("30000"),
DEV_OTEL_LOG_MAX_QUEUE_SIZE: z.string().default("512"),
DEV_OTEL_METRICS_EXPORT_INTERVAL_MILLIS: z.string().optional(),
DEV_OTEL_METRICS_EXPORT_TIMEOUT_MILLIS: z.string().optional(),
DEV_OTEL_METRICS_COLLECTION_INTERVAL_MILLIS: z.string().optional(),
PROD_OTEL_BATCH_PROCESSING_ENABLED: z.string().default("0"),
PROD_OTEL_SPAN_MAX_EXPORT_BATCH_SIZE: z.string().default("64"),
PROD_OTEL_SPAN_SCHEDULED_DELAY_MILLIS: z.string().default("200"),
PROD_OTEL_SPAN_EXPORT_TIMEOUT_MILLIS: z.string().default("30000"),
PROD_OTEL_SPAN_MAX_QUEUE_SIZE: z.string().default("512"),
PROD_OTEL_LOG_MAX_EXPORT_BATCH_SIZE: z.string().default("64"),
PROD_OTEL_LOG_SCHEDULED_DELAY_MILLIS: z.string().default("200"),
PROD_OTEL_LOG_EXPORT_TIMEOUT_MILLIS: z.string().default("30000"),
PROD_OTEL_LOG_MAX_QUEUE_SIZE: z.string().default("512"),
PROD_OTEL_METRICS_EXPORT_INTERVAL_MILLIS: z.string().optional(),
PROD_OTEL_METRICS_EXPORT_TIMEOUT_MILLIS: z.string().optional(),
PROD_OTEL_METRICS_COLLECTION_INTERVAL_MILLIS: z.string().optional(),
TRIGGER_OTEL_SPAN_ATTRIBUTE_COUNT_LIMIT: z.string().default("1024"),
TRIGGER_OTEL_LOG_ATTRIBUTE_COUNT_LIMIT: z.string().default("1024"),
TRIGGER_OTEL_SPAN_ATTRIBUTE_VALUE_LENGTH_LIMIT: z.string().default("131072"),
TRIGGER_OTEL_LOG_ATTRIBUTE_VALUE_LENGTH_LIMIT: z.string().default("131072"),
TRIGGER_OTEL_SPAN_EVENT_COUNT_LIMIT: z.string().default("10"),
TRIGGER_OTEL_LINK_COUNT_LIMIT: z.string().default("2"),
TRIGGER_OTEL_ATTRIBUTE_PER_LINK_COUNT_LIMIT: z.string().default("10"),
TRIGGER_OTEL_ATTRIBUTE_PER_EVENT_COUNT_LIMIT: z.string().default("10"),
CHECKPOINT_THRESHOLD_IN_MS: z.coerce.number().int().default(30000),
// Internal OTEL environment variables
INTERNAL_OTEL_TRACE_EXPORTER_URL: z.string().optional(),
INTERNAL_OTEL_TRACE_EXPORTER_AUTH_HEADERS: z.string().optional(),
INTERNAL_OTEL_TRACE_LOGGING_ENABLED: z.string().default("1"),
// this means 1/20 traces or 5% of traces will be sampled (sampled = recorded)
INTERNAL_OTEL_TRACE_SAMPLING_RATE: z.string().default("20"),
INTERNAL_OTEL_TRACE_INSTRUMENT_PRISMA_ENABLED: z.string().default("0"),
INTERNAL_OTEL_TRACE_DISABLED: z.string().default("0"),
DISABLE_HTTP_INSTRUMENTATION: BoolEnv.default(false),
INTERNAL_OTEL_LOG_EXPORTER_URL: z.string().optional(),
INTERNAL_OTEL_METRIC_EXPORTER_URL: z.string().optional(),
INTERNAL_OTEL_METRIC_EXPORTER_AUTH_HEADERS: z.string().optional(),
INTERNAL_OTEL_METRIC_EXPORTER_ENABLED: z.string().default("0"),
INTERNAL_OTEL_METRIC_EXPORTER_INTERVAL_MS: z.coerce.number().int().default(30_000),
INTERNAL_OTEL_HOST_METRICS_ENABLED: BoolEnv.default(true),
INTERNAL_OTEL_NODEJS_METRICS_ENABLED: BoolEnv.default(true),
INTERNAL_OTEL_ADDITIONAL_DETECTORS_ENABLED: BoolEnv.default(true),
ORG_SLACK_INTEGRATION_CLIENT_ID: z.string().optional(),
ORG_SLACK_INTEGRATION_CLIENT_SECRET: z.string().optional(),
/** Vercel integration OAuth credentials */
VERCEL_INTEGRATION_CLIENT_ID: z.string().optional(),
VERCEL_INTEGRATION_CLIENT_SECRET: z.string().optional(),
VERCEL_INTEGRATION_APP_SLUG: z.string().optional(),
/** These enable the alerts feature in v3 */
ALERT_EMAIL_TRANSPORT: z.enum(["resend", "smtp", "aws-ses"]).optional(),
ALERT_FROM_EMAIL: z.string().optional(),
ALERT_REPLY_TO_EMAIL: z.string().optional(),
ALERT_RESEND_API_KEY: z.string().optional(),
ALERT_SMTP_HOST: z.string().optional(),
ALERT_SMTP_PORT: z.coerce.number().optional(),
ALERT_SMTP_SECURE: BoolEnv.optional(),
ALERT_SMTP_USER: z.string().optional(),
ALERT_SMTP_PASSWORD: z.string().optional(),
ALERT_RATE_LIMITER_EMISSION_INTERVAL: z.coerce.number().int().default(2_500),
ALERT_RATE_LIMITER_BURST_TOLERANCE: z.coerce.number().int().default(10_000),
ALERT_RATE_LIMITER_REDIS_HOST: z
.string()
.optional()
.transform((v) => v ?? process.env.REDIS_HOST),
ALERT_RATE_LIMITER_REDIS_READER_HOST: z
.string()
.optional()
.transform((v) => v ?? process.env.REDIS_READER_HOST),
ALERT_RATE_LIMITER_REDIS_READER_PORT: z.coerce
.number()
.optional()
.transform(
(v) =>
v ?? (process.env.REDIS_READER_PORT ? parseInt(process.env.REDIS_READER_PORT) : undefined)
),
ALERT_RATE_LIMITER_REDIS_PORT: z.coerce
.number()
.optional()
.transform(
(v) => v ?? (process.env.REDIS_PORT ? parseInt(process.env.REDIS_PORT) : undefined)
),
ALERT_RATE_LIMITER_REDIS_USERNAME: z
.string()
.optional()
.transform((v) => v ?? process.env.REDIS_USERNAME),
ALERT_RATE_LIMITER_REDIS_PASSWORD: z
.string()
.optional()
.transform((v) => v ?? process.env.REDIS_PASSWORD),
ALERT_RATE_LIMITER_REDIS_TLS_DISABLED: z
.string()
.default(process.env.REDIS_TLS_DISABLED ?? "false"),
ALERT_RATE_LIMITER_REDIS_CLUSTER_MODE_ENABLED: z.string().default("0"),
LOOPS_API_KEY: z.string().optional(),
MARQS_DISABLE_REBALANCING: BoolEnv.default(false),
MARQS_VISIBILITY_TIMEOUT_MS: z.coerce
.number()
.int()
.default(60 * 1000 * 15),
MARQS_SHARED_QUEUE_LIMIT: z.coerce.number().int().default(1000),
MARQS_MAXIMUM_QUEUE_PER_ENV_COUNT: z.coerce.number().int().default(50),
MARQS_DEV_QUEUE_LIMIT: z.coerce.number().int().default(1000),
MARQS_MAXIMUM_NACK_COUNT: z.coerce.number().int().default(64),
MARQS_CONCURRENCY_LIMIT_BIAS: z.coerce.number().default(0.75),
MARQS_AVAILABLE_CAPACITY_BIAS: z.coerce.number().default(0.3),
MARQS_QUEUE_AGE_RANDOMIZATION_BIAS: z.coerce.number().default(0.25),
MARQS_REUSE_SNAPSHOT_COUNT: z.coerce.number().int().default(0),
MARQS_MAXIMUM_ENV_COUNT: z.coerce.number().int().optional(),
MARQS_SHARED_WORKER_QUEUE_CONSUMER_INTERVAL_MS: z.coerce.number().int().default(250),
MARQS_SHARED_WORKER_QUEUE_MAX_MESSAGE_COUNT: z.coerce.number().int().default(10),
MARQS_SHARED_WORKER_QUEUE_EAGER_DEQUEUE_ENABLED: z.string().default("0"),
MARQS_WORKER_ENABLED: z.string().default("0"),
MARQS_WORKER_COUNT: z.coerce.number().int().default(2),
MARQS_WORKER_CONCURRENCY_LIMIT: z.coerce.number().int().default(50),
MARQS_WORKER_CONCURRENCY_TASKS_PER_WORKER: z.coerce.number().int().default(5),
MARQS_WORKER_POLL_INTERVAL_MS: z.coerce.number().int().default(100),
MARQS_WORKER_IMMEDIATE_POLL_INTERVAL_MS: z.coerce.number().int().default(100),
MARQS_WORKER_SHUTDOWN_TIMEOUT_MS: z.coerce.number().int().default(60_000),
MARQS_SHARED_WORKER_QUEUE_COOLOFF_COUNT_THRESHOLD: z.coerce.number().int().default(10),
MARQS_SHARED_WORKER_QUEUE_COOLOFF_PERIOD_MS: z.coerce.number().int().default(5_000),
PROD_TASK_HEARTBEAT_INTERVAL_MS: z.coerce.number().int().optional(),
VERBOSE_GRAPHILE_LOGGING: z.string().default("false"),
V2_MARQS_ENABLED: z.string().default("0"),
V2_MARQS_CONSUMER_POOL_ENABLED: z.string().default("0"),
V2_MARQS_CONSUMER_POOL_SIZE: z.coerce.number().int().default(10),
V2_MARQS_CONSUMER_POLL_INTERVAL_MS: z.coerce.number().int().default(1000),
V2_MARQS_QUEUE_SELECTION_COUNT: z.coerce.number().int().default(36),
V2_MARQS_VISIBILITY_TIMEOUT_MS: z.coerce
.number()
.int()
.default(60 * 1000 * 15),
V2_MARQS_DEFAULT_ENV_CONCURRENCY: z.coerce.number().int().default(100),
V2_MARQS_VERBOSE: z.string().default("0"),
V3_MARQS_CONCURRENCY_MONITOR_ENABLED: z.string().default("0"),
V2_MARQS_CONCURRENCY_MONITOR_ENABLED: z.string().default("0"),
/* Usage settings */
USAGE_EVENT_URL: z.string().optional(),
PROD_USAGE_HEARTBEAT_INTERVAL_MS: z.coerce.number().int().optional(),
CENTS_PER_RUN: z.coerce.number().default(0),
EVENT_LOOP_MONITOR_ENABLED: z.string().default("1"),
RESOURCE_MONITOR_ENABLED: z.string().default("0"),
MAXIMUM_LIVE_RELOADING_EVENTS: z.coerce.number().int().default(1000),
MAXIMUM_TRACE_SUMMARY_VIEW_COUNT: z.coerce.number().int().default(25_000),
MAXIMUM_TRACE_DETAILED_SUMMARY_VIEW_COUNT: z.coerce.number().int().default(10_000),
TASK_PAYLOAD_OFFLOAD_THRESHOLD: z.coerce.number().int().default(524_288), // 512KB
BATCH_PAYLOAD_OFFLOAD_THRESHOLD: z.coerce.number().int().optional(), // Defaults to TASK_PAYLOAD_OFFLOAD_THRESHOLD if not set
TASK_PAYLOAD_MAXIMUM_SIZE: z.coerce.number().int().default(3_145_728), // 3MB
BATCH_TASK_PAYLOAD_MAXIMUM_SIZE: z.coerce.number().int().default(1_000_000), // 1MB
TASK_RUN_METADATA_MAXIMUM_SIZE: z.coerce.number().int().default(262_144), // 256KB
MAXIMUM_DEV_QUEUE_SIZE: z.coerce.number().int().optional(),
MAXIMUM_DEPLOYED_QUEUE_SIZE: z.coerce.number().int().optional(),
QUEUE_SIZE_CACHE_TTL_MS: z.coerce.number().int().optional().default(1_000), // 1 second
QUEUE_SIZE_CACHE_MAX_SIZE: z.coerce.number().int().optional().default(5_000),
QUEUE_SIZE_CACHE_ENABLED: z.coerce.number().int().optional().default(1),
MAX_BATCH_V2_TRIGGER_ITEMS: z.coerce.number().int().default(500),
MAX_BATCH_AND_WAIT_V2_TRIGGER_ITEMS: z.coerce.number().int().default(500),
// 2-phase batch API settings
STREAMING_BATCH_MAX_ITEMS: z.coerce.number().int().default(1_000), // Max items in streaming batch
STREAMING_BATCH_ITEM_MAXIMUM_SIZE: z.coerce.number().int().default(3_145_728),
BATCH_RATE_LIMIT_REFILL_RATE: z.coerce.number().int().default(100),
BATCH_RATE_LIMIT_MAX: z.coerce.number().int().default(1200),
BATCH_RATE_LIMIT_REFILL_INTERVAL: z.string().default("10s"),
BATCH_CONCURRENCY_LIMIT_DEFAULT: z.coerce.number().int().default(5),
REALTIME_STREAM_VERSION: z.enum(["v1", "v2"]).default("v1"),
REALTIME_STREAM_MAX_LENGTH: z.coerce.number().int().default(1000),
REALTIME_STREAM_TTL: z.coerce
.number()
.int()
.default(60 * 60 * 24), // 1 day in seconds
BATCH_METADATA_OPERATIONS_FLUSH_INTERVAL_MS: z.coerce.number().int().default(1000),
BATCH_METADATA_OPERATIONS_FLUSH_ENABLED: z.string().default("1"),
BATCH_METADATA_OPERATIONS_FLUSH_LOGGING_ENABLED: z.string().default("1"),
// Run Engine 2.0
RUN_ENGINE_WORKER_COUNT: z.coerce.number().int().default(4),
RUN_ENGINE_TASKS_PER_WORKER: z.coerce.number().int().default(10),
RUN_ENGINE_WORKER_CONCURRENCY_LIMIT: z.coerce.number().int().default(10),
RUN_ENGINE_WORKER_POLL_INTERVAL: z.coerce.number().int().default(100),
RUN_ENGINE_WORKER_IMMEDIATE_POLL_INTERVAL: z.coerce.number().int().default(100),
RUN_ENGINE_TIMEOUT_PENDING_EXECUTING: z.coerce.number().int().default(60_000),
RUN_ENGINE_TIMEOUT_PENDING_CANCEL: z.coerce.number().int().default(60_000),
RUN_ENGINE_TIMEOUT_EXECUTING: z.coerce.number().int().default(300_000), // 5 minutes
RUN_ENGINE_TIMEOUT_EXECUTING_WITH_WAITPOINTS: z.coerce.number().int().default(300_000), // 5 minutes
RUN_ENGINE_TIMEOUT_SUSPENDED: z.coerce
.number()
.int()
.default(60_000 * 10),
RUN_ENGINE_DEBUG_WORKER_NOTIFICATIONS: BoolEnv.default(false),
RUN_ENGINE_PARENT_QUEUE_LIMIT: z.coerce.number().int().default(1000),
RUN_ENGINE_CONCURRENCY_LIMIT_BIAS: z.coerce.number().default(0.75),
RUN_ENGINE_AVAILABLE_CAPACITY_BIAS: z.coerce.number().default(0.3),
RUN_ENGINE_QUEUE_AGE_RANDOMIZATION_BIAS: z.coerce.number().default(0.25),
RUN_ENGINE_REUSE_SNAPSHOT_COUNT: z.coerce.number().int().default(0),
RUN_ENGINE_MAXIMUM_ENV_COUNT: z.coerce.number().int().optional(),
RUN_ENGINE_RUN_QUEUE_SHARD_COUNT: z.coerce.number().int().default(4),
RUN_ENGINE_WORKER_SHUTDOWN_TIMEOUT_MS: z.coerce.number().int().default(60_000),
RUN_ENGINE_RETRY_WARM_START_THRESHOLD_MS: z.coerce.number().int().default(30_000),
RUN_ENGINE_PROCESS_WORKER_QUEUE_DEBOUNCE_MS: z.coerce.number().int().default(200),
RUN_ENGINE_DEQUEUE_BLOCKING_TIMEOUT_SECONDS: z.coerce.number().int().default(10),
RUN_ENGINE_MASTER_QUEUE_CONSUMERS_INTERVAL_MS: z.coerce.number().int().default(1000),
RUN_ENGINE_MASTER_QUEUE_COOLOFF_PERIOD_MS: z.coerce.number().int().default(10_000),
RUN_ENGINE_MASTER_QUEUE_COOLOFF_COUNT_THRESHOLD: z.coerce.number().int().default(10),
RUN_ENGINE_MASTER_QUEUE_CONSUMER_DEQUEUE_COUNT: z.coerce.number().int().default(10),
RUN_ENGINE_CONCURRENCY_SWEEPER_SCAN_SCHEDULE: z.string().optional(),
RUN_ENGINE_CONCURRENCY_SWEEPER_PROCESS_MARKED_SCHEDULE: z.string().optional(),
RUN_ENGINE_CONCURRENCY_SWEEPER_SCAN_JITTER_IN_MS: z.coerce.number().int().optional(),
RUN_ENGINE_CONCURRENCY_SWEEPER_PROCESS_MARKED_JITTER_IN_MS: z.coerce.number().int().optional(),
// TTL System settings for automatic run expiration
RUN_ENGINE_TTL_SYSTEM_DISABLED: BoolEnv.default(false),
RUN_ENGINE_TTL_SYSTEM_SHARD_COUNT: z.coerce.number().int().optional(),
RUN_ENGINE_TTL_SYSTEM_POLL_INTERVAL_MS: z.coerce.number().int().default(1_000),
RUN_ENGINE_TTL_SYSTEM_BATCH_SIZE: z.coerce.number().int().default(100),
RUN_ENGINE_TTL_WORKER_CONCURRENCY: z.coerce.number().int().default(1),
RUN_ENGINE_TTL_WORKER_BATCH_MAX_SIZE: z.coerce.number().int().default(50),
RUN_ENGINE_TTL_CONSUMERS_DISABLED: BoolEnv.default(false),
RUN_ENGINE_TTL_WORKER_BATCH_MAX_WAIT_MS: z.coerce.number().int().default(5_000),
/** Optional maximum TTL for all runs (e.g. "14d"). If set, runs without an explicit TTL
* will use this as their TTL, and runs with a TTL larger than this will be clamped. */
RUN_ENGINE_DEFAULT_MAX_TTL: z.string().optional(),
RUN_ENGINE_RUN_LOCK_DURATION: z.coerce.number().int().default(5000),
RUN_ENGINE_RUN_LOCK_AUTOMATIC_EXTENSION_THRESHOLD: z.coerce.number().int().default(1000),
RUN_ENGINE_RUN_LOCK_MAX_RETRIES: z.coerce.number().int().default(10),
RUN_ENGINE_RUN_LOCK_BASE_DELAY: z.coerce.number().int().default(100),
RUN_ENGINE_RUN_LOCK_MAX_DELAY: z.coerce.number().int().default(3000),
RUN_ENGINE_RUN_LOCK_BACKOFF_MULTIPLIER: z.coerce.number().default(1.8),
RUN_ENGINE_RUN_LOCK_JITTER_FACTOR: z.coerce.number().default(0.15),
RUN_ENGINE_RUN_LOCK_MAX_TOTAL_WAIT_TIME: z.coerce.number().int().default(15000),
RUN_ENGINE_SUSPENDED_HEARTBEAT_RETRIES_MAX_COUNT: z.coerce.number().int().default(12),
RUN_ENGINE_SUSPENDED_HEARTBEAT_RETRIES_MAX_DELAY_MS: z.coerce
.number()
.int()
.default(60_000 * 60 * 6),
RUN_ENGINE_SUSPENDED_HEARTBEAT_RETRIES_INITIAL_DELAY_MS: z.coerce
.number()
.int()
.default(60_000),
RUN_ENGINE_SUSPENDED_HEARTBEAT_RETRIES_FACTOR: z.coerce.number().default(2),
/** Maximum duration in milliseconds that a run can be debounced. Default: 1 hour (3,600,000ms) */
RUN_ENGINE_MAXIMUM_DEBOUNCE_DURATION_MS: z.coerce
.number()
.int()
.default(60_000 * 60), // 1 hour
/**
* Bucket size in milliseconds used to quantize the newly computed `delayUntil`
* in the debounce system. Quantization collapses concurrent triggers on the
* same hot debounce key onto the same target time so the unlocked fast-path
* skip is effective. Set to 0 to disable. Default: 1000ms (1s).
*/
RUN_ENGINE_DEBOUNCE_QUANTIZE_NEW_DELAY_UNTIL_MS: z.coerce.number().int().min(0).default(1000),
/**
* Whether the unlocked fast-path skip is enabled in the debounce system.
* Acts as a kill switch in case the fast-path needs to be disabled in
* production without a redeploy. Default: "1" (enabled).
*/
RUN_ENGINE_DEBOUNCE_FAST_PATH_SKIP_ENABLED: z.string().default("1"),
RUN_ENGINE_WORKER_REDIS_HOST: z
.string()
.optional()
.transform((v) => v ?? process.env.REDIS_HOST),
RUN_ENGINE_WORKER_REDIS_READER_HOST: z
.string()
.optional()
.transform((v) => v ?? process.env.REDIS_READER_HOST),
RUN_ENGINE_WORKER_REDIS_READER_PORT: z.coerce
.number()
.optional()
.transform(
(v) =>
v ?? (process.env.REDIS_READER_PORT ? parseInt(process.env.REDIS_READER_PORT) : undefined)
),
RUN_ENGINE_WORKER_REDIS_PORT: z.coerce
.number()
.optional()
.transform(
(v) => v ?? (process.env.REDIS_PORT ? parseInt(process.env.REDIS_PORT) : undefined)
),
RUN_ENGINE_WORKER_REDIS_USERNAME: z
.string()
.optional()
.transform((v) => v ?? process.env.REDIS_USERNAME),
RUN_ENGINE_WORKER_REDIS_PASSWORD: z
.string()
.optional()
.transform((v) => v ?? process.env.REDIS_PASSWORD),
RUN_ENGINE_WORKER_REDIS_TLS_DISABLED: z
.string()
.default(process.env.REDIS_TLS_DISABLED ?? "false"),
RUN_ENGINE_RUN_QUEUE_REDIS_HOST: z
.string()
.optional()
.transform((v) => v ?? process.env.REDIS_HOST),
RUN_ENGINE_RUN_QUEUE_REDIS_READER_HOST: z
.string()
.optional()
.transform((v) => v ?? process.env.REDIS_READER_HOST),
RUN_ENGINE_RUN_QUEUE_REDIS_READER_PORT: z.coerce
.number()
.optional()
.transform(
(v) =>
v ?? (process.env.REDIS_READER_PORT ? parseInt(process.env.REDIS_READER_PORT) : undefined)
),
RUN_ENGINE_RUN_QUEUE_REDIS_PORT: z.coerce
.number()
.optional()
.transform(
(v) => v ?? (process.env.REDIS_PORT ? parseInt(process.env.REDIS_PORT) : undefined)
),
RUN_ENGINE_RUN_QUEUE_REDIS_USERNAME: z
.string()
.optional()
.transform((v) => v ?? process.env.REDIS_USERNAME),
RUN_ENGINE_RUN_QUEUE_REDIS_PASSWORD: z
.string()
.optional()
.transform((v) => v ?? process.env.REDIS_PASSWORD),
RUN_ENGINE_RUN_QUEUE_REDIS_TLS_DISABLED: z
.string()
.default(process.env.REDIS_TLS_DISABLED ?? "false"),
RUN_ENGINE_RUN_LOCK_REDIS_HOST: z
.string()
.optional()
.transform((v) => v ?? process.env.REDIS_HOST),
RUN_ENGINE_RUN_LOCK_REDIS_READER_HOST: z
.string()
.optional()
.transform((v) => v ?? process.env.REDIS_READER_HOST),
RUN_ENGINE_RUN_LOCK_REDIS_READER_PORT: z.coerce
.number()
.optional()
.transform(
(v) =>
v ?? (process.env.REDIS_READER_PORT ? parseInt(process.env.REDIS_READER_PORT) : undefined)
),
RUN_ENGINE_RUN_LOCK_REDIS_PORT: z.coerce
.number()
.optional()
.transform(
(v) => v ?? (process.env.REDIS_PORT ? parseInt(process.env.REDIS_PORT) : undefined)
),
RUN_ENGINE_RUN_LOCK_REDIS_USERNAME: z
.string()
.optional()
.transform((v) => v ?? process.env.REDIS_USERNAME),
RUN_ENGINE_RUN_LOCK_REDIS_PASSWORD: z
.string()
.optional()
.transform((v) => v ?? process.env.REDIS_PASSWORD),
RUN_ENGINE_RUN_LOCK_REDIS_TLS_DISABLED: z
.string()
.default(process.env.REDIS_TLS_DISABLED ?? "false"),
RUN_ENGINE_DEV_PRESENCE_REDIS_HOST: z
.string()
.optional()
.transform((v) => v ?? process.env.REDIS_HOST),
RUN_ENGINE_DEV_PRESENCE_REDIS_READER_HOST: z
.string()
.optional()
.transform((v) => v ?? process.env.REDIS_READER_HOST),
RUN_ENGINE_DEV_PRESENCE_REDIS_READER_PORT: z.coerce
.number()
.optional()
.transform(
(v) =>
v ?? (process.env.REDIS_READER_PORT ? parseInt(process.env.REDIS_READER_PORT) : undefined)
),
RUN_ENGINE_DEV_PRESENCE_REDIS_PORT: z.coerce
.number()
.optional()
.transform(
(v) => v ?? (process.env.REDIS_PORT ? parseInt(process.env.REDIS_PORT) : undefined)
),
RUN_ENGINE_DEV_PRESENCE_REDIS_USERNAME: z
.string()
.optional()
.transform((v) => v ?? process.env.REDIS_USERNAME),
RUN_ENGINE_DEV_PRESENCE_REDIS_PASSWORD: z
.string()
.optional()
.transform((v) => v ?? process.env.REDIS_PASSWORD),
RUN_ENGINE_DEV_PRESENCE_REDIS_TLS_DISABLED: z
.string()
.default(process.env.REDIS_TLS_DISABLED ?? "false"),
//API Rate limiting
/**
* @example "60s"
* @example "1m"
* @example "1h"
* @example "1d"
* @example "1000ms"
* @example "1000s"
*/
RUN_ENGINE_RATE_LIMIT_REFILL_INTERVAL: z.string().default("10s"), // refill 250 tokens every 10 seconds
RUN_ENGINE_RATE_LIMIT_MAX: z.coerce.number().int().default(1200), // allow bursts of 750 requests
RUN_ENGINE_RATE_LIMIT_REFILL_RATE: z.coerce.number().int().default(400), // refix 250 tokens every 10 seconds
RUN_ENGINE_RATE_LIMIT_REQUEST_LOGS_ENABLED: z.string().default("0"),
RUN_ENGINE_RATE_LIMIT_REJECTION_LOGS_ENABLED: z.string().default("1"),
RUN_ENGINE_RATE_LIMIT_LIMITER_LOGS_ENABLED: z.string().default("0"),
RUN_ENGINE_RELEASE_CONCURRENCY_ENABLED: z.string().default("0"),
RUN_ENGINE_RELEASE_CONCURRENCY_DISABLE_CONSUMERS: z.string().default("0"),
RUN_ENGINE_RELEASE_CONCURRENCY_MAX_TOKENS_RATIO: z.coerce.number().default(1),
RUN_ENGINE_RELEASE_CONCURRENCY_RELEASINGS_MAX_AGE: z.coerce
.number()
.int()
.default(60_000 * 30),
RUN_ENGINE_RELEASE_CONCURRENCY_RELEASINGS_POLL_INTERVAL: z.coerce
.number()
.int()
.default(60_000),
RUN_ENGINE_RELEASE_CONCURRENCY_MAX_RETRIES: z.coerce.number().int().default(3),
RUN_ENGINE_RELEASE_CONCURRENCY_CONSUMERS_COUNT: z.coerce.number().int().default(1),
RUN_ENGINE_RELEASE_CONCURRENCY_POLL_INTERVAL: z.coerce.number().int().default(500),
RUN_ENGINE_RELEASE_CONCURRENCY_BATCH_SIZE: z.coerce.number().int().default(10),
RUN_ENGINE_WORKER_ENABLED: z.string().default("1"),
RUN_ENGINE_WORKER_LOG_LEVEL: z.enum(["log", "error", "warn", "info", "debug"]).default("info"),
RUN_ENGINE_RUN_QUEUE_LOG_LEVEL: z
.enum(["log", "error", "warn", "info", "debug"])
.default("info"),
RUN_ENGINE_TREAT_PRODUCTION_EXECUTION_STALLS_AS_OOM: z.string().default("0"),
RUN_ENGINE_READ_REPLICA_SNAPSHOTS_SINCE_ENABLED: z.string().default("0"),
RUN_ENGINE_DEBOUNCE_USE_REPLICA_FOR_FAST_PATH_READ: z.string().default("0"),
/** How long should the presence ttl last */
DEV_PRESENCE_SSE_TIMEOUT: z.coerce.number().int().default(30_000),
DEV_PRESENCE_TTL_MS: z.coerce.number().int().default(5_000),
DEV_PRESENCE_POLL_MS: z.coerce.number().int().default(1_000),
/** How many ms to wait until dequeuing again, if there was a run last time */
DEV_DEQUEUE_INTERVAL_WITH_RUN: z.coerce.number().int().default(250),
/** How many ms to wait until dequeuing again, if there was no run last time */
DEV_DEQUEUE_INTERVAL_WITHOUT_RUN: z.coerce.number().int().default(1_000),
/** The max number of runs per API call that we'll dequeue in DEV */
DEV_DEQUEUE_MAX_RUNS_PER_PULL: z.coerce.number().int().default(10),
/** The maximum concurrent local run processes executing at once in dev. This is a hard limit */
DEV_MAX_CONCURRENT_RUNS: z.coerce.number().int().optional(),
/** The CLI should connect to this for dev runs */
DEV_ENGINE_URL: z.string().default(process.env.APP_ORIGIN ?? "http://localhost:3030"),
LEGACY_RUN_ENGINE_WORKER_ENABLED: z.string().default(process.env.WORKER_ENABLED ?? "true"),
LEGACY_RUN_ENGINE_WORKER_CONCURRENCY_WORKERS: z.coerce.number().int().default(2),
LEGACY_RUN_ENGINE_WORKER_CONCURRENCY_TASKS_PER_WORKER: z.coerce.number().int().default(1),
LEGACY_RUN_ENGINE_WORKER_POLL_INTERVAL: z.coerce.number().int().default(1000),
LEGACY_RUN_ENGINE_WORKER_IMMEDIATE_POLL_INTERVAL: z.coerce.number().int().default(50),
LEGACY_RUN_ENGINE_WORKER_CONCURRENCY_LIMIT: z.coerce.number().int().default(50),
LEGACY_RUN_ENGINE_WORKER_SHUTDOWN_TIMEOUT_MS: z.coerce.number().int().default(60_000),
LEGACY_RUN_ENGINE_WORKER_LOG_LEVEL: z
.enum(["log", "error", "warn", "info", "debug"])
.default("info"),
LEGACY_RUN_ENGINE_WORKER_REDIS_HOST: z
.string()
.optional()
.transform((v) => v ?? process.env.REDIS_HOST),
LEGACY_RUN_ENGINE_WORKER_REDIS_READER_HOST: z
.string()
.optional()
.transform((v) => v ?? process.env.REDIS_READER_HOST),
LEGACY_RUN_ENGINE_WORKER_REDIS_READER_PORT: z.coerce
.number()
.optional()
.transform(
(v) =>
v ?? (process.env.REDIS_READER_PORT ? parseInt(process.env.REDIS_READER_PORT) : undefined)
),
LEGACY_RUN_ENGINE_WORKER_REDIS_PORT: z.coerce
.number()
.optional()
.transform(
(v) => v ?? (process.env.REDIS_PORT ? parseInt(process.env.REDIS_PORT) : undefined)
),
LEGACY_RUN_ENGINE_WORKER_REDIS_USERNAME: z
.string()
.optional()
.transform((v) => v ?? process.env.REDIS_USERNAME),
LEGACY_RUN_ENGINE_WORKER_REDIS_PASSWORD: z
.string()
.optional()
.transform((v) => v ?? process.env.REDIS_PASSWORD),
LEGACY_RUN_ENGINE_WORKER_REDIS_TLS_DISABLED: z
.string()
.default(process.env.REDIS_TLS_DISABLED ?? "false"),
LEGACY_RUN_ENGINE_WORKER_REDIS_CLUSTER_MODE_ENABLED: z.string().default("0"),
LEGACY_RUN_ENGINE_WAITING_FOR_DEPLOY_BATCH_SIZE: z.coerce.number().int().default(100),
LEGACY_RUN_ENGINE_WAITING_FOR_DEPLOY_BATCH_STAGGER_MS: z.coerce.number().int().default(1_000),
LEGACY_RUN_ENGINE_WAITING_FOR_DEPLOY_DISABLED: z.string().default("0"),
COMMON_WORKER_ENABLED: z.string().default(process.env.WORKER_ENABLED ?? "true"),
COMMON_WORKER_CONCURRENCY_WORKERS: z.coerce.number().int().default(2),
COMMON_WORKER_CONCURRENCY_TASKS_PER_WORKER: z.coerce.number().int().default(10),
COMMON_WORKER_POLL_INTERVAL: z.coerce.number().int().default(1000),
COMMON_WORKER_IMMEDIATE_POLL_INTERVAL: z.coerce.number().int().default(50),
COMMON_WORKER_CONCURRENCY_LIMIT: z.coerce.number().int().default(50),
COMMON_WORKER_SHUTDOWN_TIMEOUT_MS: z.coerce.number().int().default(60_000),
COMMON_WORKER_LOG_LEVEL: z.enum(["log", "error", "warn", "info", "debug"]).default("info"),
COMMON_WORKER_REDIS_HOST: z
.string()
.optional()
.transform((v) => v ?? process.env.REDIS_HOST),
COMMON_WORKER_REDIS_READER_HOST: z
.string()
.optional()
.transform((v) => v ?? process.env.REDIS_READER_HOST),
COMMON_WORKER_REDIS_READER_PORT: z.coerce
.number()
.optional()
.transform(
(v) =>
v ?? (process.env.REDIS_READER_PORT ? parseInt(process.env.REDIS_READER_PORT) : undefined)
),
COMMON_WORKER_REDIS_PORT: z.coerce
.number()
.optional()
.transform(
(v) => v ?? (process.env.REDIS_PORT ? parseInt(process.env.REDIS_PORT) : undefined)
),
COMMON_WORKER_REDIS_USERNAME: z
.string()
.optional()
.transform((v) => v ?? process.env.REDIS_USERNAME),
COMMON_WORKER_REDIS_PASSWORD: z
.string()
.optional()
.transform((v) => v ?? process.env.REDIS_PASSWORD),
COMMON_WORKER_REDIS_TLS_DISABLED: z.string().default(process.env.REDIS_TLS_DISABLED ?? "false"),
COMMON_WORKER_REDIS_CLUSTER_MODE_ENABLED: z.string().default("0"),
TRIGGER_MOLLIFIER_ENABLED: z.string().default("0"),
// Separate switch for the drainer (consumer side) so it can be split
// off onto a dedicated worker service. Unset → inherits
// TRIGGER_MOLLIFIER_ENABLED, so single-container self-hosters don't have to
// flip two switches. Multi-replica drainers are correct — `popAndMarkDraining`
// is an atomic ZPOPMIN + status flip in one Lua call, so only one replica
// can win any given entry — but inefficient: polling load (SMEMBERS +
// per-env scans) multiplies by N, and `TRIGGER_MOLLIFIER_DRAIN_CONCURRENCY`
// is per-process so engine load also multiplies. Splitting the drainer
// onto a dedicated worker keeps that traffic off the request-serving
// replicas. `TRIGGER_MOLLIFIER_ENABLED` is still the master kill switch;
// setting this to "1" while `TRIGGER_MOLLIFIER_ENABLED` is "0" is a
// no-op because the gate-side singleton refuses to construct a buffer
// when the system is off.
TRIGGER_MOLLIFIER_DRAINER_ENABLED: z.string().default(process.env.TRIGGER_MOLLIFIER_ENABLED ?? "0"),
TRIGGER_MOLLIFIER_SHADOW_MODE: z.string().default("0"),
TRIGGER_MOLLIFIER_REDIS_HOST: z
.string()
.optional()
.transform((v) => v ?? process.env.REDIS_HOST),
TRIGGER_MOLLIFIER_REDIS_PORT: z.coerce
.number()
.optional()
.transform(
(v) => v ?? (process.env.REDIS_PORT ? parseInt(process.env.REDIS_PORT) : undefined),
),
TRIGGER_MOLLIFIER_REDIS_USERNAME: z
.string()
.optional()
.transform((v) => v ?? process.env.REDIS_USERNAME),
TRIGGER_MOLLIFIER_REDIS_PASSWORD: z
.string()
.optional()
.transform((v) => v ?? process.env.REDIS_PASSWORD),
TRIGGER_MOLLIFIER_REDIS_TLS_DISABLED: z.string().default(process.env.REDIS_TLS_DISABLED ?? "false"),
TRIGGER_MOLLIFIER_TRIP_WINDOW_MS: z.coerce.number().int().positive().default(200),
TRIGGER_MOLLIFIER_TRIP_THRESHOLD: z.coerce.number().int().positive().default(100),
TRIGGER_MOLLIFIER_HOLD_MS: z.coerce.number().int().positive().default(500),
TRIGGER_MOLLIFIER_DRAIN_CONCURRENCY: z.coerce.number().int().positive().default(50),
TRIGGER_MOLLIFIER_DRAIN_MAX_ATTEMPTS: z.coerce.number().int().positive().default(3),
TRIGGER_MOLLIFIER_DRAIN_SHUTDOWN_TIMEOUT_MS: z.coerce.number().int().positive().default(30_000),
TRIGGER_MOLLIFIER_DRAIN_MAX_ORGS_PER_TICK: z.coerce.number().int().positive().default(500),
// Per-env per-tick pop cap. The drainer rotates one env per org per
// tick; this bounds how many entries it pops from that env before
// dispatching them through the shared `DRAIN_CONCURRENCY`-bounded
// limiter. Default matches `DRAIN_CONCURRENCY` so a single-env burst
// uses the full handler-parallelism budget — for 20k buffered on one
// env this is the difference between ~17m (one-pop-per-tick × ~50ms)
// and ~20s (400 ticks × concurrent engine.trigger). Org/env fairness
// is preserved because the per-tick env selection is unchanged; only
// the in-env pop count grows.
TRIGGER_MOLLIFIER_DRAIN_BATCH_SIZE: z.coerce.number().int().positive().default(50),
// Periodic sweep that scans buffer queue LISTs for entries whose
// dwell exceeds the stale threshold. Independent of the drainer —
// its job is exactly to make a stuck/offline drainer visible to
// ops. Defaults: explicitly opt-in (a separate kill switch from
// the mollifier itself), run every 5 minutes, alert on anything
// that's been dwelling for 5+ minutes (matches the sweep interval
// — "anything still here when we check" is the simplest threshold
// that converges).
//
// The sweep was previously defaulting to inherit
// `TRIGGER_MOLLIFIER_ENABLED`, which meant any deployment already
// running with the mollifier on would auto-start the sweep worker
// on upgrade — turning on new background load with no explicit
// rollout step. Hard-defaulting to "0" preserves the intent of
// exposing the sweep as a separate switch.
TRIGGER_MOLLIFIER_STALE_SWEEP_ENABLED: z.string().default("0"),
TRIGGER_MOLLIFIER_STALE_SWEEP_INTERVAL_MS: z.coerce
.number()
.int()
.positive()
.default(5 * 60_000),
TRIGGER_MOLLIFIER_STALE_SWEEP_THRESHOLD_MS: z.coerce
.number()
.int()
.positive()
.default(5 * 60_000),
BATCH_TRIGGER_PROCESS_JOB_VISIBILITY_TIMEOUT_MS: z.coerce
.number()
.int()
.default(60_000 * 5), // 5 minutes
BATCH_TRIGGER_CACHED_RUNS_CHECK_ENABLED: BoolEnv.default(false),
BATCH_TRIGGER_WORKER_ENABLED: z.string().default(process.env.WORKER_ENABLED ?? "true"),
BATCH_TRIGGER_WORKER_CONCURRENCY_WORKERS: z.coerce.number().int().default(2),
BATCH_TRIGGER_WORKER_CONCURRENCY_TASKS_PER_WORKER: z.coerce.number().int().default(10),
BATCH_TRIGGER_WORKER_POLL_INTERVAL: z.coerce.number().int().default(1000),
BATCH_TRIGGER_WORKER_IMMEDIATE_POLL_INTERVAL: z.coerce.number().int().default(50),
BATCH_TRIGGER_WORKER_CONCURRENCY_LIMIT: z.coerce.number().int().default(20),
BATCH_TRIGGER_WORKER_SHUTDOWN_TIMEOUT_MS: z.coerce.number().int().default(60_000),
BATCH_TRIGGER_WORKER_LOG_LEVEL: z
.enum(["log", "error", "warn", "info", "debug"])
.default("info"),
BATCH_TRIGGER_WORKER_REDIS_HOST: z
.string()
.optional()
.transform((v) => v ?? process.env.REDIS_HOST),
BATCH_TRIGGER_WORKER_REDIS_READER_HOST: z
.string()
.optional()
.transform((v) => v ?? process.env.REDIS_READER_HOST),
BATCH_TRIGGER_WORKER_REDIS_READER_PORT: z.coerce
.number()
.optional()
.transform(
(v) =>
v ?? (process.env.REDIS_READER_PORT ? parseInt(process.env.REDIS_READER_PORT) : undefined)
),
BATCH_TRIGGER_WORKER_REDIS_PORT: z.coerce
.number()
.optional()
.transform(
(v) => v ?? (process.env.REDIS_PORT ? parseInt(process.env.REDIS_PORT) : undefined)
),
BATCH_TRIGGER_WORKER_REDIS_USERNAME: z
.string()
.optional()
.transform((v) => v ?? process.env.REDIS_USERNAME),
BATCH_TRIGGER_WORKER_REDIS_PASSWORD: z
.string()
.optional()
.transform((v) => v ?? process.env.REDIS_PASSWORD),
BATCH_TRIGGER_WORKER_REDIS_TLS_DISABLED: z
.string()
.default(process.env.REDIS_TLS_DISABLED ?? "false"),
BATCH_TRIGGER_WORKER_REDIS_CLUSTER_MODE_ENABLED: z.string().default("0"),
// BatchQueue DRR settings (Run Engine v2)
BATCH_QUEUE_DRR_QUANTUM: z.coerce.number().int().default(25),
BATCH_QUEUE_MAX_DEFICIT: z.coerce.number().int().default(100),
BATCH_QUEUE_CONSUMER_COUNT: z.coerce.number().int().default(3),
BATCH_QUEUE_CONSUMER_INTERVAL_MS: z.coerce.number().int().default(50),
BATCH_QUEUE_WORKER_ENABLED: BoolEnv.default(true),
// Number of master queue shards for horizontal scaling
BATCH_QUEUE_SHARD_COUNT: z.coerce.number().int().default(1),
// Maximum queues to fetch from master queue per iteration
BATCH_QUEUE_MASTER_QUEUE_LIMIT: z.coerce.number().int().default(1000),
// Enable worker queue for two-stage processing (claim messages, push to worker queue, process from worker queue)
BATCH_QUEUE_WORKER_QUEUE_ENABLED: BoolEnv.default(true),
// Worker queue blocking timeout in seconds (for two-stage processing, only used when BATCH_QUEUE_WORKER_QUEUE_ENABLED is true)
BATCH_QUEUE_WORKER_QUEUE_TIMEOUT_SECONDS: z.coerce.number().int().default(10),
// Global rate limit: max items processed per second across all consumers
// If not set, no global rate limiting is applied
BATCH_QUEUE_GLOBAL_RATE_LIMIT: z.coerce.number().int().positive().optional(),
// Max items in the worker queue before claiming pauses (protects visibility timeouts)
// If not set, no depth limit is applied
BATCH_QUEUE_WORKER_QUEUE_MAX_DEPTH: z.coerce.number().int().positive().optional(),
ADMIN_WORKER_ENABLED: z.string().default(process.env.WORKER_ENABLED ?? "true"),
ADMIN_WORKER_CONCURRENCY_WORKERS: z.coerce.number().int().default(2),
ADMIN_WORKER_CONCURRENCY_TASKS_PER_WORKER: z.coerce.number().int().default(10),
ADMIN_WORKER_POLL_INTERVAL: z.coerce.number().int().default(1000),
ADMIN_WORKER_IMMEDIATE_POLL_INTERVAL: z.coerce.number().int().default(50),
ADMIN_WORKER_CONCURRENCY_LIMIT: z.coerce.number().int().default(20),
ADMIN_WORKER_SHUTDOWN_TIMEOUT_MS: z.coerce.number().int().default(60_000),
ADMIN_WORKER_LOG_LEVEL: z.enum(["log", "error", "warn", "info", "debug"]).default("info"),
ADMIN_WORKER_REDIS_HOST: z
.string()
.optional()
.transform((v) => v ?? process.env.REDIS_HOST),
ADMIN_WORKER_REDIS_READER_HOST: z
.string()
.optional()
.transform((v) => v ?? process.env.REDIS_READER_HOST),
ADMIN_WORKER_REDIS_READER_PORT: z.coerce
.number()
.optional()
.transform(
(v) =>
v ?? (process.env.REDIS_READER_PORT ? parseInt(process.env.REDIS_READER_PORT) : undefined)
),
ADMIN_WORKER_REDIS_PORT: z.coerce
.number()
.optional()
.transform(
(v) => v ?? (process.env.REDIS_PORT ? parseInt(process.env.REDIS_PORT) : undefined)
),
ADMIN_WORKER_REDIS_USERNAME: z
.string()
.optional()
.transform((v) => v ?? process.env.REDIS_USERNAME),
ADMIN_WORKER_REDIS_PASSWORD: z
.string()
.optional()
.transform((v) => v ?? process.env.REDIS_PASSWORD),
ADMIN_WORKER_REDIS_TLS_DISABLED: z.string().default(process.env.REDIS_TLS_DISABLED ?? "false"),
ADMIN_WORKER_REDIS_CLUSTER_MODE_ENABLED: z.string().default("0"),
ALERTS_WORKER_ENABLED: z.string().default(process.env.WORKER_ENABLED ?? "true"),
ALERTS_WORKER_CONCURRENCY_WORKERS: z.coerce.number().int().default(2),
ALERTS_WORKER_CONCURRENCY_TASKS_PER_WORKER: z.coerce.number().int().default(10),
ALERTS_WORKER_POLL_INTERVAL: z.coerce.number().int().default(1000),
ALERTS_WORKER_IMMEDIATE_POLL_INTERVAL: z.coerce.number().int().default(100),
ALERTS_WORKER_CONCURRENCY_LIMIT: z.coerce.number().int().default(50),
ALERTS_WORKER_SHUTDOWN_TIMEOUT_MS: z.coerce.number().int().default(60_000),
ALERTS_WORKER_LOG_LEVEL: z.enum(["log", "error", "warn", "info", "debug"]).default("info"),
ALERTS_WORKER_REDIS_HOST: z
.string()
.optional()
.transform((v) => v ?? process.env.REDIS_HOST),
ALERTS_WORKER_REDIS_READER_HOST: z
.string()
.optional()
.transform((v) => v ?? process.env.REDIS_READER_HOST),
ALERTS_WORKER_REDIS_READER_PORT: z.coerce
.number()
.optional()
.transform(
(v) =>
v ?? (process.env.REDIS_READER_PORT ? parseInt(process.env.REDIS_READER_PORT) : undefined)
),
ALERTS_WORKER_REDIS_PORT: z.coerce
.number()
.optional()
.transform(
(v) => v ?? (process.env.REDIS_PORT ? parseInt(process.env.REDIS_PORT) : undefined)
),
ALERTS_WORKER_REDIS_USERNAME: z
.string()
.optional()
.transform((v) => v ?? process.env.REDIS_USERNAME),
ALERTS_WORKER_REDIS_PASSWORD: z
.string()
.optional()
.transform((v) => v ?? process.env.REDIS_PASSWORD),
ALERTS_WORKER_REDIS_TLS_DISABLED: z.string().default(process.env.REDIS_TLS_DISABLED ?? "false"),
ALERTS_WORKER_REDIS_CLUSTER_MODE_ENABLED: z.string().default("0"),
SCHEDULE_ENGINE_LOG_LEVEL: z.enum(["log", "error", "warn", "info", "debug"]).default("info"),
SCHEDULE_WORKER_ENABLED: z.string().default(process.env.WORKER_ENABLED ?? "true"),
SCHEDULE_WORKER_CONCURRENCY_WORKERS: z.coerce.number().int().default(2),
SCHEDULE_WORKER_CONCURRENCY_TASKS_PER_WORKER: z.coerce.number().int().default(10),
SCHEDULE_WORKER_POLL_INTERVAL: z.coerce.number().int().default(1000),
SCHEDULE_WORKER_IMMEDIATE_POLL_INTERVAL: z.coerce.number().int().default(50),
SCHEDULE_WORKER_CONCURRENCY_LIMIT: z.coerce.number().int().default(50),
SCHEDULE_WORKER_SHUTDOWN_TIMEOUT_MS: z.coerce.number().int().default(30_000),
SCHEDULE_WORKER_DISTRIBUTION_WINDOW_SECONDS: z.coerce.number().int().default(30),
SCHEDULE_WORKER_REDIS_HOST: z
.string()
.optional()
.transform((v) => v ?? process.env.REDIS_HOST),
SCHEDULE_WORKER_REDIS_READER_HOST: z
.string()
.optional()
.transform((v) => v ?? process.env.REDIS_READER_HOST),
SCHEDULE_WORKER_REDIS_READER_PORT: z.coerce
.number()
.optional()
.transform(
(v) =>
v ?? (process.env.REDIS_READER_PORT ? parseInt(process.env.REDIS_READER_PORT) : undefined)
),
SCHEDULE_WORKER_REDIS_PORT: z.coerce
.number()
.optional()
.transform(
(v) => v ?? (process.env.REDIS_PORT ? parseInt(process.env.REDIS_PORT) : undefined)
),
SCHEDULE_WORKER_REDIS_USERNAME: z
.string()
.optional()
.transform((v) => v ?? process.env.REDIS_USERNAME),
SCHEDULE_WORKER_REDIS_PASSWORD: z
.string()
.optional()
.transform((v) => v ?? process.env.REDIS_PASSWORD),
SCHEDULE_WORKER_REDIS_TLS_DISABLED: z
.string()
.default(process.env.REDIS_TLS_DISABLED ?? "false"),
SCHEDULE_WORKER_REDIS_CLUSTER_MODE_ENABLED: z.string().default("0"),
TASK_EVENT_PARTITIONING_ENABLED: z.string().default("0"),
TASK_EVENT_PARTITIONED_WINDOW_IN_SECONDS: z.coerce.number().int().default(60), // 1 minute
DEPLOYMENTS_AUTORELOAD_POLL_INTERVAL_MS: z.coerce.number().int().default(5_000),
BULK_ACTION_AUTORELOAD_POLL_INTERVAL_MS: z.coerce.number().int().default(1_000),
QUEUES_AUTORELOAD_POLL_INTERVAL_MS: z.coerce.number().int().default(5_000),
SLACK_BOT_TOKEN: z.string().optional(),
SLACK_SIGNUP_REASON_CHANNEL_ID: z.string().optional(),
// kapa.ai
KAPA_AI_WEBSITE_ID: z.string().optional(),
// BetterStack
BETTERSTACK_API_KEY: z.string().optional(),
BETTERSTACK_STATUS_PAGE_ID: z.string().optional(),
RUN_REPLICATION_REDIS_HOST: z
.string()
.optional()
.transform((v) => v ?? process.env.REDIS_HOST),
RUN_REPLICATION_REDIS_READER_HOST: z
.string()
.optional()
.transform((v) => v ?? process.env.REDIS_READER_HOST),
RUN_REPLICATION_REDIS_READER_PORT: z.coerce
.number()
.optional()
.transform(
(v) =>
v ?? (process.env.REDIS_READER_PORT ? parseInt(process.env.REDIS_READER_PORT) : undefined)
),
RUN_REPLICATION_REDIS_PORT: z.coerce
.number()
.optional()
.transform(
(v) => v ?? (process.env.REDIS_PORT ? parseInt(process.env.REDIS_PORT) : undefined)
),
RUN_REPLICATION_REDIS_USERNAME: z
.string()
.optional()
.transform((v) => v ?? process.env.REDIS_USERNAME),
RUN_REPLICATION_REDIS_PASSWORD: z
.string()
.optional()
.transform((v) => v ?? process.env.REDIS_PASSWORD),
RUN_REPLICATION_REDIS_TLS_DISABLED: z
.string()
.default(process.env.REDIS_TLS_DISABLED ?? "false"),
RUN_REPLICATION_CLICKHOUSE_URL: z.string().optional(),
RUN_REPLICATION_ENABLED: z.string().default("0"),
RUN_REPLICATION_SLOT_NAME: z.string().default("task_runs_to_clickhouse_v1"),
RUN_REPLICATION_PUBLICATION_NAME: z.string().default("task_runs_to_clickhouse_v1_publication"),
RUN_REPLICATION_MAX_FLUSH_CONCURRENCY: z.coerce.number().int().default(2),
RUN_REPLICATION_FLUSH_INTERVAL_MS: z.coerce.number().int().default(1000),
RUN_REPLICATION_FLUSH_BATCH_SIZE: z.coerce.number().int().default(100),
RUN_REPLICATION_LEADER_LOCK_TIMEOUT_MS: z.coerce.number().int().default(30_000),
RUN_REPLICATION_LEADER_LOCK_EXTEND_INTERVAL_MS: z.coerce.number().int().default(10_000),
RUN_REPLICATION_ACK_INTERVAL_SECONDS: z.coerce.number().int().default(10),
RUN_REPLICATION_LOG_LEVEL: z.enum(["log", "error", "warn", "info", "debug"]).default("info"),
RUN_REPLICATION_CLICKHOUSE_LOG_LEVEL: z
.enum(["log", "error", "warn", "info", "debug"])
.default("info"),
RUN_REPLICATION_LEADER_LOCK_ADDITIONAL_TIME_MS: z.coerce.number().int().default(10_000),
RUN_REPLICATION_LEADER_LOCK_RETRY_INTERVAL_MS: z.coerce.number().int().default(500),
RUN_REPLICATION_WAIT_FOR_ASYNC_INSERT: z.string().default("0"),
RUN_REPLICATION_KEEP_ALIVE_ENABLED: z.string().default("0"),
RUN_REPLICATION_KEEP_ALIVE_IDLE_SOCKET_TTL_MS: z.coerce.number().int().optional(),
RUN_REPLICATION_MAX_OPEN_CONNECTIONS: z.coerce.number().int().default(10),
// Retry configuration for insert operations
RUN_REPLICATION_INSERT_MAX_RETRIES: z.coerce.number().int().default(3),
RUN_REPLICATION_INSERT_BASE_DELAY_MS: z.coerce.number().int().default(100),
RUN_REPLICATION_INSERT_MAX_DELAY_MS: z.coerce.number().int().default(2000),
RUN_REPLICATION_INSERT_STRATEGY: z.enum(["insert", "insert_async"]).default("insert"),
RUN_REPLICATION_DISABLE_PAYLOAD_INSERT: z.string().default("0"),
RUN_REPLICATION_DISABLE_ERROR_FINGERPRINTING: z.string().default("0"),
// Session replication (Postgres → ClickHouse sessions_v1). Shares Redis
// with the runs replicator for leader locking but has its own slot and
// publication so the two consume independently.
SESSION_REPLICATION_CLICKHOUSE_URL: z.string().optional(),
SESSION_REPLICATION_ENABLED: z.string().default("0"),
SESSION_REPLICATION_SLOT_NAME: z.string().default("sessions_to_clickhouse_v1"),
SESSION_REPLICATION_PUBLICATION_NAME: z
.string()
.default("sessions_to_clickhouse_v1_publication"),
SESSION_REPLICATION_MAX_FLUSH_CONCURRENCY: z.coerce.number().int().default(1),
SESSION_REPLICATION_FLUSH_INTERVAL_MS: z.coerce.number().int().default(1000),
SESSION_REPLICATION_FLUSH_BATCH_SIZE: z.coerce.number().int().default(100),
SESSION_REPLICATION_LEADER_LOCK_TIMEOUT_MS: z.coerce.number().int().default(30_000),
SESSION_REPLICATION_LEADER_LOCK_EXTEND_INTERVAL_MS: z.coerce.number().int().default(10_000),
SESSION_REPLICATION_LEADER_LOCK_ADDITIONAL_TIME_MS: z.coerce.number().int().default(10_000),
SESSION_REPLICATION_LEADER_LOCK_RETRY_INTERVAL_MS: z.coerce.number().int().default(500),
SESSION_REPLICATION_ACK_INTERVAL_SECONDS: z.coerce.number().int().default(10),
SESSION_REPLICATION_LOG_LEVEL: z
.enum(["log", "error", "warn", "info", "debug"])
.default("info"),
SESSION_REPLICATION_CLICKHOUSE_LOG_LEVEL: z
.enum(["log", "error", "warn", "info", "debug"])
.default("info"),
SESSION_REPLICATION_WAIT_FOR_ASYNC_INSERT: z.string().default("0"),
SESSION_REPLICATION_KEEP_ALIVE_ENABLED: z.string().default("0"),
SESSION_REPLICATION_KEEP_ALIVE_IDLE_SOCKET_TTL_MS: z.coerce.number().int().optional(),
SESSION_REPLICATION_MAX_OPEN_CONNECTIONS: z.coerce.number().int().default(10),
SESSION_REPLICATION_INSERT_STRATEGY: z.enum(["insert", "insert_async"]).default("insert"),
SESSION_REPLICATION_INSERT_MAX_RETRIES: z.coerce.number().int().default(3),
SESSION_REPLICATION_INSERT_BASE_DELAY_MS: z.coerce.number().int().default(100),
SESSION_REPLICATION_INSERT_MAX_DELAY_MS: z.coerce.number().int().default(2000),
// Clickhouse
CLICKHOUSE_URL: z.string(),
CLICKHOUSE_KEEP_ALIVE_ENABLED: z.string().default("1"),
CLICKHOUSE_KEEP_ALIVE_IDLE_SOCKET_TTL_MS: z.coerce.number().int().optional(),
CLICKHOUSE_MAX_OPEN_CONNECTIONS: z.coerce.number().int().default(10),
CLICKHOUSE_LOG_LEVEL: z.enum(["log", "error", "warn", "info", "debug"]).default("info"),
CLICKHOUSE_COMPRESSION_REQUEST: z.string().default("1"),
// Logs Query Settings
CLICKHOUSE_LOGS_LIST_MAX_MEMORY_USAGE: z.coerce.number().int().default(1_000_000_000),
CLICKHOUSE_LOGS_LIST_MAX_BYTES_BEFORE_EXTERNAL_SORT: z.coerce
.number()
.int()
.default(256_000_000),
CLICKHOUSE_LOGS_LIST_MAX_THREADS: z.coerce.number().int().default(2),
CLICKHOUSE_LOGS_LIST_MAX_ROWS_TO_READ: z.coerce.number().int().default(10_000_000),
CLICKHOUSE_LOGS_LIST_MAX_EXECUTION_TIME: z.coerce.number().int().default(120),
// Query feature flag
QUERY_FEATURE_ENABLED: z.string().default("1"),
// AI features (Prompts, Models, AI Metrics sidebar section)
AI_FEATURES_ENABLED: z.string().default("0"),
// Logs page ClickHouse URL (for logs queries)
LOGS_CLICKHOUSE_URL: z
.string()
.optional()
.transform((v) => v ?? process.env.CLICKHOUSE_URL),
// Query page ClickHouse limits (for TSQL queries)
QUERY_CLICKHOUSE_URL: z
.string()
.optional()
.transform((v) => v ?? process.env.CLICKHOUSE_URL),
QUERY_CLICKHOUSE_MAX_EXECUTION_TIME: z.coerce.number().int().default(10),
QUERY_CLICKHOUSE_MAX_MEMORY_USAGE: z.coerce.number().int().default(1_073_741_824), // 1GB in bytes
QUERY_CLICKHOUSE_MAX_AST_ELEMENTS: z.coerce.number().int().default(4_000_000),
QUERY_CLICKHOUSE_MAX_EXPANDED_AST_ELEMENTS: z.coerce.number().int().default(4_000_000),
QUERY_CLICKHOUSE_MAX_BYTES_BEFORE_EXTERNAL_GROUP_BY: z.coerce.number().int().default(0),
QUERY_CLICKHOUSE_MAX_RETURNED_ROWS: z.coerce.number().int().default(10_000),
// Query page concurrency limits
QUERY_DEFAULT_ORG_CONCURRENCY_LIMIT: z.coerce.number().int().default(3),
QUERY_GLOBAL_CONCURRENCY_LIMIT: z.coerce.number().int().default(100),
// Metric widget concurrency limits
METRIC_WIDGET_DEFAULT_ORG_CONCURRENCY_LIMIT: z.coerce.number().int().default(30),
// Admin ClickHouse URL (for admin dashboard queries like missing models)
ADMIN_CLICKHOUSE_URL: z
.string()
.optional()
.transform((v) => v ?? process.env.CLICKHOUSE_URL),
EVENTS_CLICKHOUSE_URL: z
.string()
.optional()
.transform((v) => v ?? process.env.CLICKHOUSE_URL),
EVENTS_CLICKHOUSE_KEEP_ALIVE_ENABLED: z.string().default("1"),
EVENTS_CLICKHOUSE_KEEP_ALIVE_IDLE_SOCKET_TTL_MS: z.coerce.number().int().optional(),
EVENTS_CLICKHOUSE_MAX_OPEN_CONNECTIONS: z.coerce.number().int().default(10),
EVENTS_CLICKHOUSE_LOG_LEVEL: z.enum(["log", "error", "warn", "info", "debug"]).default("info"),
EVENTS_CLICKHOUSE_COMPRESSION_REQUEST: z.string().default("1"),
// ClickHouse client used by @internal/run-engine's PendingVersionSystem.
// Kept on its own URL + pool so this low-QPS path can't contend with
// the main analytics client (CLICKHOUSE_URL). Falls back to the main
// URL when unset so unconfigured environments still work.
RUN_ENGINE_CLICKHOUSE_URL: z
.string()
.optional()
.transform((v) => v ?? process.env.CLICKHOUSE_URL),
RUN_ENGINE_CLICKHOUSE_KEEP_ALIVE_ENABLED: z.string().default("1"),
RUN_ENGINE_CLICKHOUSE_KEEP_ALIVE_IDLE_SOCKET_TTL_MS: z.coerce.number().int().optional(),
RUN_ENGINE_CLICKHOUSE_MAX_OPEN_CONNECTIONS: z.coerce.number().int().default(5),
RUN_ENGINE_CLICKHOUSE_LOG_LEVEL: z
.enum(["log", "error", "warn", "info", "debug"])
.default("info"),
RUN_ENGINE_CLICKHOUSE_COMPRESSION_REQUEST: z.string().default("1"),
EVENTS_CLICKHOUSE_BATCH_SIZE: z.coerce.number().int().default(1000),
EVENTS_CLICKHOUSE_FLUSH_INTERVAL_MS: z.coerce.number().int().default(1000),
METRICS_CLICKHOUSE_BATCH_SIZE: z.coerce.number().int().default(10000),
METRICS_CLICKHOUSE_FLUSH_INTERVAL_MS: z.coerce.number().int().default(1000),
METRICS_CLICKHOUSE_MAX_CONCURRENCY: z.coerce.number().int().default(3),
EVENTS_CLICKHOUSE_INSERT_STRATEGY: z.enum(["insert", "insert_async"]).default("insert"),
EVENTS_CLICKHOUSE_WAIT_FOR_ASYNC_INSERT: z.string().default("1"),
EVENTS_CLICKHOUSE_ASYNC_INSERT_MAX_DATA_SIZE: z.coerce.number().int().default(10485760),
EVENTS_CLICKHOUSE_ASYNC_INSERT_BUSY_TIMEOUT_MS: z.coerce.number().int().default(5000),
EVENTS_CLICKHOUSE_START_TIME_MAX_AGE_MS: z.coerce
.number()
.int()
.default(60_000 * 5), // 5 minutes
EVENT_REPOSITORY_DEFAULT_STORE: z
.enum(["postgres", "clickhouse", "clickhouse_v2"])
.default("postgres"),
EVENT_REPOSITORY_DEBUG_LOGS_DISABLED: BoolEnv.default(false),
EVENTS_CLICKHOUSE_MAX_TRACE_SUMMARY_VIEW_COUNT: z.coerce.number().int().default(25_000),
EVENTS_CLICKHOUSE_MAX_TRACE_DETAILED_SUMMARY_VIEW_COUNT: z.coerce.number().int().default(5_000),
EVENTS_CLICKHOUSE_MAX_LIVE_RELOADING_SETTING: z.coerce.number().int().default(2000),
// Organization data stores registry
ORGANIZATION_DATA_STORES_RELOAD_INTERVAL_MS: z.coerce
.number()
.int()
.default(60 * 1000), // 1 minute
// LLM cost tracking
LLM_COST_TRACKING_ENABLED: BoolEnv.default(true),
LLM_PRICING_RELOAD_INTERVAL_MS: z.coerce
.number()
.int()
.default(5 * 60 * 1000), // 5 minutes
LLM_PRICING_RELOAD_CHANNEL: z.string().default("llm-registry:reload"),
LLM_PRICING_RELOAD_DEBOUNCE_MS: z.coerce.number().int().default(1000),
// Whether to subscribe this process to the LLM_PRICING_RELOAD_CHANNEL.
// Default off — only OTel-ingesting services need real-time pricing
// freshness; dashboard/worker processes are fine on the existing
// 5-minute periodic reload. In multi-service deployments, set this to
// true on the span-ingesting services.
LLM_PRICING_RELOAD_PUBSUB_ENABLED: BoolEnv.default(false),
LLM_PRICING_SEED_ON_STARTUP: BoolEnv.default(false),
LLM_PRICING_READY_TIMEOUT_MS: z.coerce.number().int().default(500),
LLM_METRICS_BATCH_SIZE: z.coerce.number().int().default(5000),
LLM_METRICS_FLUSH_INTERVAL_MS: z.coerce.number().int().default(2000),
LLM_METRICS_MAX_BATCH_SIZE: z.coerce.number().int().default(10000),
LLM_METRICS_MAX_CONCURRENCY: z.coerce.number().int().default(2),
// Bootstrap
TRIGGER_BOOTSTRAP_ENABLED: z.string().default("0"),
TRIGGER_BOOTSTRAP_WORKER_GROUP_NAME: z.string().optional(),
TRIGGER_BOOTSTRAP_WORKER_TOKEN_PATH: z.string().optional(),
// Machine presets
MACHINE_PRESETS_OVERRIDE_PATH: z.string().optional(),
// CLI package tag (e.g. "latest", "v4-beta", "4.0.0") - used for setup commands
TRIGGER_CLI_TAG: z.string().default("latest"),
HEALTHCHECK_DATABASE_DISABLED: z.string().default("0"),
REQUEST_IDEMPOTENCY_REDIS_HOST: z
.string()
.optional()
.transform((v) => v ?? process.env.REDIS_HOST),
REQUEST_IDEMPOTENCY_REDIS_READER_HOST: z
.string()
.optional()
.transform((v) => v ?? process.env.REDIS_READER_HOST),
REQUEST_IDEMPOTENCY_REDIS_READER_PORT: z.coerce
.number()
.optional()
.transform(
(v) =>
v ?? (process.env.REDIS_READER_PORT ? parseInt(process.env.REDIS_READER_PORT) : undefined)
),
REQUEST_IDEMPOTENCY_REDIS_PORT: z.coerce
.number()
.optional()
.transform(
(v) => v ?? (process.env.REDIS_PORT ? parseInt(process.env.REDIS_PORT) : undefined)
),
REQUEST_IDEMPOTENCY_REDIS_USERNAME: z
.string()
.optional()
.transform((v) => v ?? process.env.REDIS_USERNAME),
REQUEST_IDEMPOTENCY_REDIS_PASSWORD: z
.string()
.optional()
.transform((v) => v ?? process.env.REDIS_PASSWORD),
REQUEST_IDEMPOTENCY_REDIS_TLS_DISABLED: z
.string()
.default(process.env.REDIS_TLS_DISABLED ?? "false"),
REQUEST_IDEMPOTENCY_LOG_LEVEL: z
.enum(["log", "error", "warn", "info", "debug"])
.default("info"),
REQUEST_IDEMPOTENCY_TTL_IN_MS: z.coerce
.number()
.int()
.default(60_000 * 60 * 24),
// Bulk action
BULK_ACTION_BATCH_SIZE: z.coerce.number().int().default(100),
BULK_ACTION_BATCH_DELAY_MS: z.coerce.number().int().default(200),
BULK_ACTION_SUBBATCH_CONCURRENCY: z.coerce.number().int().default(5),
// AI Run Filter
AI_RUN_FILTER_MODEL: z.string().optional(),
EVENT_LOOP_MONITOR_THRESHOLD_MS: z.coerce.number().int().default(100),
EVENT_LOOP_MONITOR_UTILIZATION_INTERVAL_MS: z.coerce.number().int().default(1000),
EVENT_LOOP_MONITOR_UTILIZATION_SAMPLE_RATE: z.coerce.number().default(0.05),
EVENT_LOOP_MONITOR_NOTIFY_ENABLED: z.string().default("0"),
VERY_SLOW_QUERY_THRESHOLD_MS: z.coerce.number().int().optional(),
REALTIME_STREAMS_S2_BASIN: z.string().optional(),
REALTIME_STREAMS_S2_ACCESS_TOKEN: z.string().optional(),
REALTIME_STREAMS_S2_ENDPOINT: z.string().optional(),
REALTIME_STREAMS_S2_SKIP_ACCESS_TOKENS: z.enum(["true", "false"]).default("false"),
REALTIME_STREAMS_S2_ACCESS_TOKEN_EXPIRATION_IN_MS: z.coerce
.number()
.int()
.default(60_000 * 60 * 24), // 1 day
REALTIME_STREAMS_S2_LOG_LEVEL: z
.enum(["log", "error", "warn", "info", "debug"])
.default("info"),
REALTIME_STREAMS_S2_FLUSH_INTERVAL_MS: z.coerce.number().int().default(100),
REALTIME_STREAMS_S2_MAX_RETRIES: z.coerce.number().int().default(10),
REALTIME_STREAMS_S2_WAIT_SECONDS: z.coerce.number().int().default(60),
// When "true", provision a dedicated S2 basin per org and stamp
// `streamBasinName` on new rows. Off keeps everything on the single
// basin defined by `REALTIME_STREAMS_S2_BASIN`.
REALTIME_STREAMS_PER_ORG_BASINS_ENABLED: z.enum(["true", "false"]).default("false"),
// Per-org basin name = `{prefix}-{env}-org-{orgId}`.
REALTIME_STREAMS_BASIN_NAME_PREFIX: z.string().default("triggerdotdev"),
REALTIME_STREAMS_BASIN_NAME_ENV: z.string().default("dev"),
REALTIME_STREAMS_BASIN_DEFAULT_RETENTION: durationString().default("30d"),
REALTIME_STREAMS_BASIN_STORAGE_CLASS: z.enum(["express", "standard"]).default("express"),
REALTIME_STREAMS_BASIN_DELETE_ON_EMPTY_MIN_AGE: durationString().default("1h"),
REALTIME_STREAMS_DEFAULT_VERSION: z.enum(["v1", "v2"]).default("v1"),
WAIT_UNTIL_TIMEOUT_MS: z.coerce.number().int().default(600_000),
// Private connections
PRIVATE_CONNECTIONS_ENABLED: z.string().optional(),
PRIVATE_CONNECTIONS_AWS_ACCOUNT_IDS: z.string().optional(),
// Force RBAC to not use the plugin
RBAC_FORCE_FALLBACK: BoolEnv.default(false),
})
.and(GithubAppEnvSchema)
.and(S2EnvSchema)
.superRefine((env, ctx) => {
const presets = new Set(env.COMPUTE_TEMPLATE_MACHINE_PRESETS);
for (const required of env.COMPUTE_TEMPLATE_MACHINE_PRESETS_REQUIRED) {
if (!presets.has(required)) {
ctx.addIssue({
code: z.ZodIssueCode.custom,
path: ["COMPUTE_TEMPLATE_MACHINE_PRESETS_REQUIRED"],
message: `"${required}" is not in COMPUTE_TEMPLATE_MACHINE_PRESETS`,
});
}
}
});
export type Environment = z.infer<typeof EnvironmentSchema>;
export const env = EnvironmentSchema.parse(process.env);
Reference in New Issue View Git Blame Copy Permalink