Files
Daniel Sutton 092b9ef07a fix(run-ops): DNS-safe, sortable base32hex run id (replace base62 KSUID) (#4154)
## Problem

The run-ops split mints NEW-store run ids as **27-char base62 KSUIDs**.
The supervisor writes the run id into the Kubernetes pod name
(`runner-<id>`), and pod names must be DNS-1123 labels (lowercase
`[a-z0-9-]`) — so uppercase base62 ids make k8s reject the pod (422) and
**those runs never launch** (they loop in `PENDING_EXECUTING` until the
heartbeat-stall handler nacks them, forever). `.toLowerCase()` can't fix
it: base62 has both `A`(10) and `a`(36) as distinct symbols, so folding
collides distinct ids and destroys sort order.

## Fix: change the encoding, not the structure

Mint a **26-char lowercase base32hex** run id:

```
run_<24-char base32hex core><region char><version char>
      [ 6-byte ms timestamp ][ 9 CSPRNG bytes ]
```

- **base32hex** (RFC 4648 §7, alphabet `0-9a-v`): lowercase,
order-preserving, DNS-safe; 15 bytes → exactly 24 chars, no padding.
Hand-rolled encode/decode (no new dependency).
- **48-bit ms timestamp** in the leading bytes → plain string sort ==
creation order at millisecond resolution.
- **72 bits CSPRNG** entropy; PK unique constraint is the backstop (no
retry loop).
- **region / version** are raw positional chars (read via one `charAt`
before decoding/routing), version = `"1"`.

DNS-safe from birth and hyphen-free, so **firekeeper is unchanged** —
`runner-<id>-attempt-N` → strip `runner-`, cut at first hyphen still
recovers the exact id incl. region+version.

## Residency discriminator: length → version char

`classifyKind`/`classifyResidency` (`runOpsResidency.ts`) previously
distinguished NEW vs LEGACY by **id length**. That gets ambiguous with a
third format. It now discriminates on the **version char at a fixed
position** (`isRunOpsIdBody`: 26 chars, `[25] === "1"`, base32hex
alphabet) → NEW; everything else → LEGACY. Total, never throws. The
`Residency` (NEW/LEGACY) contract the routing store consumes is
unchanged; the `"ksuid"` `ResidencyKind` label is retained only because
it's the persisted `runOpsMintKsuid` feature-flag value.

## Scope / verification

- Generator + discriminator in `@trigger.dev/core` isomorphic; mint path
+ all id-shape call sites swept (~40 webapp files); changeset added
(`@trigger.dev/core` patch).
- Core unit tests (encode/decode round-trip + property, generator shape,
ms sort-order incl. intra-second, parse partitioned-vs-legacy,
firekeeper round-trip): **24 pass**. `@trigger.dev/core` builds; webapp
typechecks; format/lint clean.

## Open decisions (flagged, not silently chosen)

1. **Backward-compat**: existing 27-char base62 KSUID runs now classify
LEGACY. On test cloud these are the broken/looping runs that never
completed, so this is acceptable — but worth a conscious call before
prod. No transitional length-recognition added (keeps the discriminator
clean).
2. **Storage collation**: the sort guarantee is byte-order — if the
run-ops id column is `TEXT` with default locale collation it's silently
not honored. Confirm whether `COLLATE "C"` / `BYTEA` is needed on the
run-ops schema.
3. **Region sourcing** wiring — see `regionCharForRegion` /
`REGION_CODES`.


---

## ⚠️ Required migration — deploy in lockstep

This PR renames a persisted feature-flag key/value and an env var. These
are **not** changed by the code alone and must be migrated when this
deploys, or affected orgs silently fall back to `cuid` minting (no crash
— `defaultValue: "cuid"`):

1. **Env var** (terraform): `RUN_OPS_MINT_KSUID_ENABLED` →
`RUN_OPS_MINT_ENABLED` (carry the value over).
2. **DB** `organization.featureFlags`: migrate both the key and value
together:
   - key `runOpsMintKsuid` → `runOpsMintKind`
   - value `"ksuid"` → `"runOpsId"`

Until an org's flag row is migrated, its `runOpsMintKind` lookup misses
and it mints `cuid` (legacy) — so no NEW-store ids for that org until
the data lands.

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-05 10:05:54 +01:00

85 lines
2.7 KiB
TypeScript

/**
* isSplitEnabled() is the Wave-0 gate. The entire migration/routing/FK-drop family
* MUST be unreachable when this returns false. Default is false (single-DB). Never
* infer split-vs-single from URL string-equality — distinctness is proven by the
* runtime sentinel.
*/
import { env } from "~/env.server";
import { logger } from "~/services/logger.server";
import { probeDistinctDatabases as defaultProbe } from "./distinctDbSentinel.server";
export type SplitModeConfig = {
flagEnabled: boolean;
legacyUrl?: string;
newUrl?: string;
};
export type SplitModeDeps = {
probe?: typeof defaultProbe;
logger?: { warn: (msg: string, meta?: Record<string, unknown>) => void };
};
export async function computeSplitEnabled(
config: SplitModeConfig,
deps: SplitModeDeps = {}
): Promise<boolean> {
// Hard gate #1: explicit positive opt-in. OFF by default -> never probe.
if (!config.flagEnabled) {
return false;
}
// Both URLs are required to even consider a split.
if (!config.legacyUrl || !config.newUrl) {
deps.logger?.warn(
"RUN_OPS_SPLIT_ENABLED is on but RUN_OPS_LEGACY_DATABASE_URL / RUN_OPS_DATABASE_URL are not both set; staying single-DB."
);
return false;
}
// Hard gate #2: runtime sentinel must confirm physically-distinct DBs.
const probe = deps.probe ?? defaultProbe;
const result = await probe(config.legacyUrl, config.newUrl, { logger: deps.logger });
return result.distinct === true;
}
export type SplitRealtimeInterlockConfig = {
splitEnabled: boolean;
nativeRealtimeEnabled: boolean;
};
/**
* Boot-time realtime interlock (pure predicate). Split mode puts NEW-resident
* (run-ops id) runs on the dedicated run-ops DB, but Electric replicates only from the
* control-plane DB — with the native realtime backend OFF those runs are invisible
* and every realtime subscription hangs. Refuse split unless native is on; split-off
* is always allowed regardless of the realtime backend.
*/
export function assertSplitRealtimeInterlock(config: SplitRealtimeInterlockConfig): void {
if (!config.splitEnabled) {
return;
}
if (!config.nativeRealtimeEnabled) {
throw new Error(
"RUN_OPS_SPLIT_ENABLED is on but the native realtime backend (REALTIME_BACKEND_NATIVE_ENABLED) is not enabled — Electric cannot serve NEW-resident runs; refusing to enable split."
);
}
}
let cached: Promise<boolean> | undefined;
export function isSplitEnabled(): Promise<boolean> {
if (!cached) {
cached = computeSplitEnabled(
{
flagEnabled: env.RUN_OPS_SPLIT_ENABLED,
legacyUrl: env.RUN_OPS_LEGACY_DATABASE_URL,
newUrl: env.RUN_OPS_DATABASE_URL,
},
{ logger }
);
}
return cached;
}
export function __resetSplitModeCacheForTests(): void {
cached = undefined;
}