920892bc11
Gives read-through and idempotency their gen-2 shard arms, so an id that names its own shard is read there and nowhere else. #4764 has landed, so this now targets `main` directly and no longer depends on an unmerged branch. It builds on what that PR supplied: `resolveShard`, `runOpsShardHandles` and the keyed router. TRI-13431 ## What changes **Read-through routes by `resolveShard`, not by the binary residency classifier.** A gen-2 id reads its own shard's replica once and probes no other store. A gen-1 v1 id still reads new only. **Callers now declare `idKind`.** A cuid gives no way to tell a run id from a waitpoint id, and the two must route differently: - a legacy-classified **run** id reads the legacy replica only — there is no cuid run migration, so the new-store probe cannot find it; - a cuid **waitpoint** keeps the new-first pair probe, which is load-bearing because a cuid waitpoint can be co-located with its run on the new store. There is no default, because a default would pick one of those arms silently. The field `runId` is renamed to `id`, since it carried both kinds already. **`ReadThroughResult` carries `found`.** `source` is an open-ended union once shards exist, so a consumer testing found-ness by listing the hit sources reads a gen-2 hit as a miss. One consumer did exactly that. Discriminating on `found` makes that class of bug a compile error rather than something a reviewer has to spot. **Idempotency resolves its client through one shard-keyed map.** Both call sites go through `clientForShardKey`, so they cannot disagree about which store owns an id. An absent key takes an explicit logged branch to the fallback, not a silent legacy default. The `classify` seam is retyped to return a `ShardKey`: `Residency` (`"NEW"`) and the reserved shard keys (`"new"`) differ only by case, and `ShardKey` collapses to `string`, so the compiler would not have caught feeding one into the other. The dead `isMigrated` branch is deleted. Nothing implemented it, and the one production comment recorded that omitting it was deliberate. **`PostgresRunStore._residency` widens to `ShardKey`.** Still unused; the store stays unaware of its siblings. ## Two behaviour fixes found while doing the above **An unconfigured shard key logs and returns not-found instead of throwing.** The waitpoint route takes the id from a URL parameter, and any base32hex core plus `[a-z0-9]` plus `"2"` parses as gen-2. The route turns a throw into a 500, so throwing here would let any authenticated client generate 500s and error logs by guessing shard chars, of which there are 36. An error-logged not-found is neither silent nor a misroute. Throwing stays correct on the router path, where ids are minted rather than received. **The two cross-seam batch hydration sites were gen-2 blind.** `hydrateRunsAcrossSeam` and `ApiBatchResultsPresenter` classified with the binary `ownerEngine`, so a gen-2 run id joined the gen-1 `new` group, missed there, and — classifying dedicated-family — never reached the legacy probe either. The id was dropped from a bulk-action page and from batch results with no error. Both now partition ids by shard key and read each configured shard once. Also: a gen-2 waitpoint that missed its shard replica fell back to the gen-1 new writer, a different database, silently disabling read-your-writes for the freshly minted token that fallback exists to serve. It now falls back to its own shard's writer. ## Merge safety Inert while `RUN_OPS_SHARDS` is unset: the shard maps are empty, so every gen-2 arm is unreachable, and gen-2 minting is not live yet. The one live change is the gen-1 run arm, and it removes work rather than adding it. `RoutingRunStore.findRun` never forwards the caller's client object — it routes by id and reads only the client's presence and replica brand — so `readRunForEvent`'s "new" closure already resolved a legacy-classified run id to the legacy store. The arm removes a duplicated read of the legacy replica. A test pins this, because a future caller passing a raw client and a run id would lose the pre-cutover 27-char case, which is new-resident but classifies legacy. ## Testing 14 tests added, testcontainers throughout, no mocks. 22 affected test files pass; typecheck, lint, format and knip are clean. Both arms were verified by neutralising them and confirming the new tests fail. The batch-results test needed rewriting after that check: the first version passed with the fix neutralised, because it used one container as both the gen-1 new client and the shard replica, so it was not testing what it claimed. Note for review: run testcontainer suites in small batches. Sixteen at once starves Docker and everything times out at 60 seconds. The run-ops legacy-guard baseline is refreshed in its own commit. The baseline is keyed by line number, so partitioning the batch-results read shifted four pre-existing entries and added one. Baselined violations in that file go from four to five, all reads; the new one is the shard read beside two gen-1 reads already there. No changeset and no `.server-changes` entry: a user notices nothing while the flag is unset.
97 lines
3.8 KiB
TypeScript
97 lines
3.8 KiB
TypeScript
import { resolveShard, type ShardKey } from "@trigger.dev/core/v3/isomorphic";
|
|
import type { PrismaReplicaClient } from "~/db.server";
|
|
import {
|
|
runOpsLegacyReplica as defaultLegacyReplica,
|
|
runOpsNewPrisma as defaultNewPrimary,
|
|
runOpsNewReplica as defaultNewClient,
|
|
runOpsSplitReadEnabled as defaultSplitReadEnabled,
|
|
} from "~/db.server";
|
|
import {
|
|
runOpsShardReplicas as defaultShardReplicas,
|
|
runOpsShardWriters as defaultShardWriters,
|
|
} from "~/v3/runOpsMigration/shardHandles.server";
|
|
import { readThroughRun } from "~/v3/runOpsMigration/readThrough.server";
|
|
|
|
type ResolveWaitpointDeps = {
|
|
newClient?: PrismaReplicaClient;
|
|
legacyReplica?: PrismaReplicaClient;
|
|
newPrimary?: PrismaReplicaClient;
|
|
shardReplicas?: ReadonlyMap<ShardKey, PrismaReplicaClient>;
|
|
shardWriters?: ReadonlyMap<ShardKey, PrismaReplicaClient>;
|
|
splitEnabled?: boolean;
|
|
isPastRetention?: (id: string) => boolean;
|
|
};
|
|
|
|
// Safe defaults matching the deps `complete`/`callback` pass, so a bare caller still fans
|
|
// out to the dedicated run-ops replica (NEW-resident waitpoints) before control-plane.
|
|
export type ResolveWaitpointReadThroughDefaults = {
|
|
newClient: PrismaReplicaClient;
|
|
legacyReplica: PrismaReplicaClient;
|
|
newPrimary: PrismaReplicaClient;
|
|
shardReplicas: ReadonlyMap<ShardKey, PrismaReplicaClient>;
|
|
shardWriters: ReadonlyMap<ShardKey, PrismaReplicaClient>;
|
|
splitEnabled: boolean;
|
|
};
|
|
|
|
const productionDefaults: ResolveWaitpointReadThroughDefaults = {
|
|
newClient: defaultNewClient,
|
|
legacyReplica: defaultLegacyReplica,
|
|
newPrimary: defaultNewPrimary as unknown as PrismaReplicaClient,
|
|
shardReplicas: defaultShardReplicas,
|
|
shardWriters: defaultShardWriters as unknown as ReadonlyMap<ShardKey, PrismaReplicaClient>,
|
|
splitEnabled: defaultSplitReadEnabled,
|
|
};
|
|
|
|
export async function resolveWaitpointThroughReadThrough<T>(opts: {
|
|
waitpointId: string;
|
|
environmentId: string;
|
|
read: (client: PrismaReplicaClient) => Promise<T | null>;
|
|
deps?: ResolveWaitpointDeps;
|
|
defaults?: ResolveWaitpointReadThroughDefaults;
|
|
}): Promise<T | null> {
|
|
const defaults = opts.defaults ?? productionDefaults;
|
|
|
|
const splitEnabled = opts.deps?.splitEnabled ?? defaults.splitEnabled;
|
|
|
|
const result = await readThroughRun({
|
|
id: opts.waitpointId,
|
|
idKind: "waitpoint",
|
|
environmentId: opts.environmentId,
|
|
readNew: (client) => opts.read(client),
|
|
readLegacy: (replica) => opts.read(replica),
|
|
deps: {
|
|
splitEnabled,
|
|
newClient: opts.deps?.newClient ?? defaults.newClient,
|
|
legacyReplica: opts.deps?.legacyReplica ?? defaults.legacyReplica,
|
|
shardReplicas: opts.deps?.shardReplicas ?? defaults.shardReplicas,
|
|
isPastRetention: opts.deps?.isPastRetention,
|
|
},
|
|
});
|
|
|
|
if (result.found) {
|
|
return result.value;
|
|
}
|
|
// past-retention is an intentional not-found: the token is gone.
|
|
if (result.reason === "past-retention") {
|
|
return null;
|
|
}
|
|
|
|
// Read-your-writes fallback for a token completed immediately after mint, before it replicated:
|
|
// re-read from the owning store's PRIMARY only. We deliberately never read the control-plane/legacy
|
|
// primary here (that is the load the replica-only read-through exists to shed), so a legacy-resident
|
|
// token that misses its replica stays a miss and the caller retries, rather than adding primary load.
|
|
const shardKey = resolveShard(opts.waitpointId);
|
|
if (shardKey !== "new" && shardKey !== "legacy") {
|
|
// A gen-2 token's primary is its OWN shard's writer. The gen-1 new writer is a different
|
|
// database, so reading it would miss and silently disable read-your-writes here.
|
|
const shardWriter = (opts.deps?.shardWriters ?? defaults.shardWriters).get(shardKey);
|
|
return shardWriter ? await opts.read(shardWriter) : null;
|
|
}
|
|
|
|
const fromNewPrimary = await opts.read(opts.deps?.newPrimary ?? defaults.newPrimary);
|
|
if (fromNewPrimary != null) {
|
|
return fromNewPrimary;
|
|
}
|
|
return null;
|
|
}
|