feat(webapp): per-org S2 basin migration (#3516)

## Summary

Move from a single shared S2 basin to **per-org basins** with retention
tied to the org's billing plan. Stops S2 from deleting streams out from
under live chat sessions when basin retention fires before the chat
ends, and unlocks per-org cost attribution.

OSS / s2-lite installs are unaffected: provisioning is gated by
`REALTIME_STREAMS_PER_ORG_BASINS_ENABLED` (default `false`), and the
read precedence falls back to the global basin env var when an entity
has no stamped basin.

```
basin = run.streamBasinName ?? session.streamBasinName ?? env.REALTIME_STREAMS_S2_BASIN
```

## Design

Three nullable `streamBasinName` columns (`Organization`, `TaskRun`,
`Session`) plus a provisioner that idempotently creates the basin and
reconfigures retention on plan changes. The trigger and session-create
paths stamp the org's basin onto new rows; the realtime read path picks
the basin from the entity context.

Admin routes back-fill existing orgs and force-reconfigure a single org.

## Test plan

- [x] `pnpm run typecheck --filter webapp --filter @internal/run-engine`
- [x] Backfill admin route end-to-end (provision + DB stamp + S2 basin
config).
- [x] Reconfigure on plan change (all retention tiers).
- [x] chat.agent multi-turn drives streams into the per-org basin.
- [x] Legacy fallback when entity has no stamped basin.
- [x] Provisioner is a no-op when the flag is off.
This commit is contained in:
Eric Allam
2026-05-05 10:06:58 +01:00
committed by GitHub
parent 3d418a9482
commit 386b4f65ff
23 changed files with 548 additions and 59 deletions
@@ -0,0 +1,8 @@
-- AlterTable
ALTER TABLE "public"."Organization" ADD COLUMN IF NOT EXISTS "streamBasinName" TEXT;
-- AlterTable
ALTER TABLE "public"."Session" ADD COLUMN IF NOT EXISTS "streamBasinName" TEXT;
-- AlterTable
ALTER TABLE "public"."TaskRun" ADD COLUMN IF NOT EXISTS "streamBasinName" TEXT;
@@ -251,6 +251,13 @@ model Organization {
platformNotifications PlatformNotification[]
errorGroupStates ErrorGroupState[]
/// S2 basin that holds this org's realtime streams. Null until the
/// per-org basin has been provisioned (OSS / s2-lite installs leave
/// it null forever; reads fall back to the global basin env var).
/// Set once at provisioning time; retention is reconfigured in-place
/// when the org's plan changes.
streamBasinName String?
}
model OrgMember {
@@ -758,6 +765,12 @@ model Session {
createdAt DateTime @default(now())
updatedAt DateTime @updatedAt
/// S2 basin where this session's stream pair lives. Stamped at create
/// time from `Organization.streamBasinName` so reads can resolve the
/// basin without joining org. Null when the org has no per-org basin
/// (OSS, or pre-backfill); reads fall back to the global basin.
streamBasinName String?
runs SessionRun[]
/// Idempotency: `(env, externalId)` uniquely identifies a session.
@@ -992,6 +1005,11 @@ model TaskRun {
realtimeStreamsVersion String @default("v1")
/// Store the stream keys that are being used by the run
realtimeStreams String[] @default([])
/// S2 basin where this run's realtime streams live. Stamped at create
/// time from `Organization.streamBasinName` so reads can resolve the
/// basin without joining org. Null when the org has no per-org basin
/// (OSS, or pre-backfill); reads fall back to the global basin.
streamBasinName String?
@@unique([oneTimeUseToken])
@@unique([runtimeEnvironmentId, taskIdentifier, idempotencyKey])