Matt Aitken
c2c6e5c705
## Summary Runs created for a Session were triggered without a realtime streams version, so they fell through to the `realtimeStreamsVersion` column default of `v1`. A Session's own `.in` / `.out` channels are always `v2`, so any run-scoped `streams.append()` or `streams.pipe()` call made inside a session run wrote to a different backend than the session it belongs to, and stayed there for the life of the run. The API trigger routes were never affected. They call `determineRealtimeStreamsVersion` with the client's `x-trigger-realtime-streams-version` header and always pass an explicit value, so a current SDK asking for v2 gets it. Only the internal callers that build trigger options by hand were leaning on the column default, which no env var can influence because that path never calls the resolver at all. ## The version resolver Fixing the call site exposed a second problem in `determineRealtimeStreamsVersion`. Its two paths disagreed: an explicit `v2` was checked against the S2 configuration first, but when the caller expressed no preference it returned `REALTIME_STREAMS_DEFAULT_VERSION` verbatim with no check. A deployment that set the default to `v2` without configuring S2 therefore stamped runs `v2`, nothing failed at trigger time, and every later read or write against those runs' streams threw `Realtime streams v2 is required for this run but S2 configuration is missing` for the life of the run. Both paths now resolve through one pure function that takes its configuration rather than reading `env`: ```ts const requested = streamVersion ?? config.defaultVersion; if (requested !== "v2") return "v1"; const hasCredentials = Boolean(config.accessToken) || config.skipAccessTokens; return hasCredentials && Boolean(config.basin) ? "v2" : "v1"; ``` ## The basin requirement `resolveStreamBasin` resolves run, session and organization basins ahead of the global setting, so a deployment that provisions a basin per organization can serve v2 with no global basin at all. Gating purely on the global setting would degrade every run there to `v1`. `determineRealtimeStreamsVersion` therefore takes an optional organization basin, and every caller that holds one passes it, including the session path: ```ts basin: organizationBasinName ?? env.REALTIME_STREAMS_S2_BASIN, ``` This is deliberately the resolved basin and not the `REALTIME_STREAMS_PER_ORG_BASINS_ENABLED` flag. The flag says the feature is on, not that a given organization has been provisioned, and provisioning happens out of band. Keying off the flag would stamp `v2` on runs for unprovisioned organizations, recreating the failure this removes. **This widens behaviour for explicit `v2` requests**, which previously required the global basin: a provisioned organization on a per-org deployment now resolves `v2` where it used to get `v1`. That is intentional, and it makes every path agree. ## Scope Only newly created runs change. A run already stamped `v1` keeps that version for its lifetime by design, since readers resolve the backend from the same column and its existing streams have to stay readable. Scheduled runs reach the same column default through `scheduleEngine.server.ts` and are deliberately left alone: that one is a policy question about `REALTIME_STREAMS_DEFAULT_VERSION` rather than an inconsistency inside a single feature. ## Verification A full-stack e2e boots the real webapp plus Postgres, Redis and s2-lite, creates a Session through the public API so the run comes from the real trigger path, appends records the way `streams.append()` does, and asserts three things at once: the version stamped on the run, that the payload is readable from S2, and that no key exists in Redis. It appends at a realistic record size so the route's body cap and S2's per-record cap are both exercised. Reverting the session-path change flips all three observations, so it fails against the old behaviour rather than passing vacuously. Unit tests cover the resolver matrix, including organization-basin-only and credential-only configurations; two of them fail against the previous resolver. Also verified by hand against a local stack: a real `chat.agent` session run writing 8 records of 250KB through `streams.append()` put 2,049,072 bytes into S2 with no Redis key, while the same agent with the session-path change removed put 2,102,360 bytes into Redis and nothing into S2.
Server Changes
This directory tracks changes to server-only components (webapp, supervisor, etc.) that are not captured by changesets. Changesets only track published npm packages — server changes would otherwise go undocumented.
When to add a file
Server-only PRs: If your PR only changes apps/webapp/, apps/supervisor/, or other server components (and does NOT change anything in packages/), add a .server-changes/ file.
Mixed PRs (both packages and server): Just add a changeset as usual. No .server-changes/ file needed — the changeset covers it.
Package-only PRs: Just add a changeset as usual.
File format
Create a markdown file with a descriptive name:
.server-changes/fix-batch-queue-stalls.md
With this format:
---
area: webapp
type: fix
---
Speed up batch queue processing by removing stalls and fixing retry race
Fields
- area (required):
webapp|supervisor - type (required):
feature|fix|improvement|breaking
Description
The body text (below the frontmatter) is a one-line description of the change. Keep it concise — it will appear in release notes.
Writing guidance
These entries are public-facing - they ship verbatim in user-visible release notes. A few rules to keep them clean:
- Write for the user, not the reviewer. Lead with what the user notices or has to do. If a reader who doesn't know the codebase can't tell what changed for them, rewrite it.
- One sentence is usually enough. The body is the bullet in the changelog. If you need a paragraph, you're probably describing the implementation rather than the change.
- Describe behavior, not implementation. Skip internal scopes, middleware names, library specifics, framework internals. Users care about what's different for them, not how it's wired.
- Never name internal tools or infra. Observability stacks, internal services, infra components, monitoring backends, CI surfaces, AWS specifics - none of these belong in user-facing notes.
Before / after:
- ❌ "The image verification step now parses the manifest's layer media types and returns a new result the finalizer rejects." (describes the wiring; a user can't act on it)
- ✅ "Deploying with an outdated CLI could produce an image that fails to start on every run. These deploys are now stopped before going live, with a message asking you to upgrade the CLI and re-deploy." (what the user sees and does)
Lifecycle
- Engineer adds a
.server-changes/file in their PR - Files accumulate on
mainas PRs merge - The changeset release PR includes these in its summary
- After the release merges, CI cleans up the consumed files
Examples
New feature:
---
area: webapp
type: feature
---
TRQL query language and the Query page
Bug fix:
---
area: webapp
type: fix
---
Fix schedule limit counting for orgs with custom limits
Improvement:
---
area: webapp
type: improvement
---
Use the replica for API auth queries to reduce primary load