fix(webapp): stop replica lag from double-triggering session runs and 404ing fresh sessions (#3914)

## Summary

Two read-replica races on the session APIs could break chats whose first
activity lands inside the replication window (or any time the replica
lags):

1. A session's first `.in` append or `.out` subscribe could fail with a
404 for a session that exists on the writer, because the route resolved
the Session row on the replica only.
2. `ensureRunForSession` probed run liveness on the replica, so a probe
miss on a run triggered moments earlier was judged "run is dead" and a
second live run was spawned for the same session. Both runs then
consumed the same input stream, producing duplicated turns and doubled
responses (and doubled LLM cost).

## Fix

Liveness now re-probes the writer before declaring the current run dead
(the old code already fell back to the writer, but only to recover the
friendlyId, after the wrong verdict was made). Session resolution on the
append and subscribe/init routes goes through a new
`resolveSessionWithWriterFallback`, which stays replica-first on the hot
path and only touches the writer on a miss.

Reproduced and verified against a local streaming replica with an
artificial apply delay: pre-fix, a send immediately after session
creation reliably produced either the 404 or two executing runs with a
doubled response; post-fix, the same flow produces exactly one run and
one response.

Also rides along: the local docker replica's default apply delay drops
from 150ms to a realistic 20ms (override via `REPLICA_APPLY_DELAY` when
you want to deliberately widen the race window).
This commit is contained in:
Eric Allam
2026-06-12 14:07:36 +01:00
committed by GitHub
parent eb498d137f
commit a04cdffda6
6 changed files with 53 additions and 32 deletions
+3 -2
View File
@@ -58,7 +58,8 @@ services:
# docker exec database bash -c 'grep -q "host replication" "$PGDATA/pg_hba.conf" || echo "host replication all all md5" >> "$PGDATA/pg_hba.conf"'
# docker exec database psql -U postgres -c "SELECT pg_reload_conf()"
# Then point the webapp at it: DATABASE_READ_REPLICA_URL=postgresql://postgres:postgres@localhost:5433/postgres
# Tune the lag via REPLICA_APPLY_DELAY (e.g. 150ms, 2s). Wipe database-replica-data to re-init.
# Tune the lag via REPLICA_APPLY_DELAY (default 20ms ~ realistic prod lag; crank to 150ms/2s to
# shake out replica races). Wipe database-replica-data to re-init.
database-replica:
container_name: ${CONTAINER_PREFIX:-}database-replica
profiles: ["replica"]
@@ -72,7 +73,7 @@ services:
- ${DB_REPLICA_VOLUME:-database-replica-data}:/var/lib/postgresql/data/
environment:
PGPASSWORD: postgres
REPLICA_APPLY_DELAY: ${REPLICA_APPLY_DELAY:-150ms}
REPLICA_APPLY_DELAY: ${REPLICA_APPLY_DELAY:-20ms}
networks:
- app_network
ports: