appendChatMessageOnce already verifies organizationId, but left it optional
because these call sites never threaded one — so the wake and the consented
investigation wrote durable user-facing messages with the organization check
skipped. Both paths now pass it on every append, the retry repair included, and
the chat id and the organization have to agree for a row to land.
The transcript moved out of `chats.messages` into `chat_messages`. TypeScript
already rejects a reference through the Drizzle schema, but a raw-SQL reference
compiles fine and only fails at runtime, and the earlier guard test went away
with the column. Zero hits today is the point: the test exists so a
reintroduction is caught rather than deployed.
`storeChatMessages` ended in `onConflictDoUpdate`, so `persistMessages` and
`persistTurn` — which are handed a whole snapshot — treated any differing body
under an existing message id as a deliberate finalisation. A stale snapshot
carrying `wake:watch_1:fired`, the watch consent record, the deterministic
confirmation or an investigation settlement card with a different body would
overwrite the durable row that was already recorded. The proxy caps body size
and metadata but does not rewrite message ids, so this was not an
internal-bug-only exposure. The same clause updated only the `message` JSONB and
never the `role` column, so `chat_messages.role` could end up disagreeing with
`message.role` — and the UI reads one while the quota query reads the other.
Ordinary transcript writes are now insert-only. Changing a stored message is its
own operation, `finalizeChatMessage`, guarded on chat id, message id and role.
`role` is verified rather than updated, and verified on both sides: the stored
column must match `expectedRole` and so must the incoming body's own `role`, so
the two cannot drift. A finalisation that matches nothing returns false; one
whose body contradicts `expectedRole` throws.
No production caller depended on the implicit finalisation. Every existing
finalisation-shaped path already writes through an insert-only append:
`settleInvestigationAndCloseCard`, `settleInvestigationStateAndCloseCard` and
the watch request/confirmation/refusal records all use
`appendChatMessageOnce(ByChatId)`.
Also: re-sending a snapshot no longer reserves positions for messages that are
already stored. The chat row is held, the missing ids are read under that lock,
and only those get slots. A 40-message chat grown one turn at a time used to
burn 1+2+…+40 = 820 slots for its 40 rows; it now burns 40. Deltas would be the
proper fix, but that reaches into the agent's turn hooks and is a larger change
than this pass.
Two smaller repairs in the same file: `messageIdOf`/`messageRoleOf` now fail
fast and name the chat and the offending message instead of casting unchecked
and surfacing a `NOT NULL` violation from the driver; and a batch carrying the
same message id twice throws instead of silently keeping the first, since that
is an impossible state and a silent pick is how the upstream bug would stay
invisible.
The comment on `reserveMessagePositions` claiming the row lock is "released with
the statement" was wrong — Postgres holds it to commit — and now says what is
true.
The wake and the consented investigation both stream their message before they
append the display copy, and the streamed copy is durable on session.out from
that moment. An append that failed therefore left the retry booting with the
message already in its history, taking the dedupe branch and never writing the
row: the model saw the message, the History panel didn't.
Both dedupe branches now re-append the message they found. The append is
id-deduped, so repairing when nothing is broken writes nothing.
The concurrency test only exercised the single-message append, which
allocates its position inside one statement. Replacing the batch
allocator's atomic `next_message_position` bump with a read-then-write
left every test passing — so the invariant that concurrent writers get
disjoint ranges was not actually covered on the path a turn takes.
Four concurrent three-message batches now assert twelve distinct
positions and that each batch's own messages stayed contiguous and in
order. Against the read-then-write version this fails on
`chat_messages_chat_position_key`, which is the constraint doing the
work rather than the application code.
The feature is unreleased and this branch is local, so there is no rolling
deploy in which an old instance still selects `chats.messages` — the usual
reason to keep a replaced column for one more release does not apply, and
after release the drop would cost two deploys. So it goes now.
With no data to preserve there is no backfill, and with no backfill there
is no such thing as a stored message without an id: `message_id` is
`NOT NULL` and the writers read `message.id` with no fallback, so a
malformed message fails the insert instead of being given an invented
identity by untested code.
Removing the column from the drizzle schema is also the enforcement that
nothing writes it — a surviving TypeScript reference stops compiling. The
scan for raw SQL naming the column found nothing to guard, so there is no
test for it.
The four idempotency invariants each get their own test against a real
table: a repeated message id creates no row and keeps its position; eight
genuinely concurrent appends take eight distinct positions, with a raw
insert past the query layer proving it is `UNIQUE (chat_id, position)`
doing the work; a controlled update changes one body and leaves identity,
position and every other row alone; and a mid-turn append survives the
turn's write, landing where it happened rather than at the end.
`dashboardAgentTranscriptMerge.test.ts` becomes
`dashboardAgentTranscriptStore.test.ts`: it no longer tests a merge, and
the ordering it asserted has deliberately changed.
The whole transcript lived in one JSONB array on `chats` and every turn
rewrote it, so bytes written grew as N squared: a 40-message chat wrote
~3.2 MB plus comparable WAL for one appended message. Dedupe was a
`jsonb_array_elements` scan, the quota count scanned every chat the user
owns, and a wholesale snapshot write could delete a message another
process had appended — patched until now by reading the row `FOR UPDATE`
and merging in JS.
`chat_messages` replaces it: primary key `(chat_id, message_id)`, so a
redelivered message is a conflict rather than a duplicate, and
`UNIQUE (chat_id, position)`, so a lost or duplicated position is
impossible rather than unlikely.
Positions come from `chats.next_message_position`, bumped by the same
single statement that reads it. Concurrent writers therefore get disjoint
contiguous ranges, and the row lock is released with the statement instead
of being held across a JS merge — the lock was never the problem, reading
and rewriting a large blob under it was. A batch reserves one contiguous
range and keeps its incoming order; a message already stored keeps the
position it was first given, which is why a wake appended mid-turn now
sits before the turn's later messages rather than after them.
`role` is lifted out of the payload so `countUserMessages` is a `count(*)`
over an index instead of an array explosion. No counter column on `chats`:
that would be one more thing to keep in sync across insert, backfill and
controlled update, for no proven need.
The writers now split by intent. A new message is an INSERT. A repeat of
the same durable event — `appendChatMessageOnce`, the sweep's closing card
— is ON CONFLICT DO NOTHING and writes nothing at all, not the row, not
the position, not the chat's timestamps. A turn's batch is ON CONFLICT DO
UPDATE guarded by `is distinct from`, so re-persisting an unchanged
40-message transcript writes zero rows; when a message does need
finalising, only its body changes and its identity and position stand.
There is no wholesale replacement of a chat left anywhere.
`mergeStoredMessages` and `writeMergedMessages` are deleted rather than
left beside the new API, so there is no ready-made way back to the bug.
`appendChatMessage` goes with them: nothing outside its own test called
it, and a non-deduped append has no meaning now that identity is the key.
Migration 0019 is additive. It creates the table, backfills from
`jsonb_array_elements … WITH ORDINALITY` using each message's own id
(deriving a stable one from the chat and ordinal for a legacy message
without one, and keeping the first of a duplicated id exactly as the old
merge did), then advances each chat's allocator past what it wrote.
`chats.messages` stays for the transition and nothing reads or writes it;
dropping it is a separate later migration.
Both places that roll the per-turn cache breakpoint onto the last message wrote
`anthropic: { cacheControl }` wholesale, replacing the namespace and silently
dropping every other Anthropic option that message carried.
`withCacheBreakpointOnLast()` now spreads the existing options and overwrites only
`cacheControl`. The agent's `prepareMessages` hook had the same write inlined, so
it goes through that one function instead, as `prepareTurnMessages()` — which also
gives the hook's behaviour a name a test can reach.
The system block's breakpoint in `chat.prompt.set` is left alone: it builds a fresh
options object, so there is nothing to merge into.
Nothing calls it. It settles investigation rows in their own writes and leaves the
caller to append the closing cards afterwards — the model that produced terminal
rows whose card never landed, which the stale sweep no longer selects. Keeping a
ready-made path back to that bug next to the atomic API is the whole risk.
`settlementCardMessages()` and the `SettledInvestigationCard` type stay: the
ordinary `persistTurn` path still uses both.
`settleInvestigationStateAndCloseCard()` bumped the revision before it looked at
the transcript, so a redelivered or replayed action settled the row a second time
while the append refused the duplicate card. The row then held revision 2 and the
transcript's terminal card revision 1: the live run rendered 2, a refresh rendered
1, for a tool advertised as idempotent on the action id.
The transaction now locks the investigation, checks the tenancy triple, locks the
chat, and looks for the message id before it writes anything. An action already in
the transcript returns the stored card and the current revision untouched.
Lock order stays investigation then chat, matching `persistTurn` and the sweep's
`settleInvestigationAndCloseCard`; taking the chat first here would deadlock
against them.
A missing or deleted chat was indistinguishable from an already-closed card —
both came back `closed: false` — and the settle committed anyway, which is exactly
the terminal-row-without-a-card the transaction exists to prevent. It is now
`{ ok: false, error: "chat_missing" }`, and the watch lane logs it as the race it
is rather than a fault.
The watch lane settled the investigation with `settleOpenInvestigations`, then
appended the terminal card as a separate write — and swallowed that write's
error, logging it and reporting success. So the row went terminal, the card
never arrived, the stale sweep stopped selecting the row because it was no
longer `in_progress`, and the user was left on "Working…" with nothing able to
repair it. Nothing in production calls the action again on its own, which is
exactly why the swallowed error mattered.
The lane now writes through `settleInvestigationStateAndCloseCard`: the terminal
revision and the closing card commit in one transaction, under the lane's own
message id, and the error propagates so the task's retry is a real retry. If the
card cannot be rendered the settle rolls back, leaving the `in_progress` row the
sweep still selects. The duplicated revision bump is gone with it — one outcome
is now one revision.
The regression test drives a failing close in one action; it no longer proves
recovery by calling the action a second time by hand.
`persistTurn` and `persistMessages` stored the whole `messages` array they were
handed. The array is the snapshot the turn started from, so anything another
process appended in between was deleted: a wake delivery, a watch consent
record, or the terminal card of an investigation the stale sweep had just
settled. That last one is unrecoverable — the row is already terminal, so the
sweep never selects it again, and the panel is back to "Working…" for ever.
Both writes now read the row under `select ... for update` inside the
transaction and merge by stable message id: incoming order is kept, a stored
message the snapshot does not have goes at the end, and no id appears twice. A
message with no id falls back to its content so it cannot be carried over twice.
An append-only `chat_messages` table is the better long-term shape; merging
under the lock is enough for this architecture and needs no migration.
Covers the two halves the earlier pass left open. Against a real database, a
stale card that cannot be rendered now proves the sweep rolls the settle back
with it: the row stays `in_progress` at revision 0, the chat stays empty, and
the row is still in the next run's selection.
The panel half is covered over `liveProgress`, the code that decides whether
"Working…" is shown: a mounted panel holding the unconcluded card re-reads the
stored transcript, merges by stable id, and the progress line goes away without
a reload. Re-reading repeatedly cannot add a second copy of the card.
The row settle, the transcript and the session state were three separate
operations. Once the row was terminal, a failed transcript write left a card
that read `in_progress` forever: the stale sweep only selects `in_progress`
rows, so nothing was left to repair it.
Both lanes now commit the pair atomically. The live turn hands its pending
settlements to `persistTurn`, which upserts the revisions, appends their cards
and writes the session in one transaction; the process-local entry survives
until that commits, so a retried `onTurnComplete` still settles. The sweep goes
through `settleInvestigationAndCloseCard`, whose rollback restores the
`in_progress` row the sweep already selects.
That also removes the last reader of the per-run `chatOwners` map, which had no
`delete` and grew for the life of the worker: the failure record now travels in
the transcript write, which needs no userId.
Separately, the consented watch investigation gets the rolling step cache. Its
ten-step `streamText` re-sent every accumulated tool output uncached; the
breakpoint helper and the per-step cache telemetry now live in `step-cache.ts`
and both lanes use them, wrapping any `prepareStep` the resolved options carry.
Binding a delegated token to its environment claim on the project-wide
routes also refused any token that carries no claim at all, which the
public PAT exchange used by MCP and the CLI is allowed to issue. Line the
project-wide helper up with its neighbours: a claimless dashboard-agent
token is still refused, everything else stays project-wide.
The sweep settled the row and appended nothing, so it visibly fixed nothing: the
chat kept rendering the last card it had, which was still "Working…". The settle
now returns the state and revision it wrote, and the sweep appends that as the
closing card revision on the chat — id-deduped on
`investigation-settlement:{id}:{revision}`, so a retried run can neither stack a
second card nor open a second investigation.
The append is scoped by chat id: a sweep runs off any session and has no user in
context, unlike the turn lane.
Settling the investigations row was invisible to the user. The panel builds the
winning revision from the transcript's own `tool-render_view` parts and never
reads that table, so a turn that ran out of steps left the card at
`in_progress` forever: the database believed the investigation had finished
while a refresh still showed "Working...".
`settleOpenInvestigations` now returns the revisions it committed, and
`onTurnComplete` appends each as one more card revision — after the transcript
write and id-deduped on `investigation-settlement:{id}:{revision}`, so a failed
append leaves the card visibly unclosed rather than silently lost, and a retry
can't stack a second card.
The card-building and the latest-revision reader move out of the watch lane and
into the runtime both lanes share, so there is one shape, not two. The watch
lane keeps its own message id: it dedupes on the action, not the revision.
The create-watch response matched any watch-alert channel in the project, so a
second member was told they were subscribed while the mail went to the first.
The channel's deduplication key is the only record of whose it is, so state
resolution, subscribe and unsubscribe now share one owner lookup.
replay() re-ran subscribe() for an already-recorded `created` submission. A retry that
succeeded could flip the ledger to `enabled`, but the confirmation in the transcript is
append-once, so the user kept being told email was unavailable while the system believed
it was on. The replay now reads recordedExternalNotification() and takes no external
decision. An attempt that dies before subscribing or before its outcome is recorded
leaves the row `pending`, and the normal creation path subscribes on the retry.
A user-actor token declaring no scope cap could mint an environment JWT with any
scopes it asked for. The exchange now clamps the minted scopes to the actor's own
ability, so a capless token is read-only, and only mints for its claimed environment.
The direct PAT authentication path returned identity only, so a user-actor token
reaching it (admin routes and other direct callers) lost its environment scope. The
claims now ride on the authentication result itself.
The environments and runs listings are project-wide, so an environment-scoped
user-actor token could read every environment its user can reach. Both routes now
resolve the claim into a mandatory filter, and a conflicting request filter is
refused rather than overridden.
A tick that could read nothing now moves the group's fairness key only, so a watch
with a permanently broken reader stops crowding out the rest of an over-cap group.
Dueness and the streak facts still follow the last real check.
A per-condition fallback identified the condition rather than the submit, so a
re-watch could replay a stale terminal outcome from the retention window.
Classify a failed tool result locally into one of seven categories and send only
the label; the message is dropped with every other free-text field. Unrecognised
failures are unknown rather than guessed, and a bare string error field is now
withheld too.
Replaces https://*.googleusercontent.com (a host with public write access)
with CSP_IMG_SRC_ALLOWLIST: exact origins only, https outside development,
deduplicated, bad entries warned about instead of failing the boot.
Route conflicts were the tab-title work meeting the agent page-context handle: both
sides kept, duplicate meta exports resolved to pageMeta, duplicate imports merged
with unused bindings dropped. Lockfile regenerated from the merged manifests.
The hand-built row stopped satisfying every column the authenticated-environment
mapper reads, so both proxy cases failed. Stubbing the lookup keeps the fixture
independent of the row's columns.
Katia Bulatova