Soft-deleted chats are hard-deleted with their children after a 30d
window by the maintenance sweep, and organization deletion enqueues a
job that soft-deletes the org's chats so the same sweep removes them.
The FK-free cascade lives in a single reusable deleteChatsByIds helper.
At the apiBuilder verify site the user-actor claims can come from the RBAC plugin; a plugin image predating the pat claim would deliver pat-less claims and no-op revocation there. Re-verify the bearer locally so the recheck reads the token's authoritative pat. Add recheck tests for the apiBuilder and UAT-preamble sites.
Extract the limiter override into an exported function so the actor?.sub decision is testable without redis, and assert both branches (act present -> jwt-actor key; no act -> hashed-token fallback).
A delegated env JWT rotates its token value every turn, so keying the limiter on the token handed each turn a fresh bucket. Key on env+acting-user instead, namespaced so it can't collide with private-key buckets.
Stamp the source PAT id on a minted user-actor token and recheck it is still live at every verify site, so revoking the PAT invalidates any token created from it. Tokens with no source PAT (e.g. the dashboard agent's) skip the recheck.
The mint route rejected TTLs only above 365 days; shorten the ceiling to 7 days so a delegated token can't be minted long-lived. Update the CLI --ttl help to match.
A CSP_IMG_SRC_ALLOWLIST entry containing ';' passed URL parsing and landed verbatim in the space-joined img-src directive, injecting or truncating a directive. Reject entries containing ';' or ',' alongside the existing wildcard and whitespace checks.
Model-authored markdown could emit a remote image whose URL the browser fetches on render — a zero-click data beacon. Constrain the shared markdown renderer with a urlTransform that strips the src of any non-local image and limits links to safe schemes.
A user-actor token could be exchanged for an env JWT that outlived it by
requesting a longer expirationTime. Surface the token's exp as expiresAt
and clamp the minted JWT to it. Non-UAT exchanges are unchanged.
A delegated user-actor token with no scope cap could exchange for an env
JWT carrying any requested scopes (admin included). Ceiling every UAT
exchange by its cap, or read-only when it has none, projecting requested
scopes through the scope grammar so read:all still admits reads.
Resolve conflicts in report-health: keep our richer fallback/trust model
(telemetry_absent trust, snapshot fallback, finished/completed throughput)
and adopt main's start-latency-unknown behavior so an absent measurement
reports as unknown instead of a confident 0ms. renderMarkdown keeps our
layout rewrite; main's compactFact change is covered by report-layout's
metricValue. Adapt main's two render assertions to our current output.
## What
Follow-up to #4539. The driver-adapter work is inert until a client
flips to the pg driver adapter, but the moment one does, our database
observability degrades: the OTel metrics pipeline reads pool stats from
Prisma's `$metrics`, which is owned by the Rust engine's `quaint` pool.
Under the adapter, `pg.Pool` owns the pool, so those gauges read zero.
The pipeline also only ever scraped a single client (the control-plane
writer singleton).
This PR makes database metrics driver-agnostic and per-client:
- Every configured client registers a metrics source: control-plane
writer/replica, run-ops writer/replica, legacy writer/replica.
Previously only the control-plane writer singleton was scraped.
- Each OTel instrument is observed per client with `db_client` and
`db_driver` (`quaint` | `pg-adapter`) attributes. `db_client` uses our
canonical datasource-role labels (`control-plane-writer`,
`control-plane-replica`, `run-ops-writer`, `run-ops-replica`,
`legacy-run-ops-writer`, `legacy-run-ops-replica`) — the same strings
used for the `db.datasource` span attribute, so a metric and a trace
point at the same pool.
- Pool figures come from the authoritative source per driver:
- **pg-adapter**: `pg.Pool` (`totalCount`/`idleCount`/`waitingCount`,
plus cumulative opened/closed from `connect`/`remove` events).
- **quaint**: the Rust engine's `$metrics` pool gauges/counters, exactly
as before.
- Query counters and duration histograms still come from `$metrics` for
both drivers (the Rust engine executes queries in both cases).
- New `db.pool.connections.waiting` gauge (pg.Pool exposes this; quaint
reports 0).
- Stops exporting Prisma metrics from the Prometheus `/metrics` route.
Pool observability now lives entirely in the OTel pipeline, per driver,
per client.
## Why
So we can flip any client (including the control-plane writer, the
primary desync-fix target) to the driver adapter without losing pool
visibility. Existing dashboards keyed on the same metric names keep
working; they gain a per-client dimension.
## Testing
Unit (`apps/webapp/app/utils/databaseMetrics.server.test.ts`): the pure
normalizer — quaint reads pool from `$metrics`; adapter reads pool from
`pg.Pool` and keeps engine query metrics; `busy` never goes negative;
graceful zeroing when `$metrics` is unavailable (adapter still reports
live pool figures).
Live smoke test against a prod-shaped local stack: three
physically-distinct Postgres DBs (control-plane, run-ops, legacy) behind
dual PgBouncers, split mode on, with a mix of adapter and quaint
clients. Reading the actual emitted OTel metrics, every pool shows up as
its own series:
```
db.pool.connections.total{db_client="control-plane-writer", db_driver="pg-adapter"} = 1
db.pool.connections.total{db_client="control-plane-replica", db_driver="quaint"} = 1
db.pool.connections.total{db_client="run-ops-writer", db_driver="pg-adapter"} = 1
db.pool.connections.total{db_client="run-ops-replica", db_driver="quaint"} = 1
db.pool.connections.total{db_client="legacy-run-ops-writer", db_driver="quaint"} = 1
db.pool.connections.total{db_client="legacy-run-ops-replica",db_driver="quaint"} = 1
db.client.queries.total{db_client="control-plane-writer",db_driver="pg-adapter"} = incrementing
db.client.queries.duration.count{db_client="control-plane-writer",db_driver="pg-adapter"} = incrementing
```
Confirms: metrics are attributed per pool with the correct driver;
adapter pools' figures come from `pg.Pool`; and query counters/duration
histograms keep incrementing under the pg adapter. Also verified
`/metrics` (Prometheus) now returns zero `prisma_*` series while still
serving the app's own metrics.
`pnpm run typecheck --filter webapp` passes.
## Notes
- `/metrics` (Prometheus) no longer includes `prisma_*` series. Anything
scraping that endpoint for Prisma metrics should read the equivalent
`db.*` metrics from the OTel exporter instead.
- **PgBouncer + `?schema=` gotcha (separate from this PR, worth flagging
for rollout):** since #4539 parses `?schema=` from the DSN and passes `{
schema }` to the adapter, node-postgres sends `search_path` as a startup
parameter. A transaction-mode PgBouncer rejects that with `FATAL:
unsupported startup parameter: search_path`. Our prod control-plane DSNs
use the default `public` schema with no `?schema=` param, so this is
latent, but any client we flip to the adapter must not carry `?schema=`
in its DSN (or the pooler needs `ignore_startup_parameters =
search_path`).
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
When the health report had no start-latency measurement for the window,
it printed a confident "p95 0ms" and graded it healthy. It now shows
"unknown" for that metric and skips grading it, so an absent measurement
can't read as a green signal.
A genuinely measured 0ms is still shown as 0ms: the loader keeps "no
measurement" distinct from a measured zero instead of coercing both to
0.
Katia Bulatova