## Summary
Batch triggers that use per-item idempotency keys could take seconds
instead of milliseconds when the target task had a large run history.
This keeps the idempotency lookup fast regardless of how many runs a
task has accumulated.
## Root cause
The batch path checks which items already have runs by looking up their
idempotency keys with a single `WHERE runtimeEnvironmentId = ? AND
taskIdentifier = ? AND idempotencyKey IN (...)` query. On a very large
`TaskRun` table Postgres underestimates the row count of a specific
`(environment, task)` pair, so once the `IN` list grows past a handful
of keys it stops doing per-key index probes and instead scans every run
for that `(environment, task)` and filters the keys in memory. The cost
is then flat and large regardless of how many keys are being checked,
and a routine `ANALYZE` does not correct the estimate at that table
size.
## Fix
Look each idempotency key up on its own, batched into a `UNION ALL` of
point lookups (chunked, run with bounded concurrency). Each branch is an
equality on all three columns of the unique index, so the planner can
only do a per-key index probe and can never fall back to the range scan.
Same results, same columns, confined to the batch trigger path.
Eric Allam
1ab5066ed0