The global rate limiter was being applied at the FairQueue claim phase,
consuming 1 token per queue-claim-attempt rather than per item
processed.
With many small queues (each batch is its own queue), consumers burned
through tokens on empty or single-item queues, causing aggressive
throttling well below the intended items/sec limit.
Changes:
- Move rate limiter from FairQueue claim phase to BatchQueue worker
queue
consumer loop (before blockingPop), so each token = 1 item processed
- Replace the FairQueue rate limiter with a worker queue depth cap to
prevent unbounded growth that could cause visibility timeouts
- Add BATCH_QUEUE_WORKER_QUEUE_MAX_DEPTH env var (optional, disabled by
default)
Eric Allam
dee6f1d09e