perf(webapp): avoid unindexed fileId scan in get-background-worker-by-version (#4245)

## What

The `GET
/api/v1/projects/:projectRef/background-workers/:envSlug/:version`
endpoint loaded each file's tasks through the nested `files.tasks`
relation. Prisma resolves that as a separate query:

```sql
SELECT id, slug, "fileId" FROM "BackgroundWorkerTask" WHERE "fileId" IN (...)
```

`BackgroundWorkerTask.fileId` is not indexed — the FK constraint exists,
but Postgres does not auto-create an index for foreign keys — so on a
large table this can only run as a sequential scan, which gets
progressively slower as the table grows and was observed taking minutes
per call in production.

The loader already loads every task for the worker via `tasks: true`,
which uses the indexed `workerId` relation, and those rows already
include `fileId`. This PR groups task slugs by `fileId` in memory from
that already-loaded data and drops the `files.tasks` include entirely.

## Behavior change (latent bug fix)

The response shape is unchanged, but there is a semantic correction for
**source files reused across worker versions** (files are de-duplicated
by `@@unique([projectId, contentHash])`, so one file row can be linked
to many workers).

- **Before:** `file.tasks` came from the `BackgroundWorkerFile.tasks`
relation, i.e. *every* `BackgroundWorkerTask` with that `fileId` —
across all workers sharing the file. So a worker's manifest could list
tasks it doesn't actually have.
- **After:** `file.tasks` is grouped from the queried worker's own
tasks, so it reflects only that worker version's tasks.

Verified on a local DB: 460 files are referenced by tasks from more than
one worker; of 6819 (worker, file) pairs, 6 differ — all one file where
the old union leaked a task slug (`cancellation-test`) into worker
versions that never had it. The new per-worker behavior is the correct
one for a worker-version manifest. (Thanks to the automated review for
flagging this.)

## Analysis

Captured the exact SQL before/after by instrumenting Prisma against real
data (a worker with 62 files):

- **Before:** 5 statements, including the `WHERE "fileId" IN (...)`
scan.
- **After:** 4 statements; the `fileId` query is gone and the other four
are identical.

EXPLAIN of the two access paths:

```
Before  WHERE "fileId" IN (...)
  Seq Scan on "BackgroundWorkerTask"
    Filter: ("fileId" = ANY (...))          -- reads the whole table, scales with table size

After   WHERE "workerId" IN (...)
  Index Scan using "BackgroundWorkerTask_workerId_slug_key"
    Index Cond: ("workerId" = ...)          -- bounded by matching rows, scale-independent
```

No new index is required: the `workerId` access path is already covered
by the existing `BackgroundWorkerTask_workerId_slug_key` unique index.

## Testing

- `pnpm run typecheck --filter webapp` passes.
- Query capture + EXPLAIN performed against a local database seeded with
real worker/file/task data.
This commit is contained in:
Daniel Sutton
2026-07-13 16:14:20 +01:00
committed by GitHub
parent 703a6dcb4c
commit e0b42a88d6
2 changed files with 21 additions and 10 deletions
@@ -0,0 +1,6 @@
---
area: webapp
type: improvement
---
Speed up retrieving a background worker by version. The endpoint no longer runs a slow lookup that scanned the full task table for large deployments; it now reuses data it already loads, so the response is the same but returns much faster.
@@ -45,15 +45,7 @@ export async function loader({ params, request }: LoaderFunctionArgs) {
},
include: {
tasks: true,
files: {
include: {
tasks: {
select: {
slug: true,
},
},
},
},
files: true,
},
});
@@ -61,6 +53,19 @@ export async function loader({ params, request }: LoaderFunctionArgs) {
return json({ error: "Background worker not found" }, { status: 404 });
}
// Group task slugs by fileId from the already-loaded tasks (which are fetched
// via the indexed workerId relation) instead of loading files.tasks, which
// queries BackgroundWorkerTask by the unindexed fileId column.
const taskSlugsByFileId = new Map<string, Set<string>>();
for (const task of backgroundWorker.tasks) {
if (!task.fileId) {
continue;
}
const slugs = taskSlugsByFileId.get(task.fileId) ?? new Set<string>();
slugs.add(task.slug);
taskSlugsByFileId.set(task.fileId, slugs);
}
return json({
id: backgroundWorker.friendlyId,
version: backgroundWorker.version,
@@ -82,7 +87,7 @@ export async function loader({ params, request }: LoaderFunctionArgs) {
filePath: file.filePath,
contentHash: file.contentHash,
contents: decompressContent(file.contents),
tasks: Array.from(new Set(file.tasks.map((task) => task.slug))),
tasks: Array.from(taskSlugsByFileId.get(file.id) ?? []),
})),
});
} catch (error) {