perf(webapp): avoid unindexed fileId scan in get-background-worker-by-version (#4245)
## What
The `GET
/api/v1/projects/:projectRef/background-workers/:envSlug/:version`
endpoint loaded each file's tasks through the nested `files.tasks`
relation. Prisma resolves that as a separate query:
```sql
SELECT id, slug, "fileId" FROM "BackgroundWorkerTask" WHERE "fileId" IN (...)
```
`BackgroundWorkerTask.fileId` is not indexed — the FK constraint exists,
but Postgres does not auto-create an index for foreign keys — so on a
large table this can only run as a sequential scan, which gets
progressively slower as the table grows and was observed taking minutes
per call in production.
The loader already loads every task for the worker via `tasks: true`,
which uses the indexed `workerId` relation, and those rows already
include `fileId`. This PR groups task slugs by `fileId` in memory from
that already-loaded data and drops the `files.tasks` include entirely.
## Behavior change (latent bug fix)
The response shape is unchanged, but there is a semantic correction for
**source files reused across worker versions** (files are de-duplicated
by `@@unique([projectId, contentHash])`, so one file row can be linked
to many workers).
- **Before:** `file.tasks` came from the `BackgroundWorkerFile.tasks`
relation, i.e. *every* `BackgroundWorkerTask` with that `fileId` —
across all workers sharing the file. So a worker's manifest could list
tasks it doesn't actually have.
- **After:** `file.tasks` is grouped from the queried worker's own
tasks, so it reflects only that worker version's tasks.
Verified on a local DB: 460 files are referenced by tasks from more than
one worker; of 6819 (worker, file) pairs, 6 differ — all one file where
the old union leaked a task slug (`cancellation-test`) into worker
versions that never had it. The new per-worker behavior is the correct
one for a worker-version manifest. (Thanks to the automated review for
flagging this.)
## Analysis
Captured the exact SQL before/after by instrumenting Prisma against real
data (a worker with 62 files):
- **Before:** 5 statements, including the `WHERE "fileId" IN (...)`
scan.
- **After:** 4 statements; the `fileId` query is gone and the other four
are identical.
EXPLAIN of the two access paths:
```
Before WHERE "fileId" IN (...)
Seq Scan on "BackgroundWorkerTask"
Filter: ("fileId" = ANY (...)) -- reads the whole table, scales with table size
After WHERE "workerId" IN (...)
Index Scan using "BackgroundWorkerTask_workerId_slug_key"
Index Cond: ("workerId" = ...) -- bounded by matching rows, scale-independent
```
No new index is required: the `workerId` access path is already covered
by the existing `BackgroundWorkerTask_workerId_slug_key` unique index.
## Testing
- `pnpm run typecheck --filter webapp` passes.
- Query capture + EXPLAIN performed against a local database seeded with
real worker/file/task data.
This commit is contained in:
@@ -0,0 +1,6 @@
|
||||
---
|
||||
area: webapp
|
||||
type: improvement
|
||||
---
|
||||
|
||||
Speed up retrieving a background worker by version. The endpoint no longer runs a slow lookup that scanned the full task table for large deployments; it now reuses data it already loads, so the response is the same but returns much faster.
|
||||
+15
-10
@@ -45,15 +45,7 @@ export async function loader({ params, request }: LoaderFunctionArgs) {
|
||||
},
|
||||
include: {
|
||||
tasks: true,
|
||||
files: {
|
||||
include: {
|
||||
tasks: {
|
||||
select: {
|
||||
slug: true,
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
files: true,
|
||||
},
|
||||
});
|
||||
|
||||
@@ -61,6 +53,19 @@ export async function loader({ params, request }: LoaderFunctionArgs) {
|
||||
return json({ error: "Background worker not found" }, { status: 404 });
|
||||
}
|
||||
|
||||
// Group task slugs by fileId from the already-loaded tasks (which are fetched
|
||||
// via the indexed workerId relation) instead of loading files.tasks, which
|
||||
// queries BackgroundWorkerTask by the unindexed fileId column.
|
||||
const taskSlugsByFileId = new Map<string, Set<string>>();
|
||||
for (const task of backgroundWorker.tasks) {
|
||||
if (!task.fileId) {
|
||||
continue;
|
||||
}
|
||||
const slugs = taskSlugsByFileId.get(task.fileId) ?? new Set<string>();
|
||||
slugs.add(task.slug);
|
||||
taskSlugsByFileId.set(task.fileId, slugs);
|
||||
}
|
||||
|
||||
return json({
|
||||
id: backgroundWorker.friendlyId,
|
||||
version: backgroundWorker.version,
|
||||
@@ -82,7 +87,7 @@ export async function loader({ params, request }: LoaderFunctionArgs) {
|
||||
filePath: file.filePath,
|
||||
contentHash: file.contentHash,
|
||||
contents: decompressContent(file.contents),
|
||||
tasks: Array.from(new Set(file.tasks.map((task) => task.slug))),
|
||||
tasks: Array.from(taskSlugsByFileId.get(file.id) ?? []),
|
||||
})),
|
||||
});
|
||||
} catch (error) {
|
||||
|
||||
Reference in New Issue
Block a user