perf(webapp): paginate the environment variables settings page (#4597)

## What The environment variables settings page loaded **every** variable in the project in one shot, with a nested `values` read plus a `valueReference` (SecretReference) sub-load that was selected but never read. For a project with many variables this pulled `variables × environments` value rows (~18k for large projects) on every page load, plus a matching ~18k-row `SecretReference IN` query. This paginates the presenter by variable key and removes the dead include. - Remove the never-read `valueReference: { select: { key } }` include → the `SecretReference` query is gone entirely. - Paginate the parent variable query: `count` + `orderBy key` + `skip/take`, page size 50 → the value read is bounded to `pageSize × environments` per page. - Scope the count and the page to variables that have a value in a displayed environment (`values: { some: { environmentId: { in } } }`), so `totalCount`/`totalPages` and the `skip/take` window match what actually renders (no phantom empty pages from variables that live only in archived branches or another member's dev env). - Display order comes from the DB `orderBy: { key: "asc" }` — the presenter no longer re-sorts each page with `localeCompare`, which under pagination could disagree with the DB collation at page boundaries. - The secret-value lookup (`SecretStore` keys) and the updater lookup (`user` by id) are now scoped to the current page instead of the whole project. - Search moves server-side (variable key, case-insensitive) and drives both the count and the page; the UI gains standard pagination controls. ## Why The two correlated ~18k-row control-plane queries flagged in the ticket come from this settings-page presenter, not from any hot path. Both are index-covered (`rows_read == rows_returned`); the issue is the sheer volume fetched in one burst. Bounding it per page removes the burst. ## Evidence Measured on an isolated stack with a seeded project of 1000 variables × 3 environments (3000 value rows), using Prisma's emitted-SQL log: | | SecretReference query | value rows fetched | | --- | --- | --- | | before | 1 | 3000 | | after | **0** | **150** (page 1) + one `count` | `EXPLAIN` on Prisma's verbatim statements (index confirmed via `enable_seqscan=off`; the local table is too small for the planner to choose them by default): - `count` (`WHERE projectId AND EXISTS(values in displayed envs)`) → Hash Join: Index Scan on `EnvironmentVariable_pkey` + Bitmap Index Scan on `EnvironmentVariableValue_environmentId_idx` - paginated parent (`WHERE projectId AND EXISTS(...) ORDER BY key LIMIT/OFFSET`) → Nested Loop Semi Join: Index Scan on `EnvironmentVariable_projectId_key_key` (**no Sort node**) driving an Index-Only Scan on `EnvironmentVariableValue_variableId_environmentId_key` - nested values (`variableId = ANY … AND environmentId = ANY …`) → index scan on `EnvironmentVariableValue_environmentId_idx` - `SecretStore` keys (`key = ANY …`) → index scan on `SecretStore_key_idx` No new index required. Verified in the browser on the seeded project: 20 pages, page navigation, server-side search (matches across all pages), last page renders, no app console errors. `typecheck`, `oxlint`, `oxfmt` all clean. ## Behavior change The previous client-side search matched variable **name and value** (and environment type / branch name). Values are encrypted at rest and resolved separately, so they cannot be searched server-side under pagination. Search is now **variable-name only**, server-side, case-insensitive. Projects with fewer than one page of variables see no pagination bar and no visible change. ## Rollout / rollback Pure read-path change on a dashboard loader, no schema or data migration. Rollback is a straight revert. ## Screenshots <img width="2400" height="1794" alt="01-page1" src="https://github.com/user-attachments/assets/d4a7effd-d167-4dd6-92f4-6e9174818acd" /> <img width="2400" height="1794" alt="02-search-single" src="https://github.com/user-attachments/assets/cd113ca9-ff87-431f-b2f6-7f7d36f2b32a" />
2026-08-12 23:51:14 +01:00
parent 8d0f693186
commit c6ef5f3959
3 changed files with 113 additions and 68 deletions
@@ -0,0 +1,6 @@
+---
+area: webapp
+type: improvement
+---
+
+The environment variables page now loads a page at a time, keeping it fast for projects with a large number of variables. Search matches variable names across every page.
@@ -8,10 +8,12 @@ import type { SyncEnvVarsMapping, EnvSlug } from "~/v3/vercel/vercelProjectInteg
 import { VercelIntegrationService } from "~/services/vercelIntegration.server";
 import { loadEnvironmentVariablesEnvironments } from "./environmentVariablesEnvironments.server";

-import { boundedIn } from "@trigger.dev/database";
+import { boundedIn, type Prisma } from "@trigger.dev/database";
 type Result = Awaited<ReturnType<EnvironmentVariablesPresenter["call"]>>;
 export type EnvironmentVariableWithSetValues = Result["environmentVariables"][number];

+export const DEFAULT_ENV_VARS_PAGE_SIZE = 50;
+
 export class EnvironmentVariablesPresenter {
  #prismaClient: PrismaClient;
  #replicaClient: PrismaReplicaClient;
@@ -21,7 +23,19 @@ export class EnvironmentVariablesPresenter {
    this.#replicaClient = replicaClient;
  }

-  public async call({ userId, projectSlug }: { userId: User["id"]; projectSlug: Project["slug"] }) {
+  public async call({
+    userId,
+    projectSlug,
+    page = 1,
+    pageSize = DEFAULT_ENV_VARS_PAGE_SIZE,
+    search,
+  }: {
+    userId: User["id"];
+    projectSlug: Project["slug"];
+    page?: number;
+    pageSize?: number;
+    search?: string;
+  }) {
    const project = await this.#replicaClient.project.findFirst({
      select: {
        id: true,
@@ -53,6 +67,18 @@ export class EnvironmentVariablesPresenter {
    // values in archived branch environments, which would otherwise all be loaded here.
    const environmentIds = sortedEnvironments.map((env) => env.id);

+    const variableWhere: Prisma.EnvironmentVariableWhereInput = {
+      projectId: project.id,
+      values: { some: { environmentId: { in: boundedIn(environmentIds) } } },
+      ...(search ? { key: { contains: search, mode: "insensitive" } } : {}),
+    };
+
+    const totalCount = await this.#replicaClient.environmentVariable.count({
+      where: variableWhere,
+    });
+    const totalPages = Math.max(1, Math.ceil(totalCount / pageSize));
+    const currentPage = Math.min(Math.max(1, page), totalPages);
+
    const environmentVariables = await this.#replicaClient.environmentVariable.findMany({
      select: {
        id: true,
@@ -64,11 +90,6 @@ export class EnvironmentVariablesPresenter {
            version: true,
            lastUpdatedBy: true,
            updatedAt: true,
-            valueReference: {
-              select: {
-                key: true,
-              },
-            },
            isSecret: true,
          },
          where: {
@@ -78,9 +99,12 @@ export class EnvironmentVariablesPresenter {
          },
        },
      },
-      where: {
-        projectId: project.id,
+      where: variableWhere,
+      orderBy: {
+        key: "asc",
      },
+      skip: (currentPage - 1) * pageSize,
+      take: pageSize,
    });

    const userIds = new Set(
@@ -152,58 +176,61 @@ export class EnvironmentVariablesPresenter {
    }

    return {
-      environmentVariables: environmentVariables
-        .flatMap((environmentVariable) => {
-          return sortedEnvironments.flatMap((env) => {
-            const valueRecord = environmentVariable.values.find((v) => v.environmentId === env.id);
-            const isSecret = valueRecord?.isSecret ?? false;
+      environmentVariables: environmentVariables.flatMap((environmentVariable) => {
+        return sortedEnvironments.flatMap((env) => {
+          const valueRecord = environmentVariable.values.find((v) => v.environmentId === env.id);
+          const isSecret = valueRecord?.isSecret ?? false;

-            if (!valueRecord) {
-              return [];
-            }
+          if (!valueRecord) {
+            return [];
+          }

-            const val = isSecret
-              ? undefined
-              : variableValuesByEnvAndKey.get(`${env.id}:${environmentVariable.key}`);
+          const val = isSecret
+            ? undefined
+            : variableValuesByEnvAndKey.get(`${env.id}:${environmentVariable.key}`);

-            if (!isSecret && val === undefined) {
-              return [];
-            }
+          if (!isSecret && val === undefined) {
+            return [];
+          }

-            const lastUpdatedBy = valueRecord.lastUpdatedBy as EnvironmentVariableUpdater | null;
+          const lastUpdatedBy = valueRecord.lastUpdatedBy as EnvironmentVariableUpdater | null;

-            const updatedByUser =
-              lastUpdatedBy?.type === "user"
-                ? (() => {
-                    const user = usersRecord[lastUpdatedBy.userId];
-                    return user
-                      ? {
-                          id: user.id,
-                          name: user.displayName || user.name || "Unknown",
-                          avatarUrl: user.avatarUrl,
-                        }
-                      : null;
-                  })()
-                : null;
+          const updatedByUser =
+            lastUpdatedBy?.type === "user"
+              ? (() => {
+                  const user = usersRecord[lastUpdatedBy.userId];
+                  return user
+                    ? {
+                        id: user.id,
+                        name: user.displayName || user.name || "Unknown",
+                        avatarUrl: user.avatarUrl,
+                      }
+                    : null;
+                })()
+              : null;

-            return [
-              {
-                id: environmentVariable.id,
-                key: environmentVariable.key,
-                environment: { type: env.type, id: env.id, branchName: env.branchName },
-                value: isSecret ? "" : val!,
-                isSecret,
-                version: valueRecord.version,
-                lastUpdatedBy,
-                updatedByUser,
-                updatedAt: valueRecord.updatedAt,
-              },
-            ];
-          });
-        })
-        .sort((a, b) => a.key.localeCompare(b.key)),
+          return [
+            {
+              id: environmentVariable.id,
+              key: environmentVariable.key,
+              environment: { type: env.type, id: env.id, branchName: env.branchName },
+              value: isSecret ? "" : val!,
+              isSecret,
+              version: valueRecord.version,
+              lastUpdatedBy,
+              updatedByUser,
+              updatedAt: valueRecord.updatedAt,
+            },
+          ];
+        });
+      }),
      environments: sortedEnvironments,
      hasStaging,
+      pagination: {
+        currentPage,
+        totalPages,
+        totalCount,
+      },
      // Vercel integration data
      vercelIntegration: vercelIntegration
        ? {
@@ -40,6 +40,7 @@ import { Input } from "~/components/primitives/Input";
 import { InputGroup } from "~/components/primitives/InputGroup";
 import { Label } from "~/components/primitives/Label";
 import { NavBar, PageAccessories, PageTitle } from "~/components/primitives/PageHeader";
+import { PaginationControls } from "~/components/primitives/Pagination";
 import { Paragraph } from "~/components/primitives/Paragraph";
 import { SearchInput } from "~/components/primitives/SearchInput";
 import { Switch } from "~/components/primitives/Switch";
@@ -55,10 +56,8 @@ import {
 import { SimpleTooltip } from "~/components/primitives/Tooltip";
 import { prisma } from "~/db.server";
 import { useEnvironment } from "~/hooks/useEnvironment";
-import { useFuzzyFilter } from "~/hooks/useFuzzyFilter";
 import { useOrganization } from "~/hooks/useOrganizations";
 import { useProject } from "~/hooks/useProject";
-import { useSearchParams } from "~/hooks/useSearchParam";
 import { redirectWithSuccessMessage } from "~/models/message.server";
 import { resolveOrgIdFromSlug } from "~/models/organization.server";
 import {
@@ -117,6 +116,8 @@ export type EnvironmentVariablesPageLoaderData = {
  accessibleEnvironmentIds: string[];
  // Environment ids whose env vars the current role can write (create/edit/delete).
  writableEnvironmentIds: string[];
+  pagination: { currentPage: number; totalPages: number; totalCount: number };
+  search?: string;
 };

 export const environmentVariablesRouteId =
@@ -125,6 +126,10 @@ export const environmentVariablesRouteId =
 export const loader = dashboardLoader(
  {
    params: EnvironmentParamSchema,
+    searchParams: z.object({
+      page: z.coerce.number().int().min(1).catch(1),
+      search: z.string().trim().min(1).optional().catch(undefined),
+    }),
    context: async (params) => {
      const organizationId = await resolveOrgIdFromSlug(params.organizationSlug);
      return organizationId ? { organizationId } : {};
@@ -132,15 +137,17 @@ export const loader = dashboardLoader(
    // No hard authorization: the page lists every environment. Values in
    // environments the role can't read are masked per-tier below.
  },
-  async ({ params, user, ability }) => {
+  async ({ params, searchParams, user, ability }) => {
    const { projectParam } = params;

    try {
      const presenter = new EnvironmentVariablesPresenter();
-      const { environmentVariables, environments, hasStaging, vercelIntegration } =
+      const { environmentVariables, environments, hasStaging, vercelIntegration, pagination } =
        await presenter.call({
          userId: user.id,
          projectSlug: projectParam,
+          page: searchParams.page,
+          search: searchParams.search,
        });

      const accessibleEnvironmentIds = environments
@@ -176,6 +183,8 @@ export const loader = dashboardLoader(
        vercelIntegration,
        accessibleEnvironmentIds,
        writableEnvironmentIds,
+        pagination,
+        search: searchParams.search,
      });
    } catch (error) {
      console.error(error);
@@ -392,17 +401,12 @@ function EnvironmentVariablesListPage({
  loaderData: EnvironmentVariablesPageLoaderData;
 }) {
  const [revealAll, setRevealAll] = useState(false);
-  const { environmentVariables, vercelIntegration } = loaderData;
+  const { environmentVariables, vercelIntegration, pagination, search } = loaderData;
+  const hasSearch = Boolean(search);
  const organization = useOrganization();
  const project = useProject();
  const environment = useEnvironment();
-  const { value } = useSearchParams();
-  const urlSearch = value("search") ?? "";
-  const { filteredItems } = useFuzzyFilter<EnvironmentVariableWithSetValues>({
-    items: environmentVariables,
-    keys: ["key", "value", "environment.type", "environment.branchName"],
-    filterText: urlSearch,
-  });
+  const filteredItems = environmentVariables;

  const tableScrollRef = useRef<HTMLDivElement>(null);

@@ -477,9 +481,9 @@ function EnvironmentVariablesListPage({
      </NavBar>
      <PageBody scrollable={false}>
        <div className={cn("flex h-full min-h-0 flex-col")}>
-          {environmentVariables.length > 0 && (
+          {(environmentVariables.length > 0 || hasSearch) && (
            <div className="flex items-center justify-between gap-2 px-2 py-2">
-              <SearchInput placeholder="Search variables…" autoFocus />
+              <SearchInput placeholder="Search variables…" resetParams={["page"]} autoFocus />
              <div className="flex items-center justify-end gap-1.5">
                <Switch
                  variant="secondary/small"
@@ -574,7 +578,7 @@ function EnvironmentVariablesListPage({
                <TableBody>
                  <TableRow>
                    <TableCell colSpan={vercelColumnCount}>
-                      {environmentVariables.length === 0 ? (
+                      {!hasSearch ? (
                        <div className="flex flex-col items-center justify-center gap-y-4 py-8">
                          <Header2>You haven't set any environment variables yet.</Header2>
                          <LinkButton
@@ -597,6 +601,14 @@ function EnvironmentVariablesListPage({
              )}
            </Table>
          </div>
+          {pagination.totalPages > 1 && (
+            <div className="flex items-center justify-end border-t border-grid-dimmed px-2 py-2">
+              <PaginationControls
+                currentPage={pagination.currentPage}
+                totalPages={pagination.totalPages}
+              />
+            </div>
+          )}
        </div>
      </PageBody>
      <Outlet />