Matt Aitken
0e14b6d750
## Summary - Drop all 8 foreign key constraints on TaskRun. The run listing path is now fully ClickHouse-backed so we no longer need Postgres to enforce referential integrity on this table. The FK constraints add write overhead on every insert/update with no remaining benefit. Prisma queries are unaffected. - Remove PostgresRunsRepository and its associated feature flag (runsListRepository), which was the last remaining code path querying TaskRun directly for list/count operations. - Drop three indexes that were only useful for the Postgres run list path and have no remaining query consumers: - TaskRun_runtimeEnvironmentId_id_idx — was the cursor pagination index for PostgresRunsRepository; superseded by the (runtimeEnvironmentId, createdAt DESC) composite index - TaskRun_scheduleId_idx — redundant with the (scheduleId, createdAt DESC) composite index; no direct Postgres queries filter by scheduleId alone - TaskRun_rootTaskRunId_idx — no queries filter TaskRun by rootTaskRunId as a WHERE clause anywhere in the codebase All index drops use CONCURRENTLY IF EXISTS to avoid table locks in production. ## Test plan - pnpm run db:migrate:deploy applies all migrations cleanly - pnpm run typecheck --filter webapp passes - Run list pages load correctly in the dashboard (ClickHouse path) - Scheduled task runs still trigger and appear correctly
@trigger.dev/database
This is the internal database package for the Trigger.dev project. It exports a generated prisma client that can be instantiated with a connection string.
How to switch branches when you've done migrations
Sometimes you've applied migrations and then want to switch branches without wiping out your local database.
To do this you can run the following command:
DB_VOLUME=database-data-alt pnpm run docker
This will switch to the alt volume for your local database. This database will be blank if you haven't switched to this volume before, so you'll need to follow the normal steps (in the Contributing guide) to get setup, e.g. apply migrations and seed.
To switch back to the original volume, run the following command:
pnpm run docker
How to add a new index on a large table
- Modify the Prisma.schema with a single index change (no other changes, just one index at a time)
- Create a Prisma migration using
cd internal-packages/database && pnpm run db:migrate:dev:create - Modify the SQL file: add IF NOT EXISTS to it and CONCURRENTLY:
CREATE INDEX CONCURRENTLY IF NOT EXISTS "JobRun_eventId_idx" ON "JobRun" ("eventId");
- Don’t apply the Prisma migration locally yet. This is a good opportunity to test the flow.
- Manually apply the index to your database, by running the index command.
- Then locally run
pnpm run db:migrate:deploy
Before deploying
Run the index creation before deploying
CREATE INDEX CONCURRENTLY IF NOT EXISTS "JobRun_eventId_idx" ON "JobRun" ("eventId");
These commands are useful:
-- creates an index safely, this can take a long time (2 mins maybe)
CREATE INDEX CONCURRENTLY IF NOT EXISTS "JobRun_eventId_idx" ON "JobRun" ("eventId");
-- checks the status of an index
SELECT * FROM pg_stat_progress_create_index WHERE relid = '"JobRun"'::regclass;
-- checks if the index is there
SELECT * FROM pg_indexes WHERE tablename = 'JobRun' AND indexname = 'JobRun_eventId_idx';
Now, when you deploy and prisma runs the migration, it will skip the index creation because it already exists. If you don't do this, there will be pain.