Eric Allam
6df8069c0e
* WIP batch trigger v2 * Fix for the DateField being one month out… getUTCMonth() is zero indexed 🤦♂️ * Added a custom date range filter * Deal with closing the custom date range * Child runs filter * Fix for the clear button untoggling the child runs * WIP batchTriggerV2 * Finished removing rate limit from the webapp * Added an index TaskRun to make useRealtimeBatch performant * Renamed the period filter labels to be “Last X mins” * Denormalize background worker columns into TaskRun * Use the runTags column on TaskRun * Add TaskRun ("projectId", "id" DESC) index * Improved the v2 batch trigger endpoint to process items in parallel and also added a threshold, below which the processing of items is async * Added a runId filter, and WIP for batchId filter * WIP triggerAll * Add new batch methods for triggering multiple different tasks in a single batch * Disabled switch styling * Batch filtering, force child runs to show if filtering by batch/run * Added schedule ID filtering * Force child runs to show when filtering by scheduleId, for consistency * realtime: allow setting enabled: false on useApiClient * Batches page * Always complete batches, not only batchTriggerAndWait in deployed tasks * Add batch.retrieve and allow filtering by batch in runs.list * Renamed pending to “In progress” * Tidied up the table a bit * Deal with old batches: “Legacy batch” * Added the Batch to the run inspector * Fixed the migration that created the new idempotency key index on BatchTaskRun * Fixed the name of the idempotencyKeyExpiresAt option and now default idempotency key TTL is 30 days, not 24 hours * Timezone fix: wrong month in Usage page dropdown * The DateField now defaults to local time, but can be overriden to use utc with an option * Don’t allow the task icon to get squished * BatchFilters removed unused imports * In the batch filtering, use `id` instead of `batchId` in the URL * BatchFilters: we don’t need a child tasks hidden input field * Creates some common filter components/functions * Fix for batchVersion check when filtering by batch status * Add additional logging around telemetry and more attributes for trigger spans * Show clear button for specific id filters * Batch list: only allow environments that are part of this project * Unnecessary optional chain Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * Add JSDocs --------- Co-authored-by: Matt Aitken <matt@mattaitken.com> Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
@trigger.dev/database
This is the internal database package for the Trigger.dev project. It exports a generated prisma client that can be instantiated with a connection string.
How to switch branches when you've done migrations
Sometimes you've applied migrations and then want to switch branches without wiping out your local database.
To do this you can run the following command:
DB_VOLUME=database-data-alt pnpm run docker
This will switch to the alt volume for your local database. This database will be blank if you haven't switched to this volume before, so you'll need to follow the normal steps (in the Contributing guide) to get setup, e.g. apply migrations and seed.
To switch back to the original volume, run the following command:
pnpm run docker
How to add a new index on a large table
- Modify the Prisma.schema with a single index change (no other changes, just one index at a time)
- Create a Prisma migration using
cd internal-packages/database && pnpm run db:migrate:dev --create-only - Modify the SQL file: add IF NOT EXISTS to it and CONCURRENTLY:
CREATE INDEX CONCURRENTLY IF NOT EXISTS "JobRun_eventId_idx" ON "JobRun" ("eventId");
- Don’t apply the Prisma migration locally yet. This is a good opportunity to test the flow.
- Manually apply the index to your database, by running the index command.
- Then locally run
pnpm run db:migrate:deploy
Before deploying
Run the index creation before deploying
CREATE INDEX CONCURRENTLY IF NOT EXISTS "JobRun_eventId_idx" ON "JobRun" ("eventId");
These commands are useful:
-- creates an index safely, this can take a long time (2 mins maybe)
CREATE INDEX CONCURRENTLY IF NOT EXISTS "JobRun_eventId_idx" ON "JobRun" ("eventId");
-- checks the status of an index
SELECT * FROM pg_stat_progress_create_index WHERE relid = '"JobRun"'::regclass;
-- checks if the index is there
SELECT * FROM pg_indexes WHERE tablename = 'JobRun' AND indexname = 'JobRun_eventId_idx';
Now, when you deploy and prisma runs the migration, it will skip the index creation because it already exists. If you don't do this, there will be pain.