Eric Allam
b1b9321ad2
* WIP job run performance improvements - Added a `perf` tool to better measure job run performance under heavy load - Removed `runFinished` job (not really needed) - startQueuedRuns now uses a jobKey with replace - Fixed an issue with ZodWorker when using jobKey * Publish improvement docker images * fixed the improvement docker publishing * Downgrade back to prisma 4.16.0 because 5.1.x broke docker builds * Changes to how queued runs work - Split the worker into two different workers, one dedicated to performRunExecution - Schedule performRunExecution in a single place, with a queue and using a round robin manually controlled concurrency - Remove startQueuedRuns - All runs are queued before they are started - Setting the worker maxPoolSize to the same as the worker concurrency - Starting to be able to split the docker image * Remove queue name from startRun graphile job * Make the prisma connection pool stuff configurable through env vars * Hardcode (for now) the max concurrent runs limit * Rewrite performRunExecution to be more performant PerformRunExecutionV2: - Does not create and manage jobRunExecution records - Does not reimplement retrying, uses graphile worker retrying instead I’ve kept around PerformRunExecutionV1 so this works when deploying. Definitely needs LOTS of testing * Fix issues with cached tasks - Limit the size of the cached tasks sent when executing a run, using the knapsack problem dynamic programming approach - Actually USE the cached tasks in IO by using the idempotencyKey instead of the task ID - Remove output from all logs - Added a stress test job catalog * Forgot to commit the logger updates * Never log connectionString * Login to docker hub to get around rate limits * Add additional logging to the graphile workers * Fix the *_ENABLED env vars * Allow adding and removing jobs to be done from the webapp * Don’t set the job to failed if it’s being retried * Deprecated queue options in the job and removed startPosition. Now using the job/env combo as the job queue name * Dequeung jobs doesn’t check if the runner is initialized * Fixed issues with retrying a run getting stuck on a cancelled task, and errors from parsing the results of dequeing a job * Remove queued round robin thing that isn’t used anymore * Added slack to job catalog * Better forwards compat * Added long delay * Fixed lock file
21 lines
485 B
JSON
21 lines
485 B
JSON
{
|
|
"name": "@trigger.dev/database",
|
|
"private": true,
|
|
"version": "0.0.0",
|
|
"main": "./src/index.ts",
|
|
"types": "./src/index.ts",
|
|
"dependencies": {
|
|
"@prisma/client": "4.16.0",
|
|
"typescript": "^4.8.4"
|
|
},
|
|
"devDependencies": {
|
|
"prisma": "4.16.0"
|
|
},
|
|
"scripts": {
|
|
"generate": "prisma generate",
|
|
"db:migrate:dev": "prisma migrate dev",
|
|
"db:migrate:deploy": "prisma migrate deploy",
|
|
"db:studio": "prisma studio",
|
|
"typecheck": "tsc --noEmit"
|
|
}
|
|
} |