Commit Graph

3435 Commits

Author SHA1 Message Date
James Ritchie 1d58ef97bc Min size for resizable panel on test page 2024-08-29 17:47:04 +01:00
James Ritchie 4510fc0ed0 Inline code blocks behave nicer when text wraps 2024-08-24 12:10:08 +01:00
James Ritchie 3952421f4a Updated resizable storybook 2024-08-23 18:08:06 +01:00
James Ritchie 1d3cd1d8e9 Added isStaticAtRest prop to resizable panel 2024-08-23 17:47:24 +01:00
James Ritchie 442e3b466e styled the handle focus state 2024-08-23 17:47:06 +01:00
James Ritchie 1e68485298 Small padding fix 2024-08-23 16:38:17 +01:00
James Ritchie bb93adca35 Callout if runs don’t start right away now has some top margin 2024-08-23 16:33:08 +01:00
James Ritchie d36f07b79c Updated to the latest version: react-window-splitter 2024-08-23 16:18:49 +01:00
James Ritchie 075623007f Merge remote-tracking branch 'origin/main' into react-window-splitter
# Conflicts:
#	.changeset/hungry-sloths-promise.md
#	apps/webapp/app/models/runtimeEnvironment.server.ts
#	apps/webapp/app/presenters/v3/ConcurrencyPresenter.server.ts
#	apps/webapp/app/routes/_app.orgs.$organizationSlug.projects.v3.$projectParam.schedules/route.tsx
#	apps/webapp/app/v3/services/completeAttempt.server.ts
#	apps/webapp/app/v3/services/createCheckpoint.server.ts
#	apps/webapp/app/v3/services/resumeBatchRun.server.ts
#	apps/webapp/app/v3/services/resumeTaskDependency.server.ts
#	apps/webapp/app/v3/taskStatus.ts
#	docs/troubleshooting.mdx
#	integrations/airtable/CHANGELOG.md
#	integrations/airtable/package.json
#	integrations/github/CHANGELOG.md
#	integrations/github/package.json
#	integrations/linear/CHANGELOG.md
#	integrations/linear/package.json
#	integrations/openai/CHANGELOG.md
#	integrations/openai/package.json
#	integrations/plain/CHANGELOG.md
#	integrations/plain/package.json
#	integrations/replicate/CHANGELOG.md
#	integrations/replicate/package.json
#	integrations/resend/CHANGELOG.md
#	integrations/resend/package.json
#	integrations/sendgrid/CHANGELOG.md
#	integrations/sendgrid/package.json
#	integrations/shopify/CHANGELOG.md
#	integrations/shopify/package.json
#	integrations/slack/CHANGELOG.md
#	integrations/slack/package.json
#	integrations/stripe/CHANGELOG.md
#	integrations/stripe/package.json
#	integrations/supabase/CHANGELOG.md
#	integrations/supabase/package.json
#	integrations/typeform/CHANGELOG.md
#	integrations/typeform/package.json
#	packages/astro/CHANGELOG.md
#	packages/astro/package.json
#	packages/cli-v3/CHANGELOG.md
#	packages/cli-v3/package.json
#	packages/cli-v3/src/entryPoints/deploy-run-controller.ts
#	packages/cli/CHANGELOG.md
#	packages/cli/package.json
#	packages/core-apps/CHANGELOG.md
#	packages/core-apps/package.json
#	packages/core-backend/CHANGELOG.md
#	packages/core-backend/package.json
#	packages/core/CHANGELOG.md
#	packages/core/package.json
#	packages/eslint-plugin/CHANGELOG.md
#	packages/eslint-plugin/package.json
#	packages/express/CHANGELOG.md
#	packages/express/package.json
#	packages/hono/CHANGELOG.md
#	packages/hono/package.json
#	packages/integration-kit/CHANGELOG.md
#	packages/integration-kit/package.json
#	packages/nestjs/CHANGELOG.md
#	packages/nestjs/package.json
#	packages/nextjs/CHANGELOG.md
#	packages/nextjs/package.json
#	packages/otlp-importer/CHANGELOG.md
#	packages/otlp-importer/package.json
#	packages/react/CHANGELOG.md
#	packages/react/package.json
#	packages/remix/CHANGELOG.md
#	packages/remix/package.json
#	packages/sveltekit/CHANGELOG.md
#	packages/sveltekit/package.json
#	packages/testing/CHANGELOG.md
#	packages/testing/package.json
#	packages/trigger-sdk/CHANGELOG.md
#	packages/trigger-sdk/package.json
#	packages/yalt/CHANGELOG.md
#	packages/yalt/package.json
#	pnpm-lock.yaml
2024-08-23 16:09:08 +01:00
James Ritchie 6496539c01 Usage doesn’t show as a red bar if you’re beyond the plan 2024-08-23 14:04:57 +01:00
Matt Aitken a9810b0fd8 Stop heartbeats when creating andWait checkpoints, fix for concurrency tracker (#1268)
* RESUME messages were missing some data that is needed for the concurrency tracker

* Cancel heartbeats when checkpoints are created for triggerAndWait and batchTriggerAndWait

* Added some logs when canceling heartbeats

* Improved the nested dependencies test task
2024-08-23 13:10:39 +01:00
Eric Allam f9ec66c562 v3: new build system (#1265)
* upgrade @opentelemetry packages to the latest versions

* remove v2 only packages, will be moved to a dedicated repo

* remove more v2 code and run pnpm install

* use the npm yalt package in the webapp

* convert @trigger.dev/core to tshy

* Switch from jest to vitest in @trigger.dev/core

* Fixed core test

* move core-backend code into core subpath export

* convert @trigger.dev/sdk to tshy

* Removed hono

* move core-apps to core/v3/apps, remove core-apps, start converting cli-v3

* Fix up some of the commands

* cli now building and loadable

* using package-json-from-dist to get package version now in core and cli

* dev command WIP

* cleaned up some repetition and structure of the entry point stuff

* bringing back the background worker stuff

* Indexing of the v3 catalog

* getting closer to executing dev runs...

* centralize dev logging using event emitter

* Move indexing to it’s own entry point, simplify code

* dev runs working

* Get instrumentation to work with openai

* debugging achieved internally

* provide worker files as part of the worker creation on the server

* support for cjs and esm javascript

* Fixed timeout

* worker manifest now has the config path

* auto-upgrade config to non-deprecated alternatives

* Adding package preview release

* deployment WIP

* improve the syncEnvVars output and adapt resolveEnvVars

* WIP bun runtime

* WIP bun support

* seed tasks with the machine preset if listed in the config

* deploy run executions WIP, extracted TaskRunProcess into 1 place

* deployed tasks running and executing 🎉

* support for waits and better flushing & process cleanup

* Fixed the heartbeating

* Better warning messages

* Improve and unify the indexing between dev and deploy

* Support for external deps that need node-gyp to build

* build extensions can now install custom packages and run instructions in the image. Also prisma extension now works and also works with multiple schema files

* Add back in the main/types/module to sdk

* dev no longer is Ink/React, grace period for disconnections in dev

* Fix the changeset config

* More changeset fixes

* Remove config packages

* More changeset fixes

* Fixed typescript issues (needed to revert back to zod 3.22.3

* Fix pr_checks workflow

* Remove the prepare script

* Fixed tests and package versions

* Remove cli test script

* Remove packages from tailwind watch paths

* Add repo to public packages

* Just commit the generated files and do the building at dev time

* Try and get pkg.pr.new working

* Try again

* Fix emitDecoratorMetadata importing named export from typescript

* config file backwards compat with export const config

* Fixed issue where import errors weren’t coming through

* p-retry is a prod dep

* typescript needs to be a prod dependency for emitDecoratorMetadata

* Add better debug logging to help track down import-in-the-middle bug

* An external is only considered resolvable if it resolves to the same path as the collected external

* Fix runtime checks to allow >=18.20

* Move extensions to a new build package

* Fixed building packages in dockerfile

* Remove the e2e test from publish workflow for now

* Don’t treat pkg.pr.new versions has needing upgrading

* making sure config handleError works, and discovered path aliases don’t work in config files

* Strip empty string env vars so they accidentally override real values

* Couple of things

* Update version to use preview instead of beta

* Hopefully fix re-attempts with >30s delay

* Match socket emit messages to current latest in main

* Initial guide

* Go back to beta

* Go back to the preview, and update guide to use pr preview tags

* Go back to beta

---------

Co-authored-by: Matt Aitken <matt@mattaitken.com>
2024-08-23 13:10:15 +01:00
James Ritchie 2cd8658190 If an org has been deleted, show a ☠️ icon on the admin page 2024-08-22 13:15:00 +01:00
Matt Aitken ba7bbdbbdb Dashboard query improvements (#1266)
* Remove run count from JobPresenter

* Remove job/integration counts from org/projects query

* Removed v2 counts from ProjectPresenter query

* JobRun index idx_jobrun_organizationId_createdAt

* JobRun index idx_jobrun_versionId

* BatchTaskRunItem index taskRunAttemptId

* BatchTaskRunItem index taskRunId
2024-08-21 15:49:18 +01:00
Matt Aitken 7746302b51 Run failed alerts (#1262)
* Add new ProjectAlertType ‘TASK_RUN’ and then migrate channels to it

* WIP new alerts for run failures

* Consolidate failed run status into taskStatus.ts

* Added the read replica into BaseService, could be useful

* Send task run alerts code

* Revert "Added the read replica into BaseService, could be useful"

This reverts commit cc2348a40254ea3f9e11c0feaf303ecd8794433a.

* Allow adding task run alerts in the UI

* Use task run alert not attempt alert…

* Use the primary
2024-08-21 14:53:48 +01:00
Matt Aitken 6558e5f8eb Fix for permanently frozen tasks when at max concurrency
Heartbeats now free up concurrency for paused/frozen runs, then they don’t get crashed but get acked by the sharedQueueConsumer
2024-08-21 11:37:02 +01:00
Matt Aitken 9f07df8abc Don’t nack in requeueTaskRun if it’s WAITING_TO_RESUME or PAUSED 2024-08-20 19:00:29 +01:00
Matt Aitken 8ad599f90e Try set the checkpointEventId when resuming batches and dependency 2024-08-20 16:09:43 +01:00
Matt Aitken d48eb9bac8 Fix: was showing deleted team member dev environments (#1261)
* Env vars: don’t show deleted dev environments

* Concurrency limits: don’t show deleted dev environments

* Create/edit schedule: don’t show deleted dev environments
2024-08-20 14:03:24 +01:00
Matt Aitken 0591db5f2b Fix for short runs inside (batch)triggerAndWait (#1263)
* Test for checkpoints

* Make sourceTaskAttemptId optional on resumeBatchRun

* Removed all completions/executions logic from the shared queue consumer

* Removed the sourceTaskAttemptId from ResumeBatchRunService

* Revert "Removed all completions/executions logic from the shared queue consumer"

This reverts commit d35398d50463c81a1975bb5d5bcfca66a24b8ede.

* WIP on triggerAndWait…

* Fixed triggerAndWait continuing when a checkpoint completes

* Remove the ResumeAttempt code that fails attempts (was protecting against infinite restores)

* Removed messageBody.data.completedAttemptIds.length === 0 commented out code

* Don’t ack if there’s no batchRun

* Added the marqs?.replaceMessage back in but NOT when there’s no checkpoint. More logging

This is a fix for when some attempts fail

* Improvement to the test task that now randomly fails attempts

* When a checkpoint happens, only continue the attempt if it’s in the correct state

* Changeset for rollback in branch

* Set keepRunAlive to false when the dependent task isn’t finished

* Changeset manual version (to get inline with the hotfix branch)

* Changeset: Fixes for continuing after waits

* Latest lockfile (after manual changeset version)
2024-08-20 13:57:08 +01:00
nicktrn a64064965d Prevent runs that fail fast from never sending a heartbeat 2024-08-19 13:17:33 +01:00
Matt Aitken 2416f426b8 Latest version of react-window-splitter (0.2.5) 2024-08-18 12:46:24 +01:00
Matt Aitken 41ec450bee Revert "Remove the RequeueTaskRunService"
This reverts commit 3421f5ec0b.
2024-08-15 20:43:47 +01:00
Matt Aitken 9a9bdc20c9 Revert "Disable upserting task queues when triggering a task"
This reverts commit 6652edf7a7.
2024-08-15 20:42:50 +01:00
Matt Aitken 6652edf7a7 Disable upserting task queues when triggering a task 2024-08-15 19:34:22 +01:00
Matt Aitken 369bc38560 Instead of upserting taskQueues, do a findFirst then create/update 2024-08-15 18:48:50 +01:00
Matt Aitken 27f7132509 Don’t do the auto-increment inside a transaction, this means triggerTask isn’t inside one 2024-08-15 17:44:10 +01:00
Matt Aitken b6fc68aaae Don’t use Graphile for invocation usage reporting 2024-08-15 17:22:52 +01:00
Matt Aitken 3421f5ec0b Remove the RequeueTaskRunService 2024-08-15 17:20:40 +01:00
Matt Aitken 15097a3a67 Use the replica for the v2 run and task presenters 2024-08-15 15:34:57 +01:00
Matt Aitken 1b7bb90852 Only show the first 1000 task on the v2 run page 2024-08-15 15:26:12 +01:00
Matt Aitken 7e68c02636 Don’t reload v2 run page if more than 100 tasks 2024-08-15 15:25:59 +01:00
nicktrn 01e01376f8 Fix attempt retry mechanics for new workers 2024-08-15 12:13:13 +01:00
Matt Aitken 4027045f32 Schedules page 2024-08-15 11:11:11 +01:00
Matt Aitken c3298f974f Test page 2024-08-15 11:11:08 +01:00
Matt Aitken 68945a4e4b Deployments page 2024-08-15 10:56:46 +01:00
Matt Aitken 194c56aa62 Set sensible defaults for the run page 2024-08-15 10:34:39 +01:00
Matt Aitken d3810fc5bc The latest react-window-splitter fixes the ESM issues 2024-08-15 10:34:31 +01:00
James Ritchie cd10d47ae0 Added a description panel to the Concurrency admin page 2024-08-15 10:10:53 +01:00
James Ritchie 494e1eadaa Fix for schedule page not scrolling 2024-08-15 10:10:53 +01:00
nicktrn cbaaa3bb1f Fix restored container names 2024-08-15 10:10:53 +01:00
Matt Aitken 26e872aa54 Include the old message data when replacing, so we get the projectId etc. 2024-08-15 10:10:53 +01:00
Matt Aitken 41c551de3d Added messageReplaced to concurrency tracking (when freezing) 2024-08-15 10:10:53 +01:00
Matt Aitken 6b0c37ae6d Call subscriber.messageDequeued in dequeueMessageInSharedQueue 2024-08-15 10:10:53 +01:00
Matt Aitken 0e990fd2b5 Added more logging to TaskRunConcurrencyTracker and some more try/catches 2024-08-15 10:10:52 +01:00
nicktrn 4148921dff v3: fix unfreezable state crashes for runs with multiple waits (#1253)
* support named capture groups

* write crash errors to attempt.error

* make restored pod names unique per checkpoint

* use last eight characters of checkpoint id instead

* add more chaos monkey env vars

* Ignore unfreezable states

* prevent excessive queue config parsing errors

* handle dependency resume edge case

* better entry point logging

* ignore checkpoint cancellation timeouts

* add missing idempotency keys to wait for dep replays

* remove checkpoints between attempts

* fix retry container names on kubernetes

* add changeset

* fix types

* bring back internal duration timers
2024-08-15 10:10:52 +01:00
Matt Aitken 2c80e9a715 Concurrency page and more accurate tracking (#1252)
* Initial TaskRunConcurrencyTracker implementation

* MARQS calls a subscriber to events

* When enqueuing add the extra required metadata

* Track concurrency per environment for tasks too

* Admin page for global concurrency

* Use the new concurrency tracker on the tasks page

* Useful performance test task

* getAllTaskIdentifiers()

* New page for concurrency

* BackgroundWorkerTask index for quick lookup of task identifiers

* Added a way to get concurrency for environments

* Added upgrade/request more concurrency button

* Queued task column working

* Use defer and suspense

* Added queue column to the concurrency environments table

* Some comments added for clarity

* Fixed bad log message

* Sidemenu: move lower and rename to “Concurrency limits”

* Only show the environments, not tasks. Renamed to “Concurrency limits”
2024-08-15 10:10:52 +01:00
James Ritchie 37fd0882ae Fixed v2 usage meter displaying when on paid plan (#1255)
* Fixed v2 usage meter displaying when on paid plan

* Show the free usage panel only for v3 projects
2024-08-15 10:10:52 +01:00
nicktrn a8f725d643 Increase indexing memory limit 2024-08-15 10:10:52 +01:00
James Ritchie 2940b07430 Added a description panel to the Concurrency admin page 2024-08-14 11:33:15 +01:00