Commit Graph

5307 Commits

Author SHA1 Message Date
nicktrn 7c57d0e2aa add github action test readme 2024-09-16 13:56:53 +01:00
nicktrn 404e18cb52 add docs checks 2024-09-16 13:50:07 +01:00
nicktrn 19f4a3cccf add changesets action mock 2024-09-16 12:42:38 +01:00
nicktrn db06c77477 add test events 2024-09-16 08:36:49 +01:00
nicktrn d2d0716ed8 move test action 2024-09-16 08:36:32 +01:00
nicktrn 483162a825 clarify image tag action errors 2024-09-16 08:30:41 +01:00
nicktrn 5df1b7e331 fix inputs and published package parsing 2024-09-16 08:17:59 +01:00
nicktrn d36f0476e4 call publish after successful package release 2024-09-15 22:25:13 +01:00
nicktrn d5b916777d rename publish workflows 2024-09-15 22:04:30 +01:00
nicktrn cb87cb565c add tests for custom action 2024-09-15 21:55:08 +01:00
nicktrn ce949094a4 make it possible to call publish 2024-09-15 21:53:51 +01:00
nicktrn 8518b6f6e5 simplify semver checks 2024-09-15 19:34:28 +01:00
nicktrn eef94fac3c update publish trigger paths 2024-09-15 19:31:10 +01:00
nicktrn f58df38ae8 image action updates 2024-09-15 19:29:14 +01:00
nicktrn 90f2f78e04 ignore all docs changes for release trigger 2024-09-15 19:28:02 +01:00
nicktrn b2d1c3f3f2 replace cancel action with native concurrency control 2024-09-15 19:23:46 +01:00
nicktrn fee02a49e9 remove publish infra bloat 2024-09-15 14:12:26 +01:00
nicktrn 107b888b9a add custom image tag action 2024-09-15 14:11:04 +01:00
nicktrn 28050adbcb lint and action updates 2024-09-15 14:08:09 +01:00
Niels 55b7ffeb72 Don't run release workflow on mdx changes (#1304) 2024-09-15 11:24:35 +01:00
Jacob Paris cfe0abd95e Update remix examples (#1303)
* Update remix examples

* fix: remove trailing space
2024-09-15 11:10:01 +01:00
James Ritchie c1690bd1ae New React to PDF example task (#1300)
* Better code snippet to clarify your framework

* New page for react-pdf

* Added 3 more examples

* New react to PDF example

* Alphabeticalise the side menu

* Removed github link and tweaked title
2024-09-13 16:46:52 +01:00
Matt Aitken 9a7ad926aa ElectricSQL run page (hidden page for now) (#1297)
* WIP on using react-window-splitter

* WIP with new resizable panels and SSR

* Use the cookie package

* Resizable storybook page

* Increase indexing memory limit

* Fixed v2 usage meter displaying when on paid plan (#1255)

* Fixed v2 usage meter displaying when on paid plan

* Show the free usage panel only for v3 projects

* Concurrency page and more accurate tracking (#1252)

* Initial TaskRunConcurrencyTracker implementation

* MARQS calls a subscriber to events

* When enqueuing add the extra required metadata

* Track concurrency per environment for tasks too

* Admin page for global concurrency

* Use the new concurrency tracker on the tasks page

* Useful performance test task

* getAllTaskIdentifiers()

* New page for concurrency

* BackgroundWorkerTask index for quick lookup of task identifiers

* Added a way to get concurrency for environments

* Added upgrade/request more concurrency button

* Queued task column working

* Use defer and suspense

* Added queue column to the concurrency environments table

* Some comments added for clarity

* Fixed bad log message

* Sidemenu: move lower and rename to “Concurrency limits”

* Only show the environments, not tasks. Renamed to “Concurrency limits”

* v3: fix unfreezable state crashes for runs with multiple waits (#1253)

* support named capture groups

* write crash errors to attempt.error

* make restored pod names unique per checkpoint

* use last eight characters of checkpoint id instead

* add more chaos monkey env vars

* Ignore unfreezable states

* prevent excessive queue config parsing errors

* handle dependency resume edge case

* better entry point logging

* ignore checkpoint cancellation timeouts

* add missing idempotency keys to wait for dep replays

* remove checkpoints between attempts

* fix retry container names on kubernetes

* add changeset

* fix types

* bring back internal duration timers

* Added more logging to TaskRunConcurrencyTracker and some more try/catches

* Call subscriber.messageDequeued in dequeueMessageInSharedQueue

* Added messageReplaced to concurrency tracking (when freezing)

* Added depenenciesToBundle guide to bundle all packages

* Include the old message data when replacing, so we get the projectId etc.

* Fix restored container names

* Fix for schedule page not scrolling

* Added a description panel to the Concurrency admin page

* chore: Update version for release (beta) (#1256)

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

* Release 3.0.0-beta.53

* Added a note to use batchTrigger() instead of trigger()

* The latest react-window-splitter fixes the ESM issues

* Set sensible defaults for the run page

* Deployments page

* Test page

* Schedules page

* Latest version of react-window-splitter (0.2.5)

* Updated to the latest version: react-window-splitter

* Callout if runs don’t start right away now has some top margin

* Small padding fix

* styled the handle focus state

* Added isStaticAtRest prop to resizable panel

* Updated resizable storybook

* Inline code blocks behave nicer when text wraps

* Added ElectricSQL to docker-compose, available on 3060

* Extracted some logic out of the eventRepository for getting a trace. This will be used on the frontend

* Use the new util

* More restructuring ready to use the trace summary from the frontend

* Using ElectricSQL for the run page data

* Min size for resizable panel on test page

* Don’t load the trace in the RunPresenter anymore

* Fix for the resizable panels on the run page

* Added overflow hidden to the panel group

* min size for the test page left hand panel

* Updated to latest window-splitter version

* Removed unused const

* One fix for client-server mismatch

* Slight improvement in the loading state

* Restructured the page so the loading is better

* Improvement to the loading states

* Improved the loading behaviour with the inspector

* WIP on auth, having problems with it

* Upgrade Remix to 2.9.1 (same as PR #1096)

* Switched structure around again so we only call the useTrace hook from the client

* Added auth to the sync

* Overscan more rows in the tree view

* Fix for TS error

* Remove duplicate import

* Revert "Upgrade Remix to 2.9.1 (same as PR #1096)"

This reverts commit e63ee9e924.

* save cookie only when id is used

* Deployment table now scrolls

* removed imports

* A lot of changes to make the inspector live too… WIP

* More major overhauls to get the synced version of the run page working…

* If a span is completed show that

* Set the debounce much lower for selecting the span view

* Load the details run inspector data on demand

* Delete the SpanPresenter

* Use the async payload because it deals with superjson

* Fixed weird merge conflict

* Share some inspector timeline components

* A couple of layout tweaks

* Improved the run inspector loading states

* Fix for paragaph errors

* Fix for focusing on a span

* Undefined typre for useSyncedShape

* ELECTRIC_ORIGIN env var doesn’t have a default, added to the examples

* Updated @electric-sql/react package to the latest

* Fix the timeline duration stretching

* Added some better error handling for the electric sync

* More logging

* Better error when there are bad responses

* Turn off resizable snapshots, there’s a bug

* Added getSpan back

* Added SpanPresenter back

* Updated to the new Electric hooks package

* Made a copy so we have the old run page and the new electric one

* Put the main eventRepository back for now

---------

Co-authored-by: nicktrn <55853254+nicktrn@users.noreply.github.com>
Co-authored-by: James Ritchie <james@trigger.dev>
Co-authored-by: James Ritchie <james@jamesritchie.co.uk>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-09-13 11:45:24 +01:00
Matt Aitken a66db33dee If there’s no attempt and no worker when finalizing a run, don’t try to create an attempt 2024-09-12 15:08:37 +01:00
Matt Aitken f90e23d503 Fix so failed cancels in the dashboard redirect back with an error message 2024-09-12 15:08:06 +01:00
Matt Aitken 0cc56040f3 batchTriggerAndWait checkpoint race condition when at max concurrency (#1296)
* Ignore /packages/cli-v3/src/package.json

* Added more logs when resuming a dependency, added the runId

* A task for reproducing a race condition with checkpoints

* Fix for doing remote image build when not self-hosting

* Set team members, alerts and schedule limits to 100m for self-hosting

* Import fix

* Set the checkpointEventId in marqs when the checkpoint is created for batchTriggerAndWait

This should fix a horrible race condition when at max concurrency
2024-09-12 14:27:42 +01:00
Eric Allam 67547d2521 Remove payload from task run task events 2024-09-12 14:12:16 +01:00
Eric Allam a50063ce04 Always insert the dirs option when initializing a new project in the trigger.config.ts 2024-09-12 14:12:16 +01:00
Eric Allam 8c690a960c Make sure BuildManifest is exported from @trigger.dev/build 2024-09-12 14:12:16 +01:00
D-K-P 511a24b9f3 Supabase docs improvements 2024-09-12 11:28:46 +01:00
Dan 7ca2b628f7 Added tutorial docs and images (#1295) 2024-09-12 10:10:51 +01:00
James Ritchie 9c61a25402 Docs/task examples (#1293)
* WIP first example task guide

* Added example link to github repo

* Created a top level Examples docs section

* Renamed side menu to examples for now
2024-09-11 17:37:49 +01:00
James Ritchie 972b5e34b8 Docs/more improvements (#1292)
* WIP creating a CLI section

* WIP adding more CLI commands in a new section

* New CLI section with introduction and more CLI commands

* More improvement to the CLI options

* Each CLI command has its own page

* fixed broken links

* Removed outdated copy
2024-09-11 17:14:39 +01:00
Eric Allam 47ca24aab8 Update new build sytem docs 2024-09-11 15:57:22 +01:00
Eric Allam 8578c9b281 v3: new build system fixes round 2 (#1283)
* Fixed empty env vars overriding in dev runs

* Don’t import package.json anymore

* fix node10 moduleResolution in @trigger.dev/core

* Support self-hosters pushing to a custom registry when running deploy

* dev: Fixed stuck runs when a child run fails with a process exit

* Make some doc notes about known issues and docker hub private repos

* Fix --project-ref when running deploy

* Fix —config option when deploying

* Fixing the flushing/killing process with the new build system

* Add monorepo-react-email e2e test fixture

* Fix issue with emitDecoratorMetadata and tsconfigs with extends

* Got the emit decorator metadata fixture working

* Fixed typechecking yarn e2e CLI tests in monorepos

* Add remote forced externals system, in case we come across another package that cannot be bundled (spurred on by header-generator)

* Remote externals now powered by JSON Hero to be easier to update

* resolve config source files

* Add a —javascript option to init, defaults to typescript

* Add support for prisma typed sql

* Remove msw and retry.interceptFetch

* Add missing code to the openai retries example

* Don’t generate the v3 catalog prisma client during CI

* Fixed v3-catalog task imports

* Remove interceptor usage in task file

* Only import import-in-the-middle hook if there are instrumented packages

* Fix yarn.lock file
2024-09-11 15:48:20 +01:00
Matt Aitken 8bb81ec73f The usage tasks table was showing double all values if you had a staging env… 🤦‍♂️ 2024-09-11 11:39:08 +01:00
Matt Aitken c46e3d528e Fix for BigInt values getting stripped from log properties 2024-09-11 11:20:22 +01:00
Matt Aitken 93bc32ca87 Checkpoint restoring should of triggerAndWait should use the child run 2024-09-10 15:46:24 +01:00
Matt Aitken 36397d0cb1 Added more logging when finalizing runs and when messages don’t exist in marqs 2024-09-10 14:52:32 +01:00
nicktrn 90bf01fb23 Decrease cpu requests by 25% 2024-09-10 14:28:05 +01:00
Matt Peck 15ed74b63e Allow custom docker network (#1291) 2024-09-09 22:54:27 +01:00
Matt Aitken 392453e8a6 Frozen run fixes (#1286)
* When resuming a batch, only do marqs operations once

* Made TaskRunDependency clearer in the Prisma schema

* New ResumeDependentParentsService service, use it from checkpoints

* WIP on making resuming more robust

* Turn the declarative schedules off because they make debugging other runs painful

* Resuming batches when there’s an attempt is working

* If there’s no attempt then create one

* Added a log if there are no span events to complete

* If Graphile addJob doesn’t return a row, log and return undefined. No throw

* Pass prisma into the ResumeDependentParentsService

* Removed the todos

* Pass Prisma through to the checkpoint service

* Fix for not checking the batch item correctly

* Fix for when a log flush times out and the process is checkpointed

* Fix for when a log flush times out and the process is checkpointed

* Another test run that does batches with failed subtasks

* Don’t call ResumeTaskRunDependenciesService anymore (we have a new service)

* Only resume if the run is in a final state

* If an attempt doesn’t exist, fix for creating queue with sanitized name

* If DEV then don’t resume using marqs/batches. The CLI manages it

* We don’t need to check the run status again, it’s in the main function now

* Added TaskRunAttempt taskRunId index

* Only allow calling ResumeDependentParentsService with a run ID

* Put the flushing back to what it was
2024-09-09 16:03:36 +01:00
Elie Steinbock e5f0aafb14 Include uploadthing in .env.example for references (#1288) 2024-09-08 10:28:16 +01:00
Elie Steinbock 05cdc6993c Remove astro as recommended extension (#1285) 2024-09-06 16:36:15 +01:00
James Ritchie 569dcf4ef4 Docs improvements (#1284)
* Separated the triggering docs page into 2 sections

* Added link to GitHub Actions from deploy page

* Improved the batchTriggerAndWait docs for handling errors

* Added docs for using google credential files

* Improved the triggering page copy some more

* Improved the google credentials copy
2024-09-06 14:26:58 +01:00
Eric Allam 87e46d497b projects now have a dedicated build server (#1281) 2024-09-06 09:36:03 +01:00
James Ritchie 022bbe1371 Fixed type issues with emails 2024-09-05 17:22:58 +01:00
James Ritchie 3e0881c2e9 Added project name to run alert email 2024-09-05 15:33:55 +01:00
Matt Aitken ec3c300cac Put the database command back in with wal_level logical 2024-09-05 14:18:21 +01:00
Matt Aitken 3f182cd145 Latest lockfile 2024-09-05 14:18:01 +01:00