Commit Graph

3086 Commits

Author SHA1 Message Date
Eric Allam 2dea8dec35 Implement simple 7 day log retention and blank states (#976) 2024-03-27 17:22:16 +00:00
Matt Aitken 252f5b3967 Fix for janky scrolling on the run page 2024-03-27 16:55:03 +00:00
Matt Aitken 426316aa3c There was an error about a div being in a paragraph 2024-03-27 16:23:11 +00:00
Eric Allam 74236ae7c1 Fix changeset workflow (thought I’d already fixed this?) 2024-03-27 14:47:42 +00:00
Eric Allam 055d468b38 Setup socket.io redis adapter and fix deployment builds for superjson error 2024-03-27 14:42:10 +00:00
Matt Aitken 3a35d60091 Change the npm tag from "latest" to "beta" in docs/cli/onboarding (#974)
* The v3 init command defaults to the “@v3” tag

* Change it to beta

* Updated the docs and in-app docs

* Updated GitHub action
2024-03-27 13:39:53 +00:00
Eric Allam f11c0f991b Log out the headers and url when no key is found 2024-03-27 13:30:47 +00:00
Eric Allam 1cf181fd02 Control api rate limiter logging through env vars 2024-03-27 12:42:05 +00:00
James Ritchie db611bebba Text truncation fixes (#973)
* Span titles and code blocks truncate when window is small

* Tree view toggle arrow doesn’t have red hover state if row is an error
2024-03-27 12:16:20 +00:00
Eric Allam 215b60c742 v3: superjson output support (#971)
* superjson output support

* Better support for superjson in the task events, limiting output attributes

* Offload large outputs to object store (r2)

* Finishing up the offloading of large outputs/payloads to an object store
2024-03-27 11:57:22 +00:00
Matt Aitken 9c89ab40aa Throttle the live streaming of the run to 1 second updates (#972) 2024-03-26 14:20:10 +00:00
Matt Aitken b342e06583 Fix for scrolling with long run traces 2024-03-26 11:30:05 +00:00
Eric Allam 4418de515b Whitelist api endpoints that don’t use an authorization endpoint from the rate limiter 2024-03-25 21:18:05 +00:00
Eric Allam 0d2a71c5cf handle when traces/logs don’t have traceIds/spanIds 2024-03-25 21:10:31 +00:00
Eric Allam 395abe1b92 v3: Various fixes to support papermark use-case (#970)
* Adding a changeset for v3

* Add a version field to @trigger.dev/core-apps package.json

* Build trigger.dev when doing a prerelease

* bundle @trigger.dev/core-apps with trigger.dev cli

* Fix the init command config template

* Don’t use * for the @trigger.dev/core dep version specifier

* strip workspace: from the package version before installing it

* Added dependenciesToBundle config option to bundle ESM only packages

* Added logging around resolving dependency paths

* Try again

* Resolve dependencies based on the project dir first

* flip the bundled default

* Adding some logs around dev task completion notifications

* Adding some additional logs

* Add more logs

* Write out the log using process.stdout

* Store pending completion notifications and resume them when awaited (fixes race condition)

* Cleanup some of the logs

* Copy over the postinstall step from the projects package.json

* Add support for including additional files when deploying (e.g. prisma schema)

* Don’t run scripts when resolving deps

* copy all the files just in case anything is needed in postinstall

* Remove duplicate option

* Use the tag when outputting the dev command

* Remove the postinstall script

* add the trigger dir to the config if the default is not chosen

* Remove the “hud” display in the dev command

* trigger file names with dashes now work

* Better file watching in dev

* Much better duplicate ID experience now

* Much better “Project not found” error

* Export the handleError function types from sdk

* Add support for configuring instrumentation

* Upgrade and unify @opentelemetry/* packages (and remove storybook from the webapp)

* Fix typescript error in react package

* Upgrade react types in webapp

* Allow span icons to be determined based on the span name (e.g. prisma:)

* Ignore built-in env vars when checking for env vars, and allow continuing the deployment even if missing env vars were detected

* Improve the retry.fetch default behavior and option structure

* Fixed typescript errors with packages/email react types

* Update the retry.fetch docs
2024-03-25 17:02:49 +00:00
Matt Aitken b35eebb666 Rate limit API requests and changed SQS reading speed (#969)
* Trying to use the @upstash/ratelimit package with ioredis…

* WIP using the redis package instead

* Revert the action back

* Removed redis

* Started refactoring

* SQS setting for the poll interval. Set the default queue reading to be slower

* API rate limiter as Express middleware

* Organise imports

* Fixed spelling mistake “limitter”

* No authorization header response is problem+json
2024-03-25 16:39:20 +00:00
nicktrn 9ecf07731a v3: small features and fixes for prod infra (#968)
* add checkpoint restore events

* fix retries.enabledInDev

* fix tsconfig paths

* fix provider build and dev

* update kubernetes provider and fix builds again

* upgrade prod base to node 20

* update infra publish workflow

* rethrow k8s errors after logging

* shorten index container names

* fix error type assertion

* rename type assertion

* remove resource limits for now

* add missing run id on create

* add push to deploy command for self-hosting

* checkpointing fixes

* update coordinator image

* ensure valid registry login

* delete checkpoint archive after successful push

* log options on error

* structured logs for socket connections

* fix structured log merge

* exit process after checkpointing

* update restore pull secret name

* append shortcode to restore names

* log handler payload

* disable post start lifecycle hook

* pass in coordinator host via volume

* replace dapi with taskinfo

* add missing restore label

* don't restart restored containers

* remove init container from create

* atomic post-completion checkpoints

* switch to run id for container names

* improve wait accuracy

* measure basic checkpoint perf

* always log disconnect reason

* use system clock to end wait spans

* checkpoint readiness and cancel signals

* restore from checkpoint events and fix statuses

* remove attempt id env var

* restore dependencies from events

* reconnect wip

* lifecycle hooks are back

* fix hooks and improve reconnect

* make docker send postStart hook

* only checkpoint for retry if large delay

* fix a few more resume issues

* lifecycle hook fixes

* skip connection handler when waiting for post start hook

---------

Co-authored-by: Eric Allam <eric@trigger.dev>
2024-03-25 16:28:56 +00:00
Matt Aitken b890a64046 The in-progress stripes are much more subtle 2024-03-25 16:20:51 +00:00
Matt Aitken 4266b308e7 Link from trigger() to a run. Link back to the triggering run. (#967)
* Schema for SpanMessagingEvents

* RunFilters status map was missing the key

* Link from a trigger to the run

* Two-way links working
2024-03-25 11:28:35 +00:00
Matt Aitken 67c62f7d25 v3 is restricted to approved orgs (#965)
* v3 projects can only be created if the org has permissions to do it

* Admin page

* Removed unused classes

* Admin org page, with links between user and org pages

* Set v3 enabled or not
2024-03-25 11:28:21 +00:00
Matt Aitken c6d6a9191c Hide show.actions from the properties panel (#964) 2024-03-22 19:53:28 +00:00
Kritik Jiyaviya a49b659701 feat: Ability to revoke an invite to an org (#902)
* feat: Ability to revoke an invite to an org

* Resolve a conflict with main

---------

Co-authored-by: Matt Aitken <matt@mattaitken.com>
2024-03-22 14:35:49 +00:00
Eric Allam 737ff9928c When cancelling a run in the UI make sure the task run status is cancelled not interrupted 2024-03-20 13:22:50 +00:00
Eric Allam fc4b95d69e A bunch of deployment fixes (#960)
- Time out deployments to prevent never-ending building/deploying
- Use the CURRENT worker deployment when determining which version to execute
- Fix some of the deployment error handling logic
2024-03-20 13:15:48 +00:00
James Ritchie 6c546936f3 Improved v3 feedback button (#959)
* Removed feedback button from PageHeader

* Show a different feedback button in the side menu for v3 projects

* Fixed issue with button wrapping on smaller width browser
2024-03-20 12:16:20 +00:00
James Ritchie 08e4fc002f V3 UI improvements (#958)
* Adding a v3 feedback button top-right

* Changed paused to frozen for attempts

* Reordered and renamed run statuses

* Dropdown status menu shows tool tips on hover

* Added a new DP feedback button in the header

* Created a more useful feedback sheet

* Added loading bar animation to storybook

* Switch the integration page to point at the “feature” help instead of “integration” which has been removed

---------

Co-authored-by: Matt Aitken <matt@mattaitken.com>
2024-03-19 20:36:04 +00:00
Matt Aitken 2e56af9742 Tasks page live-reloads with new tasks (#957)
* DevPubSub: use the schema that’s already been defined

* Publish a message to projectPubSub when a new DEV worker is created

* Live reloading of the tasks table (for dev tasks)

* Live notifications for deployed workers as well as local workers
2024-03-19 15:42:57 +00:00
Eric Allam dc5eb68a0b Fix typecheck errors because of resolveJsonModule 2024-03-19 15:33:09 +00:00
Eric Allam 41e1ac9e0b v3: nanosecond precision task events and immediate mode for dev events (#956)
* Store startTime in nanoseconds since epoch, ensure logs using nanosecond precision for their timestamp (so they are not out of order)

* export task events immediately in dev
2024-03-19 15:06:47 +00:00
Matt Aitken c5aaabdb4e The run timeline updates on the client if there's no new data (#955)
* Use “@v3” instead of “@latest” for the npx commands for v3

* Split the Timeline into a component so it can live refresh without re-rendering everything

* Timeline now live refreshes on the client every 500ms when run is executing

* The timeline bars now animate their position/width when it changes

* Export some more types from TreeView
2024-03-19 12:13:48 +00:00
Eric Allam 86c24afb44 Support json traces and logs in the otel endpoints (#954) 2024-03-19 10:59:50 +00:00
nicktrn b585d1f2c5 v3: checkpoint restore events (#951)
* add checkpoint restore events

* fix retries.enabledInDev

---------

Co-authored-by: Eric Allam <eric@trigger.dev>
2024-03-19 09:02:56 +00:00
Eric Allam ba0ffacb15 Set the OTEL_EXPORTER_OTLP_ENDPOINT env var on dev workers 2024-03-18 20:16:25 +00:00
Matt Aitken 9304c447be Test, output and canceling improvements (#952)
* The vertical lines in the run tree view are now brighter

* Blank state when you click a task with no runs

* Canceling from the runs table

* Fix for styles coming through incorrectly to properties

* Fix for the OTEL output column having a $output prefix if it’s an object
2024-03-18 17:27:52 +00:00
Eric Allam ebec14093d Add healthcheck endpoint outside of remix if remix routes are disabled 2024-03-18 17:19:56 +00:00
Eric Allam 67cac97b26 Add DEPLOY_REGISTRY_HOST to allow configuring the deploy registry via the server 2024-03-18 15:56:28 +00:00
Eric Allam 479d355b7b Setting DASHBOARD_AND_API_DISABLED=true will now disable the remix routes 2024-03-18 14:59:21 +00:00
Matt Aitken cae3298564 Cancel run confirmation modal (#949)
* Tasks get the SHOW_ACTIONS property

* Override properties with the passed in ones in eventRepository

* Link from deployments to environment variables page, no need to link to docs

* “Show parent items” keeps the selected item selected

* LinkButton can accept onMouseDown, onMouseEnter and onMouseLeave

* “Show parent items” keeps any items focused and has a keyboard shortcut

* Action bar with focus button

* Canceling is working

* Nicer layout and button loading and disabled state

* US english spelling “Cancelling” -> “Canceling”

* Added a confirm modal to canceling

* Copy tweaks

* Fix for bad import

---------

Co-authored-by: Eric Allam <eric@trigger.dev>
2024-03-18 12:22:06 +00:00
nicktrn 1f190fe680 v3: fix dependency resume (#946)
* fix basic resume mechanics

* fix dependency resume

* fix deploy page sort

* remove task list console log

* checkpoint restore between attempts

* add checkpoint requirement checks to docker provider

* switch to raw execa for long statements

* catch and report pre-connection index errors

* graceful worker shutdown after completion

* fix final completion

* use same task process for each attempt

* update retry catalog entry

* add immediate return task

---------

Co-authored-by: Eric Allam <eric@trigger.dev>
2024-03-18 12:21:40 +00:00
Matt Aitken 85ec1af198 Don’t add proj_ to the front of the project ref 2024-03-18 11:05:09 +00:00
Eric Allam 52c364d691 v3: cli init command + CLI telemetry (#948)
* Adding telemetry to the deploy command

* Add ability to opt-out of telemetry in the CLI

* Remove log

* Unifying some telemetry stuff and adding it to the login command

* init command

* Remove extra trigger dirs

* Couple tweaks to the init CLI

* Don’t use npm init -y to create the package.json file
2024-03-18 10:12:11 +00:00
Matt Aitken a5d83971aa Cancel a run from the span view (#947)
* Tasks get the SHOW_ACTIONS property

* Override properties with the passed in ones in eventRepository

* Link from deployments to environment variables page, no need to link to docs

* “Show parent items” keeps the selected item selected

* LinkButton can accept onMouseDown, onMouseEnter and onMouseLeave

* “Show parent items” keeps any items focused and has a keyboard shortcut

* Action bar with focus button

* Canceling is working

* Nicer layout and button loading and disabled state

* US english spelling “Cancelling” -> “Canceling”
2024-03-17 11:42:30 +00:00
Matt Aitken 5b9c67aeef v3 deployments page (#945)
* Set the “current” deployment in WorkerDeploymentPromotion when a new deployment is completed

* Deployments page added to sidenav

* Basic deployments page working

* Pagination for deployments

* Right-align the pagination controls

* Improved the blank state

* Selected deployment panel

* Improved the side panel
2024-03-15 16:04:01 +00:00
Matt Aitken 1c98eb2b61 Use TaskRunStatus (instead of TaskRunAttemptStatus) (#944)
* Using the new run status instead of run attempt status

* The test page now uses the TaskRunStatus as well

* Organized imports

* Use TaskRun icons instead of attempt ones

* 10x zoom on the run timeline
2024-03-15 10:32:07 +00:00
Eric Allam f424c81086 No staging by default, and project external refs now are prefixed with proj_ 2024-03-14 15:11:31 +00:00
Eric Allam 0a33bf7206 v3: handle errors and customizing retrying (#943)
* Add the ability to handle errors at the task and project level

* Cancel in progress runs when disconnecting the dev CLI (in between attempts)

* unify the task executor across dev & prod
2024-03-14 14:38:13 +00:00
Matt Aitken 819b663ad7 Test page environment selection (#942)
* useLinkStatus hook

* Now you can customize the appearance of InlineCode

* Created <TaskFunctionName/>

* The test page now has an environment selector at the top
2024-03-14 13:54:47 +00:00
Eric Allam 478ce006cb v3: task run status and canceling runs (#941)
* WIP task run status, revamped resuming task dependencies

* Don’t select a span when toggling collapsing

* Cancel runs and attempts, in prod and dev
2024-03-13 15:51:43 +00:00
Eric Allam 851b1769cb v3: use structured logging in a few places 2024-03-11 15:53:24 +00:00
Eric Allam e8381d5341 v3: Correctly handle tasks that have no output when resuming 2024-03-11 15:43:45 +00:00