Run Engine 2.0 (WIP) (#1575)
* bump worker version * Suggested glossary for the RunEngine, TBC * Removed BatchTaskRun changes from this branch, they were done in main * Set the BatchTaskRun status to completed when all runs are completed * When dequeuing respect passed in maxResources * Ported over the new run props: idempotencyKeyExpiresAt, versions, oneTimeUseToken, maxDurationInSeconds * Didn’t hit save… the new props when triggering tasks passed through * Idempotency expiration + waitpoint edge case * WIP on creating checkpoint, parking for now * fix worker routes * upgrade webapp node types to support generic event emitter * separate event bus handler singleton and run failure alerts * duration waits * fix execution snapshot debug spans * task waits * fix event bus types * temporary fix for react hook run handle type * disable run notifications for now * convert any typecasts to expect errors to more easily fix later * fix webapp types after node types upgrade * updateEnvConcurrencyLimits across marqs and the runqueue * Pass proper values into the run engine * RunQueue settings and removed unused rebalancing workers * Remove rebalancing prop * Tidied more things up * Update/remove queue limits for MARQS and RunQueue * taskQueue/concurrencyLimit changes ported back into the RunEngine * Reworked completing waitpoints to improve performance and reduce race conditions * Improved test robustness * Down to a single run lock only when a run is totally unblocked and ready to continue * warm starts, worker notifications, wait fixes * Fix for Run Engine poll interval env var * Expect the waitpoint to be completed quickly * If a run is locked then it’s too late to expire it * Added VALKEY_ env vars and plugged them into the run engine * Extracted and updated the guard queue function so it can be used when batching * Added logging and universal concurrency changes to trigger task v1 * Added notes back in * Bump @trigger.dev/worker to 3.3.7 * reportInvocationUsage for the runAttemptStarted event * improve execution snapshot span debug span start times * Unfriendly IDs * update lockfile * Created a shared determineEngineVersion function * disable unfinished commands * save new cli config to different location, misc fixes * add basic engine version check via current deploy * new run engine will default to node 22 runtime * block some actions for projects on previous run engine * fix worker group tests * fix triggerAndWait test * one typescript version to rule them all * redlock type patch * fix type issues caused by ts-reset * improve cleanup scripts * add missing socket.io dep * fix run notification handler type * fix worker group test again * generate prisma client for e2e tests * remove worker group tests for now * prevent image pull rate limits during unit tests * increase timeout for queue concurrency limit test * generate prisma client for preview release * same node types everywhere * Updated engine readme, removed legacy system notes * use default machine preset from platform package * worker instances plural in schema * disable pnpm update notifications * return worker group details from connect call * add workers admin route * fix heartbeat route return type * move deployment labels to core apps * refactor run controller env schema * Add firstAttemptStartedAt to TaskRun * RunEngine 2.0 batch trigger support (#1581) * Make it clear when BatchTriggerV2Service is used * Copy of BatchTriggerV2Service * WIP batch triggering * Allow blocking a run with multiple waitpoints at once. Made it atomic * Removed unused param * New batch service * Pass through the parentRunId and resumeParentOnCompletion * Use the new batch service, and correct trigger task version * Force V1 engine if using BatchTriggerV2Service, we’ve already done the check at this point * Removed the $transaction and early exit if nothing changed * Adedd a simple batch task to the hello world reference catalog * Fix for batch waits not working * Added parentRunId in a couple more places * Removed waitForBatch log * Added another parentRunId * Expanded the example to include all the different triggers * More changes to blocking to support continuing after idempotent completed runs * Fix for the wrong type when blocking a run * remove @map * optimise worker auth query * add engine version header to core api client requests * remove unique constraint for default group id * consolidate migrations * the first managed worker becomes the global default * Debug events off by default, added an admin toggle to show them * worker group name can't be an empty string * add exec helper to core * move machine resources to core * add pre-dequeue callback to determine max resources * optionally skip dequeue * bump worker package * move worker to core * fix ReadableStream type error * fix another type issue * update a few more tsconfigs * add metadata changes introduced in #1563 * Run Engine 2.0 trigger idempotency (#1613) * Return isCached from the trigger API endpoint * Fix for the wrong type when blocking a run * Render the idempotent run in the inspector * Event repository for idempotency * Debug events off by default, added an admin toggle to show them * triggerAndWait idempotency span * Some improvements to the reference idempotency task * Removed the cached tracing from the SDK * Server-side creating cached span * Improved idempotency test task * Create cached task spans in a better way * Idempotency span support inc batch trigger * Simplified how the spans are done, using more of the existing code * Improved the idempotency test task * Added Waitpoint Batch type, add to TaskRunWaitpoint with order * Pass batch ids through to the run engine when triggering * Added batchIndex * Better batch support in the run engine * Added settings to batch trigger service, before major overhaul * Allow the longer run/batch ids in the filters * Changed how batching works, includes breaking changes in CLI * Removed batch idempotency because it gets put on the runs instead * Added `runs` to the batch.retrieve call/API * Set firstAttemptStartedAt when creating the first attempt * Do nothing when receiving a BATCH waitpoint * Some fixes in the new batch trigger service… mostly just passing missing optional params through * Tweaked the idempotency test task for more situations * Only block with a batch if it’s a batchTriggerAndWait… 🤦♂️ * Added another case to the idempotency test task: multiple of the same idempotencyKey in a single batch * Support for the same run multiple times in the same batch * Small tweaks * Make sure to complete batches, even if they’re not andWait ones * Export RunDuplicateIdempotencyKeyError from the run engine * Latest lockfile * Trigger with a machine (old run engine) * RE2, allow setting machine when triggering * Fix for new glob patterns * add max run count to dequeue from version route * add worker instance name env var and header * queue consumer pre skip callback * poll for more runs after final execution errors * fix dequeue search param schema * add shortcut to debug switch * expose run engine timeouts as env vars * make warm start durations configurable * add optional status to json reply helper * fix preSkip hook, add debug logs * BLOCKED_BY_WAITPOINTS -> SUSPENDED * exit controller when run suspended * check if already replied before http reply * run controller will wait for next run after the current one is suspended * cancel run button shortcut * minimal event repository environment type * fix update metadata call * run suspension and misc fixes wip * change debug shortcut to shift + D * Started work on the Dev supervisor * Formatting * Fix for bad imports * Before rebuilding SSE * Presence updating from the CLI working via SSE * add worker notification debug logs * send run:stop when exiting run phase * skip current snapshot poll on worker notification * add more logs and route to submit run debug logs * add worker and runner ids to snapshots * improve run notification debug logs * add workload debug log route * misc run controller fixes and refactor * prevent parallel execution of critical functions * update bun to 1.2.1 * WIP with dev dequeuing * Method to convert friendlyIds to non-friendly, do nothing with actual ids * Set the engine on BackgroundWorker, lazily upgrade projects to engine V2 * Runs with ttls were getting immediately expired… oops. * Pass the Waiting for deploy reason through, so we have it on the execution snapshots * Fixed the logic for getting the right background worker for a run * Use the correct ID when dequeuing… * determineEngineVersion is now fully functional * Rate limiter ignores the dev endpoints * Retrieving a batch gives you the runIds * Set a unique version for the RE2 BatchTaskRun * add provisional changeset * The start of dev run execution is working * First dev run working * Moved the dev run controller closer to what Nick did with the managed one * export exec output type * Heartbeat fix: don’t heartbeat if _isHeartbeating == false * Dev runs get notifications, some dev bug fixes * Improved logging or dequeuing * We need to dequeue runs from the latest version too, for triggerAndWait * Ported Eric’s validateWorkerManifest with nicer errors * When flattening an idempotency key if part is undefined, return undefined * Dev logging fixes * Remove sigterm listener * Deprecating workers. Don’t specify a BackgroundWorker when dequeuing an environment * Deleted some old files. Renamed “managed” to “deploy” * When a build finishes, always copy the build dir (otherwise the first one gets trampled on by the 2nd) * Dev master queues should work differently * Deleting old workers * Added debounce function to core * Improvement to canceling * WIP on debounce canceling on socket disconnection * Added environment data to execution snapshots * Dev runs that have stalled get “Canceled” with a reason explaining why * Show CLI messaged when a connection to the platform is lost/restored * Fix TriggerTask after merge * Add trigger task v2 max attempts, replace some findUniques * Port the new queue logic to the run engine * More fixes post-merge * We weren’t setting a `retryConfig` up for the tests… it’s now required * Start the Redis worker inside the Run Engine… 🤦♂️ * Trying to make the testcontainers more reliable * Added keyPrefix: "engine:” * Badly placed bracket in trigger task * Better Redis namespacing * Fix for expired run not getting removed from the queue * Don’t create a redis client in the testcontainers, return the redisOptions instead * Cleanup redis client in the run lock tests * Fix for the RunQueue not supporting keyPrefix * Updated more of the RunQueue scripts rebalancing * Trying to make Redis more robust in the tests… * Improved test resiliciency more * Fix for delays (checkpoint check) * Increase the timeout slightly to fix ttl test * Added priority support when triggering * More wip trying to make test containers more reliable * batchTriggerAndWait test is still failing… some wip to try fix it * Fixed redis tests now we’re not providing a client * Separate Redis clients for the run engine worker/queue/runlock * Made the wait for duration test more resilient * Added idempotencyKeyExpiresAt to Waitpoints * Waitpoint timeouts and idempotency expiry * Use finishWaitpoint, removed extra worker job * Added waitpoint idempotency tests * Creating resume tokens is working * Some improvements to the resume tokens * Moved resumeTokens to just be wait functions 🥳 * Delete old RuntimeManagers * Wait for token is working * Better test for the wait tokens * Improved the test task some more * Hide the accessories in the span inspector * WIP on waitpoint inspector * WIP on complete waitpoint form * Span overview panel can be changed based on the entity type * Improved the waitpoint display * WIP on completing waitpoint form * Use the existing CodeBlock for the tip * Style improvements * Complete waitpoint * All waitpoint sidebar variants * Waits now use a pause icon * Durations waits use the API to create/block with a waitpoint, not the runtime * Fix for engine.blockRunWithWaitpoint required org id * Removed old wait code from the run controllers/task run process * Form action for skipping a datetime waitpoint * Move testDockerCheckpoint to a separate core package export (it can’t be bundled on the client) * Fix for glitchy hourglass animation * Completed waitpoints display better * Increase Redis maxRetriesPerRequest to 20 (default) * Completing and skipping waitpoints is working * Remove the database prisma dev command, since we need to use create only now. Updated docs * Added skip timeout, reworked the UI * Tweaked spacing * Added payload limit to waitpoint token completion from dashboard * Test idempotency works on wait.for and wait.until * Moved the worker-actions to /engine/ from /api/ * Moved dev engine endpoints to /engine/ from /api/ * Separate /engine/ rate limiter * Added parallel wait prevention, it’s working for duration waits but not well for triggerAndWait yet * WIP post-merge conflicts * Set taskEventStore column in the new engine * Remove duplicate keys * Post-merge fixes * Fix for span merge layout * Use executedAt instead of firstAttemptStartedAt --------- Co-authored-by: Matt Aitken <matt@mattaitken.com>
This commit is contained in:
nicktrn
@@ -0,0 +1,10 @@
|
||||
---
|
||||
"@trigger.dev/react-hooks": patch
|
||||
"@trigger.dev/sdk": patch
|
||||
"trigger.dev": patch
|
||||
"@trigger.dev/build": patch
|
||||
"@trigger.dev/core": patch
|
||||
"@trigger.dev/rsc": patch
|
||||
---
|
||||
|
||||
Run Engine 2.0 (alpha)
|
||||
@@ -1,7 +1,7 @@
|
||||
{
|
||||
"compilerOptions": {
|
||||
"target": "es2022",
|
||||
"lib": ["ES2022", "DOM", "DOM.Iterable"],
|
||||
"lib": ["ES2022", "DOM", "DOM.Iterable", "DOM.AsyncIterable"],
|
||||
"module": "NodeNext",
|
||||
"moduleResolution": "NodeNext",
|
||||
"moduleDetection": "force",
|
||||
|
||||
@@ -38,6 +38,9 @@ jobs:
|
||||
- name: 📥 Download deps
|
||||
run: pnpm install --frozen-lockfile --filter trigger.dev...
|
||||
|
||||
- name: 📀 Generate Prisma Client
|
||||
run: pnpm run generate
|
||||
|
||||
- name: 🔧 Build v3 cli monorepo dependencies
|
||||
run: pnpm run build --filter trigger.dev^...
|
||||
|
||||
|
||||
@@ -24,6 +24,13 @@ jobs:
|
||||
node-version: 20.11.1
|
||||
cache: "pnpm"
|
||||
|
||||
# ..to avoid rate limits when pulling images
|
||||
- name: 🐳 Login to DockerHub
|
||||
uses: docker/login-action@v3
|
||||
with:
|
||||
username: ${{ secrets.DOCKERHUB_USERNAME }}
|
||||
password: ${{ secrets.DOCKERHUB_TOKEN }}
|
||||
|
||||
- name: 📥 Download deps
|
||||
run: pnpm install --frozen-lockfile
|
||||
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
link-workspace-packages=false
|
||||
public-hoist-pattern[]=*prisma*
|
||||
prefer-workspace-packages=true
|
||||
prefer-workspace-packages=true
|
||||
update-notifier=false
|
||||
Vendored
+8
@@ -133,6 +133,14 @@
|
||||
"command": "pnpm exec trigger dev",
|
||||
"cwd": "${workspaceFolder}/references/hello-world",
|
||||
"sourceMaps": true
|
||||
},
|
||||
{
|
||||
"type": "node-terminal",
|
||||
"request": "launch",
|
||||
"name": "Debug RunEngine tests",
|
||||
"command": "pnpm run test --filter @internal/run-engine",
|
||||
"cwd": "${workspaceFolder}",
|
||||
"sourceMaps": true
|
||||
}
|
||||
]
|
||||
}
|
||||
|
||||
+11
-3
@@ -230,13 +230,21 @@ pnpm run db:studio
|
||||
cd packages/database
|
||||
```
|
||||
|
||||
3. Create and apply the migrations
|
||||
3. Create a migration
|
||||
|
||||
```
|
||||
pnpm run db:migrate:dev
|
||||
pnpm run db:migrate:dev:create
|
||||
```
|
||||
|
||||
This creates a migration file and executes the migrations against your database and applies changes to the database schema(s)
|
||||
This creates a migration file. Check the migration file does only what you want. If you're adding any database indexes they must use `CONCURRENTLY`, otherwise they'll lock the table when executed.
|
||||
|
||||
4. Run the migration.
|
||||
|
||||
```
|
||||
pnpm run db:migrate:deploy
|
||||
pnpm run generate
|
||||
```
|
||||
This executes the migrations against your database and applies changes to the database schema(s), and then regenerates the Prisma client.
|
||||
|
||||
4. Commit generated migrations as well as changes to the schema.prisma file
|
||||
5. If you're using VSCode you may need to restart the Typescript server in the webapp to get updated type inference. Open a TypeScript file, then open the Command Palette (View > Command Palette) and run `TypeScript: Restart TS server`.
|
||||
|
||||
@@ -23,10 +23,8 @@
|
||||
"tinyexec": "^0.3.0"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@types/node": "^18",
|
||||
"dotenv": "^16.4.2",
|
||||
"esbuild": "^0.19.11",
|
||||
"tsx": "^4.7.0",
|
||||
"typescript": "^5.3.3"
|
||||
"tsx": "^4.7.0"
|
||||
}
|
||||
}
|
||||
@@ -1,5 +1,5 @@
|
||||
import { ExponentialBackoff } from "@trigger.dev/core/v3/apps";
|
||||
import { testDockerCheckpoint } from "@trigger.dev/core/v3/apps";
|
||||
import { testDockerCheckpoint } from "@trigger.dev/core/v3/checkpoints";
|
||||
import { nanoid } from "nanoid";
|
||||
import fs from "node:fs/promises";
|
||||
import { ChaosMonkey } from "./chaosMonkey";
|
||||
|
||||
@@ -1,8 +1,6 @@
|
||||
{
|
||||
"include": ["./src/**/*.ts"],
|
||||
"exclude": ["node_modules"],
|
||||
"compilerOptions": {
|
||||
"target": "es2016",
|
||||
"target": "es2018",
|
||||
"module": "commonjs",
|
||||
"esModuleInterop": true,
|
||||
"resolveJsonModule": true,
|
||||
|
||||
@@ -20,10 +20,8 @@
|
||||
"execa": "^8.0.1"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@types/node": "^18.19.8",
|
||||
"dotenv": "^16.4.2",
|
||||
"esbuild": "^0.19.11",
|
||||
"tsx": "^4.7.0",
|
||||
"typescript": "^5.3.3"
|
||||
"tsx": "^4.7.0"
|
||||
}
|
||||
}
|
||||
@@ -7,7 +7,8 @@ import {
|
||||
TaskOperationsRestoreOptions,
|
||||
} from "@trigger.dev/core/v3/apps";
|
||||
import { SimpleLogger } from "@trigger.dev/core/v3/apps";
|
||||
import { isExecaChildProcess, testDockerCheckpoint } from "@trigger.dev/core/v3/apps";
|
||||
import { isExecaChildProcess } from "@trigger.dev/core/v3/apps";
|
||||
import { testDockerCheckpoint } from "@trigger.dev/core/v3/checkpoints";
|
||||
import { setTimeout } from "node:timers/promises";
|
||||
import { PostStartCauses, PreStopCauses } from "@trigger.dev/core/v3";
|
||||
|
||||
|
||||
@@ -23,7 +23,6 @@
|
||||
"devDependencies": {
|
||||
"dotenv": "^16.4.2",
|
||||
"esbuild": "^0.19.11",
|
||||
"tsx": "^4.7.0",
|
||||
"typescript": "^5.3.3"
|
||||
"tsx": "^4.7.0"
|
||||
}
|
||||
}
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"compilerOptions": {
|
||||
"target": "es2016",
|
||||
"target": "es2018",
|
||||
"module": "commonjs",
|
||||
"esModuleInterop": true,
|
||||
"forceConsistentCasingInFileNames": true,
|
||||
|
||||
@@ -9,7 +9,6 @@
|
||||
},
|
||||
"devDependencies": {
|
||||
"@cloudflare/workers-types": "^4.20240512.0",
|
||||
"typescript": "^5.0.4",
|
||||
"wrangler": "^3.57.1"
|
||||
},
|
||||
"dependencies": {
|
||||
|
||||
@@ -0,0 +1,27 @@
|
||||
import { useAnimate } from "framer-motion";
|
||||
import { HourglassIcon } from "lucide-react";
|
||||
import { useEffect } from "react";
|
||||
|
||||
export function AnimatedHourglassIcon({
|
||||
className,
|
||||
delay,
|
||||
}: {
|
||||
className?: string;
|
||||
delay?: number;
|
||||
}) {
|
||||
const [scope, animate] = useAnimate();
|
||||
|
||||
useEffect(() => {
|
||||
animate(
|
||||
[
|
||||
[scope.current, { rotate: 0 }, { duration: 0.7 }],
|
||||
[scope.current, { rotate: 180 }, { duration: 0.3 }],
|
||||
[scope.current, { rotate: 180 }, { duration: 0.7 }],
|
||||
[scope.current, { rotate: 360 }, { duration: 0.3 }],
|
||||
],
|
||||
{ repeat: Infinity, delay }
|
||||
);
|
||||
}, []);
|
||||
|
||||
return <HourglassIcon ref={scope} className={className} />;
|
||||
}
|
||||
@@ -0,0 +1,12 @@
|
||||
export function PauseIcon({ className }: { className?: string }) {
|
||||
return (
|
||||
<svg className={className} viewBox="0 0 20 20" fill="none" xmlns="http://www.w3.org/2000/svg">
|
||||
<path
|
||||
fillRule="evenodd"
|
||||
clipRule="evenodd"
|
||||
d="M0 10C0 4.47715 4.47715 0 10 0C15.5228 0 20 4.47715 20 10C20 15.5228 15.5228 20 10 20C4.47715 20 0 15.5228 0 10ZM6.5 6C6.5 5.44772 6.94772 5 7.5 5C8.05229 5 8.5 5.44772 8.5 6V14C8.5 14.5523 8.05229 15 7.5 15C6.94772 15 6.5 14.5523 6.5 14V6ZM12.5 5C11.9477 5 11.5 5.44772 11.5 6V14C11.5 14.5523 11.9477 15 12.5 15C13.0523 15 13.5 14.5523 13.5 14V6C13.5 5.44772 13.0523 5 12.5 5Z"
|
||||
fill="currentColor"
|
||||
/>
|
||||
</svg>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,49 @@
|
||||
export function TaskCachedIcon({ className }: { className?: string }) {
|
||||
return (
|
||||
<svg
|
||||
className={className}
|
||||
width="16"
|
||||
height="16"
|
||||
viewBox="0 0 16 16"
|
||||
fill="none"
|
||||
xmlns="http://www.w3.org/2000/svg"
|
||||
>
|
||||
<g clipPath="url(#clip0_15584_76102)">
|
||||
<path
|
||||
d="M0.5 3.5L0.5 2.5C0.5 1.39543 1.39543 0.5 2.5 0.5H3.5"
|
||||
stroke="#3B82F6"
|
||||
strokeLinecap="square"
|
||||
strokeLinejoin="round"
|
||||
/>
|
||||
<path
|
||||
d="M15.5 12.5L15.5 13.5C15.5 14.6046 14.6046 15.5 13.5 15.5L12.5 15.5"
|
||||
stroke="#3B82F6"
|
||||
strokeLinecap="square"
|
||||
strokeLinejoin="round"
|
||||
/>
|
||||
<path
|
||||
d="M12.5 0.5L13.5 0.5C14.6046 0.5 15.5 1.39543 15.5 2.5L15.5 3.5"
|
||||
stroke="#3B82F6"
|
||||
strokeLinecap="square"
|
||||
strokeLinejoin="round"
|
||||
/>
|
||||
<path
|
||||
d="M3.5 15.5L2.5 15.5C1.39543 15.5 0.5 14.6046 0.5 13.5L0.5 12.5"
|
||||
stroke="#3B82F6"
|
||||
strokeLinecap="square"
|
||||
strokeLinejoin="round"
|
||||
/>
|
||||
<path d="M11.1799 4.19V5.598H8.8479V12H7.1649V5.598H4.8219V4.19H11.1799Z" fill="#3B82F6" />
|
||||
<line x1="6" y1="15.5" x2="10" y2="15.5" stroke="#3B82F6" />
|
||||
<line x1="6" y1="0.5" x2="10" y2="0.5" stroke="#3B82F6" />
|
||||
<line x1="15.5" y1="6" x2="15.5" y2="10" stroke="#3B82F6" />
|
||||
<line x1="0.5" y1="6" x2="0.5" y2="10" stroke="#3B82F6" />
|
||||
</g>
|
||||
<defs>
|
||||
<clipPath id="clip0_15584_76102">
|
||||
<rect width="16" height="16" fill="white" />
|
||||
</clipPath>
|
||||
</defs>
|
||||
</svg>
|
||||
);
|
||||
}
|
||||
@@ -53,7 +53,7 @@ type CodeBlockProps = {
|
||||
fileName?: string;
|
||||
|
||||
/** title text for the Title row */
|
||||
rowTitle?: string;
|
||||
rowTitle?: ReactNode;
|
||||
|
||||
/** Whether to show the open in modal button */
|
||||
showOpenInModal?: boolean;
|
||||
|
||||
@@ -3,6 +3,7 @@
|
||||
import * as React from "react";
|
||||
import * as SwitchPrimitives from "@radix-ui/react-switch";
|
||||
import { cn } from "~/utils/cn";
|
||||
import { ShortcutDefinition, useShortcutKeys } from "~/hooks/useShortcutKeys";
|
||||
|
||||
const variations = {
|
||||
large: {
|
||||
@@ -23,14 +24,34 @@ const variations = {
|
||||
type SwitchProps = React.ComponentPropsWithoutRef<typeof SwitchPrimitives.Root> & {
|
||||
label?: React.ReactNode;
|
||||
variant: keyof typeof variations;
|
||||
shortcut?: ShortcutDefinition;
|
||||
};
|
||||
|
||||
export const Switch = React.forwardRef<React.ElementRef<typeof SwitchPrimitives.Root>, SwitchProps>(
|
||||
({ className, variant, label, ...props }, ref) => {
|
||||
const innerRef = React.useRef<HTMLButtonElement>(null);
|
||||
React.useImperativeHandle(ref, () => innerRef.current as HTMLButtonElement);
|
||||
|
||||
const { container, root, thumb, text } = variations[variant];
|
||||
|
||||
if (props.shortcut) {
|
||||
useShortcutKeys({
|
||||
shortcut: props.shortcut,
|
||||
action: () => {
|
||||
if (innerRef.current) {
|
||||
innerRef.current.click();
|
||||
}
|
||||
},
|
||||
disabled: props.disabled,
|
||||
});
|
||||
}
|
||||
|
||||
return (
|
||||
<SwitchPrimitives.Root className={cn("group", container, className)} {...props} ref={ref}>
|
||||
<SwitchPrimitives.Root
|
||||
className={cn("group", container, className)}
|
||||
{...props}
|
||||
ref={innerRef}
|
||||
>
|
||||
{label ? (
|
||||
<label className={cn("whitespace-nowrap", text)}>
|
||||
{typeof label === "string" ? <span>{label}</span> : label}
|
||||
|
||||
@@ -359,8 +359,8 @@ function BatchIdDropdown({
|
||||
if (batchId) {
|
||||
if (!batchId.startsWith("batch_")) {
|
||||
error = "Batch IDs start with 'batch_'";
|
||||
} else if (batchId.length !== 27) {
|
||||
error = "Batch IDs are 27 characters long";
|
||||
} else if (batchId.length !== 27 && batchId.length !== 31) {
|
||||
error = "Batch IDs are 27/32 characters long";
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -66,3 +66,37 @@ export function LiveCountUp({
|
||||
</>
|
||||
);
|
||||
}
|
||||
|
||||
export function LiveCountdown({
|
||||
endTime,
|
||||
updateInterval = 100,
|
||||
}: {
|
||||
endTime: Date;
|
||||
updateInterval?: number;
|
||||
}) {
|
||||
const [now, setNow] = useState<Date>();
|
||||
|
||||
useEffect(() => {
|
||||
const interval = setInterval(() => {
|
||||
const date = new Date();
|
||||
setNow(date);
|
||||
|
||||
if (date > endTime) {
|
||||
clearInterval(interval);
|
||||
}
|
||||
}, updateInterval);
|
||||
|
||||
return () => clearInterval(interval);
|
||||
}, [endTime]);
|
||||
|
||||
return (
|
||||
<>
|
||||
{formatDuration(now, endTime, {
|
||||
style: "short",
|
||||
maxDecimalPoints: 0,
|
||||
units: ["d", "h", "m", "s"],
|
||||
maxUnits: 4,
|
||||
})}
|
||||
</>
|
||||
);
|
||||
}
|
||||
|
||||
@@ -763,8 +763,8 @@ function RunIdDropdown({
|
||||
if (runId) {
|
||||
if (!runId.startsWith("run_")) {
|
||||
error = "Run IDs start with 'run_'";
|
||||
} else if (runId.length !== 25) {
|
||||
error = "Run IDs are 25 characters long";
|
||||
} else if (runId.length !== 25 && runId.length !== 29) {
|
||||
error = "Run IDs are 25/30 characters long";
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -7,8 +7,10 @@ import {
|
||||
} from "@heroicons/react/20/solid";
|
||||
import { AttemptIcon } from "~/assets/icons/AttemptIcon";
|
||||
import { TaskIcon } from "~/assets/icons/TaskIcon";
|
||||
import { TaskCachedIcon } from "~/assets/icons/TaskCachedIcon";
|
||||
import { NamedIcon } from "~/components/primitives/NamedIcon";
|
||||
import { cn } from "~/utils/cn";
|
||||
import { PauseIcon } from "~/assets/icons/PauseIcon";
|
||||
|
||||
type TaskIconProps = {
|
||||
name: string | undefined;
|
||||
@@ -41,12 +43,14 @@ export function RunIcon({ name, className, spanName }: TaskIconProps) {
|
||||
switch (name) {
|
||||
case "task":
|
||||
return <TaskIcon className={cn(className, "text-blue-500")} />;
|
||||
case "task-cached":
|
||||
return <TaskCachedIcon className={cn(className, "text-blue-500")} />;
|
||||
case "scheduled":
|
||||
return <ClockIcon className={cn(className, "text-sun-500")} />;
|
||||
case "attempt":
|
||||
return <AttemptIcon className={cn(className, "text-text-dimmed")} />;
|
||||
case "wait":
|
||||
return <ClockIcon className={cn(className, "text-teal-500")} />;
|
||||
return <PauseIcon className={cn(className, "text-teal-500")} />;
|
||||
case "trace":
|
||||
return <Squares2X2Icon className={cn(className, "text-text-dimmed")} />;
|
||||
case "tag":
|
||||
|
||||
@@ -11,13 +11,16 @@ type SpanTitleProps = {
|
||||
level: TaskEventLevel;
|
||||
isPartial: boolean;
|
||||
size: "small" | "large";
|
||||
hideAccessory?: boolean;
|
||||
};
|
||||
|
||||
export function SpanTitle(event: SpanTitleProps) {
|
||||
return (
|
||||
<span className={cn("flex items-center gap-x-2 overflow-x-hidden", eventTextClassName(event))}>
|
||||
<span className="truncate">{event.message}</span>{" "}
|
||||
<SpanAccessory accessory={event.style.accessory} size={event.size} />
|
||||
{!event.hideAccessory && (
|
||||
<SpanAccessory accessory={event.style.accessory} size={event.size} />
|
||||
)}
|
||||
</span>
|
||||
);
|
||||
}
|
||||
|
||||
@@ -1,6 +1,5 @@
|
||||
export const LIVE_ENVIRONMENT = "live";
|
||||
export const DEV_ENVIRONMENT = "development";
|
||||
export const CURRENT_DEPLOYMENT_LABEL = "current";
|
||||
export const MAX_LIVE_PROJECTS = 1;
|
||||
export const DEFAULT_MAX_CONCURRENT_RUNS = 10;
|
||||
export const MAX_CONCURRENT_RUNS_LIMIT = 20;
|
||||
|
||||
@@ -212,7 +212,10 @@ process.on("uncaughtException", (error, origin) => {
|
||||
|
||||
const sqsEventConsumer = singleton("sqsEventConsumer", getSharedSqsEventConsumer);
|
||||
|
||||
singleton("RunEngineEventBusHandlers", registerRunEngineEventBusHandlers);
|
||||
|
||||
export { apiRateLimiter } from "./services/apiRateLimit.server";
|
||||
export { engineRateLimiter } from "./services/engineRateLimit.server";
|
||||
export { socketIo } from "./v3/handleSocketIo.server";
|
||||
export { wss } from "./v3/handleWebsockets.server";
|
||||
export { registryProxy } from "./v3/registryProxy.server";
|
||||
@@ -221,6 +224,7 @@ import { eventLoopMonitor } from "./eventLoopMonitor.server";
|
||||
import { env } from "./env.server";
|
||||
import { logger } from "./services/logger.server";
|
||||
import { Prisma } from "./db.server";
|
||||
import { registerRunEngineEventBusHandlers } from "./v3/runEngineHandlers.server";
|
||||
|
||||
if (env.EVENT_LOOP_MONITOR_ENABLED === "1") {
|
||||
eventLoopMonitor.enable();
|
||||
|
||||
@@ -259,6 +259,8 @@ const EnvironmentSchema = z.object({
|
||||
SHARED_QUEUE_CONSUMER_EMIT_RESUME_DEPENDENCY_TIMEOUT_MS: z.coerce.number().int().default(1000),
|
||||
SHARED_QUEUE_CONSUMER_RESOLVE_PAYLOADS_BATCH_SIZE: z.coerce.number().int().default(25),
|
||||
|
||||
MANAGED_WORKER_SECRET: z.string().default("managed-secret"),
|
||||
|
||||
// Development OTEL environment variables
|
||||
DEV_OTEL_EXPORTER_OTLP_ENDPOINT: z.string().optional(),
|
||||
// If this is set to 1, then the below variables are used to configure the batch processor for spans and logs
|
||||
@@ -408,6 +410,167 @@ const EnvironmentSchema = z.object({
|
||||
BATCH_METADATA_OPERATIONS_FLUSH_ENABLED: z.string().default("1"),
|
||||
BATCH_METADATA_OPERATIONS_FLUSH_LOGGING_ENABLED: z.string().default("1"),
|
||||
|
||||
// Run Engine 2.0
|
||||
RUN_ENGINE_WORKER_COUNT: z.coerce.number().int().default(4),
|
||||
RUN_ENGINE_TASKS_PER_WORKER: z.coerce.number().int().default(10),
|
||||
RUN_ENGINE_WORKER_POLL_INTERVAL: z.coerce.number().int().default(100),
|
||||
RUN_ENGINE_TIMEOUT_PENDING_EXECUTING: z.coerce.number().int().default(60_000),
|
||||
RUN_ENGINE_TIMEOUT_PENDING_CANCEL: z.coerce.number().int().default(60_000),
|
||||
RUN_ENGINE_TIMEOUT_EXECUTING: z.coerce.number().int().default(60_000),
|
||||
RUN_ENGINE_TIMEOUT_EXECUTING_WITH_WAITPOINTS: z.coerce.number().int().default(60_000),
|
||||
RUN_ENGINE_DEBUG_WORKER_NOTIFICATIONS: z.coerce.boolean().default(false),
|
||||
|
||||
RUN_ENGINE_WORKER_REDIS_HOST: z
|
||||
.string()
|
||||
.optional()
|
||||
.transform((v) => v ?? process.env.REDIS_HOST),
|
||||
RUN_ENGINE_WORKER_REDIS_READER_HOST: z
|
||||
.string()
|
||||
.optional()
|
||||
.transform((v) => v ?? process.env.REDIS_READER_HOST),
|
||||
RUN_ENGINE_WORKER_REDIS_READER_PORT: z.coerce
|
||||
.number()
|
||||
.optional()
|
||||
.transform(
|
||||
(v) =>
|
||||
v ?? (process.env.REDIS_READER_PORT ? parseInt(process.env.REDIS_READER_PORT) : undefined)
|
||||
),
|
||||
RUN_ENGINE_WORKER_REDIS_PORT: z.coerce
|
||||
.number()
|
||||
.optional()
|
||||
.transform((v) => v ?? (process.env.REDIS_PORT ? parseInt(process.env.REDIS_PORT) : undefined)),
|
||||
RUN_ENGINE_WORKER_REDIS_USERNAME: z
|
||||
.string()
|
||||
.optional()
|
||||
.transform((v) => v ?? process.env.REDIS_USERNAME),
|
||||
RUN_ENGINE_WORKER_REDIS_PASSWORD: z
|
||||
.string()
|
||||
.optional()
|
||||
.transform((v) => v ?? process.env.REDIS_PASSWORD),
|
||||
RUN_ENGINE_WORKER_REDIS_TLS_DISABLED: z
|
||||
.string()
|
||||
.default(process.env.REDIS_TLS_DISABLED ?? "false"),
|
||||
|
||||
RUN_ENGINE_RUN_QUEUE_REDIS_HOST: z
|
||||
.string()
|
||||
.optional()
|
||||
.transform((v) => v ?? process.env.REDIS_HOST),
|
||||
RUN_ENGINE_RUN_QUEUE_REDIS_READER_HOST: z
|
||||
.string()
|
||||
.optional()
|
||||
.transform((v) => v ?? process.env.REDIS_READER_HOST),
|
||||
RUN_ENGINE_RUN_QUEUE_REDIS_READER_PORT: z.coerce
|
||||
.number()
|
||||
.optional()
|
||||
.transform(
|
||||
(v) =>
|
||||
v ?? (process.env.REDIS_READER_PORT ? parseInt(process.env.REDIS_READER_PORT) : undefined)
|
||||
),
|
||||
RUN_ENGINE_RUN_QUEUE_REDIS_PORT: z.coerce
|
||||
.number()
|
||||
.optional()
|
||||
.transform((v) => v ?? (process.env.REDIS_PORT ? parseInt(process.env.REDIS_PORT) : undefined)),
|
||||
RUN_ENGINE_RUN_QUEUE_REDIS_USERNAME: z
|
||||
.string()
|
||||
.optional()
|
||||
.transform((v) => v ?? process.env.REDIS_USERNAME),
|
||||
RUN_ENGINE_RUN_QUEUE_REDIS_PASSWORD: z
|
||||
.string()
|
||||
.optional()
|
||||
.transform((v) => v ?? process.env.REDIS_PASSWORD),
|
||||
RUN_ENGINE_RUN_QUEUE_REDIS_TLS_DISABLED: z
|
||||
.string()
|
||||
.default(process.env.REDIS_TLS_DISABLED ?? "false"),
|
||||
|
||||
RUN_ENGINE_RUN_LOCK_REDIS_HOST: z
|
||||
.string()
|
||||
.optional()
|
||||
.transform((v) => v ?? process.env.REDIS_HOST),
|
||||
RUN_ENGINE_RUN_LOCK_REDIS_READER_HOST: z
|
||||
.string()
|
||||
.optional()
|
||||
.transform((v) => v ?? process.env.REDIS_READER_HOST),
|
||||
RUN_ENGINE_RUN_LOCK_REDIS_READER_PORT: z.coerce
|
||||
.number()
|
||||
.optional()
|
||||
.transform(
|
||||
(v) =>
|
||||
v ?? (process.env.REDIS_READER_PORT ? parseInt(process.env.REDIS_READER_PORT) : undefined)
|
||||
),
|
||||
RUN_ENGINE_RUN_LOCK_REDIS_PORT: z.coerce
|
||||
.number()
|
||||
.optional()
|
||||
.transform((v) => v ?? (process.env.REDIS_PORT ? parseInt(process.env.REDIS_PORT) : undefined)),
|
||||
RUN_ENGINE_RUN_LOCK_REDIS_USERNAME: z
|
||||
.string()
|
||||
.optional()
|
||||
.transform((v) => v ?? process.env.REDIS_USERNAME),
|
||||
RUN_ENGINE_RUN_LOCK_REDIS_PASSWORD: z
|
||||
.string()
|
||||
.optional()
|
||||
.transform((v) => v ?? process.env.REDIS_PASSWORD),
|
||||
RUN_ENGINE_RUN_LOCK_REDIS_TLS_DISABLED: z
|
||||
.string()
|
||||
.default(process.env.REDIS_TLS_DISABLED ?? "false"),
|
||||
|
||||
RUN_ENGINE_DEV_PRESENCE_REDIS_HOST: z
|
||||
.string()
|
||||
.optional()
|
||||
.transform((v) => v ?? process.env.REDIS_HOST),
|
||||
RUN_ENGINE_DEV_PRESENCE_REDIS_READER_HOST: z
|
||||
.string()
|
||||
.optional()
|
||||
.transform((v) => v ?? process.env.REDIS_READER_HOST),
|
||||
RUN_ENGINE_DEV_PRESENCE_REDIS_READER_PORT: z.coerce
|
||||
.number()
|
||||
.optional()
|
||||
.transform(
|
||||
(v) =>
|
||||
v ?? (process.env.REDIS_READER_PORT ? parseInt(process.env.REDIS_READER_PORT) : undefined)
|
||||
),
|
||||
RUN_ENGINE_DEV_PRESENCE_REDIS_PORT: z.coerce
|
||||
.number()
|
||||
.optional()
|
||||
.transform((v) => v ?? (process.env.REDIS_PORT ? parseInt(process.env.REDIS_PORT) : undefined)),
|
||||
RUN_ENGINE_DEV_PRESENCE_REDIS_USERNAME: z
|
||||
.string()
|
||||
.optional()
|
||||
.transform((v) => v ?? process.env.REDIS_USERNAME),
|
||||
RUN_ENGINE_DEV_PRESENCE_REDIS_PASSWORD: z
|
||||
.string()
|
||||
.optional()
|
||||
.transform((v) => v ?? process.env.REDIS_PASSWORD),
|
||||
RUN_ENGINE_DEV_PRESENCE_REDIS_TLS_DISABLED: z
|
||||
.string()
|
||||
.default(process.env.REDIS_TLS_DISABLED ?? "false"),
|
||||
|
||||
//API Rate limiting
|
||||
/**
|
||||
* @example "60s"
|
||||
* @example "1m"
|
||||
* @example "1h"
|
||||
* @example "1d"
|
||||
* @example "1000ms"
|
||||
* @example "1000s"
|
||||
*/
|
||||
RUN_ENGINE_RATE_LIMIT_REFILL_INTERVAL: z.string().default("10s"), // refill 250 tokens every 10 seconds
|
||||
RUN_ENGINE_RATE_LIMIT_MAX: z.coerce.number().int().default(1200), // allow bursts of 750 requests
|
||||
RUN_ENGINE_RATE_LIMIT_REFILL_RATE: z.coerce.number().int().default(400), // refix 250 tokens every 10 seconds
|
||||
RUN_ENGINE_RATE_LIMIT_REQUEST_LOGS_ENABLED: z.string().default("0"),
|
||||
RUN_ENGINE_RATE_LIMIT_REJECTION_LOGS_ENABLED: z.string().default("1"),
|
||||
RUN_ENGINE_RATE_LIMIT_LIMITER_LOGS_ENABLED: z.string().default("0"),
|
||||
|
||||
/** How long should the presence ttl last */
|
||||
DEV_PRESENCE_TTL_MS: z.coerce.number().int().default(30_000),
|
||||
DEV_PRESENCE_POLL_INTERVAL_MS: z.coerce.number().int().default(5_000),
|
||||
DEV_PRESENCE_RECONNECT_THRESHOLD_MS: z.coerce.number().int().default(2_000),
|
||||
/** How many ms to wait until dequeuing again, if there was a run last time */
|
||||
DEV_DEQUEUE_INTERVAL_WITH_RUN: z.coerce.number().int().default(250),
|
||||
/** How many ms to wait until dequeuing again, if there was no run last time */
|
||||
DEV_DEQUEUE_INTERVAL_WITHOUT_RUN: z.coerce.number().int().default(1_000),
|
||||
/** The max number of runs per API call that we'll dequeue in DEV */
|
||||
DEV_DEQUEUE_MAX_RUNS_PER_PULL: z.coerce.number().int().default(10),
|
||||
|
||||
LEGACY_RUN_ENGINE_WORKER_ENABLED: z.string().default(process.env.WORKER_ENABLED ?? "true"),
|
||||
LEGACY_RUN_ENGINE_WORKER_CONCURRENCY_WORKERS: z.coerce.number().int().default(2),
|
||||
LEGACY_RUN_ENGINE_WORKER_CONCURRENCY_TASKS_PER_WORKER: z.coerce.number().int().default(1),
|
||||
|
||||
@@ -1,10 +1,13 @@
|
||||
import type { Prisma, RuntimeEnvironment } from "@trigger.dev/database";
|
||||
import { AuthenticatedEnvironment } from "@internal/testcontainers";
|
||||
import type { Prisma, PrismaClientOrTransaction, RuntimeEnvironment } from "@trigger.dev/database";
|
||||
import { prisma } from "~/db.server";
|
||||
import { getUsername } from "~/utils/username";
|
||||
|
||||
export type { RuntimeEnvironment };
|
||||
|
||||
export async function findEnvironmentByApiKey(apiKey: string) {
|
||||
export async function findEnvironmentByApiKey(
|
||||
apiKey: string
|
||||
): Promise<AuthenticatedEnvironment | null> {
|
||||
const environment = await prisma.runtimeEnvironment.findFirst({
|
||||
where: {
|
||||
apiKey,
|
||||
@@ -24,7 +27,9 @@ export async function findEnvironmentByApiKey(apiKey: string) {
|
||||
return environment;
|
||||
}
|
||||
|
||||
export async function findEnvironmentByPublicApiKey(apiKey: string) {
|
||||
export async function findEnvironmentByPublicApiKey(
|
||||
apiKey: string
|
||||
): Promise<AuthenticatedEnvironment | null> {
|
||||
const environment = await prisma.runtimeEnvironment.findFirst({
|
||||
where: {
|
||||
pkApiKey: apiKey,
|
||||
@@ -44,7 +49,7 @@ export async function findEnvironmentByPublicApiKey(apiKey: string) {
|
||||
return environment;
|
||||
}
|
||||
|
||||
export async function findEnvironmentById(id: string) {
|
||||
export async function findEnvironmentById(id: string): Promise<AuthenticatedEnvironment | null> {
|
||||
const environment = await prisma.runtimeEnvironment.findFirst({
|
||||
where: {
|
||||
id,
|
||||
@@ -64,6 +69,32 @@ export async function findEnvironmentById(id: string) {
|
||||
return environment;
|
||||
}
|
||||
|
||||
export async function findEnvironmentFromRun(
|
||||
runId: string,
|
||||
tx?: PrismaClientOrTransaction
|
||||
): Promise<AuthenticatedEnvironment | null> {
|
||||
const taskRun = await (tx ?? prisma).taskRun.findFirst({
|
||||
where: {
|
||||
id: runId,
|
||||
},
|
||||
include: {
|
||||
runtimeEnvironment: {
|
||||
include: {
|
||||
project: true,
|
||||
organization: true,
|
||||
orgMember: true,
|
||||
},
|
||||
},
|
||||
},
|
||||
});
|
||||
|
||||
if (!taskRun) {
|
||||
return null;
|
||||
}
|
||||
|
||||
return taskRun?.runtimeEnvironment;
|
||||
}
|
||||
|
||||
export async function createNewSession(environment: RuntimeEnvironment, ipAddress: string) {
|
||||
const session = await prisma.runtimeEnvironmentSession.create({
|
||||
data: {
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
import { WorkerDeploymentStatus } from "@trigger.dev/database";
|
||||
import { WorkerDeploymentStatus, WorkerInstanceGroupType } from "@trigger.dev/database";
|
||||
import { sqlDatabaseSchema, PrismaClient, prisma } from "~/db.server";
|
||||
import { Organization } from "~/models/organization.server";
|
||||
import { Project } from "~/models/project.server";
|
||||
@@ -95,29 +95,31 @@ export class DeploymentListPresenter {
|
||||
userName: string | null;
|
||||
userDisplayName: string | null;
|
||||
userAvatarUrl: string | null;
|
||||
type: WorkerInstanceGroupType;
|
||||
}[]
|
||||
>`
|
||||
SELECT
|
||||
wd."id",
|
||||
wd."shortCode",
|
||||
wd."version",
|
||||
(SELECT COUNT(*) FROM ${sqlDatabaseSchema}."BackgroundWorkerTask" WHERE "BackgroundWorkerTask"."workerId" = wd."workerId") AS "tasksCount",
|
||||
wd."environmentId",
|
||||
wd."status",
|
||||
u."id" AS "userId",
|
||||
u."name" AS "userName",
|
||||
u."displayName" AS "userDisplayName",
|
||||
u."avatarUrl" AS "userAvatarUrl",
|
||||
SELECT
|
||||
wd."id",
|
||||
wd."shortCode",
|
||||
wd."version",
|
||||
(SELECT COUNT(*) FROM ${sqlDatabaseSchema}."BackgroundWorkerTask" WHERE "BackgroundWorkerTask"."workerId" = wd."workerId") AS "tasksCount",
|
||||
wd."environmentId",
|
||||
wd."status",
|
||||
u."id" AS "userId",
|
||||
u."name" AS "userName",
|
||||
u."displayName" AS "userDisplayName",
|
||||
u."avatarUrl" AS "userAvatarUrl",
|
||||
wd."builtAt",
|
||||
wd."deployedAt"
|
||||
FROM
|
||||
wd."deployedAt",
|
||||
wd."type"
|
||||
FROM
|
||||
${sqlDatabaseSchema}."WorkerDeployment" as wd
|
||||
INNER JOIN
|
||||
${sqlDatabaseSchema}."User" as u ON wd."triggeredById" = u."id"
|
||||
WHERE
|
||||
INNER JOIN
|
||||
${sqlDatabaseSchema}."User" as u ON wd."triggeredById" = u."id"
|
||||
WHERE
|
||||
wd."projectId" = ${project.id}
|
||||
ORDER BY
|
||||
string_to_array(wd."version", '.')::int[] DESC
|
||||
ORDER BY
|
||||
string_to_array(wd."version", '.')::int[] DESC
|
||||
LIMIT ${pageSize} OFFSET ${pageSize * (page - 1)};`;
|
||||
|
||||
return {
|
||||
@@ -146,6 +148,7 @@ LIMIT ${pageSize} OFFSET ${pageSize * (page - 1)};`;
|
||||
isCurrent: label?.label === "current",
|
||||
isDeployed: deployment.status === "DEPLOYED",
|
||||
isLatest: page === 1 && index === 0,
|
||||
type: deployment.type,
|
||||
environment: {
|
||||
id: environment.id,
|
||||
type: environment.type,
|
||||
|
||||
@@ -66,6 +66,7 @@ export class DeploymentPresenter {
|
||||
imageReference: true,
|
||||
externalBuildData: true,
|
||||
projectId: true,
|
||||
type: true,
|
||||
environment: {
|
||||
select: {
|
||||
id: true,
|
||||
@@ -152,6 +153,7 @@ export class DeploymentPresenter {
|
||||
organizationId: project.organizationId,
|
||||
errorData: DeploymentPresenter.prepareErrorData(deployment.errorData),
|
||||
isBuilt: !!deployment.builtAt,
|
||||
type: deployment.type,
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
@@ -0,0 +1,16 @@
|
||||
|
||||
const PRESENCE_KEY_PREFIX = "dev-presence:connection:";
|
||||
const PRESENCE_CHANNEL_PREFIX = "dev-presence:updates:";
|
||||
|
||||
export class DevPresenceStream {
|
||||
static getPresenceKey(environmentId: string) {
|
||||
return `${PRESENCE_KEY_PREFIX}${environmentId}`;
|
||||
}
|
||||
|
||||
static getPresenceChannel(environmentId: string) {
|
||||
return `${PRESENCE_CHANNEL_PREFIX}${environmentId}`;
|
||||
}
|
||||
|
||||
//todo create a Redis client for each function call to subscribe
|
||||
//todo you can get the redis options, or there might be a clone function
|
||||
}
|
||||
@@ -39,7 +39,7 @@ export class RunStreamPresenter {
|
||||
traceId: run.traceId,
|
||||
});
|
||||
|
||||
let pinger: NodeJS.Timer | undefined = undefined;
|
||||
let pinger: NodeJS.Timeout | undefined = undefined;
|
||||
|
||||
const { unsubscribe, eventEmitter } = await eventRepository.subscribeToTrace(run.traceId);
|
||||
|
||||
|
||||
@@ -1,7 +1,9 @@
|
||||
import {
|
||||
isWaitpointOutputTimeout,
|
||||
MachinePresetName,
|
||||
parsePacket,
|
||||
prettyPrintPacket,
|
||||
SemanticInternalAttributes,
|
||||
TaskRunError,
|
||||
} from "@trigger.dev/core/v3";
|
||||
import { RUNNING_STATUSES } from "~/components/runs/v3/TaskRunStatus";
|
||||
@@ -9,8 +11,10 @@ import { eventRepository } from "~/v3/eventRepository.server";
|
||||
import { machinePresetFromName } from "~/v3/machinePresets.server";
|
||||
import { FINAL_ATTEMPT_STATUSES, isFailedRunStatus, isFinalRunStatus } from "~/v3/taskStatus";
|
||||
import { BasePresenter } from "./basePresenter.server";
|
||||
import { getMaxDuration } from "~/v3/utils/maxDuration";
|
||||
import { getTaskEventStoreTableForRun } from "~/v3/taskEventStore.server";
|
||||
import { getMaxDuration } from "@trigger.dev/core/v3/apps";
|
||||
import { logger } from "~/services/logger.server";
|
||||
import { getTaskEventStoreTableForRun, TaskEventStoreTable } from "~/v3/taskEventStore.server";
|
||||
import { Pi } from "lucide-react";
|
||||
|
||||
type Result = Awaited<ReturnType<SpanPresenter["call"]>>;
|
||||
export type Span = NonNullable<NonNullable<Result>["span"]>;
|
||||
@@ -40,7 +44,34 @@ export class SpanPresenter extends BasePresenter {
|
||||
throw new Error("Project not found");
|
||||
}
|
||||
|
||||
const run = await this.#getRun(spanId);
|
||||
const parentRun = await this._prisma.taskRun.findFirst({
|
||||
select: {
|
||||
traceId: true,
|
||||
runtimeEnvironmentId: true,
|
||||
taskEventStore: true,
|
||||
createdAt: true,
|
||||
completedAt: true,
|
||||
},
|
||||
where: {
|
||||
friendlyId: runFriendlyId,
|
||||
},
|
||||
});
|
||||
|
||||
if (!parentRun) {
|
||||
return;
|
||||
}
|
||||
|
||||
const { traceId } = parentRun;
|
||||
|
||||
const eventStore = getTaskEventStoreTableForRun(parentRun);
|
||||
|
||||
const run = await this.#getRun({
|
||||
eventStore,
|
||||
traceId,
|
||||
spanId,
|
||||
createdAt: parentRun.createdAt,
|
||||
completedAt: parentRun.completedAt,
|
||||
});
|
||||
if (run) {
|
||||
return {
|
||||
type: "run" as const,
|
||||
@@ -49,7 +80,14 @@ export class SpanPresenter extends BasePresenter {
|
||||
}
|
||||
|
||||
//get the run
|
||||
const span = await this.#getSpan(runFriendlyId, spanId);
|
||||
const span = await this.#getSpan({
|
||||
eventStore,
|
||||
traceId,
|
||||
spanId,
|
||||
environmentId: parentRun.runtimeEnvironmentId,
|
||||
createdAt: parentRun.createdAt,
|
||||
completedAt: parentRun.completedAt,
|
||||
});
|
||||
|
||||
if (!span) {
|
||||
throw new Error("Span not found");
|
||||
@@ -61,10 +99,35 @@ export class SpanPresenter extends BasePresenter {
|
||||
};
|
||||
}
|
||||
|
||||
async #getRun(spanId: string) {
|
||||
async #getRun({
|
||||
eventStore,
|
||||
traceId,
|
||||
spanId,
|
||||
createdAt,
|
||||
completedAt,
|
||||
}: {
|
||||
eventStore: TaskEventStoreTable;
|
||||
traceId: string;
|
||||
spanId: string;
|
||||
createdAt: Date;
|
||||
completedAt: Date | null;
|
||||
}) {
|
||||
const span = await eventRepository.getSpan(
|
||||
eventStore,
|
||||
spanId,
|
||||
traceId,
|
||||
createdAt,
|
||||
completedAt ?? undefined
|
||||
);
|
||||
|
||||
if (!span) {
|
||||
return;
|
||||
}
|
||||
|
||||
const run = await this._replica.taskRun.findFirst({
|
||||
select: {
|
||||
id: true,
|
||||
spanId: true,
|
||||
traceId: true,
|
||||
//metadata
|
||||
number: true,
|
||||
@@ -85,6 +148,12 @@ export class SpanPresenter extends BasePresenter {
|
||||
sdkVersion: true,
|
||||
},
|
||||
},
|
||||
engine: true,
|
||||
masterQueue: true,
|
||||
secondaryMasterQueue: true,
|
||||
error: true,
|
||||
output: true,
|
||||
outputType: true,
|
||||
//status + duration
|
||||
status: true,
|
||||
startedAt: true,
|
||||
@@ -96,6 +165,7 @@ export class SpanPresenter extends BasePresenter {
|
||||
logsDeletedAt: true,
|
||||
//idempotency
|
||||
idempotencyKey: true,
|
||||
idempotencyKeyExpiresAt: true,
|
||||
//delayed
|
||||
delayUntil: true,
|
||||
//ttl
|
||||
@@ -119,9 +189,6 @@ export class SpanPresenter extends BasePresenter {
|
||||
metadata: true,
|
||||
metadataType: true,
|
||||
maxAttempts: true,
|
||||
output: true,
|
||||
outputType: true,
|
||||
error: true,
|
||||
project: {
|
||||
include: {
|
||||
organization: true,
|
||||
@@ -155,9 +222,13 @@ export class SpanPresenter extends BasePresenter {
|
||||
},
|
||||
},
|
||||
},
|
||||
where: {
|
||||
spanId,
|
||||
},
|
||||
where: span.originalRun
|
||||
? {
|
||||
friendlyId: span.originalRun,
|
||||
}
|
||||
: {
|
||||
spanId,
|
||||
},
|
||||
});
|
||||
|
||||
if (!run) {
|
||||
@@ -165,7 +236,6 @@ export class SpanPresenter extends BasePresenter {
|
||||
}
|
||||
|
||||
const isFinished = isFinalRunStatus(run.status);
|
||||
|
||||
const output = !isFinished
|
||||
? undefined
|
||||
: run.outputType === "application/store"
|
||||
@@ -182,6 +252,7 @@ export class SpanPresenter extends BasePresenter {
|
||||
: undefined;
|
||||
|
||||
let error: TaskRunError | undefined = undefined;
|
||||
|
||||
if (run?.error) {
|
||||
const result = TaskRunError.safeParse(run.error);
|
||||
if (result.success) {
|
||||
@@ -262,6 +333,8 @@ export class SpanPresenter extends BasePresenter {
|
||||
sdkVersion: run.lockedToVersion?.sdkVersion,
|
||||
isTest: run.isTest,
|
||||
environmentId: run.runtimeEnvironment.id,
|
||||
idempotencyKey: run.idempotencyKey,
|
||||
idempotencyKeyExpiresAt: run.idempotencyKeyExpiresAt,
|
||||
schedule: await this.resolveSchedule(run.scheduleId ?? undefined),
|
||||
queue: {
|
||||
name: run.queue,
|
||||
@@ -294,6 +367,11 @@ export class SpanPresenter extends BasePresenter {
|
||||
metadata,
|
||||
maxDurationInSeconds: getMaxDuration(run.maxDurationInSeconds),
|
||||
batch: run.batch ? { friendlyId: run.batch.friendlyId } : undefined,
|
||||
engine: run.engine,
|
||||
masterQueue: run.masterQueue,
|
||||
secondaryMasterQueue: run.secondaryMasterQueue,
|
||||
spanId: run.spanId,
|
||||
isCached: !!span.originalRun,
|
||||
};
|
||||
}
|
||||
|
||||
@@ -326,31 +404,28 @@ export class SpanPresenter extends BasePresenter {
|
||||
};
|
||||
}
|
||||
|
||||
async #getSpan(runFriendlyId: string, spanId: string) {
|
||||
const run = await this._prisma.taskRun.findFirst({
|
||||
select: {
|
||||
traceId: true,
|
||||
createdAt: true,
|
||||
completedAt: true,
|
||||
taskEventStore: true,
|
||||
},
|
||||
where: {
|
||||
friendlyId: runFriendlyId,
|
||||
},
|
||||
});
|
||||
|
||||
if (!run) {
|
||||
return;
|
||||
}
|
||||
|
||||
async #getSpan({
|
||||
eventStore,
|
||||
traceId,
|
||||
spanId,
|
||||
environmentId,
|
||||
createdAt,
|
||||
completedAt,
|
||||
}: {
|
||||
traceId: string;
|
||||
spanId: string;
|
||||
environmentId: string;
|
||||
eventStore: TaskEventStoreTable;
|
||||
createdAt: Date;
|
||||
completedAt: Date | null;
|
||||
}) {
|
||||
const span = await eventRepository.getSpan(
|
||||
getTaskEventStoreTableForRun(run),
|
||||
eventStore,
|
||||
spanId,
|
||||
run.traceId,
|
||||
run.createdAt,
|
||||
run.completedAt ?? undefined
|
||||
traceId,
|
||||
createdAt,
|
||||
completedAt ?? undefined
|
||||
);
|
||||
|
||||
if (!span) {
|
||||
return;
|
||||
}
|
||||
@@ -373,12 +448,78 @@ export class SpanPresenter extends BasePresenter {
|
||||
},
|
||||
});
|
||||
|
||||
return {
|
||||
const data = {
|
||||
...span,
|
||||
events: span.events,
|
||||
properties: span.properties ? JSON.stringify(span.properties, null, 2) : undefined,
|
||||
triggeredRuns,
|
||||
showActionBar: span.show?.actions === true,
|
||||
};
|
||||
|
||||
switch (span.entity.type) {
|
||||
case "waitpoint":
|
||||
const waitpoint = await this._replica.waitpoint.findFirst({
|
||||
where: {
|
||||
friendlyId: span.entity.id,
|
||||
},
|
||||
select: {
|
||||
friendlyId: true,
|
||||
type: true,
|
||||
status: true,
|
||||
idempotencyKey: true,
|
||||
userProvidedIdempotencyKey: true,
|
||||
idempotencyKeyExpiresAt: true,
|
||||
output: true,
|
||||
outputType: true,
|
||||
outputIsError: true,
|
||||
completedAfter: true,
|
||||
},
|
||||
});
|
||||
|
||||
if (!waitpoint) {
|
||||
logger.error(`SpanPresenter: Waitpoint not found`, {
|
||||
spanId,
|
||||
waitpointFriendlyId: span.entity.id,
|
||||
});
|
||||
return { ...data, entity: null };
|
||||
}
|
||||
|
||||
const output =
|
||||
waitpoint.outputType === "application/store"
|
||||
? `/resources/packets/${environmentId}/${waitpoint.output}`
|
||||
: typeof waitpoint.output !== "undefined" && waitpoint.output !== null
|
||||
? await prettyPrintPacket(waitpoint.output, waitpoint.outputType ?? undefined)
|
||||
: undefined;
|
||||
|
||||
let isTimeout = false;
|
||||
if (waitpoint.outputIsError && output) {
|
||||
if (isWaitpointOutputTimeout(output)) {
|
||||
isTimeout = true;
|
||||
}
|
||||
}
|
||||
|
||||
return {
|
||||
...data,
|
||||
entity: {
|
||||
type: "waitpoint" as const,
|
||||
object: {
|
||||
friendlyId: waitpoint.friendlyId,
|
||||
type: waitpoint.type,
|
||||
status: waitpoint.status,
|
||||
idempotencyKey: waitpoint.idempotencyKey,
|
||||
userProvidedIdempotencyKey: waitpoint.userProvidedIdempotencyKey,
|
||||
idempotencyKeyExpiresAt: waitpoint.idempotencyKeyExpiresAt,
|
||||
output: output,
|
||||
outputType: waitpoint.outputType,
|
||||
outputIsError: waitpoint.outputIsError,
|
||||
completedAfter: waitpoint.completedAfter,
|
||||
isTimeout,
|
||||
},
|
||||
},
|
||||
};
|
||||
|
||||
default:
|
||||
return { ...data, entity: null };
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -19,8 +19,8 @@ import {
|
||||
import { logger } from "~/services/logger.server";
|
||||
import { BasePresenter } from "./basePresenter.server";
|
||||
import { TaskRunStatus } from "~/database-types";
|
||||
import { CURRENT_DEPLOYMENT_LABEL } from "~/consts";
|
||||
import { concurrencyTracker } from "~/v3/services/taskRunConcurrencyTracker.server";
|
||||
import { CURRENT_DEPLOYMENT_LABEL } from "@trigger.dev/core/v3/apps";
|
||||
|
||||
export type Task = {
|
||||
slug: string;
|
||||
|
||||
@@ -58,7 +58,7 @@ export class TasksStreamPresenter {
|
||||
projectSlug,
|
||||
});
|
||||
|
||||
let pinger: NodeJS.Timer | undefined = undefined;
|
||||
let pinger: NodeJS.Timeout | undefined = undefined;
|
||||
|
||||
const subscriber = await projectPubSub.subscribe(`project:${project.id}:*`);
|
||||
|
||||
|
||||
+5
@@ -29,6 +29,7 @@ import { DeploymentPresenter } from "~/presenters/v3/DeploymentPresenter.server"
|
||||
import { requireUserId } from "~/services/session.server";
|
||||
import { cn } from "~/utils/cn";
|
||||
import { v3DeploymentParams, v3DeploymentsPath } from "~/utils/pathBuilder";
|
||||
import { capitalizeWord } from "~/utils/string";
|
||||
|
||||
export const loader = async ({ request, params }: LoaderFunctionArgs) => {
|
||||
const userId = await requireUserId(request);
|
||||
@@ -164,6 +165,10 @@ export default function Page() {
|
||||
{deployment.cliVersion ? deployment.cliVersion : "–"}
|
||||
</Property.Value>
|
||||
</Property.Item>
|
||||
<Property.Item>
|
||||
<Property.Label>Worker type</Property.Label>
|
||||
<Property.Value>{capitalizeWord(deployment.type)}</Property.Value>
|
||||
</Property.Item>
|
||||
<Property.Item>
|
||||
<Property.Label>Started at</Property.Label>
|
||||
<Property.Value>
|
||||
|
||||
+1
@@ -7,6 +7,7 @@ import {
|
||||
} from "@heroicons/react/20/solid";
|
||||
import { MetaFunction, Outlet, useLocation, useParams } from "@remix-run/react";
|
||||
import { LoaderFunctionArgs } from "@remix-run/server-runtime";
|
||||
import { WorkerInstanceGroupType } from "@trigger.dev/database";
|
||||
import { typedjson, useTypedLoaderData } from "remix-typedjson";
|
||||
import { z } from "zod";
|
||||
import { UserAvatar } from "~/components/UserProfilePhoto";
|
||||
|
||||
+15
-4
@@ -70,7 +70,7 @@ import { useProject } from "~/hooks/useProject";
|
||||
import { useReplaceSearchParams } from "~/hooks/useReplaceSearchParams";
|
||||
import { Shortcut, useShortcutKeys } from "~/hooks/useShortcutKeys";
|
||||
import { useHasAdminAccess, useUser } from "~/hooks/useUser";
|
||||
import { RunPresenter } from "~/presenters/v3/RunPresenter.server";
|
||||
import { Run, RunPresenter } from "~/presenters/v3/RunPresenter.server";
|
||||
import { getImpersonationId } from "~/services/impersonation.server";
|
||||
import { getResizableSnapshot } from "~/services/resizablePanel.server";
|
||||
import { requireUserId } from "~/services/session.server";
|
||||
@@ -225,8 +225,8 @@ export default function Page() {
|
||||
{run.isFinished ? null : (
|
||||
<Dialog key={`cancel-${run.friendlyId}`}>
|
||||
<DialogTrigger asChild>
|
||||
<Button variant="danger/small" LeadingIcon={StopCircleIcon}>
|
||||
Cancel run
|
||||
<Button variant="danger/small" LeadingIcon={StopCircleIcon} shortcut={{ key: "C" }}>
|
||||
Cancel run…
|
||||
</Button>
|
||||
</DialogTrigger>
|
||||
<CancelRunDialog
|
||||
@@ -466,8 +466,10 @@ function TasksTreeView({
|
||||
maximumLiveReloadingSetting,
|
||||
rootRun,
|
||||
}: TasksTreeViewProps) {
|
||||
const isAdmin = useHasAdminAccess();
|
||||
const [filterText, setFilterText] = useState("");
|
||||
const [errorsOnly, setErrorsOnly] = useState(false);
|
||||
const [showDebug, setShowDebug] = useState(false);
|
||||
const [showDurations, setShowDurations] = useState(true);
|
||||
const [scale, setScale] = useState(0);
|
||||
const parentRef = useRef<HTMLDivElement>(null);
|
||||
@@ -487,7 +489,7 @@ function TasksTreeView({
|
||||
scrollToNode,
|
||||
virtualizer,
|
||||
} = useTree({
|
||||
tree: events,
|
||||
tree: showDebug ? events : events.filter((event) => !event.data.isDebug),
|
||||
selectedId,
|
||||
// collapsedIds,
|
||||
onSelectedIdChanged,
|
||||
@@ -512,6 +514,15 @@ function TasksTreeView({
|
||||
<div className="grid h-full grid-rows-[2.5rem_1fr_3.25rem] overflow-hidden">
|
||||
<div className="flex items-center justify-between gap-2 border-b border-grid-dimmed px-2">
|
||||
<SearchField onChange={setFilterText} />
|
||||
{isAdmin && (
|
||||
<Switch
|
||||
variant="small"
|
||||
label="Debug"
|
||||
shortcut={{ modifiers: ["shift"], key: "D" }}
|
||||
checked={showDebug}
|
||||
onCheckedChange={(e) => setShowDebug(e.valueOf())}
|
||||
/>
|
||||
)}
|
||||
<Switch
|
||||
variant="small"
|
||||
label="Errors only"
|
||||
|
||||
+1
-1
@@ -121,7 +121,7 @@ export const action: ActionFunction = async ({ request, params }) => {
|
||||
);
|
||||
}
|
||||
|
||||
logger.error("Failed to start a test run", { error: e });
|
||||
logger.error("Failed to start a test run", { error: e instanceof Error ? e.message : e });
|
||||
|
||||
return redirectBackWithErrorMessage(
|
||||
request,
|
||||
|
||||
@@ -3,6 +3,7 @@ import { z } from "zod";
|
||||
import { prisma } from "~/db.server";
|
||||
import { authenticateApiRequestWithPersonalAccessToken } from "~/services/personalAccessToken.server";
|
||||
import { marqs } from "~/v3/marqs/index.server";
|
||||
import { updateEnvConcurrencyLimits } from "~/v3/runQueue.server";
|
||||
|
||||
const ParamsSchema = z.object({
|
||||
environmentId: z.string(),
|
||||
@@ -60,7 +61,7 @@ export async function action({ request, params }: ActionFunctionArgs) {
|
||||
},
|
||||
});
|
||||
|
||||
await marqs?.updateEnvConcurrencyLimits(environment);
|
||||
await updateEnvConcurrencyLimits(environment);
|
||||
|
||||
return json({ success: true });
|
||||
}
|
||||
|
||||
@@ -3,6 +3,7 @@ import { z } from "zod";
|
||||
import { prisma } from "~/db.server";
|
||||
import { authenticateApiRequestWithPersonalAccessToken } from "~/services/personalAccessToken.server";
|
||||
import { marqs } from "~/v3/marqs/index.server";
|
||||
import { updateEnvConcurrencyLimits } from "~/v3/runQueue.server";
|
||||
|
||||
const ParamsSchema = z.object({
|
||||
organizationId: z.string(),
|
||||
@@ -97,7 +98,7 @@ export async function action({ request, params }: ActionFunctionArgs) {
|
||||
},
|
||||
});
|
||||
|
||||
await marqs?.updateEnvConcurrencyLimits({ ...modifiedEnvironment, organization });
|
||||
await updateEnvConcurrencyLimits({ ...modifiedEnvironment, organization });
|
||||
}
|
||||
|
||||
return json({ success: true });
|
||||
|
||||
@@ -4,6 +4,7 @@ import { prisma } from "~/db.server";
|
||||
import { createEnvironment } from "~/models/organization.server";
|
||||
import { authenticateApiRequestWithPersonalAccessToken } from "~/services/personalAccessToken.server";
|
||||
import { marqs } from "~/v3/marqs/index.server";
|
||||
import { updateEnvConcurrencyLimits } from "~/v3/runQueue.server";
|
||||
|
||||
const ParamsSchema = z.object({
|
||||
organizationId: z.string(),
|
||||
@@ -58,10 +59,10 @@ export async function action({ request, params }: ActionFunctionArgs) {
|
||||
|
||||
if (!stagingEnvironment) {
|
||||
const staging = await createEnvironment(organization, project, "STAGING");
|
||||
await marqs?.updateEnvConcurrencyLimits({ ...staging, organization, project });
|
||||
await updateEnvConcurrencyLimits({ ...staging, organization, project });
|
||||
created++;
|
||||
} else {
|
||||
await marqs?.updateEnvConcurrencyLimits({ ...stagingEnvironment, organization, project });
|
||||
await updateEnvConcurrencyLimits({ ...stagingEnvironment, organization, project });
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,65 @@
|
||||
import { ActionFunctionArgs, json } from "@remix-run/server-runtime";
|
||||
import { z } from "zod";
|
||||
import { prisma } from "~/db.server";
|
||||
import { authenticateApiRequestWithPersonalAccessToken } from "~/services/personalAccessToken.server";
|
||||
import { WorkerGroupService } from "~/v3/services/worker/workerGroupService.server";
|
||||
|
||||
const RequestBodySchema = z.object({
|
||||
name: z.string().optional(),
|
||||
description: z.string().optional(),
|
||||
projectId: z.string().optional(),
|
||||
makeDefault: z.boolean().optional(),
|
||||
});
|
||||
|
||||
export async function action({ request }: ActionFunctionArgs) {
|
||||
// Next authenticate the request
|
||||
const authenticationResult = await authenticateApiRequestWithPersonalAccessToken(request);
|
||||
|
||||
if (!authenticationResult) {
|
||||
return json({ error: "Invalid or Missing API key" }, { status: 401 });
|
||||
}
|
||||
|
||||
const user = await prisma.user.findUnique({
|
||||
where: {
|
||||
id: authenticationResult.userId,
|
||||
},
|
||||
});
|
||||
|
||||
if (!user) {
|
||||
return json({ error: "Invalid or Missing API key" }, { status: 401 });
|
||||
}
|
||||
|
||||
if (!user.admin) {
|
||||
return json({ error: "You must be an admin to perform this action" }, { status: 403 });
|
||||
}
|
||||
|
||||
try {
|
||||
const rawBody = await request.json();
|
||||
const { name, description, projectId, makeDefault } = RequestBodySchema.parse(rawBody ?? {});
|
||||
|
||||
const service = new WorkerGroupService();
|
||||
const { workerGroup, token } = await service.createWorkerGroup({
|
||||
name,
|
||||
description,
|
||||
});
|
||||
|
||||
if (makeDefault && projectId) {
|
||||
await prisma.project.update({
|
||||
where: {
|
||||
id: projectId,
|
||||
},
|
||||
data: {
|
||||
defaultWorkerGroupId: workerGroup.id,
|
||||
engine: "V2",
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
return json({
|
||||
token,
|
||||
workerGroup,
|
||||
});
|
||||
} catch (error) {
|
||||
return json({ error: error instanceof Error ? error.message : error }, { status: 400 });
|
||||
}
|
||||
}
|
||||
@@ -34,6 +34,7 @@ export const loader = createLoaderApiRoute(
|
||||
createdAt: batch.createdAt,
|
||||
updatedAt: batch.updatedAt,
|
||||
runCount: batch.runCount,
|
||||
runs: batch.runIds,
|
||||
});
|
||||
}
|
||||
);
|
||||
|
||||
@@ -0,0 +1,41 @@
|
||||
import { LoaderFunctionArgs, json } from "@remix-run/server-runtime";
|
||||
import { WorkerInstanceGroupType } from "@trigger.dev/database";
|
||||
import { prisma } from "~/db.server";
|
||||
import { authenticateApiRequest } from "~/services/apiAuth.server";
|
||||
import { logger } from "~/services/logger.server";
|
||||
|
||||
export async function loader({ request }: LoaderFunctionArgs) {
|
||||
// Next authenticate the request
|
||||
const authenticationResult = await authenticateApiRequest(request);
|
||||
|
||||
if (!authenticationResult) {
|
||||
logger.info("Invalid or missing api key", { url: request.url });
|
||||
return json({ error: "Invalid or Missing API key" }, { status: 401 });
|
||||
}
|
||||
|
||||
const authenticatedEnv = authenticationResult.environment;
|
||||
|
||||
const deployment = await prisma.workerDeployment.findFirst({
|
||||
where: {
|
||||
type: WorkerInstanceGroupType.UNMANAGED,
|
||||
environmentId: authenticatedEnv.id,
|
||||
},
|
||||
orderBy: {
|
||||
createdAt: "desc",
|
||||
},
|
||||
});
|
||||
|
||||
if (!deployment) {
|
||||
return json({ error: "Deployment not found" }, { status: 404 });
|
||||
}
|
||||
|
||||
return json({
|
||||
id: deployment.friendlyId,
|
||||
status: deployment.status,
|
||||
contentHash: deployment.contentHash,
|
||||
shortCode: deployment.shortCode,
|
||||
version: deployment.version,
|
||||
imageReference: deployment.imageReference,
|
||||
errorData: deployment.errorData,
|
||||
});
|
||||
}
|
||||
@@ -6,6 +6,7 @@ import {
|
||||
import { env } from "~/env.server";
|
||||
import { authenticateApiRequest } from "~/services/apiAuth.server";
|
||||
import { logger } from "~/services/logger.server";
|
||||
import { ServiceValidationError } from "~/v3/services/baseService.server";
|
||||
import { InitializeDeploymentService } from "~/v3/services/initializeDeployment.server";
|
||||
|
||||
export async function action({ request, params }: ActionFunctionArgs) {
|
||||
@@ -33,18 +34,30 @@ export async function action({ request, params }: ActionFunctionArgs) {
|
||||
|
||||
const service = new InitializeDeploymentService();
|
||||
|
||||
const { deployment, imageTag } = await service.call(authenticatedEnv, body.data);
|
||||
try {
|
||||
const { deployment, imageTag } = await service.call(authenticatedEnv, body.data);
|
||||
|
||||
const responseBody: InitializeDeploymentResponseBody = {
|
||||
id: deployment.friendlyId,
|
||||
contentHash: deployment.contentHash,
|
||||
shortCode: deployment.shortCode,
|
||||
version: deployment.version,
|
||||
externalBuildData:
|
||||
deployment.externalBuildData as InitializeDeploymentResponseBody["externalBuildData"],
|
||||
imageTag,
|
||||
registryHost: body.data.registryHost ?? env.DEPLOY_REGISTRY_HOST,
|
||||
};
|
||||
const responseBody: InitializeDeploymentResponseBody = {
|
||||
id: deployment.friendlyId,
|
||||
contentHash: deployment.contentHash,
|
||||
shortCode: deployment.shortCode,
|
||||
version: deployment.version,
|
||||
externalBuildData:
|
||||
deployment.externalBuildData as InitializeDeploymentResponseBody["externalBuildData"],
|
||||
imageTag,
|
||||
registryHost: body.data.registryHost ?? env.DEPLOY_REGISTRY_HOST,
|
||||
};
|
||||
|
||||
return json(responseBody, { status: 200 });
|
||||
return json(responseBody, { status: 200 });
|
||||
} catch (error) {
|
||||
if (error instanceof ServiceValidationError) {
|
||||
return json({ error: error.message }, { status: 400 });
|
||||
} else if (error instanceof Error) {
|
||||
logger.error("Error initializing deployment", { error: error.message });
|
||||
return json({ error: `Internal server error: ${error.message}` }, { status: 500 });
|
||||
} else {
|
||||
logger.error("Error initializing deployment", { error: String(error) });
|
||||
return json({ error: "Internal server error" }, { status: 500 });
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,5 +1,9 @@
|
||||
import { json } from "@remix-run/server-runtime";
|
||||
import { generateJWT as internal_generateJWT, TriggerTaskRequestBody } from "@trigger.dev/core/v3";
|
||||
import {
|
||||
generateJWT as internal_generateJWT,
|
||||
RunEngineVersionSchema,
|
||||
TriggerTaskRequestBody,
|
||||
} from "@trigger.dev/core/v3";
|
||||
import { TaskRun } from "@trigger.dev/database";
|
||||
import { z } from "zod";
|
||||
import { env } from "~/env.server";
|
||||
@@ -21,6 +25,7 @@ export const HeadersSchema = z.object({
|
||||
"x-trigger-span-parent-as-link": z.coerce.number().nullish(),
|
||||
"x-trigger-worker": z.string().nullish(),
|
||||
"x-trigger-client": z.string().nullish(),
|
||||
"x-trigger-engine-version": RunEngineVersionSchema.nullish(),
|
||||
traceparent: z.string().optional(),
|
||||
tracestate: z.string().optional(),
|
||||
});
|
||||
@@ -49,6 +54,7 @@ const { action, loader } = createActionApiRoute(
|
||||
tracestate,
|
||||
"x-trigger-worker": isFromWorker,
|
||||
"x-trigger-client": triggerClient,
|
||||
"x-trigger-engine-version": engineVersion,
|
||||
} = headers;
|
||||
|
||||
const service = new TriggerTaskService();
|
||||
@@ -74,14 +80,20 @@ const { action, loader } = createActionApiRoute(
|
||||
|
||||
const idempotencyKeyExpiresAt = resolveIdempotencyKeyTTL(idempotencyKeyTTL);
|
||||
|
||||
const result = await service.call(params.taskId, authentication.environment, body, {
|
||||
idempotencyKey: idempotencyKey ?? undefined,
|
||||
idempotencyKeyExpiresAt: idempotencyKeyExpiresAt,
|
||||
triggerVersion: triggerVersion ?? undefined,
|
||||
traceContext,
|
||||
spanParentAsLink: spanParentAsLink === 1,
|
||||
oneTimeUseToken,
|
||||
});
|
||||
const result = await service.call(
|
||||
params.taskId,
|
||||
authentication.environment,
|
||||
body,
|
||||
{
|
||||
idempotencyKey: idempotencyKey ?? undefined,
|
||||
idempotencyKeyExpiresAt: idempotencyKeyExpiresAt,
|
||||
triggerVersion: triggerVersion ?? undefined,
|
||||
traceContext,
|
||||
spanParentAsLink: spanParentAsLink === 1,
|
||||
oneTimeUseToken,
|
||||
},
|
||||
engineVersion ?? undefined
|
||||
);
|
||||
|
||||
if (!result) {
|
||||
return json({ error: "Task not found" }, { status: 404 });
|
||||
|
||||
@@ -67,6 +67,7 @@ const { action, loader } = createActionApiRoute(
|
||||
"x-trigger-span-parent-as-link": spanParentAsLink,
|
||||
"x-trigger-worker": isFromWorker,
|
||||
"x-trigger-client": triggerClient,
|
||||
"x-trigger-engine-version": engineVersion,
|
||||
"batch-processing-strategy": batchProcessingStrategy,
|
||||
traceparent,
|
||||
tracestate,
|
||||
|
||||
@@ -0,0 +1,68 @@
|
||||
import { json } from "@remix-run/server-runtime";
|
||||
import {
|
||||
CompleteWaitpointTokenRequestBody,
|
||||
CompleteWaitpointTokenResponseBody,
|
||||
conditionallyExportPacket,
|
||||
stringifyIO,
|
||||
} from "@trigger.dev/core/v3";
|
||||
import { WaitpointId } from "@trigger.dev/core/v3/apps";
|
||||
import { z } from "zod";
|
||||
import { $replica } from "~/db.server";
|
||||
import { env } from "~/env.server";
|
||||
import { logger } from "~/services/logger.server";
|
||||
import { createActionApiRoute } from "~/services/routeBuilders/apiBuilder.server";
|
||||
import { engine } from "~/v3/runEngine.server";
|
||||
|
||||
const { action } = createActionApiRoute(
|
||||
{
|
||||
params: z.object({
|
||||
waitpointFriendlyId: z.string(),
|
||||
}),
|
||||
body: CompleteWaitpointTokenRequestBody,
|
||||
maxContentLength: env.TASK_PAYLOAD_MAXIMUM_SIZE,
|
||||
method: "POST",
|
||||
},
|
||||
async ({ authentication, body, params }) => {
|
||||
// Resume tokens are actually just waitpoints
|
||||
const waitpointId = WaitpointId.toId(params.waitpointFriendlyId);
|
||||
|
||||
try {
|
||||
//check permissions
|
||||
const waitpoint = await $replica.waitpoint.findFirst({
|
||||
where: {
|
||||
id: waitpointId,
|
||||
environmentId: authentication.environment.id,
|
||||
},
|
||||
});
|
||||
|
||||
if (!waitpoint) {
|
||||
throw json({ error: "Waitpoint not found" }, { status: 404 });
|
||||
}
|
||||
|
||||
const stringifiedData = await stringifyIO(body.data);
|
||||
const finalData = await conditionallyExportPacket(
|
||||
stringifiedData,
|
||||
`${waitpointId}/waitpoint/token`
|
||||
);
|
||||
|
||||
const result = await engine.completeWaitpoint({
|
||||
id: waitpointId,
|
||||
output: finalData.data
|
||||
? { type: finalData.dataType, value: finalData.data, isError: false }
|
||||
: undefined,
|
||||
});
|
||||
|
||||
return json<CompleteWaitpointTokenResponseBody>(
|
||||
{
|
||||
success: true,
|
||||
},
|
||||
{ status: 200 }
|
||||
);
|
||||
} catch (error) {
|
||||
logger.error("Failed to complete waitpoint token", { error });
|
||||
throw json({ error: "Failed to complete waitpoint token" }, { status: 500 });
|
||||
}
|
||||
}
|
||||
);
|
||||
|
||||
export { action };
|
||||
@@ -0,0 +1,43 @@
|
||||
import { json } from "@remix-run/server-runtime";
|
||||
import {
|
||||
CreateWaitpointTokenRequestBody,
|
||||
CreateWaitpointTokenResponseBody,
|
||||
} from "@trigger.dev/core/v3";
|
||||
import { WaitpointId } from "@trigger.dev/core/v3/apps";
|
||||
import { createActionApiRoute } from "~/services/routeBuilders/apiBuilder.server";
|
||||
import { parseDelay } from "~/utils/delays";
|
||||
import { resolveIdempotencyKeyTTL } from "~/utils/idempotencyKeys.server";
|
||||
import { engine } from "~/v3/runEngine.server";
|
||||
|
||||
const { action } = createActionApiRoute(
|
||||
{
|
||||
body: CreateWaitpointTokenRequestBody,
|
||||
maxContentLength: 1024 * 10, // 10KB
|
||||
method: "POST",
|
||||
},
|
||||
async ({ authentication, body }) => {
|
||||
const idempotencyKeyExpiresAt = body.idempotencyKeyTTL
|
||||
? resolveIdempotencyKeyTTL(body.idempotencyKeyTTL)
|
||||
: undefined;
|
||||
|
||||
const timeout = await parseDelay(body.timeout);
|
||||
|
||||
const result = await engine.createManualWaitpoint({
|
||||
environmentId: authentication.environment.id,
|
||||
projectId: authentication.environment.projectId,
|
||||
idempotencyKey: body.idempotencyKey,
|
||||
idempotencyKeyExpiresAt,
|
||||
timeout,
|
||||
});
|
||||
|
||||
return json<CreateWaitpointTokenResponseBody>(
|
||||
{
|
||||
id: WaitpointId.toFriendlyId(result.waitpoint.id),
|
||||
isCached: result.isCached,
|
||||
},
|
||||
{ status: 200 }
|
||||
);
|
||||
}
|
||||
);
|
||||
|
||||
export { action };
|
||||
@@ -0,0 +1,73 @@
|
||||
import { json, TypedResponse } from "@remix-run/server-runtime";
|
||||
import {
|
||||
WorkersCreateRequestBody,
|
||||
WorkersCreateResponseBody,
|
||||
WorkersListResponseBody,
|
||||
} from "@trigger.dev/core/v3";
|
||||
import {
|
||||
createActionApiRoute,
|
||||
createLoaderApiRoute,
|
||||
} from "~/services/routeBuilders/apiBuilder.server";
|
||||
import { WorkerGroupService } from "~/v3/services/worker/workerGroupService.server";
|
||||
|
||||
export const loader = createLoaderApiRoute(
|
||||
{
|
||||
corsStrategy: "all",
|
||||
findResource: async () => 1, // This is a dummy function, we don't need to find a resource
|
||||
},
|
||||
async ({
|
||||
authentication,
|
||||
}): Promise<TypedResponse<WorkersListResponseBody | { error: string }>> => {
|
||||
if (authentication.environment.project.engine !== "V2") {
|
||||
return json({ error: "Not supported for V1 projects" }, { status: 400 });
|
||||
}
|
||||
|
||||
const service = new WorkerGroupService();
|
||||
const workers = await service.listWorkerGroups({
|
||||
projectId: authentication.environment.projectId,
|
||||
});
|
||||
|
||||
return json(
|
||||
workers.map((w) => ({
|
||||
type: w.type,
|
||||
name: w.name,
|
||||
description: w.description,
|
||||
isDefault: w.id === authentication.environment.project.defaultWorkerGroupId,
|
||||
updatedAt: w.updatedAt,
|
||||
}))
|
||||
);
|
||||
}
|
||||
);
|
||||
|
||||
export const { action } = createActionApiRoute(
|
||||
{
|
||||
corsStrategy: "all",
|
||||
body: WorkersCreateRequestBody,
|
||||
},
|
||||
async ({
|
||||
authentication,
|
||||
body,
|
||||
}): Promise<TypedResponse<WorkersCreateResponseBody | { error: string }>> => {
|
||||
if (authentication.environment.project.engine !== "V2") {
|
||||
return json({ error: "Not supported" }, { status: 400 });
|
||||
}
|
||||
|
||||
const service = new WorkerGroupService();
|
||||
const { workerGroup, token } = await service.createWorkerGroup({
|
||||
projectId: authentication.environment.projectId,
|
||||
organizationId: authentication.environment.organizationId,
|
||||
name: body.name,
|
||||
description: body.description,
|
||||
});
|
||||
|
||||
return json({
|
||||
token: {
|
||||
plaintext: token.plaintext,
|
||||
},
|
||||
workerGroup: {
|
||||
name: workerGroup.name,
|
||||
description: workerGroup.description,
|
||||
},
|
||||
});
|
||||
}
|
||||
);
|
||||
@@ -0,0 +1,40 @@
|
||||
import { json } from "@remix-run/server-runtime";
|
||||
import { z } from "zod";
|
||||
import { $replica } from "~/db.server";
|
||||
import { createLoaderApiRoute } from "~/services/routeBuilders/apiBuilder.server";
|
||||
|
||||
const ParamsSchema = z.object({
|
||||
batchId: z.string(),
|
||||
});
|
||||
|
||||
export const loader = createLoaderApiRoute(
|
||||
{
|
||||
params: ParamsSchema,
|
||||
allowJWT: true,
|
||||
corsStrategy: "all",
|
||||
findResource: (params, auth) => {
|
||||
return $replica.batchTaskRun.findFirst({
|
||||
where: {
|
||||
friendlyId: params.batchId,
|
||||
runtimeEnvironmentId: auth.environment.id,
|
||||
},
|
||||
});
|
||||
},
|
||||
authorization: {
|
||||
action: "read",
|
||||
resource: (batch) => ({ batch: batch.friendlyId }),
|
||||
superScopes: ["read:runs", "read:all", "admin"],
|
||||
},
|
||||
},
|
||||
async ({ resource: batch }) => {
|
||||
return json({
|
||||
id: batch.friendlyId,
|
||||
status: batch.status,
|
||||
idempotencyKey: batch.idempotencyKey ?? undefined,
|
||||
createdAt: batch.createdAt,
|
||||
updatedAt: batch.updatedAt,
|
||||
runCount: batch.runCount,
|
||||
runs: batch.runIds,
|
||||
});
|
||||
}
|
||||
);
|
||||
@@ -0,0 +1,152 @@
|
||||
import { json } from "@remix-run/server-runtime";
|
||||
import {
|
||||
BatchTriggerTaskV3RequestBody,
|
||||
BatchTriggerTaskV3Response,
|
||||
generateJWT,
|
||||
} from "@trigger.dev/core/v3";
|
||||
import { env } from "~/env.server";
|
||||
import { AuthenticatedEnvironment, getOneTimeUseToken } from "~/services/apiAuth.server";
|
||||
import { logger } from "~/services/logger.server";
|
||||
import { createActionApiRoute } from "~/services/routeBuilders/apiBuilder.server";
|
||||
import { ServiceValidationError } from "~/v3/services/baseService.server";
|
||||
import { BatchProcessingStrategy } from "~/v3/services/batchTriggerV3.server";
|
||||
import { BatchTriggerV4Service } from "~/v3/services/batchTriggerV4.server";
|
||||
import { OutOfEntitlementError } from "~/v3/services/triggerTask.server";
|
||||
import { HeadersSchema } from "./api.v1.tasks.$taskId.trigger";
|
||||
|
||||
const { action, loader } = createActionApiRoute(
|
||||
{
|
||||
headers: HeadersSchema.extend({
|
||||
"batch-processing-strategy": BatchProcessingStrategy.nullish(),
|
||||
}),
|
||||
body: BatchTriggerTaskV3RequestBody,
|
||||
allowJWT: true,
|
||||
maxContentLength: env.BATCH_TASK_PAYLOAD_MAXIMUM_SIZE,
|
||||
authorization: {
|
||||
action: "batchTrigger",
|
||||
resource: (_, __, ___, body) => ({
|
||||
tasks: Array.from(new Set(body.items.map((i) => i.task))),
|
||||
}),
|
||||
superScopes: ["write:tasks", "admin"],
|
||||
},
|
||||
corsStrategy: "all",
|
||||
},
|
||||
async ({ body, headers, params, authentication }) => {
|
||||
if (!body.items.length) {
|
||||
return json({ error: "Batch cannot be triggered with no items" }, { status: 400 });
|
||||
}
|
||||
|
||||
// Check the there are fewer than MAX_BATCH_V2_TRIGGER_ITEMS items
|
||||
if (body.items.length > env.MAX_BATCH_V2_TRIGGER_ITEMS) {
|
||||
return json(
|
||||
{
|
||||
error: `Batch size of ${body.items.length} is too large. Maximum allowed batch size is ${env.MAX_BATCH_V2_TRIGGER_ITEMS}.`,
|
||||
},
|
||||
{ status: 400 }
|
||||
);
|
||||
}
|
||||
|
||||
const {
|
||||
"trigger-version": triggerVersion,
|
||||
"x-trigger-span-parent-as-link": spanParentAsLink,
|
||||
"x-trigger-worker": isFromWorker,
|
||||
"x-trigger-client": triggerClient,
|
||||
"x-trigger-engine-version": engineVersion,
|
||||
"batch-processing-strategy": batchProcessingStrategy,
|
||||
traceparent,
|
||||
tracestate,
|
||||
} = headers;
|
||||
|
||||
const oneTimeUseToken = await getOneTimeUseToken(authentication);
|
||||
|
||||
logger.debug("Batch trigger request", {
|
||||
triggerVersion,
|
||||
spanParentAsLink,
|
||||
isFromWorker,
|
||||
triggerClient,
|
||||
traceparent,
|
||||
tracestate,
|
||||
batchProcessingStrategy,
|
||||
});
|
||||
|
||||
const traceContext =
|
||||
traceparent && isFromWorker // If the request is from a worker, we should pass the trace context
|
||||
? { traceparent, tracestate }
|
||||
: undefined;
|
||||
|
||||
const service = new BatchTriggerV4Service(batchProcessingStrategy ?? undefined);
|
||||
|
||||
try {
|
||||
const batch = await service.call(authentication.environment, body, {
|
||||
triggerVersion: triggerVersion ?? undefined,
|
||||
traceContext,
|
||||
spanParentAsLink: spanParentAsLink === 1,
|
||||
oneTimeUseToken,
|
||||
});
|
||||
|
||||
const $responseHeaders = await responseHeaders(
|
||||
batch,
|
||||
authentication.environment,
|
||||
triggerClient
|
||||
);
|
||||
|
||||
return json(batch, { status: 202, headers: $responseHeaders });
|
||||
} catch (error) {
|
||||
logger.error("Batch trigger error", {
|
||||
error: {
|
||||
message: (error as Error).message,
|
||||
stack: (error as Error).stack,
|
||||
},
|
||||
});
|
||||
|
||||
if (error instanceof ServiceValidationError) {
|
||||
return json({ error: error.message }, { status: 422 });
|
||||
} else if (error instanceof OutOfEntitlementError) {
|
||||
return json({ error: error.message }, { status: 422 });
|
||||
} else if (error instanceof Error) {
|
||||
return json(
|
||||
{ error: error.message },
|
||||
{ status: 500, headers: { "x-should-retry": "false" } }
|
||||
);
|
||||
}
|
||||
|
||||
return json({ error: "Something went wrong" }, { status: 500 });
|
||||
}
|
||||
}
|
||||
);
|
||||
|
||||
async function responseHeaders(
|
||||
batch: BatchTriggerTaskV3Response,
|
||||
environment: AuthenticatedEnvironment,
|
||||
triggerClient?: string | null
|
||||
): Promise<Record<string, string>> {
|
||||
const claimsHeader = JSON.stringify({
|
||||
sub: environment.id,
|
||||
pub: true,
|
||||
});
|
||||
|
||||
if (triggerClient === "browser") {
|
||||
const claims = {
|
||||
sub: environment.id,
|
||||
pub: true,
|
||||
scopes: [`read:batch:${batch.id}`],
|
||||
};
|
||||
|
||||
const jwt = await generateJWT({
|
||||
secretKey: environment.apiKey,
|
||||
payload: claims,
|
||||
expirationTime: "1h",
|
||||
});
|
||||
|
||||
return {
|
||||
"x-trigger-jwt-claims": claimsHeader,
|
||||
"x-trigger-jwt": jwt,
|
||||
};
|
||||
}
|
||||
|
||||
return {
|
||||
"x-trigger-jwt-claims": claimsHeader,
|
||||
};
|
||||
}
|
||||
|
||||
export { action, loader };
|
||||
@@ -0,0 +1,32 @@
|
||||
import { json, TypedResponse } from "@remix-run/server-runtime";
|
||||
import { DevConfigResponseBody } from "@trigger.dev/core/v3/schemas";
|
||||
import { z } from "zod";
|
||||
import { env } from "~/env.server";
|
||||
import { logger } from "~/services/logger.server";
|
||||
import { createLoaderApiRoute } from "~/services/routeBuilders/apiBuilder.server";
|
||||
|
||||
export const loader = createLoaderApiRoute(
|
||||
{
|
||||
findResource: async () => 1,
|
||||
headers: z.object({
|
||||
"x-forwarded-for": z.string().optional(),
|
||||
}),
|
||||
},
|
||||
async ({ authentication }): Promise<TypedResponse<DevConfigResponseBody>> => {
|
||||
logger.debug("Get dev settings", { environmentId: authentication.environment.id });
|
||||
|
||||
try {
|
||||
return json({
|
||||
environmentId: authentication.environment.id,
|
||||
dequeueIntervalWithRun: env.DEV_DEQUEUE_INTERVAL_WITH_RUN,
|
||||
dequeueIntervalWithoutRun: env.DEV_DEQUEUE_INTERVAL_WITHOUT_RUN,
|
||||
});
|
||||
} catch (error) {
|
||||
logger.error("Failed to get dev settings", {
|
||||
environmentId: authentication.environment.id,
|
||||
error,
|
||||
});
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
);
|
||||
@@ -0,0 +1,92 @@
|
||||
import { json } from "@remix-run/server-runtime";
|
||||
import { DequeuedMessage, DevDequeueRequestBody, MachineResources } from "@trigger.dev/core/v3";
|
||||
import { BackgroundWorkerId } from "@trigger.dev/core/v3/apps";
|
||||
import { env } from "~/env.server";
|
||||
import { createActionApiRoute } from "~/services/routeBuilders/apiBuilder.server";
|
||||
import { engine } from "~/v3/runEngine.server";
|
||||
|
||||
const { action } = createActionApiRoute(
|
||||
{
|
||||
body: DevDequeueRequestBody,
|
||||
maxContentLength: 1024 * 10, // 10KB
|
||||
method: "POST",
|
||||
},
|
||||
async ({ authentication, body }) => {
|
||||
//we won't return more runs than this in one API call
|
||||
let maxDequeueCount = env.DEV_DEQUEUE_MAX_RUNS_PER_PULL;
|
||||
|
||||
//we can't use more than the max resources
|
||||
const availableResources = body.maxResources ?? {
|
||||
cpu: 8,
|
||||
memory: 16,
|
||||
};
|
||||
|
||||
let dequeuedMessages: DequeuedMessage[] = [];
|
||||
|
||||
//we need to check the current worker, because a run might have been locked to it
|
||||
const workers = body.oldWorkers.concat(body.currentWorker);
|
||||
|
||||
//first we want to clear out old runs
|
||||
for (const worker of workers) {
|
||||
//dequeue
|
||||
const latestResult = await engine.dequeueFromBackgroundWorkerMasterQueue({
|
||||
consumerId: authentication.environment.id,
|
||||
//specific version
|
||||
backgroundWorkerId: BackgroundWorkerId.toId(worker),
|
||||
maxRunCount: maxDequeueCount,
|
||||
maxResources: availableResources,
|
||||
});
|
||||
|
||||
//add runs to the array
|
||||
dequeuedMessages.push(...latestResult);
|
||||
|
||||
//update availableResources
|
||||
const consumedResources = latestResult.reduce(
|
||||
(acc, r) => {
|
||||
return {
|
||||
cpu: acc.cpu + r.run.machine.cpu,
|
||||
memory: acc.memory + r.run.machine.memory,
|
||||
};
|
||||
},
|
||||
{ cpu: 0, memory: 0 }
|
||||
);
|
||||
updateAvailableResources(availableResources, consumedResources);
|
||||
|
||||
//update maxDequeueCount
|
||||
maxDequeueCount -= latestResult.length;
|
||||
|
||||
//if we have no resources left, we exit the loop
|
||||
if (!hasAvailableResources(availableResources)) break;
|
||||
//we've already dequeued the max number of runs
|
||||
if (maxDequeueCount <= 0) break;
|
||||
}
|
||||
|
||||
//dequeue from the current version if we still have space
|
||||
if (hasAvailableResources(availableResources) && maxDequeueCount > 0) {
|
||||
const latestResult = await engine.dequeueFromEnvironmentMasterQueue({
|
||||
consumerId: authentication.environment.id,
|
||||
//current dev version (no specific version specified)
|
||||
environmentId: authentication.environment.id,
|
||||
maxRunCount: maxDequeueCount,
|
||||
maxResources: availableResources,
|
||||
});
|
||||
dequeuedMessages.push(...latestResult);
|
||||
}
|
||||
|
||||
return json({ dequeuedMessages }, { status: 200 });
|
||||
}
|
||||
);
|
||||
|
||||
function updateAvailableResources(
|
||||
availableResources: MachineResources,
|
||||
resources: MachineResources
|
||||
) {
|
||||
availableResources.cpu -= resources.cpu;
|
||||
availableResources.memory -= resources.memory;
|
||||
}
|
||||
|
||||
function hasAvailableResources(availableResources: MachineResources) {
|
||||
return availableResources.cpu > 0 && availableResources.memory > 0;
|
||||
}
|
||||
|
||||
export { action };
|
||||
@@ -0,0 +1,81 @@
|
||||
import { json } from "@remix-run/server-runtime";
|
||||
import { Redis } from "ioredis";
|
||||
import { env } from "~/env.server";
|
||||
import { DevPresenceStream } from "~/presenters/v3/DevPresenceStream.server";
|
||||
import { authenticateApiRequestWithFailure } from "~/services/apiAuth.server";
|
||||
import { logger } from "~/services/logger.server";
|
||||
import { createSSELoader } from "~/utils/sse";
|
||||
|
||||
const redis = new Redis({
|
||||
port: env.RUN_ENGINE_DEV_PRESENCE_REDIS_PORT ?? undefined,
|
||||
host: env.RUN_ENGINE_DEV_PRESENCE_REDIS_HOST ?? undefined,
|
||||
username: env.RUN_ENGINE_DEV_PRESENCE_REDIS_USERNAME ?? undefined,
|
||||
password: env.RUN_ENGINE_DEV_PRESENCE_REDIS_PASSWORD ?? undefined,
|
||||
enableAutoPipelining: true,
|
||||
...(env.RUN_ENGINE_DEV_PRESENCE_REDIS_TLS_DISABLED === "true" ? {} : { tls: {} }),
|
||||
});
|
||||
|
||||
export const loader = createSSELoader({
|
||||
timeout: env.DEV_PRESENCE_TTL_MS,
|
||||
interval: env.DEV_PRESENCE_POLL_INTERVAL_MS,
|
||||
debug: true,
|
||||
handler: async ({ id, controller, debug, request }) => {
|
||||
const authentication = await authenticateApiRequestWithFailure(request);
|
||||
|
||||
if (!authentication.ok) {
|
||||
throw json({ error: "Invalid or Missing API key" }, { status: 401 });
|
||||
}
|
||||
|
||||
const environmentId = authentication.environment.id;
|
||||
|
||||
const presenceKey = DevPresenceStream.getPresenceKey(environmentId);
|
||||
const presenceChannel = DevPresenceStream.getPresenceChannel(environmentId);
|
||||
|
||||
return {
|
||||
beforeStream: async () => {
|
||||
logger.debug("Start dev presence SSE session", {
|
||||
environmentId,
|
||||
presenceKey,
|
||||
presenceChannel,
|
||||
});
|
||||
},
|
||||
initStream: async ({ send }) => {
|
||||
//todo set a string instead, with the expire on the same call
|
||||
//won't need multi
|
||||
|
||||
// Set initial presence with more context
|
||||
await redis.setex(presenceKey, env.DEV_PRESENCE_TTL_MS / 1000, Date.now().toString());
|
||||
|
||||
// Publish presence update
|
||||
await redis.publish(
|
||||
presenceChannel,
|
||||
JSON.stringify({
|
||||
type: "connected",
|
||||
environmentId,
|
||||
timestamp: Date.now(),
|
||||
})
|
||||
);
|
||||
|
||||
send({ event: "start", data: `Started ${id}` });
|
||||
},
|
||||
iterator: async ({ send, date }) => {
|
||||
await redis.setex(presenceKey, env.DEV_PRESENCE_TTL_MS / 1000, date.toISOString());
|
||||
|
||||
send({ event: "time", data: new Date().toISOString() });
|
||||
},
|
||||
cleanup: async () => {
|
||||
await redis.del(presenceKey);
|
||||
|
||||
// Publish disconnect event
|
||||
await redis.publish(
|
||||
presenceChannel,
|
||||
JSON.stringify({
|
||||
type: "disconnected",
|
||||
environmentId,
|
||||
timestamp: Date.now(),
|
||||
})
|
||||
);
|
||||
},
|
||||
};
|
||||
},
|
||||
});
|
||||
@@ -0,0 +1,74 @@
|
||||
import { TypedResponse } from "@remix-run/server-runtime";
|
||||
import { assertExhaustive } from "@trigger.dev/core";
|
||||
import { RunId } from "@trigger.dev/core/v3/apps";
|
||||
import {
|
||||
WorkerApiDebugLogBody,
|
||||
WorkerApiRunAttemptStartResponseBody,
|
||||
} from "@trigger.dev/core/v3/workers";
|
||||
import { z } from "zod";
|
||||
import { prisma } from "~/db.server";
|
||||
import { logger } from "~/services/logger.server";
|
||||
import { createActionApiRoute } from "~/services/routeBuilders/apiBuilder.server";
|
||||
import { recordRunDebugLog } from "~/v3/eventRepository.server";
|
||||
|
||||
const { action } = createActionApiRoute(
|
||||
{
|
||||
params: z.object({
|
||||
runFriendlyId: z.string(),
|
||||
}),
|
||||
body: WorkerApiDebugLogBody,
|
||||
method: "POST",
|
||||
},
|
||||
async ({
|
||||
authentication,
|
||||
body,
|
||||
params,
|
||||
}): Promise<TypedResponse<WorkerApiRunAttemptStartResponseBody>> => {
|
||||
const { runFriendlyId } = params;
|
||||
|
||||
try {
|
||||
const run = await prisma.taskRun.findFirst({
|
||||
where: {
|
||||
friendlyId: params.runFriendlyId,
|
||||
runtimeEnvironmentId: authentication.environment.id,
|
||||
},
|
||||
});
|
||||
|
||||
if (!run) {
|
||||
throw new Response("You don't have permissions for this run", { status: 401 });
|
||||
}
|
||||
|
||||
const eventResult = await recordRunDebugLog(
|
||||
RunId.fromFriendlyId(runFriendlyId),
|
||||
body.message,
|
||||
{
|
||||
attributes: {
|
||||
properties: body.properties,
|
||||
},
|
||||
startTime: body.time,
|
||||
}
|
||||
);
|
||||
|
||||
if (eventResult.success) {
|
||||
return new Response(null, { status: 204 });
|
||||
}
|
||||
|
||||
switch (eventResult.code) {
|
||||
case "FAILED_TO_RECORD_EVENT":
|
||||
return new Response(null, { status: 400 }); // send a 400 to prevent retries
|
||||
case "RUN_NOT_FOUND":
|
||||
return new Response(null, { status: 404 });
|
||||
default:
|
||||
return assertExhaustive(eventResult.code);
|
||||
}
|
||||
} catch (error) {
|
||||
logger.error("Failed to record dev log", {
|
||||
environmentId: authentication.environment.id,
|
||||
error,
|
||||
});
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
);
|
||||
|
||||
export { action };
|
||||
+64
@@ -0,0 +1,64 @@
|
||||
import { json, TypedResponse } from "@remix-run/server-runtime";
|
||||
import { assertExhaustive } from "@trigger.dev/core";
|
||||
import { RunId, SnapshotId } from "@trigger.dev/core/v3/apps";
|
||||
import {
|
||||
WorkerApiDebugLogBody,
|
||||
WorkerApiRunAttemptCompleteRequestBody,
|
||||
WorkerApiRunAttemptCompleteResponseBody,
|
||||
WorkerApiRunAttemptStartResponseBody,
|
||||
WorkloadHeartbeatResponseBody,
|
||||
} from "@trigger.dev/core/v3/workers";
|
||||
import { z } from "zod";
|
||||
import { prisma } from "~/db.server";
|
||||
import { logger } from "~/services/logger.server";
|
||||
import { createActionApiRoute } from "~/services/routeBuilders/apiBuilder.server";
|
||||
import { recordRunDebugLog } from "~/v3/eventRepository.server";
|
||||
import { engine } from "~/v3/runEngine.server";
|
||||
|
||||
const { action } = createActionApiRoute(
|
||||
{
|
||||
params: z.object({
|
||||
runFriendlyId: z.string(),
|
||||
snapshotFriendlyId: z.string(),
|
||||
}),
|
||||
body: WorkerApiRunAttemptCompleteRequestBody,
|
||||
method: "POST",
|
||||
},
|
||||
async ({
|
||||
authentication,
|
||||
body,
|
||||
params,
|
||||
}): Promise<TypedResponse<WorkerApiRunAttemptCompleteResponseBody>> => {
|
||||
const { completion } = body;
|
||||
const { runFriendlyId, snapshotFriendlyId } = params;
|
||||
|
||||
try {
|
||||
const run = await prisma.taskRun.findFirst({
|
||||
where: {
|
||||
friendlyId: params.runFriendlyId,
|
||||
runtimeEnvironmentId: authentication.environment.id,
|
||||
},
|
||||
});
|
||||
|
||||
if (!run) {
|
||||
throw new Response("You don't have permissions for this run", { status: 401 });
|
||||
}
|
||||
|
||||
const completeResult = await engine.completeRunAttempt({
|
||||
runId: RunId.toId(runFriendlyId),
|
||||
snapshotId: SnapshotId.toId(snapshotFriendlyId),
|
||||
completion,
|
||||
});
|
||||
|
||||
return json({ result: completeResult });
|
||||
} catch (error) {
|
||||
logger.error("Failed to complete dev attempt", {
|
||||
environmentId: authentication.environment.id,
|
||||
error,
|
||||
});
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
);
|
||||
|
||||
export { action };
|
||||
+103
@@ -0,0 +1,103 @@
|
||||
import { json, TypedResponse } from "@remix-run/server-runtime";
|
||||
import { MachinePreset } from "@trigger.dev/core/v3";
|
||||
import { RunId, SnapshotId } from "@trigger.dev/core/v3/apps";
|
||||
import {
|
||||
WorkerApiRunAttemptStartRequestBody,
|
||||
WorkerApiRunAttemptStartResponseBody,
|
||||
} from "@trigger.dev/core/v3/workers";
|
||||
import { RuntimeEnvironment } from "@trigger.dev/database";
|
||||
import { defaultMachine } from "@trigger.dev/platform/v3";
|
||||
import { z } from "zod";
|
||||
import { prisma } from "~/db.server";
|
||||
import { generateJWTTokenForEnvironment } from "~/services/apiAuth.server";
|
||||
import { logger } from "~/services/logger.server";
|
||||
import {
|
||||
createActionApiRoute,
|
||||
createActionWorkerApiRoute,
|
||||
} from "~/services/routeBuilders/apiBuilder.server";
|
||||
import { resolveVariablesForEnvironment } from "~/v3/environmentVariables/environmentVariablesRepository.server";
|
||||
import { machinePresetFromName } from "~/v3/machinePresets.server";
|
||||
import { engine } from "~/v3/runEngine.server";
|
||||
|
||||
const { action } = createActionApiRoute(
|
||||
{
|
||||
body: WorkerApiRunAttemptStartRequestBody,
|
||||
params: z.object({
|
||||
runFriendlyId: z.string(),
|
||||
snapshotFriendlyId: z.string(),
|
||||
}),
|
||||
method: "POST",
|
||||
},
|
||||
async ({
|
||||
authentication,
|
||||
body,
|
||||
params,
|
||||
}): Promise<TypedResponse<WorkerApiRunAttemptStartResponseBody>> => {
|
||||
const { runFriendlyId, snapshotFriendlyId } = params;
|
||||
|
||||
try {
|
||||
const run = await prisma.taskRun.findFirst({
|
||||
where: {
|
||||
friendlyId: params.runFriendlyId,
|
||||
runtimeEnvironmentId: authentication.environment.id,
|
||||
},
|
||||
});
|
||||
|
||||
if (!run) {
|
||||
throw new Response("You don't have permissions for this run", { status: 401 });
|
||||
}
|
||||
|
||||
const engineResult = await engine.startRunAttempt({
|
||||
runId: RunId.toId(runFriendlyId),
|
||||
snapshotId: SnapshotId.toId(snapshotFriendlyId),
|
||||
});
|
||||
|
||||
const defaultMachinePreset = machinePresetFromName(defaultMachine);
|
||||
|
||||
const envVars = await getEnvVars(
|
||||
authentication.environment,
|
||||
engineResult.run.id,
|
||||
engineResult.execution.machine ?? defaultMachinePreset
|
||||
);
|
||||
|
||||
return json({
|
||||
...engineResult,
|
||||
envVars,
|
||||
});
|
||||
} catch (error) {
|
||||
logger.error("Failed to record dev log", {
|
||||
environmentId: authentication.environment.id,
|
||||
error,
|
||||
});
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
);
|
||||
|
||||
async function getEnvVars(
|
||||
environment: RuntimeEnvironment,
|
||||
runId: string,
|
||||
machinePreset: MachinePreset
|
||||
): Promise<Record<string, string>> {
|
||||
const variables = await resolveVariablesForEnvironment(environment);
|
||||
|
||||
const jwt = await generateJWTTokenForEnvironment(environment, {
|
||||
run_id: runId,
|
||||
machine_preset: machinePreset.name,
|
||||
});
|
||||
|
||||
variables.push(
|
||||
...[
|
||||
{ key: "TRIGGER_JWT", value: jwt },
|
||||
{ key: "TRIGGER_RUN_ID", value: runId },
|
||||
{ key: "TRIGGER_MACHINE_PRESET", value: machinePreset.name },
|
||||
]
|
||||
);
|
||||
|
||||
return variables.reduce((acc: Record<string, string>, curr) => {
|
||||
acc[curr.key] = curr.value;
|
||||
return acc;
|
||||
}, {});
|
||||
}
|
||||
|
||||
export { action };
|
||||
+59
@@ -0,0 +1,59 @@
|
||||
import { json, TypedResponse } from "@remix-run/server-runtime";
|
||||
import { assertExhaustive } from "@trigger.dev/core";
|
||||
import { RunId, SnapshotId } from "@trigger.dev/core/v3/apps";
|
||||
import {
|
||||
WorkerApiDebugLogBody,
|
||||
WorkerApiRunAttemptStartResponseBody,
|
||||
WorkloadHeartbeatResponseBody,
|
||||
} from "@trigger.dev/core/v3/workers";
|
||||
import { z } from "zod";
|
||||
import { prisma } from "~/db.server";
|
||||
import { logger } from "~/services/logger.server";
|
||||
import { createActionApiRoute } from "~/services/routeBuilders/apiBuilder.server";
|
||||
import { recordRunDebugLog } from "~/v3/eventRepository.server";
|
||||
import { engine } from "~/v3/runEngine.server";
|
||||
|
||||
const { action } = createActionApiRoute(
|
||||
{
|
||||
params: z.object({
|
||||
runFriendlyId: z.string(),
|
||||
snapshotFriendlyId: z.string(),
|
||||
}),
|
||||
method: "POST",
|
||||
},
|
||||
async ({
|
||||
authentication,
|
||||
body,
|
||||
params,
|
||||
}): Promise<TypedResponse<WorkloadHeartbeatResponseBody>> => {
|
||||
const { runFriendlyId, snapshotFriendlyId } = params;
|
||||
|
||||
try {
|
||||
const run = await prisma.taskRun.findFirst({
|
||||
where: {
|
||||
friendlyId: params.runFriendlyId,
|
||||
runtimeEnvironmentId: authentication.environment.id,
|
||||
},
|
||||
});
|
||||
|
||||
if (!run) {
|
||||
throw new Response("You don't have permissions for this run", { status: 401 });
|
||||
}
|
||||
|
||||
await engine.heartbeatRun({
|
||||
runId: RunId.toId(runFriendlyId),
|
||||
snapshotId: SnapshotId.toId(snapshotFriendlyId),
|
||||
});
|
||||
|
||||
return json({ ok: true });
|
||||
} catch (error) {
|
||||
logger.error("Failed to heartbeat dev run", {
|
||||
environmentId: authentication.environment.id,
|
||||
error,
|
||||
});
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
);
|
||||
|
||||
export { action };
|
||||
@@ -0,0 +1,55 @@
|
||||
import { json, TypedResponse } from "@remix-run/server-runtime";
|
||||
import { RunId } from "@trigger.dev/core/v3/apps";
|
||||
import { WorkerApiRunLatestSnapshotResponseBody } from "@trigger.dev/core/v3/workers";
|
||||
import { z } from "zod";
|
||||
import { prisma } from "~/db.server";
|
||||
import { logger } from "~/services/logger.server";
|
||||
import { createLoaderApiRoute } from "~/services/routeBuilders/apiBuilder.server";
|
||||
import { engine } from "~/v3/runEngine.server";
|
||||
|
||||
export const loader = createLoaderApiRoute(
|
||||
{
|
||||
findResource: async () => 1,
|
||||
params: z.object({
|
||||
runFriendlyId: z.string(),
|
||||
}),
|
||||
},
|
||||
async ({
|
||||
authentication,
|
||||
params,
|
||||
}): Promise<TypedResponse<WorkerApiRunLatestSnapshotResponseBody>> => {
|
||||
logger.debug("dev: Get latest snapshot", {
|
||||
environmentId: authentication.environment.id,
|
||||
params,
|
||||
});
|
||||
|
||||
try {
|
||||
const run = await prisma.taskRun.findFirst({
|
||||
where: {
|
||||
friendlyId: params.runFriendlyId,
|
||||
runtimeEnvironmentId: authentication.environment.id,
|
||||
},
|
||||
});
|
||||
|
||||
if (!run) {
|
||||
throw new Response("You don't have permissions for this run", { status: 401 });
|
||||
}
|
||||
|
||||
const executionData = await engine.getRunExecutionData({
|
||||
runId: RunId.toId(params.runFriendlyId),
|
||||
});
|
||||
|
||||
if (!executionData) {
|
||||
throw new Error("Failed to retrieve latest snapshot");
|
||||
}
|
||||
|
||||
return json({ execution: executionData });
|
||||
} catch (error) {
|
||||
logger.error("Failed to get latest snapshot", {
|
||||
environmentId: authentication.environment.id,
|
||||
error,
|
||||
});
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
);
|
||||
@@ -0,0 +1,75 @@
|
||||
import { json, TypedResponse } from "@remix-run/server-runtime";
|
||||
import { WaitForDurationRequestBody, WaitForDurationResponseBody } from "@trigger.dev/core/v3";
|
||||
import { RunId } from "@trigger.dev/core/v3/apps";
|
||||
|
||||
import { z } from "zod";
|
||||
import { prisma } from "~/db.server";
|
||||
import { logger } from "~/services/logger.server";
|
||||
import { createActionApiRoute } from "~/services/routeBuilders/apiBuilder.server";
|
||||
import { resolveIdempotencyKeyTTL } from "~/utils/idempotencyKeys.server";
|
||||
import { engine } from "~/v3/runEngine.server";
|
||||
|
||||
const { action } = createActionApiRoute(
|
||||
{
|
||||
body: WaitForDurationRequestBody,
|
||||
params: z.object({
|
||||
runFriendlyId: z.string(),
|
||||
}),
|
||||
method: "POST",
|
||||
},
|
||||
async ({ authentication, body, params }): Promise<TypedResponse<WaitForDurationResponseBody>> => {
|
||||
const { runFriendlyId } = params;
|
||||
const runId = RunId.toId(runFriendlyId);
|
||||
|
||||
try {
|
||||
const run = await prisma.taskRun.findFirst({
|
||||
where: {
|
||||
id: runId,
|
||||
runtimeEnvironmentId: authentication.environment.id,
|
||||
},
|
||||
});
|
||||
|
||||
if (!run) {
|
||||
throw new Response("You don't have permissions for this run", { status: 401 });
|
||||
}
|
||||
|
||||
const idempotencyKeyExpiresAt = body.idempotencyKeyTTL
|
||||
? resolveIdempotencyKeyTTL(body.idempotencyKeyTTL)
|
||||
: undefined;
|
||||
|
||||
const { waitpoint } = await engine.createDateTimeWaitpoint({
|
||||
projectId: authentication.environment.project.id,
|
||||
environmentId: authentication.environment.id,
|
||||
completedAfter: body.date,
|
||||
idempotencyKey: body.idempotencyKey,
|
||||
idempotencyKeyExpiresAt: idempotencyKeyExpiresAt,
|
||||
});
|
||||
|
||||
const waitResult = await engine.blockRunWithWaitpoint({
|
||||
runId: run.id,
|
||||
waitpoints: waitpoint.id,
|
||||
environmentId: authentication.environment.id,
|
||||
projectId: authentication.environment.project.id,
|
||||
organizationId: authentication.environment.organization.id,
|
||||
releaseConcurrency: {
|
||||
releaseQueue: true,
|
||||
},
|
||||
});
|
||||
|
||||
return json({
|
||||
waitUntil: body.date,
|
||||
waitpoint: {
|
||||
id: waitpoint.friendlyId,
|
||||
},
|
||||
});
|
||||
} catch (error) {
|
||||
logger.error("Failed to wait for duration dev", {
|
||||
environmentId: authentication.environment.id,
|
||||
error,
|
||||
});
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
);
|
||||
|
||||
export { action };
|
||||
+58
@@ -0,0 +1,58 @@
|
||||
import { json } from "@remix-run/server-runtime";
|
||||
import { WaitForWaitpointTokenResponseBody } from "@trigger.dev/core/v3";
|
||||
import { RunId, WaitpointId } from "@trigger.dev/core/v3/apps";
|
||||
import { z } from "zod";
|
||||
import { $replica } from "~/db.server";
|
||||
import { logger } from "~/services/logger.server";
|
||||
import { createActionApiRoute } from "~/services/routeBuilders/apiBuilder.server";
|
||||
import { engine } from "~/v3/runEngine.server";
|
||||
|
||||
const { action } = createActionApiRoute(
|
||||
{
|
||||
params: z.object({
|
||||
runFriendlyId: z.string(),
|
||||
waitpointFriendlyId: z.string(),
|
||||
}),
|
||||
maxContentLength: 1024 * 10, // 10KB
|
||||
method: "POST",
|
||||
},
|
||||
async ({ authentication, body, params }) => {
|
||||
// Resume tokens are actually just waitpoints
|
||||
const waitpointId = WaitpointId.toId(params.waitpointFriendlyId);
|
||||
const runId = RunId.toId(params.runFriendlyId);
|
||||
|
||||
try {
|
||||
//check permissions
|
||||
const waitpoint = await $replica.waitpoint.findFirst({
|
||||
where: {
|
||||
id: waitpointId,
|
||||
environmentId: authentication.environment.id,
|
||||
},
|
||||
});
|
||||
|
||||
if (!waitpoint) {
|
||||
throw json({ error: "Waitpoint not found" }, { status: 404 });
|
||||
}
|
||||
|
||||
const result = await engine.blockRunWithWaitpoint({
|
||||
runId,
|
||||
waitpoints: [waitpointId],
|
||||
environmentId: authentication.environment.id,
|
||||
projectId: authentication.environment.project.id,
|
||||
organizationId: authentication.environment.organization.id,
|
||||
});
|
||||
|
||||
return json<WaitForWaitpointTokenResponseBody>(
|
||||
{
|
||||
success: true,
|
||||
},
|
||||
{ status: 200 }
|
||||
);
|
||||
} catch (error) {
|
||||
logger.error("Failed to wait for waitpoint", { runId, waitpointId, error });
|
||||
throw json({ error: "Failed to wait for waitpoint token" }, { status: 500 });
|
||||
}
|
||||
}
|
||||
);
|
||||
|
||||
export { action };
|
||||
@@ -0,0 +1,19 @@
|
||||
import { json, TypedResponse } from "@remix-run/server-runtime";
|
||||
import { WorkerApiConnectRequestBody, WorkerApiConnectResponseBody } from "@trigger.dev/core/v3/workers";
|
||||
import { createActionWorkerApiRoute } from "~/services/routeBuilders/apiBuilder.server";
|
||||
|
||||
export const action = createActionWorkerApiRoute(
|
||||
{
|
||||
body: WorkerApiConnectRequestBody,
|
||||
},
|
||||
async ({ authenticatedWorker, body }): Promise<TypedResponse<WorkerApiConnectResponseBody>> => {
|
||||
await authenticatedWorker.connect(body.metadata);
|
||||
return json({
|
||||
ok: true,
|
||||
workerGroup: {
|
||||
type: authenticatedWorker.type,
|
||||
name: authenticatedWorker.name,
|
||||
},
|
||||
});
|
||||
}
|
||||
);
|
||||
+68
@@ -0,0 +1,68 @@
|
||||
import { json, TypedResponse } from "@remix-run/server-runtime";
|
||||
import { CURRENT_DEPLOYMENT_LABEL } from "@trigger.dev/core/v3/apps";
|
||||
import { WorkerApiDequeueResponseBody } from "@trigger.dev/core/v3/workers";
|
||||
import { z } from "zod";
|
||||
import { $replica, prisma } from "~/db.server";
|
||||
import { createLoaderWorkerApiRoute } from "~/services/routeBuilders/apiBuilder.server";
|
||||
|
||||
export const loader = createLoaderWorkerApiRoute(
|
||||
{
|
||||
params: z.object({
|
||||
deploymentFriendlyId: z.string(),
|
||||
}),
|
||||
searchParams: z.object({
|
||||
maxRunCount: z.coerce.number().optional(),
|
||||
}),
|
||||
},
|
||||
async ({
|
||||
authenticatedWorker,
|
||||
params,
|
||||
searchParams,
|
||||
}): Promise<TypedResponse<WorkerApiDequeueResponseBody>> => {
|
||||
const deployment = await $replica.workerDeployment.findUnique({
|
||||
where: {
|
||||
friendlyId: params.deploymentFriendlyId,
|
||||
},
|
||||
include: {
|
||||
worker: true,
|
||||
},
|
||||
});
|
||||
|
||||
if (!deployment) {
|
||||
throw new Error("Deployment not found");
|
||||
}
|
||||
|
||||
if (!deployment.worker) {
|
||||
throw new Error("Worker not found");
|
||||
}
|
||||
|
||||
const dequeuedMessages = (await isCurrentDeployment(deployment.id, deployment.environmentId))
|
||||
? await authenticatedWorker.dequeueFromEnvironment(
|
||||
deployment.worker.id,
|
||||
deployment.environmentId
|
||||
)
|
||||
: await authenticatedWorker.dequeueFromVersion(
|
||||
deployment.worker.id,
|
||||
searchParams.maxRunCount
|
||||
);
|
||||
|
||||
return json(dequeuedMessages);
|
||||
}
|
||||
);
|
||||
|
||||
async function isCurrentDeployment(deploymentId: string, environmentId: string): Promise<boolean> {
|
||||
const promotion = await prisma.workerDeploymentPromotion.findUnique({
|
||||
where: {
|
||||
environmentId_label: {
|
||||
environmentId,
|
||||
label: CURRENT_DEPLOYMENT_LABEL,
|
||||
},
|
||||
},
|
||||
});
|
||||
|
||||
if (!promotion) {
|
||||
return false;
|
||||
}
|
||||
|
||||
return promotion.deploymentId === deploymentId;
|
||||
}
|
||||
@@ -0,0 +1,16 @@
|
||||
import { json, TypedResponse } from "@remix-run/server-runtime";
|
||||
import { WorkerApiDequeueRequestBody, WorkerApiDequeueResponseBody } from "@trigger.dev/core/v3/workers";
|
||||
import { createActionWorkerApiRoute } from "~/services/routeBuilders/apiBuilder.server";
|
||||
|
||||
export const action = createActionWorkerApiRoute(
|
||||
{
|
||||
body: WorkerApiDequeueRequestBody,
|
||||
},
|
||||
async ({ authenticatedWorker, body }): Promise<TypedResponse<WorkerApiDequeueResponseBody>> => {
|
||||
return json(
|
||||
await authenticatedWorker.dequeue({
|
||||
maxResources: body.maxResources,
|
||||
})
|
||||
);
|
||||
}
|
||||
);
|
||||
@@ -0,0 +1,13 @@
|
||||
import { json, TypedResponse } from "@remix-run/server-runtime";
|
||||
import { WorkerApiHeartbeatResponseBody, WorkerApiHeartbeatRequestBody } from "@trigger.dev/core/v3/workers";
|
||||
import { createActionWorkerApiRoute } from "~/services/routeBuilders/apiBuilder.server";
|
||||
|
||||
export const action = createActionWorkerApiRoute(
|
||||
{
|
||||
body: WorkerApiHeartbeatRequestBody,
|
||||
},
|
||||
async ({ authenticatedWorker }): Promise<TypedResponse<WorkerApiHeartbeatResponseBody>> => {
|
||||
await authenticatedWorker.heartbeatWorkerInstance();
|
||||
return json({ ok: true });
|
||||
}
|
||||
);
|
||||
@@ -0,0 +1,38 @@
|
||||
import { assertExhaustive } from "@trigger.dev/core";
|
||||
import { RunId } from "@trigger.dev/core/v3/apps";
|
||||
import { WorkerApiDebugLogBody } from "@trigger.dev/core/v3/runEngineWorker";
|
||||
import { z } from "zod";
|
||||
import { createActionWorkerApiRoute } from "~/services/routeBuilders/apiBuilder.server";
|
||||
import { recordRunDebugLog } from "~/v3/eventRepository.server";
|
||||
|
||||
export const action = createActionWorkerApiRoute(
|
||||
{
|
||||
params: z.object({
|
||||
runFriendlyId: z.string(),
|
||||
}),
|
||||
body: WorkerApiDebugLogBody,
|
||||
},
|
||||
async ({ body, params }): Promise<Response> => {
|
||||
const { runFriendlyId } = params;
|
||||
|
||||
const eventResult = await recordRunDebugLog(RunId.fromFriendlyId(runFriendlyId), body.message, {
|
||||
attributes: {
|
||||
properties: body.properties,
|
||||
},
|
||||
startTime: body.time,
|
||||
});
|
||||
|
||||
if (eventResult.success) {
|
||||
return new Response(null, { status: 204 });
|
||||
}
|
||||
|
||||
switch (eventResult.code) {
|
||||
case "FAILED_TO_RECORD_EVENT":
|
||||
return new Response(null, { status: 400 }); // send a 400 to prevent retries
|
||||
case "RUN_NOT_FOUND":
|
||||
return new Response(null, { status: 404 });
|
||||
default:
|
||||
return assertExhaustive(eventResult.code);
|
||||
}
|
||||
}
|
||||
);
|
||||
+33
@@ -0,0 +1,33 @@
|
||||
import { json, TypedResponse } from "@remix-run/server-runtime";
|
||||
import {
|
||||
WorkerApiRunAttemptCompleteRequestBody,
|
||||
WorkerApiRunAttemptCompleteResponseBody,
|
||||
} from "@trigger.dev/core/v3/workers";
|
||||
import { z } from "zod";
|
||||
import { createActionWorkerApiRoute } from "~/services/routeBuilders/apiBuilder.server";
|
||||
|
||||
export const action = createActionWorkerApiRoute(
|
||||
{
|
||||
body: WorkerApiRunAttemptCompleteRequestBody,
|
||||
params: z.object({
|
||||
runFriendlyId: z.string(),
|
||||
snapshotFriendlyId: z.string(),
|
||||
}),
|
||||
},
|
||||
async ({
|
||||
authenticatedWorker,
|
||||
body,
|
||||
params,
|
||||
}): Promise<TypedResponse<WorkerApiRunAttemptCompleteResponseBody>> => {
|
||||
const { completion } = body;
|
||||
const { runFriendlyId, snapshotFriendlyId } = params;
|
||||
|
||||
const completeResult = await authenticatedWorker.completeRunAttempt({
|
||||
runFriendlyId,
|
||||
snapshotFriendlyId,
|
||||
completion,
|
||||
});
|
||||
|
||||
return json({ result: completeResult });
|
||||
}
|
||||
);
|
||||
+32
@@ -0,0 +1,32 @@
|
||||
import { json, TypedResponse } from "@remix-run/server-runtime";
|
||||
import {
|
||||
WorkerApiRunAttemptStartRequestBody,
|
||||
WorkerApiRunAttemptStartResponseBody,
|
||||
} from "@trigger.dev/core/v3/workers";
|
||||
import { z } from "zod";
|
||||
import { createActionWorkerApiRoute } from "~/services/routeBuilders/apiBuilder.server";
|
||||
|
||||
export const action = createActionWorkerApiRoute(
|
||||
{
|
||||
body: WorkerApiRunAttemptStartRequestBody,
|
||||
params: z.object({
|
||||
runFriendlyId: z.string(),
|
||||
snapshotFriendlyId: z.string(),
|
||||
}),
|
||||
},
|
||||
async ({
|
||||
authenticatedWorker,
|
||||
body,
|
||||
params,
|
||||
}): Promise<TypedResponse<WorkerApiRunAttemptStartResponseBody>> => {
|
||||
const { runFriendlyId, snapshotFriendlyId } = params;
|
||||
|
||||
const runExecutionData = await authenticatedWorker.startRunAttempt({
|
||||
runFriendlyId,
|
||||
snapshotFriendlyId,
|
||||
isWarmStart: body.isWarmStart,
|
||||
});
|
||||
|
||||
return json(runExecutionData);
|
||||
}
|
||||
);
|
||||
+34
@@ -0,0 +1,34 @@
|
||||
import { json, TypedResponse } from "@remix-run/server-runtime";
|
||||
import { WorkerApiContinueRunExecutionRequestBody } from "@trigger.dev/core/v3/workers";
|
||||
import { z } from "zod";
|
||||
import { logger } from "~/services/logger.server";
|
||||
import { createLoaderWorkerApiRoute } from "~/services/routeBuilders/apiBuilder.server";
|
||||
|
||||
export const loader = createLoaderWorkerApiRoute(
|
||||
{
|
||||
params: z.object({
|
||||
runFriendlyId: z.string(),
|
||||
snapshotFriendlyId: z.string(),
|
||||
}),
|
||||
},
|
||||
async ({
|
||||
authenticatedWorker,
|
||||
params,
|
||||
}): Promise<TypedResponse<WorkerApiContinueRunExecutionRequestBody>> => {
|
||||
const { runFriendlyId, snapshotFriendlyId } = params;
|
||||
|
||||
logger.debug("Continuing run execution", { runFriendlyId, snapshotFriendlyId });
|
||||
|
||||
try {
|
||||
const continuationResult = await authenticatedWorker.continueRunExecution({
|
||||
runFriendlyId,
|
||||
snapshotFriendlyId,
|
||||
});
|
||||
|
||||
return json(continuationResult);
|
||||
} catch (error) {
|
||||
logger.error("Failed to suspend run", { runFriendlyId, snapshotFriendlyId, error });
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
);
|
||||
+26
@@ -0,0 +1,26 @@
|
||||
import { json, TypedResponse } from "@remix-run/server-runtime";
|
||||
import { WorkloadHeartbeatResponseBody } from "@trigger.dev/core/v3/workers";
|
||||
import { z } from "zod";
|
||||
import { createActionWorkerApiRoute } from "~/services/routeBuilders/apiBuilder.server";
|
||||
|
||||
export const action = createActionWorkerApiRoute(
|
||||
{
|
||||
params: z.object({
|
||||
runFriendlyId: z.string(),
|
||||
snapshotFriendlyId: z.string(),
|
||||
}),
|
||||
},
|
||||
async ({
|
||||
authenticatedWorker,
|
||||
params,
|
||||
}): Promise<TypedResponse<WorkloadHeartbeatResponseBody>> => {
|
||||
const { runFriendlyId, snapshotFriendlyId } = params;
|
||||
|
||||
await authenticatedWorker.heartbeatRun({
|
||||
runFriendlyId,
|
||||
snapshotFriendlyId,
|
||||
});
|
||||
|
||||
return json({ ok: true });
|
||||
}
|
||||
);
|
||||
+51
@@ -0,0 +1,51 @@
|
||||
import { json, TypedResponse } from "@remix-run/server-runtime";
|
||||
import {
|
||||
WorkerApiSuspendRunRequestBody,
|
||||
WorkerApiSuspendRunResponseBody,
|
||||
} from "@trigger.dev/core/v3/workers";
|
||||
import { z } from "zod";
|
||||
import { logger } from "~/services/logger.server";
|
||||
import { createActionWorkerApiRoute } from "~/services/routeBuilders/apiBuilder.server";
|
||||
|
||||
export const action = createActionWorkerApiRoute(
|
||||
{
|
||||
params: z.object({
|
||||
runFriendlyId: z.string(),
|
||||
snapshotFriendlyId: z.string(),
|
||||
}),
|
||||
body: WorkerApiSuspendRunRequestBody,
|
||||
},
|
||||
async ({
|
||||
authenticatedWorker,
|
||||
params,
|
||||
body,
|
||||
}): Promise<TypedResponse<WorkerApiSuspendRunResponseBody>> => {
|
||||
const { runFriendlyId, snapshotFriendlyId } = params;
|
||||
|
||||
logger.debug("Restoring run", { runFriendlyId, snapshotFriendlyId, body });
|
||||
|
||||
if (!body.success) {
|
||||
// TODO: we could create a debug span here
|
||||
logger.error("Failed to restore run", {
|
||||
runFriendlyId,
|
||||
snapshotFriendlyId,
|
||||
error: body.error,
|
||||
});
|
||||
|
||||
return json({ ok: true });
|
||||
}
|
||||
|
||||
try {
|
||||
await authenticatedWorker.createCheckpoint({
|
||||
runFriendlyId,
|
||||
snapshotFriendlyId,
|
||||
checkpoint: body.checkpoint,
|
||||
});
|
||||
|
||||
return json({ ok: true });
|
||||
} catch (error) {
|
||||
logger.error("Failed to restore run", { runFriendlyId, snapshotFriendlyId, error });
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
);
|
||||
+51
@@ -0,0 +1,51 @@
|
||||
import { json, TypedResponse } from "@remix-run/server-runtime";
|
||||
import {
|
||||
WorkerApiSuspendRunRequestBody,
|
||||
WorkerApiSuspendRunResponseBody,
|
||||
} from "@trigger.dev/core/v3/workers";
|
||||
import { z } from "zod";
|
||||
import { logger } from "~/services/logger.server";
|
||||
import { createActionWorkerApiRoute } from "~/services/routeBuilders/apiBuilder.server";
|
||||
|
||||
export const action = createActionWorkerApiRoute(
|
||||
{
|
||||
params: z.object({
|
||||
runFriendlyId: z.string(),
|
||||
snapshotFriendlyId: z.string(),
|
||||
}),
|
||||
body: WorkerApiSuspendRunRequestBody,
|
||||
},
|
||||
async ({
|
||||
authenticatedWorker,
|
||||
params,
|
||||
body,
|
||||
}): Promise<TypedResponse<WorkerApiSuspendRunResponseBody>> => {
|
||||
const { runFriendlyId, snapshotFriendlyId } = params;
|
||||
|
||||
logger.debug("Suspending run", { runFriendlyId, snapshotFriendlyId, body });
|
||||
|
||||
if (!body.success) {
|
||||
// TODO: we could create a debug span here
|
||||
logger.error("Failed to suspend run", {
|
||||
runFriendlyId,
|
||||
snapshotFriendlyId,
|
||||
error: body.error,
|
||||
});
|
||||
|
||||
return json({ ok: true });
|
||||
}
|
||||
|
||||
try {
|
||||
await authenticatedWorker.createCheckpoint({
|
||||
runFriendlyId,
|
||||
snapshotFriendlyId,
|
||||
checkpoint: body.checkpoint,
|
||||
});
|
||||
|
||||
return json({ ok: true });
|
||||
} catch (error) {
|
||||
logger.error("Failed to suspend run", { runFriendlyId, snapshotFriendlyId, error });
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
);
|
||||
+28
@@ -0,0 +1,28 @@
|
||||
import { json, TypedResponse } from "@remix-run/server-runtime";
|
||||
import { WorkerApiRunLatestSnapshotResponseBody } from "@trigger.dev/core/v3/workers";
|
||||
import { z } from "zod";
|
||||
import { createLoaderWorkerApiRoute } from "~/services/routeBuilders/apiBuilder.server";
|
||||
|
||||
export const loader = createLoaderWorkerApiRoute(
|
||||
{
|
||||
params: z.object({
|
||||
runFriendlyId: z.string(),
|
||||
}),
|
||||
},
|
||||
async ({
|
||||
authenticatedWorker,
|
||||
params,
|
||||
}): Promise<TypedResponse<WorkerApiRunLatestSnapshotResponseBody>> => {
|
||||
const { runFriendlyId } = params;
|
||||
|
||||
const executionData = await authenticatedWorker.getLatestSnapshot({
|
||||
runFriendlyId,
|
||||
});
|
||||
|
||||
if (!executionData) {
|
||||
throw new Error("Failed to retrieve latest snapshot");
|
||||
}
|
||||
|
||||
return json({ execution: executionData });
|
||||
}
|
||||
);
|
||||
+410
@@ -0,0 +1,410 @@
|
||||
import { env } from "~/env.server";
|
||||
import { parse } from "@conform-to/zod";
|
||||
import { Form, useLocation, useNavigation, useSubmit } from "@remix-run/react";
|
||||
import { ActionFunctionArgs, json } from "@remix-run/server-runtime";
|
||||
import {
|
||||
conditionallyExportPacket,
|
||||
IOPacket,
|
||||
stringifyIO,
|
||||
timeoutError,
|
||||
} from "@trigger.dev/core/v3";
|
||||
import { WaitpointId } from "@trigger.dev/core/v3/apps";
|
||||
import { Waitpoint } from "@trigger.dev/database";
|
||||
import { useCallback, useRef } from "react";
|
||||
import { z } from "zod";
|
||||
import { AnimatedHourglassIcon } from "~/assets/icons/AnimatedHourglassIcon";
|
||||
import { JSONEditor } from "~/components/code/JSONEditor";
|
||||
import { Button } from "~/components/primitives/Buttons";
|
||||
import { DateTime } from "~/components/primitives/DateTime";
|
||||
import { Paragraph } from "~/components/primitives/Paragraph";
|
||||
import { InfoIconTooltip } from "~/components/primitives/Tooltip";
|
||||
import { LiveCountdown } from "~/components/runs/v3/LiveTimer";
|
||||
import { $replica } from "~/db.server";
|
||||
import { useOrganization } from "~/hooks/useOrganizations";
|
||||
import { useProject } from "~/hooks/useProject";
|
||||
import { redirectWithErrorMessage, redirectWithSuccessMessage } from "~/models/message.server";
|
||||
import { logger } from "~/services/logger.server";
|
||||
import { requireUserId } from "~/services/session.server";
|
||||
import { ProjectParamSchema, v3RunsPath } from "~/utils/pathBuilder";
|
||||
import { engine } from "~/v3/runEngine.server";
|
||||
|
||||
const CompleteWaitpointFormData = z.discriminatedUnion("type", [
|
||||
z.object({
|
||||
type: z.literal("MANUAL"),
|
||||
payload: z.string().optional(),
|
||||
isTimeout: z.string().optional(),
|
||||
successRedirect: z.string(),
|
||||
failureRedirect: z.string(),
|
||||
}),
|
||||
z.object({
|
||||
type: z.literal("DATETIME"),
|
||||
successRedirect: z.string(),
|
||||
failureRedirect: z.string(),
|
||||
}),
|
||||
]);
|
||||
|
||||
const Params = ProjectParamSchema.extend({
|
||||
waitpointFriendlyId: z.string(),
|
||||
});
|
||||
|
||||
export const action = async ({ request, params }: ActionFunctionArgs) => {
|
||||
const userId = await requireUserId(request);
|
||||
const { organizationSlug, projectParam, waitpointFriendlyId } = Params.parse(params);
|
||||
|
||||
const formData = await request.formData();
|
||||
const submission = parse(formData, { schema: CompleteWaitpointFormData });
|
||||
|
||||
if (!submission.value) {
|
||||
return json(submission);
|
||||
}
|
||||
|
||||
try {
|
||||
//first check that the user has access to the project
|
||||
const project = await $replica.project.findUnique({
|
||||
where: {
|
||||
slug: projectParam,
|
||||
organization: {
|
||||
members: {
|
||||
some: {
|
||||
userId,
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
select: { id: true },
|
||||
});
|
||||
|
||||
if (!project) {
|
||||
throw new Error("Project not found");
|
||||
}
|
||||
|
||||
const waitpointId = WaitpointId.toId(waitpointFriendlyId);
|
||||
|
||||
const waitpoint = await $replica.waitpoint.findFirst({
|
||||
select: {
|
||||
projectId: true,
|
||||
},
|
||||
where: {
|
||||
id: waitpointId,
|
||||
},
|
||||
});
|
||||
|
||||
if (waitpoint?.projectId !== project.id) {
|
||||
return redirectWithErrorMessage(
|
||||
submission.value.failureRedirect,
|
||||
request,
|
||||
"No waitpoint found"
|
||||
);
|
||||
}
|
||||
|
||||
switch (submission.value.type) {
|
||||
case "DATETIME": {
|
||||
const result = await engine.completeWaitpoint({
|
||||
id: waitpointId,
|
||||
});
|
||||
|
||||
return redirectWithSuccessMessage(
|
||||
submission.value.successRedirect,
|
||||
request,
|
||||
"Waitpoint skipped"
|
||||
);
|
||||
}
|
||||
case "MANUAL": {
|
||||
if (submission.value.isTimeout) {
|
||||
try {
|
||||
const result = await engine.completeWaitpoint({
|
||||
id: waitpointId,
|
||||
output: {
|
||||
type: "application/json",
|
||||
value: JSON.stringify(timeoutError(new Date())),
|
||||
isError: true,
|
||||
},
|
||||
});
|
||||
|
||||
return redirectWithSuccessMessage(
|
||||
submission.value.successRedirect,
|
||||
request,
|
||||
"Waitpoint timed out"
|
||||
);
|
||||
} catch (e) {
|
||||
return redirectWithErrorMessage(
|
||||
submission.value.failureRedirect,
|
||||
request,
|
||||
"Invalid payload, must be valid JSON"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
try {
|
||||
if (
|
||||
submission.value.payload &&
|
||||
submission.value.payload.length > env.TASK_PAYLOAD_MAXIMUM_SIZE
|
||||
) {
|
||||
return redirectWithErrorMessage(
|
||||
submission.value.failureRedirect,
|
||||
request,
|
||||
"Payload is too large"
|
||||
);
|
||||
}
|
||||
|
||||
const data = submission.value.payload ? JSON.parse(submission.value.payload) : {};
|
||||
const stringifiedData = await stringifyIO(data);
|
||||
const finalData = await conditionallyExportPacket(
|
||||
stringifiedData,
|
||||
`${waitpointId}/waitpoint/token`
|
||||
);
|
||||
|
||||
const result = await engine.completeWaitpoint({
|
||||
id: waitpointId,
|
||||
output: finalData.data
|
||||
? { type: finalData.dataType, value: finalData.data, isError: false }
|
||||
: undefined,
|
||||
});
|
||||
|
||||
return redirectWithSuccessMessage(
|
||||
submission.value.successRedirect,
|
||||
request,
|
||||
"Waitpoint completed"
|
||||
);
|
||||
} catch (e) {
|
||||
return redirectWithErrorMessage(
|
||||
submission.value.failureRedirect,
|
||||
request,
|
||||
"Invalid payload, must be valid JSON"
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
} catch (error: any) {
|
||||
logger.error("Failed to complete waitpoint", error);
|
||||
|
||||
const errorMessage = `Something went wrong. Please try again.`;
|
||||
return redirectWithErrorMessage(
|
||||
v3RunsPath({ slug: organizationSlug }, { slug: projectParam }),
|
||||
request,
|
||||
errorMessage
|
||||
);
|
||||
}
|
||||
};
|
||||
|
||||
type FormWaitpoint = Pick<Waitpoint, "friendlyId" | "type" | "completedAfter" | "status">;
|
||||
|
||||
export function CompleteWaitpointForm({ waitpoint }: { waitpoint: FormWaitpoint }) {
|
||||
const navigation = useNavigation();
|
||||
const submit = useSubmit();
|
||||
const isLoading = navigation.state !== "idle";
|
||||
const organization = useOrganization();
|
||||
const project = useProject();
|
||||
|
||||
return (
|
||||
<div className="space-y-3">
|
||||
{waitpoint.type === "DATETIME" ? (
|
||||
waitpoint.completedAfter ? (
|
||||
<CompleteDateTimeWaitpointForm
|
||||
waitpoint={{
|
||||
friendlyId: waitpoint.friendlyId,
|
||||
completedAfter: waitpoint.completedAfter,
|
||||
}}
|
||||
/>
|
||||
) : (
|
||||
<>Waitpoint doesn't have a complete date</>
|
||||
)
|
||||
) : (
|
||||
<CompleteManualWaitpointForm waitpoint={waitpoint} />
|
||||
)}
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
function CompleteDateTimeWaitpointForm({
|
||||
waitpoint,
|
||||
}: {
|
||||
waitpoint: { friendlyId: string; completedAfter: Date };
|
||||
}) {
|
||||
const location = useLocation();
|
||||
const navigation = useNavigation();
|
||||
const isLoading = navigation.state !== "idle";
|
||||
const organization = useOrganization();
|
||||
const project = useProject();
|
||||
|
||||
const timeToComplete = waitpoint.completedAfter.getTime() - Date.now();
|
||||
if (timeToComplete < 0) {
|
||||
return (
|
||||
<div className="flex items-center justify-center">
|
||||
<Paragraph variant="small/bright">Waitpoint completed</Paragraph>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
return (
|
||||
<Form
|
||||
action={`/resources/orgs/${organization.slug}/projects/${project.slug}/waitpoints/${waitpoint.friendlyId}/complete`}
|
||||
method="post"
|
||||
className="grid h-full max-h-full grid-rows-[2.5rem_1fr_3.25rem] overflow-hidden border-t border-grid-bright"
|
||||
>
|
||||
<div className="mx-3 flex items-center">
|
||||
<Paragraph variant="small/bright">Manually skip this waitpoint</Paragraph>
|
||||
</div>
|
||||
<div className="border-t border-grid-dimmed">
|
||||
<input type="hidden" name="type" value={"DATETIME"} />
|
||||
<input
|
||||
type="hidden"
|
||||
name="successRedirect"
|
||||
value={`${location.pathname}${location.search}`}
|
||||
/>
|
||||
<input
|
||||
type="hidden"
|
||||
name="failureRedirect"
|
||||
value={`${location.pathname}${location.search}`}
|
||||
/>
|
||||
<div className="flex flex-wrap items-center justify-between gap-1 p-2 text-sm tabular-nums">
|
||||
<div className="flex items-center gap-1">
|
||||
<AnimatedHourglassIcon
|
||||
className="text-dimmed-dimmed size-4"
|
||||
delay={(waitpoint.completedAfter.getMilliseconds() - Date.now()) / 1000}
|
||||
/>
|
||||
<span className="mt-0.5 ">
|
||||
<LiveCountdown endTime={waitpoint.completedAfter} />
|
||||
</span>
|
||||
</div>
|
||||
<DateTime date={waitpoint.completedAfter} />
|
||||
</div>
|
||||
</div>
|
||||
<div className="flex items-center justify-end border-t border-grid-dimmed bg-background-dimmed px-2">
|
||||
<Button
|
||||
variant="secondary/medium"
|
||||
type="submit"
|
||||
disabled={isLoading}
|
||||
LeadingIcon={isLoading ? "spinner" : undefined}
|
||||
>
|
||||
{isLoading ? "Completing…" : "Skip waitpoint"}
|
||||
</Button>
|
||||
</div>
|
||||
</Form>
|
||||
);
|
||||
}
|
||||
|
||||
function CompleteManualWaitpointForm({ waitpoint }: { waitpoint: { friendlyId: string } }) {
|
||||
const location = useLocation();
|
||||
const navigation = useNavigation();
|
||||
const submit = useSubmit();
|
||||
const isLoading = navigation.state !== "idle";
|
||||
const organization = useOrganization();
|
||||
const project = useProject();
|
||||
const currentJson = useRef<string>("{\n\n}");
|
||||
const formAction = `/resources/orgs/${organization.slug}/projects/${project.slug}/waitpoints/${waitpoint.friendlyId}/complete`;
|
||||
|
||||
const submitForm = useCallback(
|
||||
(e: React.FormEvent<HTMLFormElement>) => {
|
||||
const formData = new FormData(e.currentTarget);
|
||||
const data: Record<string, string> = {
|
||||
type: formData.get("type") as string,
|
||||
failureRedirect: formData.get("failureRedirect") as string,
|
||||
successRedirect: formData.get("successRedirect") as string,
|
||||
};
|
||||
|
||||
data.payload = currentJson.current;
|
||||
|
||||
submit(data, {
|
||||
action: formAction,
|
||||
method: "post",
|
||||
});
|
||||
e.preventDefault();
|
||||
},
|
||||
[currentJson]
|
||||
);
|
||||
|
||||
return (
|
||||
<>
|
||||
<Form
|
||||
action={formAction}
|
||||
method="post"
|
||||
onSubmit={(e) => submitForm(e)}
|
||||
className="grid h-full max-h-full grid-rows-[2.5rem_1fr_3.25rem] overflow-hidden border-t border-grid-bright"
|
||||
>
|
||||
<input type="hidden" name="type" value={"MANUAL"} />
|
||||
<input
|
||||
type="hidden"
|
||||
name="successRedirect"
|
||||
value={`${location.pathname}${location.search}`}
|
||||
/>
|
||||
<input
|
||||
type="hidden"
|
||||
name="failureRedirect"
|
||||
value={`${location.pathname}${location.search}`}
|
||||
/>
|
||||
<div className="mx-3 flex items-center gap-1">
|
||||
<Paragraph variant="small/bright">Manually complete this waitpoint</Paragraph>
|
||||
<InfoIconTooltip
|
||||
content={
|
||||
"This is will immediately complete this waitpoint with the payload you specify. This is useful during development for testing."
|
||||
}
|
||||
contentClassName="normal-case tracking-normal max-w-xs"
|
||||
/>
|
||||
</div>
|
||||
<div className="overflow-y-auto bg-charcoal-900 scrollbar-thin scrollbar-track-transparent scrollbar-thumb-charcoal-600">
|
||||
<div className="max-h-[70vh] min-h-40 overflow-y-auto bg-charcoal-900 scrollbar-thin scrollbar-track-transparent scrollbar-thumb-charcoal-600">
|
||||
<JSONEditor
|
||||
autoFocus
|
||||
defaultValue={currentJson.current}
|
||||
readOnly={false}
|
||||
basicSetup
|
||||
onChange={(v) => {
|
||||
currentJson.current = v;
|
||||
}}
|
||||
showClearButton={false}
|
||||
showCopyButton={false}
|
||||
height="100%"
|
||||
min-height="100%"
|
||||
max-height="100%"
|
||||
/>
|
||||
</div>
|
||||
</div>
|
||||
<div className="flex items-center justify-end gap-2 border-t border-grid-dimmed bg-background-dimmed px-2">
|
||||
<Button
|
||||
variant="secondary/medium"
|
||||
type="submit"
|
||||
disabled={isLoading}
|
||||
LeadingIcon={isLoading ? "spinner" : undefined}
|
||||
>
|
||||
{isLoading ? "Completing…" : "Complete waitpoint"}
|
||||
</Button>
|
||||
</div>
|
||||
</Form>
|
||||
</>
|
||||
);
|
||||
}
|
||||
|
||||
export function ForceTimeout({ waitpoint }: { waitpoint: { friendlyId: string } }) {
|
||||
const location = useLocation();
|
||||
const navigation = useNavigation();
|
||||
const isLoading = navigation.state !== "idle";
|
||||
const organization = useOrganization();
|
||||
const project = useProject();
|
||||
const formAction = `/resources/orgs/${organization.slug}/projects/${project.slug}/waitpoints/${waitpoint.friendlyId}/complete`;
|
||||
|
||||
return (
|
||||
<Form action={formAction} method="post">
|
||||
<input type="hidden" name="type" value={"MANUAL"} />
|
||||
<input type="hidden" name="isTimeout" value={"1"} />
|
||||
<input
|
||||
type="hidden"
|
||||
name="successRedirect"
|
||||
value={`${location.pathname}${location.search}`}
|
||||
/>
|
||||
<input
|
||||
type="hidden"
|
||||
name="failureRedirect"
|
||||
value={`${location.pathname}${location.search}`}
|
||||
/>
|
||||
<Button
|
||||
variant="tertiary/small"
|
||||
type="submit"
|
||||
disabled={isLoading}
|
||||
LeadingIcon={isLoading ? "spinner" : undefined}
|
||||
>
|
||||
{isLoading ? "Forcing timeout…" : "Force timeout"}
|
||||
</Button>
|
||||
</Form>
|
||||
);
|
||||
}
|
||||
+325
-177
@@ -59,6 +59,7 @@ import { requireUserId } from "~/services/session.server";
|
||||
import { cn } from "~/utils/cn";
|
||||
import { formatCurrencyAccurate } from "~/utils/numberFormatter";
|
||||
import {
|
||||
docsPath,
|
||||
v3BatchPath,
|
||||
v3RunDownloadLogsPath,
|
||||
v3RunPath,
|
||||
@@ -67,6 +68,11 @@ import {
|
||||
v3SchedulePath,
|
||||
v3SpanParamsSchema,
|
||||
} from "~/utils/pathBuilder";
|
||||
import { SpanLink } from "~/v3/eventRepository.server";
|
||||
import {
|
||||
CompleteWaitpointForm,
|
||||
ForceTimeout,
|
||||
} from "../resources.orgs.$organizationSlug.projects.$projectParam.waitpoints.$waitpointFriendlyId.complete/route";
|
||||
|
||||
export const loader = async ({ request, params }: LoaderFunctionArgs) => {
|
||||
const userId = await requireUserId(request);
|
||||
@@ -170,7 +176,6 @@ function SpanBody({
|
||||
runParam?: string;
|
||||
closePanel?: () => void;
|
||||
}) {
|
||||
const isAdmin = useHasAdminAccess();
|
||||
const organization = useOrganization();
|
||||
const project = useProject();
|
||||
const { value, replace } = useSearchParams();
|
||||
@@ -190,7 +195,7 @@ function SpanBody({
|
||||
className="h-4 min-h-4 w-4 min-w-4"
|
||||
/>
|
||||
<Header2 className={cn("overflow-x-hidden")}>
|
||||
<SpanTitle {...span} size="large" />
|
||||
<SpanTitle {...span} size="large" hideAccessory />
|
||||
</Header2>
|
||||
</div>
|
||||
{runParam && closePanel && (
|
||||
@@ -226,179 +231,67 @@ function SpanBody({
|
||||
</TabButton>
|
||||
</TabContainer>
|
||||
</div>
|
||||
<div className="overflow-y-auto px-3 scrollbar-thin scrollbar-track-transparent scrollbar-thumb-charcoal-600">
|
||||
<div>
|
||||
{tab === "detail" ? (
|
||||
<div className="flex flex-col gap-4 pt-3">
|
||||
<Property.Table>
|
||||
<Property.Item>
|
||||
<Property.Label>Status</Property.Label>
|
||||
<Property.Value>
|
||||
<TaskRunAttemptStatusCombo
|
||||
status={
|
||||
span.isCancelled
|
||||
? "CANCELED"
|
||||
: span.isError
|
||||
? "FAILED"
|
||||
: span.isPartial
|
||||
? "EXECUTING"
|
||||
: "COMPLETED"
|
||||
}
|
||||
className="text-sm"
|
||||
/>
|
||||
</Property.Value>
|
||||
</Property.Item>
|
||||
<Property.Item>
|
||||
<Property.Label>Task</Property.Label>
|
||||
<Property.Value>
|
||||
<SimpleTooltip
|
||||
button={
|
||||
<TextLink
|
||||
to={v3RunsPath(organization, project, { tasks: [span.taskSlug] })}
|
||||
>
|
||||
{span.taskSlug}
|
||||
</TextLink>
|
||||
}
|
||||
content={`Filter runs by ${span.taskSlug}`}
|
||||
/>
|
||||
</Property.Value>
|
||||
</Property.Item>
|
||||
{span.idempotencyKey && (
|
||||
<Property.Item>
|
||||
<Property.Label>Idempotency key</Property.Label>
|
||||
<Property.Value>{span.idempotencyKey}</Property.Value>
|
||||
</Property.Item>
|
||||
)}
|
||||
<Property.Item>
|
||||
<Property.Label>Version</Property.Label>
|
||||
<Property.Value>
|
||||
{span.workerVersion ? (
|
||||
span.workerVersion
|
||||
) : (
|
||||
<span className="flex items-center gap-1">
|
||||
<span>Never started</span>
|
||||
<InfoIconTooltip
|
||||
content={"Runs get locked to the latest version when they start."}
|
||||
contentClassName="normal-case tracking-normal"
|
||||
/>
|
||||
</span>
|
||||
)}
|
||||
</Property.Value>
|
||||
</Property.Item>
|
||||
</Property.Table>
|
||||
</div>
|
||||
) : (
|
||||
<div className="flex flex-col gap-4 pt-3">
|
||||
{span.level === "TRACE" ? (
|
||||
<>
|
||||
<div className="border-b border-grid-bright pb-3">
|
||||
<TaskRunAttemptStatusCombo
|
||||
status={
|
||||
span.isCancelled
|
||||
? "CANCELED"
|
||||
: span.isError
|
||||
? "FAILED"
|
||||
: span.isPartial
|
||||
? "EXECUTING"
|
||||
: "COMPLETED"
|
||||
}
|
||||
className="text-sm"
|
||||
/>
|
||||
</div>
|
||||
<SpanTimeline
|
||||
startTime={new Date(span.startTime)}
|
||||
duration={span.duration}
|
||||
inProgress={span.isPartial}
|
||||
isError={span.isError}
|
||||
events={createTimelineSpanEventsFromSpanEvents(span.events, isAdmin)}
|
||||
<div className="overflow-y-auto scrollbar-thin scrollbar-track-transparent scrollbar-thumb-charcoal-600">
|
||||
{tab === "detail" ? (
|
||||
<div className="flex flex-col gap-4 px-3 pt-3">
|
||||
<Property.Table>
|
||||
<Property.Item>
|
||||
<Property.Label>Status</Property.Label>
|
||||
<Property.Value>
|
||||
<TaskRunAttemptStatusCombo
|
||||
status={
|
||||
span.isCancelled
|
||||
? "CANCELED"
|
||||
: span.isError
|
||||
? "FAILED"
|
||||
: span.isPartial
|
||||
? "EXECUTING"
|
||||
: "COMPLETED"
|
||||
}
|
||||
className="text-sm"
|
||||
/>
|
||||
</>
|
||||
) : (
|
||||
<div className="min-w-fit max-w-80">
|
||||
<RunTimelineEvent
|
||||
title="Timestamp"
|
||||
subtitle={<DateTimeAccurate date={span.startTime} />}
|
||||
variant="dot-solid"
|
||||
</Property.Value>
|
||||
</Property.Item>
|
||||
<Property.Item>
|
||||
<Property.Label>Task</Property.Label>
|
||||
<Property.Value>
|
||||
<SimpleTooltip
|
||||
button={
|
||||
<TextLink to={v3RunsPath(organization, project, { tasks: [span.taskSlug] })}>
|
||||
{span.taskSlug}
|
||||
</TextLink>
|
||||
}
|
||||
content={`Filter runs by ${span.taskSlug}`}
|
||||
/>
|
||||
</div>
|
||||
)}
|
||||
<Property.Table>
|
||||
</Property.Value>
|
||||
</Property.Item>
|
||||
{span.idempotencyKey && (
|
||||
<Property.Item>
|
||||
<Property.Label>Message</Property.Label>
|
||||
<Property.Value className="whitespace-pre-wrap">{span.message}</Property.Value>
|
||||
<Property.Label>Idempotency key</Property.Label>
|
||||
<Property.Value>{span.idempotencyKey}</Property.Value>
|
||||
</Property.Item>
|
||||
{span.triggeredRuns.length > 0 && (
|
||||
<Property.Item>
|
||||
<div className="flex flex-col gap-1.5">
|
||||
<Header3>Triggered runs</Header3>
|
||||
<Table containerClassName="max-h-[12.5rem]">
|
||||
<TableHeader className="bg-background-bright">
|
||||
<TableRow>
|
||||
<TableHeaderCell>Run #</TableHeaderCell>
|
||||
<TableHeaderCell>Task</TableHeaderCell>
|
||||
<TableHeaderCell>Version</TableHeaderCell>
|
||||
<TableHeaderCell>Created at</TableHeaderCell>
|
||||
</TableRow>
|
||||
</TableHeader>
|
||||
<TableBody>
|
||||
{span.triggeredRuns.map((run) => {
|
||||
const path = v3RunSpanPath(
|
||||
organization,
|
||||
project,
|
||||
{ friendlyId: run.friendlyId },
|
||||
{ spanId: run.spanId }
|
||||
);
|
||||
return (
|
||||
<TableRow key={run.friendlyId}>
|
||||
<TableCell
|
||||
to={path}
|
||||
actionClassName="py-1.5"
|
||||
rowHoverStyle="bright"
|
||||
>
|
||||
{run.number}
|
||||
</TableCell>
|
||||
<TableCell
|
||||
to={path}
|
||||
actionClassName="py-1.5"
|
||||
rowHoverStyle="bright"
|
||||
>
|
||||
{run.taskIdentifier}
|
||||
</TableCell>
|
||||
<TableCell
|
||||
to={path}
|
||||
actionClassName="py-1.5"
|
||||
rowHoverStyle="bright"
|
||||
>
|
||||
{run.lockedToVersion?.version ?? "–"}
|
||||
</TableCell>
|
||||
<TableCell
|
||||
to={path}
|
||||
actionClassName="py-1.5"
|
||||
rowHoverStyle="bright"
|
||||
>
|
||||
<DateTime date={run.createdAt} />
|
||||
</TableCell>
|
||||
</TableRow>
|
||||
);
|
||||
})}
|
||||
</TableBody>
|
||||
</Table>
|
||||
</div>
|
||||
</Property.Item>
|
||||
)}
|
||||
</Property.Table>
|
||||
{span.events.length > 0 && <SpanEvents spanEvents={span.events} />}
|
||||
{span.properties !== undefined && (
|
||||
<CodeBlock
|
||||
rowTitle="Properties"
|
||||
code={span.properties}
|
||||
maxLines={20}
|
||||
showLineNumbers={false}
|
||||
/>
|
||||
)}
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
<Property.Item>
|
||||
<Property.Label>Version</Property.Label>
|
||||
<Property.Value>
|
||||
{span.workerVersion ? (
|
||||
span.workerVersion
|
||||
) : (
|
||||
<span className="flex items-center gap-1">
|
||||
<span>Never started</span>
|
||||
<InfoIconTooltip
|
||||
content={"Runs get locked to the latest version when they start."}
|
||||
contentClassName="normal-case tracking-normal"
|
||||
/>
|
||||
</span>
|
||||
)}
|
||||
</Property.Value>
|
||||
</Property.Item>
|
||||
</Property.Table>
|
||||
</div>
|
||||
) : (
|
||||
<SpanEntity span={span} />
|
||||
)}
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
@@ -417,6 +310,7 @@ function RunBody({
|
||||
}) {
|
||||
const organization = useOrganization();
|
||||
const project = useProject();
|
||||
const isAdmin = useHasAdminAccess();
|
||||
const { value, replace } = useSearchParams();
|
||||
const tab = value("tab");
|
||||
|
||||
@@ -427,12 +321,15 @@ function RunBody({
|
||||
<div className="flex items-center justify-between gap-2 overflow-x-hidden px-3">
|
||||
<div className="flex items-center gap-1 overflow-x-hidden">
|
||||
<RunIcon
|
||||
name={"task"}
|
||||
name={run.isCached ? "task-cached" : "task"}
|
||||
spanName={run.taskIdentifier}
|
||||
className="h-4 min-h-4 w-4 min-w-4"
|
||||
/>
|
||||
<Header2 className={cn("overflow-x-hidden text-blue-500")}>
|
||||
<span className="truncate">{run.taskIdentifier}</span>
|
||||
<span className="truncate">
|
||||
{run.taskIdentifier}
|
||||
{run.isCached ? " (cached)" : null}
|
||||
</span>
|
||||
</Header2>
|
||||
</div>
|
||||
{runParam && closePanel && (
|
||||
@@ -602,6 +499,22 @@ function RunBody({
|
||||
</Property.Value>
|
||||
</Property.Item>
|
||||
)}
|
||||
<Property.Item>
|
||||
<Property.Label>Idempotency</Property.Label>
|
||||
<Property.Value>
|
||||
<div className="break-all">{run.idempotencyKey ? run.idempotencyKey : "–"}</div>
|
||||
{run.idempotencyKey && (
|
||||
<div>
|
||||
Expires:{" "}
|
||||
{run.idempotencyKeyExpiresAt ? (
|
||||
<DateTime date={run.idempotencyKeyExpiresAt} />
|
||||
) : (
|
||||
"–"
|
||||
)}
|
||||
</div>
|
||||
)}
|
||||
</Property.Value>
|
||||
</Property.Item>
|
||||
<Property.Item>
|
||||
<Property.Label>Version</Property.Label>
|
||||
<Property.Value>
|
||||
@@ -634,6 +547,22 @@ function RunBody({
|
||||
)}
|
||||
</Property.Value>
|
||||
</Property.Item>
|
||||
<Property.Item>
|
||||
<Property.Label>Engine version</Property.Label>
|
||||
<Property.Value>{run.engine}</Property.Value>
|
||||
</Property.Item>
|
||||
{isAdmin && (
|
||||
<>
|
||||
<Property.Item>
|
||||
<Property.Label>Primary master queue</Property.Label>
|
||||
<Property.Value>{run.masterQueue}</Property.Value>
|
||||
</Property.Item>
|
||||
<Property.Item>
|
||||
<Property.Label>Secondary master queue</Property.Label>
|
||||
<Property.Value>{run.secondaryMasterQueue}</Property.Value>
|
||||
</Property.Item>
|
||||
</>
|
||||
)}
|
||||
<Property.Item>
|
||||
<Property.Label>Test run</Property.Label>
|
||||
<Property.Value>
|
||||
@@ -771,12 +700,13 @@ function RunBody({
|
||||
</div>
|
||||
<RunTimeline run={run} />
|
||||
|
||||
{run.error && <RunError error={run.error} />}
|
||||
|
||||
{run.payload !== undefined && (
|
||||
<PacketDisplay data={run.payload} dataType={run.payloadType} title="Payload" />
|
||||
)}
|
||||
{run.error !== undefined ? (
|
||||
<RunError error={run.error} />
|
||||
) : run.output !== undefined ? (
|
||||
|
||||
{run.error === undefined && run.output !== undefined ? (
|
||||
<PacketDisplay data={run.output} dataType={run.outputType} title="Output" />
|
||||
) : null}
|
||||
</div>
|
||||
@@ -787,12 +717,17 @@ function RunBody({
|
||||
<div className="flex items-center gap-4">
|
||||
{run.friendlyId !== runParam && (
|
||||
<LinkButton
|
||||
to={v3RunSpanPath(organization, project, { friendlyId: run.friendlyId }, { spanId })}
|
||||
to={v3RunSpanPath(
|
||||
organization,
|
||||
project,
|
||||
{ friendlyId: run.friendlyId },
|
||||
{ spanId: run.spanId }
|
||||
)}
|
||||
variant="minimal/medium"
|
||||
LeadingIcon={QueueListIcon}
|
||||
shortcut={{ key: "f" }}
|
||||
>
|
||||
Focus on run
|
||||
{run.isCached ? "Jump to original run" : "Focus on run"}
|
||||
</LinkButton>
|
||||
)}
|
||||
</div>
|
||||
@@ -925,3 +860,216 @@ function PacketDisplay({
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
function SpanEntity({ span }: { span: Span }) {
|
||||
const isAdmin = useHasAdminAccess();
|
||||
|
||||
const organization = useOrganization();
|
||||
const project = useProject();
|
||||
|
||||
if (!span.entity) {
|
||||
//normal span
|
||||
return (
|
||||
<div className="flex flex-col gap-4 p-3">
|
||||
{span.level === "TRACE" ? (
|
||||
<>
|
||||
<div className="border-b border-grid-bright pb-3">
|
||||
<TaskRunAttemptStatusCombo
|
||||
status={
|
||||
span.isCancelled
|
||||
? "CANCELED"
|
||||
: span.isError
|
||||
? "FAILED"
|
||||
: span.isPartial
|
||||
? "EXECUTING"
|
||||
: "COMPLETED"
|
||||
}
|
||||
className="text-sm"
|
||||
/>
|
||||
</div>
|
||||
<SpanTimeline
|
||||
startTime={new Date(span.startTime)}
|
||||
duration={span.duration}
|
||||
inProgress={span.isPartial}
|
||||
isError={span.isError}
|
||||
events={createTimelineSpanEventsFromSpanEvents(span.events, isAdmin)}
|
||||
/>
|
||||
</>
|
||||
) : (
|
||||
<div className="min-w-fit max-w-80">
|
||||
<RunTimelineEvent
|
||||
title="Timestamp"
|
||||
subtitle={<DateTimeAccurate date={span.startTime} />}
|
||||
variant="dot-solid"
|
||||
/>
|
||||
</div>
|
||||
)}
|
||||
<Property.Table>
|
||||
<Property.Item>
|
||||
<Property.Label>Message</Property.Label>
|
||||
<Property.Value className="whitespace-pre-wrap">{span.message}</Property.Value>
|
||||
</Property.Item>
|
||||
{span.triggeredRuns.length > 0 && (
|
||||
<Property.Item>
|
||||
<div className="flex flex-col gap-1.5">
|
||||
<Header3>Triggered runs</Header3>
|
||||
<Table containerClassName="max-h-[12.5rem]">
|
||||
<TableHeader className="bg-background-bright">
|
||||
<TableRow>
|
||||
<TableHeaderCell>Run #</TableHeaderCell>
|
||||
<TableHeaderCell>Task</TableHeaderCell>
|
||||
<TableHeaderCell>Version</TableHeaderCell>
|
||||
<TableHeaderCell>Created at</TableHeaderCell>
|
||||
</TableRow>
|
||||
</TableHeader>
|
||||
<TableBody>
|
||||
{span.triggeredRuns.map((run) => {
|
||||
const path = v3RunSpanPath(
|
||||
organization,
|
||||
project,
|
||||
{ friendlyId: run.friendlyId },
|
||||
{ spanId: run.spanId }
|
||||
);
|
||||
return (
|
||||
<TableRow key={run.friendlyId}>
|
||||
<TableCell to={path} actionClassName="py-1.5" rowHoverStyle="bright">
|
||||
{run.number}
|
||||
</TableCell>
|
||||
<TableCell to={path} actionClassName="py-1.5" rowHoverStyle="bright">
|
||||
{run.taskIdentifier}
|
||||
</TableCell>
|
||||
<TableCell to={path} actionClassName="py-1.5" rowHoverStyle="bright">
|
||||
{run.lockedToVersion?.version ?? "–"}
|
||||
</TableCell>
|
||||
<TableCell to={path} actionClassName="py-1.5" rowHoverStyle="bright">
|
||||
<DateTime date={run.createdAt} />
|
||||
</TableCell>
|
||||
</TableRow>
|
||||
);
|
||||
})}
|
||||
</TableBody>
|
||||
</Table>
|
||||
</div>
|
||||
</Property.Item>
|
||||
)}
|
||||
</Property.Table>
|
||||
{span.events.length > 0 && <SpanEvents spanEvents={span.events} />}
|
||||
{span.properties !== undefined ? (
|
||||
<CodeBlock
|
||||
rowTitle="Properties"
|
||||
code={span.properties}
|
||||
maxLines={20}
|
||||
showLineNumbers={false}
|
||||
/>
|
||||
) : null}
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
switch (span.entity.type) {
|
||||
case "waitpoint": {
|
||||
return (
|
||||
<div className="grid h-full grid-rows-[1fr_auto]">
|
||||
<div className="flex flex-col gap-4 overflow-y-auto px-3 pt-3 scrollbar-thin scrollbar-track-transparent scrollbar-thumb-charcoal-600">
|
||||
<div>
|
||||
<Header2>Waitpoint</Header2>
|
||||
<Paragraph variant="small">
|
||||
A waitpoint pauses your code from continuing until the conditions are met.{" "}
|
||||
<TextLink to={docsPath("wait")}>View docs</TextLink>.
|
||||
</Paragraph>
|
||||
</div>
|
||||
<Property.Table>
|
||||
<Property.Item>
|
||||
<Property.Label>Status</Property.Label>
|
||||
<Property.Value>
|
||||
<TaskRunStatusCombo
|
||||
status={
|
||||
span.entity.object.isTimeout
|
||||
? "TIMED_OUT"
|
||||
: span.entity.object.status === "PENDING"
|
||||
? "EXECUTING"
|
||||
: span.entity.object.outputIsError
|
||||
? "COMPLETED_WITH_ERRORS"
|
||||
: "COMPLETED_SUCCESSFULLY"
|
||||
}
|
||||
className="text-sm"
|
||||
/>
|
||||
</Property.Value>
|
||||
</Property.Item>
|
||||
<Property.Item>
|
||||
<Property.Label>ID</Property.Label>
|
||||
<Property.Value className="whitespace-pre-wrap">
|
||||
{span.entity.object.friendlyId}
|
||||
</Property.Value>
|
||||
</Property.Item>
|
||||
<Property.Item>
|
||||
<Property.Label>Idempotency key</Property.Label>
|
||||
<Property.Value>
|
||||
<div>
|
||||
<div>
|
||||
{span.entity.object.userProvidedIdempotencyKey
|
||||
? span.entity.object.idempotencyKey
|
||||
: "–"}
|
||||
</div>
|
||||
<div>
|
||||
{span.entity.object.idempotencyKeyExpiresAt ? (
|
||||
<>
|
||||
TTL: <DateTime date={span.entity.object.idempotencyKeyExpiresAt} />
|
||||
</>
|
||||
) : null}
|
||||
</div>
|
||||
</div>
|
||||
</Property.Value>
|
||||
</Property.Item>
|
||||
{span.entity.object.type === "MANUAL" && (
|
||||
<>
|
||||
<Property.Item>
|
||||
<Property.Label>Timeout at</Property.Label>
|
||||
<Property.Value>
|
||||
<div className="flex w-full flex-wrap items-center justify-between gap-1">
|
||||
{span.entity.object.completedAfter ? (
|
||||
<DateTimeAccurate date={span.entity.object.completedAfter} />
|
||||
) : (
|
||||
"–"
|
||||
)}
|
||||
{span.entity.object.status === "PENDING" && (
|
||||
<ForceTimeout waitpoint={span.entity.object} />
|
||||
)}
|
||||
</div>
|
||||
</Property.Value>
|
||||
</Property.Item>
|
||||
</>
|
||||
)}
|
||||
{span.entity.object.status === "PENDING" ? null : span.entity.object.isTimeout ? (
|
||||
<></>
|
||||
) : span.entity.object.output ? (
|
||||
<PacketDisplay
|
||||
title="Output"
|
||||
data={span.entity.object.output}
|
||||
dataType={span.entity.object.outputType}
|
||||
/>
|
||||
) : span.entity.object.completedAfter ? (
|
||||
<Property.Item>
|
||||
<Property.Label>Completed at</Property.Label>
|
||||
<Property.Value>
|
||||
<DateTimeAccurate date={span.entity.object.completedAfter} />
|
||||
</Property.Value>
|
||||
</Property.Item>
|
||||
) : (
|
||||
"Completed with no output"
|
||||
)}
|
||||
</Property.Table>
|
||||
</div>
|
||||
{span.entity.object.status === "PENDING" && (
|
||||
<div className="">
|
||||
<CompleteWaitpointForm waitpoint={span.entity.object} />
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
);
|
||||
}
|
||||
default: {
|
||||
return <Paragraph variant="small">No span for {span.entity.type}</Paragraph>;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,34 @@
|
||||
import { env } from "~/env.server";
|
||||
import { authenticateAuthorizationHeader } from "./apiAuth.server";
|
||||
import { authorizationRateLimitMiddleware } from "./authorizationRateLimitMiddleware.server";
|
||||
import { Duration } from "./rateLimiter.server";
|
||||
|
||||
export const engineRateLimiter = authorizationRateLimitMiddleware({
|
||||
redis: {
|
||||
port: env.RATE_LIMIT_REDIS_PORT,
|
||||
host: env.RATE_LIMIT_REDIS_HOST,
|
||||
username: env.RATE_LIMIT_REDIS_USERNAME,
|
||||
password: env.RATE_LIMIT_REDIS_PASSWORD,
|
||||
tlsDisabled: env.RATE_LIMIT_REDIS_TLS_DISABLED === "true",
|
||||
clusterMode: env.RATE_LIMIT_REDIS_CLUSTER_MODE_ENABLED === "1",
|
||||
},
|
||||
keyPrefix: "engine",
|
||||
defaultLimiter: {
|
||||
type: "tokenBucket",
|
||||
refillRate: env.RUN_ENGINE_RATE_LIMIT_REFILL_RATE,
|
||||
interval: env.RUN_ENGINE_RATE_LIMIT_REFILL_INTERVAL as Duration,
|
||||
maxTokens: env.RUN_ENGINE_RATE_LIMIT_MAX,
|
||||
},
|
||||
limiterCache: {
|
||||
fresh: 60_000 * 10, // Data is fresh for 10 minutes
|
||||
stale: 60_000 * 20, // Date is stale after 20 minutes
|
||||
},
|
||||
pathMatchers: [/^\/engine/],
|
||||
// Allow /api/v1/tasks/:id/callback/:secret
|
||||
pathWhiteList: [],
|
||||
log: {
|
||||
rejections: env.RUN_ENGINE_RATE_LIMIT_REJECTION_LOGS_ENABLED === "1",
|
||||
requests: env.RUN_ENGINE_RATE_LIMIT_REQUEST_LOGS_ENABLED === "1",
|
||||
limiter: env.RUN_ENGINE_RATE_LIMIT_LIMITER_LOGS_ENABLED === "1",
|
||||
},
|
||||
});
|
||||
@@ -129,6 +129,7 @@ export async function authenticatePersonalAccessToken(
|
||||
token: string
|
||||
): Promise<PersonalAccessTokenAuthenticationResult | undefined> {
|
||||
if (!token.startsWith(tokenPrefix)) {
|
||||
logger.warn(`PAT doesn't start with ${tokenPrefix}`);
|
||||
return;
|
||||
}
|
||||
|
||||
|
||||
@@ -17,11 +17,17 @@ import {
|
||||
PersonalAccessTokenAuthenticationResult,
|
||||
} from "../personalAccessToken.server";
|
||||
import { safeJsonParse } from "~/utils/json";
|
||||
import {
|
||||
AuthenticatedWorkerInstance,
|
||||
WorkerGroupTokenService,
|
||||
} from "~/v3/services/worker/workerGroupTokenService.server";
|
||||
|
||||
type AnyZodSchema = z.ZodFirstPartySchemaTypes | z.ZodDiscriminatedUnion<any, any>;
|
||||
|
||||
type ApiKeyRouteBuilderOptions<
|
||||
TParamsSchema extends z.AnyZodObject | undefined = undefined,
|
||||
TSearchParamsSchema extends z.AnyZodObject | undefined = undefined,
|
||||
THeadersSchema extends z.AnyZodObject | undefined = undefined,
|
||||
TParamsSchema extends AnyZodSchema | undefined = undefined,
|
||||
TSearchParamsSchema extends AnyZodSchema | undefined = undefined,
|
||||
THeadersSchema extends AnyZodSchema | undefined = undefined,
|
||||
TResource = never
|
||||
> = {
|
||||
params?: TParamsSchema;
|
||||
@@ -30,7 +36,9 @@ type ApiKeyRouteBuilderOptions<
|
||||
allowJWT?: boolean;
|
||||
corsStrategy?: "all" | "none";
|
||||
findResource: (
|
||||
params: TParamsSchema extends z.AnyZodObject ? z.infer<TParamsSchema> : undefined,
|
||||
params: TParamsSchema extends z.ZodFirstPartySchemaTypes | z.ZodDiscriminatedUnion<any, any>
|
||||
? z.infer<TParamsSchema>
|
||||
: undefined,
|
||||
authentication: ApiAuthenticationResultSuccess
|
||||
) => Promise<TResource | undefined>;
|
||||
shouldRetryNotFound?: boolean;
|
||||
@@ -38,36 +46,48 @@ type ApiKeyRouteBuilderOptions<
|
||||
action: AuthorizationAction;
|
||||
resource: (
|
||||
resource: NonNullable<TResource>,
|
||||
params: TParamsSchema extends z.AnyZodObject ? z.infer<TParamsSchema> : undefined,
|
||||
searchParams: TSearchParamsSchema extends z.AnyZodObject
|
||||
params: TParamsSchema extends z.ZodFirstPartySchemaTypes | z.ZodDiscriminatedUnion<any, any>
|
||||
? z.infer<TParamsSchema>
|
||||
: undefined,
|
||||
searchParams: TSearchParamsSchema extends
|
||||
| z.ZodFirstPartySchemaTypes
|
||||
| z.ZodDiscriminatedUnion<any, any>
|
||||
? z.infer<TSearchParamsSchema>
|
||||
: undefined,
|
||||
headers: THeadersSchema extends z.AnyZodObject ? z.infer<THeadersSchema> : undefined
|
||||
headers: THeadersSchema extends z.ZodFirstPartySchemaTypes | z.ZodDiscriminatedUnion<any, any>
|
||||
? z.infer<THeadersSchema>
|
||||
: undefined
|
||||
) => AuthorizationResources;
|
||||
superScopes?: string[];
|
||||
};
|
||||
};
|
||||
|
||||
type ApiKeyHandlerFunction<
|
||||
TParamsSchema extends z.AnyZodObject | undefined,
|
||||
TSearchParamsSchema extends z.AnyZodObject | undefined,
|
||||
THeadersSchema extends z.AnyZodObject | undefined = undefined,
|
||||
TParamsSchema extends AnyZodSchema | undefined,
|
||||
TSearchParamsSchema extends AnyZodSchema | undefined,
|
||||
THeadersSchema extends AnyZodSchema | undefined = undefined,
|
||||
TResource = never
|
||||
> = (args: {
|
||||
params: TParamsSchema extends z.AnyZodObject ? z.infer<TParamsSchema> : undefined;
|
||||
searchParams: TSearchParamsSchema extends z.AnyZodObject
|
||||
params: TParamsSchema extends z.ZodFirstPartySchemaTypes | z.ZodDiscriminatedUnion<any, any>
|
||||
? z.infer<TParamsSchema>
|
||||
: undefined;
|
||||
searchParams: TSearchParamsSchema extends
|
||||
| z.ZodFirstPartySchemaTypes
|
||||
| z.ZodDiscriminatedUnion<any, any>
|
||||
? z.infer<TSearchParamsSchema>
|
||||
: undefined;
|
||||
headers: THeadersSchema extends z.AnyZodObject ? z.infer<THeadersSchema> : undefined;
|
||||
headers: THeadersSchema extends z.ZodFirstPartySchemaTypes | z.ZodDiscriminatedUnion<any, any>
|
||||
? z.infer<THeadersSchema>
|
||||
: undefined;
|
||||
authentication: ApiAuthenticationResultSuccess;
|
||||
request: Request;
|
||||
resource: NonNullable<TResource>;
|
||||
}) => Promise<Response>;
|
||||
|
||||
export function createLoaderApiRoute<
|
||||
TParamsSchema extends z.AnyZodObject | undefined = undefined,
|
||||
TSearchParamsSchema extends z.AnyZodObject | undefined = undefined,
|
||||
THeadersSchema extends z.AnyZodObject | undefined = undefined,
|
||||
TParamsSchema extends AnyZodSchema | undefined = undefined,
|
||||
TSearchParamsSchema extends AnyZodSchema | undefined = undefined,
|
||||
THeadersSchema extends AnyZodSchema | undefined = undefined,
|
||||
TResource = never
|
||||
>(
|
||||
options: ApiKeyRouteBuilderOptions<TParamsSchema, TSearchParamsSchema, THeadersSchema, TResource>,
|
||||
@@ -226,6 +246,19 @@ export function createLoaderApiRoute<
|
||||
if (error instanceof Response) {
|
||||
return await wrapResponse(request, error, corsStrategy !== "none");
|
||||
}
|
||||
|
||||
logger.error("Error in loader", {
|
||||
error:
|
||||
error instanceof Error
|
||||
? {
|
||||
name: error.name,
|
||||
message: error.message,
|
||||
stack: error.stack,
|
||||
}
|
||||
: String(error),
|
||||
url: request.url,
|
||||
});
|
||||
|
||||
return await wrapResponse(
|
||||
request,
|
||||
json({ error: "Internal Server Error" }, { status: 500 }),
|
||||
@@ -241,9 +274,9 @@ export function createLoaderApiRoute<
|
||||
}
|
||||
|
||||
type PATRouteBuilderOptions<
|
||||
TParamsSchema extends z.AnyZodObject | undefined = undefined,
|
||||
TSearchParamsSchema extends z.AnyZodObject | undefined = undefined,
|
||||
THeadersSchema extends z.AnyZodObject | undefined = undefined
|
||||
TParamsSchema extends AnyZodSchema | undefined = undefined,
|
||||
TSearchParamsSchema extends AnyZodSchema | undefined = undefined,
|
||||
THeadersSchema extends AnyZodSchema | undefined = undefined
|
||||
> = {
|
||||
params?: TParamsSchema;
|
||||
searchParams?: TSearchParamsSchema;
|
||||
@@ -252,23 +285,29 @@ type PATRouteBuilderOptions<
|
||||
};
|
||||
|
||||
type PATHandlerFunction<
|
||||
TParamsSchema extends z.AnyZodObject | undefined,
|
||||
TSearchParamsSchema extends z.AnyZodObject | undefined,
|
||||
THeadersSchema extends z.AnyZodObject | undefined = undefined
|
||||
TParamsSchema extends AnyZodSchema | undefined,
|
||||
TSearchParamsSchema extends AnyZodSchema | undefined,
|
||||
THeadersSchema extends AnyZodSchema | undefined = undefined
|
||||
> = (args: {
|
||||
params: TParamsSchema extends z.AnyZodObject ? z.infer<TParamsSchema> : undefined;
|
||||
searchParams: TSearchParamsSchema extends z.AnyZodObject
|
||||
params: TParamsSchema extends z.ZodFirstPartySchemaTypes | z.ZodDiscriminatedUnion<any, any>
|
||||
? z.infer<TParamsSchema>
|
||||
: undefined;
|
||||
searchParams: TSearchParamsSchema extends
|
||||
| z.ZodFirstPartySchemaTypes
|
||||
| z.ZodDiscriminatedUnion<any, any>
|
||||
? z.infer<TSearchParamsSchema>
|
||||
: undefined;
|
||||
headers: THeadersSchema extends z.AnyZodObject ? z.infer<THeadersSchema> : undefined;
|
||||
headers: THeadersSchema extends z.ZodFirstPartySchemaTypes | z.ZodDiscriminatedUnion<any, any>
|
||||
? z.infer<THeadersSchema>
|
||||
: undefined;
|
||||
authentication: PersonalAccessTokenAuthenticationResult;
|
||||
request: Request;
|
||||
}) => Promise<Response>;
|
||||
|
||||
export function createLoaderPATApiRoute<
|
||||
TParamsSchema extends z.AnyZodObject | undefined = undefined,
|
||||
TSearchParamsSchema extends z.AnyZodObject | undefined = undefined,
|
||||
THeadersSchema extends z.AnyZodObject | undefined = undefined
|
||||
TParamsSchema extends AnyZodSchema | undefined = undefined,
|
||||
TSearchParamsSchema extends AnyZodSchema | undefined = undefined,
|
||||
THeadersSchema extends AnyZodSchema | undefined = undefined
|
||||
>(
|
||||
options: PATRouteBuilderOptions<TParamsSchema, TSearchParamsSchema, THeadersSchema>,
|
||||
handler: PATHandlerFunction<TParamsSchema, TSearchParamsSchema, THeadersSchema>
|
||||
@@ -374,10 +413,10 @@ export function createLoaderPATApiRoute<
|
||||
}
|
||||
|
||||
type ApiKeyActionRouteBuilderOptions<
|
||||
TParamsSchema extends z.AnyZodObject | undefined = undefined,
|
||||
TSearchParamsSchema extends z.AnyZodObject | undefined = undefined,
|
||||
THeadersSchema extends z.AnyZodObject | undefined = undefined,
|
||||
TBodySchema extends z.AnyZodObject | undefined = undefined
|
||||
TParamsSchema extends AnyZodSchema | undefined = undefined,
|
||||
TSearchParamsSchema extends AnyZodSchema | undefined = undefined,
|
||||
THeadersSchema extends AnyZodSchema | undefined = undefined,
|
||||
TBodySchema extends AnyZodSchema | undefined = undefined
|
||||
> = {
|
||||
params?: TParamsSchema;
|
||||
searchParams?: TSearchParamsSchema;
|
||||
@@ -388,12 +427,20 @@ type ApiKeyActionRouteBuilderOptions<
|
||||
authorization?: {
|
||||
action: AuthorizationAction;
|
||||
resource: (
|
||||
params: TParamsSchema extends z.AnyZodObject ? z.infer<TParamsSchema> : undefined,
|
||||
searchParams: TSearchParamsSchema extends z.AnyZodObject
|
||||
params: TParamsSchema extends z.ZodFirstPartySchemaTypes | z.ZodDiscriminatedUnion<any, any>
|
||||
? z.infer<TParamsSchema>
|
||||
: undefined,
|
||||
searchParams: TSearchParamsSchema extends
|
||||
| z.ZodFirstPartySchemaTypes
|
||||
| z.ZodDiscriminatedUnion<any, any>
|
||||
? z.infer<TSearchParamsSchema>
|
||||
: undefined,
|
||||
headers: THeadersSchema extends z.AnyZodObject ? z.infer<THeadersSchema> : undefined,
|
||||
body: TBodySchema extends z.AnyZodObject ? z.infer<TBodySchema> : undefined
|
||||
headers: THeadersSchema extends z.ZodFirstPartySchemaTypes | z.ZodDiscriminatedUnion<any, any>
|
||||
? z.infer<THeadersSchema>
|
||||
: undefined,
|
||||
body: TBodySchema extends z.ZodFirstPartySchemaTypes | z.ZodDiscriminatedUnion<any, any>
|
||||
? z.infer<TBodySchema>
|
||||
: undefined
|
||||
) => AuthorizationResources;
|
||||
superScopes?: string[];
|
||||
};
|
||||
@@ -402,26 +449,34 @@ type ApiKeyActionRouteBuilderOptions<
|
||||
};
|
||||
|
||||
type ApiKeyActionHandlerFunction<
|
||||
TParamsSchema extends z.AnyZodObject | undefined,
|
||||
TSearchParamsSchema extends z.AnyZodObject | undefined,
|
||||
THeadersSchema extends z.AnyZodObject | undefined = undefined,
|
||||
TBodySchema extends z.AnyZodObject | undefined = undefined
|
||||
TParamsSchema extends AnyZodSchema | undefined,
|
||||
TSearchParamsSchema extends AnyZodSchema | undefined,
|
||||
THeadersSchema extends AnyZodSchema | undefined = undefined,
|
||||
TBodySchema extends AnyZodSchema | undefined = undefined
|
||||
> = (args: {
|
||||
params: TParamsSchema extends z.AnyZodObject ? z.infer<TParamsSchema> : undefined;
|
||||
searchParams: TSearchParamsSchema extends z.AnyZodObject
|
||||
params: TParamsSchema extends z.ZodFirstPartySchemaTypes | z.ZodDiscriminatedUnion<any, any>
|
||||
? z.infer<TParamsSchema>
|
||||
: undefined;
|
||||
searchParams: TSearchParamsSchema extends
|
||||
| z.ZodFirstPartySchemaTypes
|
||||
| z.ZodDiscriminatedUnion<any, any>
|
||||
? z.infer<TSearchParamsSchema>
|
||||
: undefined;
|
||||
headers: THeadersSchema extends z.AnyZodObject ? z.infer<THeadersSchema> : undefined;
|
||||
body: TBodySchema extends z.AnyZodObject ? z.infer<TBodySchema> : undefined;
|
||||
headers: THeadersSchema extends z.ZodFirstPartySchemaTypes | z.ZodDiscriminatedUnion<any, any>
|
||||
? z.infer<THeadersSchema>
|
||||
: undefined;
|
||||
body: TBodySchema extends z.ZodFirstPartySchemaTypes | z.ZodDiscriminatedUnion<any, any>
|
||||
? z.infer<TBodySchema>
|
||||
: undefined;
|
||||
authentication: ApiAuthenticationResultSuccess;
|
||||
request: Request;
|
||||
}) => Promise<Response>;
|
||||
|
||||
export function createActionApiRoute<
|
||||
TParamsSchema extends z.AnyZodObject | undefined = undefined,
|
||||
TSearchParamsSchema extends z.AnyZodObject | undefined = undefined,
|
||||
THeadersSchema extends z.AnyZodObject | undefined = undefined,
|
||||
TBodySchema extends z.AnyZodObject | undefined = undefined
|
||||
TParamsSchema extends AnyZodSchema | undefined = undefined,
|
||||
TSearchParamsSchema extends AnyZodSchema | undefined = undefined,
|
||||
THeadersSchema extends AnyZodSchema | undefined = undefined,
|
||||
TBodySchema extends AnyZodSchema | undefined = undefined
|
||||
>(
|
||||
options: ApiKeyActionRouteBuilderOptions<
|
||||
TParamsSchema,
|
||||
@@ -667,3 +722,289 @@ async function wrapResponse(
|
||||
})
|
||||
: response;
|
||||
}
|
||||
|
||||
type WorkerLoaderRouteBuilderOptions<
|
||||
TParamsSchema extends AnyZodSchema | undefined = undefined,
|
||||
TSearchParamsSchema extends AnyZodSchema | undefined = undefined,
|
||||
THeadersSchema extends AnyZodSchema | undefined = undefined
|
||||
> = {
|
||||
params?: TParamsSchema;
|
||||
searchParams?: TSearchParamsSchema;
|
||||
headers?: THeadersSchema;
|
||||
};
|
||||
|
||||
type WorkerLoaderHandlerFunction<
|
||||
TParamsSchema extends AnyZodSchema | undefined,
|
||||
TSearchParamsSchema extends AnyZodSchema | undefined,
|
||||
THeadersSchema extends AnyZodSchema | undefined = undefined
|
||||
> = (args: {
|
||||
params: TParamsSchema extends z.ZodFirstPartySchemaTypes | z.ZodDiscriminatedUnion<any, any>
|
||||
? z.infer<TParamsSchema>
|
||||
: undefined;
|
||||
searchParams: TSearchParamsSchema extends
|
||||
| z.ZodFirstPartySchemaTypes
|
||||
| z.ZodDiscriminatedUnion<any, any>
|
||||
? z.infer<TSearchParamsSchema>
|
||||
: undefined;
|
||||
authenticatedWorker: AuthenticatedWorkerInstance;
|
||||
request: Request;
|
||||
headers: THeadersSchema extends z.ZodFirstPartySchemaTypes | z.ZodDiscriminatedUnion<any, any>
|
||||
? z.infer<THeadersSchema>
|
||||
: undefined;
|
||||
}) => Promise<Response>;
|
||||
|
||||
export function createLoaderWorkerApiRoute<
|
||||
TParamsSchema extends AnyZodSchema | undefined = undefined,
|
||||
TSearchParamsSchema extends AnyZodSchema | undefined = undefined,
|
||||
THeadersSchema extends AnyZodSchema | undefined = undefined
|
||||
>(
|
||||
options: WorkerLoaderRouteBuilderOptions<TParamsSchema, TSearchParamsSchema, THeadersSchema>,
|
||||
handler: WorkerLoaderHandlerFunction<TParamsSchema, TSearchParamsSchema, THeadersSchema>
|
||||
) {
|
||||
return async function loader({ request, params }: LoaderFunctionArgs) {
|
||||
const {
|
||||
params: paramsSchema,
|
||||
searchParams: searchParamsSchema,
|
||||
headers: headersSchema,
|
||||
} = options;
|
||||
|
||||
try {
|
||||
const service = new WorkerGroupTokenService();
|
||||
const authenticationResult = await service.authenticate(request);
|
||||
|
||||
if (!authenticationResult) {
|
||||
return json({ error: "Invalid or missing worker token" }, { status: 401 });
|
||||
}
|
||||
|
||||
let parsedParams: any = undefined;
|
||||
if (paramsSchema) {
|
||||
const parsed = paramsSchema.safeParse(params);
|
||||
if (!parsed.success) {
|
||||
return json(
|
||||
{ error: "Params Error", details: fromZodError(parsed.error).details },
|
||||
{ status: 400 }
|
||||
);
|
||||
}
|
||||
parsedParams = parsed.data;
|
||||
}
|
||||
|
||||
let parsedSearchParams: any = undefined;
|
||||
if (searchParamsSchema) {
|
||||
const searchParams = Object.fromEntries(new URL(request.url).searchParams);
|
||||
const parsed = searchParamsSchema.safeParse(searchParams);
|
||||
if (!parsed.success) {
|
||||
return json(
|
||||
{ error: "Query Error", details: fromZodError(parsed.error).details },
|
||||
{ status: 400 }
|
||||
);
|
||||
}
|
||||
parsedSearchParams = parsed.data;
|
||||
}
|
||||
|
||||
let parsedHeaders: any = undefined;
|
||||
if (headersSchema) {
|
||||
const rawHeaders = Object.fromEntries(request.headers);
|
||||
const headers = headersSchema.safeParse(rawHeaders);
|
||||
if (!headers.success) {
|
||||
return json(
|
||||
{ error: "Headers Error", details: fromZodError(headers.error).details },
|
||||
{ status: 400 }
|
||||
);
|
||||
}
|
||||
parsedHeaders = headers.data;
|
||||
}
|
||||
|
||||
const result = await handler({
|
||||
params: parsedParams,
|
||||
searchParams: parsedSearchParams,
|
||||
authenticatedWorker: authenticationResult,
|
||||
request,
|
||||
headers: parsedHeaders,
|
||||
});
|
||||
return result;
|
||||
} catch (error) {
|
||||
console.error("Error in API route:", error);
|
||||
if (error instanceof Response) {
|
||||
return error;
|
||||
}
|
||||
|
||||
logger.error("Error in loader", {
|
||||
error:
|
||||
error instanceof Error
|
||||
? {
|
||||
name: error.name,
|
||||
message: error.message,
|
||||
stack: error.stack,
|
||||
}
|
||||
: String(error),
|
||||
url: request.url,
|
||||
});
|
||||
|
||||
return json({ error: "Internal Server Error" }, { status: 500 });
|
||||
}
|
||||
};
|
||||
}
|
||||
|
||||
type WorkerActionRouteBuilderOptions<
|
||||
TParamsSchema extends AnyZodSchema | undefined = undefined,
|
||||
TSearchParamsSchema extends AnyZodSchema | undefined = undefined,
|
||||
THeadersSchema extends AnyZodSchema | undefined = undefined,
|
||||
TBodySchema extends AnyZodSchema | undefined = undefined
|
||||
> = {
|
||||
params?: TParamsSchema;
|
||||
searchParams?: TSearchParamsSchema;
|
||||
headers?: THeadersSchema;
|
||||
body?: TBodySchema;
|
||||
method?: "POST" | "PUT" | "DELETE" | "PATCH";
|
||||
};
|
||||
|
||||
type WorkerActionHandlerFunction<
|
||||
TParamsSchema extends AnyZodSchema | undefined,
|
||||
TSearchParamsSchema extends AnyZodSchema | undefined,
|
||||
THeadersSchema extends AnyZodSchema | undefined = undefined,
|
||||
TBodySchema extends AnyZodSchema | undefined = undefined
|
||||
> = (args: {
|
||||
params: TParamsSchema extends z.ZodFirstPartySchemaTypes | z.ZodDiscriminatedUnion<any, any>
|
||||
? z.infer<TParamsSchema>
|
||||
: undefined;
|
||||
searchParams: TSearchParamsSchema extends
|
||||
| z.ZodFirstPartySchemaTypes
|
||||
| z.ZodDiscriminatedUnion<any, any>
|
||||
? z.infer<TSearchParamsSchema>
|
||||
: undefined;
|
||||
authenticatedWorker: AuthenticatedWorkerInstance;
|
||||
request: Request;
|
||||
headers: THeadersSchema extends z.ZodFirstPartySchemaTypes | z.ZodDiscriminatedUnion<any, any>
|
||||
? z.infer<THeadersSchema>
|
||||
: undefined;
|
||||
body: TBodySchema extends z.ZodFirstPartySchemaTypes | z.ZodDiscriminatedUnion<any, any>
|
||||
? z.infer<TBodySchema>
|
||||
: undefined;
|
||||
}) => Promise<Response>;
|
||||
|
||||
export function createActionWorkerApiRoute<
|
||||
TParamsSchema extends AnyZodSchema | undefined = undefined,
|
||||
TSearchParamsSchema extends AnyZodSchema | undefined = undefined,
|
||||
THeadersSchema extends AnyZodSchema | undefined = undefined,
|
||||
TBodySchema extends AnyZodSchema | undefined = undefined
|
||||
>(
|
||||
options: WorkerActionRouteBuilderOptions<
|
||||
TParamsSchema,
|
||||
TSearchParamsSchema,
|
||||
THeadersSchema,
|
||||
TBodySchema
|
||||
>,
|
||||
handler: WorkerActionHandlerFunction<
|
||||
TParamsSchema,
|
||||
TSearchParamsSchema,
|
||||
THeadersSchema,
|
||||
TBodySchema
|
||||
>
|
||||
) {
|
||||
return async function action({ request, params }: ActionFunctionArgs) {
|
||||
if (options.method) {
|
||||
if (request.method.toUpperCase() !== options.method) {
|
||||
return json(
|
||||
{ error: "Method not allowed" },
|
||||
{ status: 405, headers: { Allow: options.method } }
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
const {
|
||||
params: paramsSchema,
|
||||
searchParams: searchParamsSchema,
|
||||
body: bodySchema,
|
||||
headers: headersSchema,
|
||||
} = options;
|
||||
|
||||
try {
|
||||
const service = new WorkerGroupTokenService();
|
||||
const authenticationResult = await service.authenticate(request);
|
||||
|
||||
if (!authenticationResult) {
|
||||
return json({ error: "Invalid or missing worker token" }, { status: 401 });
|
||||
}
|
||||
|
||||
let parsedParams: any = undefined;
|
||||
if (paramsSchema) {
|
||||
const parsed = paramsSchema.safeParse(params);
|
||||
if (!parsed.success) {
|
||||
return json(
|
||||
{ error: "Params Error", details: fromZodError(parsed.error).details },
|
||||
{ status: 400 }
|
||||
);
|
||||
}
|
||||
parsedParams = parsed.data;
|
||||
}
|
||||
|
||||
let parsedSearchParams: any = undefined;
|
||||
if (searchParamsSchema) {
|
||||
const searchParams = Object.fromEntries(new URL(request.url).searchParams);
|
||||
const parsed = searchParamsSchema.safeParse(searchParams);
|
||||
if (!parsed.success) {
|
||||
return json(
|
||||
{ error: "Query Error", details: fromZodError(parsed.error).details },
|
||||
{ status: 400 }
|
||||
);
|
||||
}
|
||||
parsedSearchParams = parsed.data;
|
||||
}
|
||||
|
||||
let parsedHeaders: any = undefined;
|
||||
if (headersSchema) {
|
||||
const rawHeaders = Object.fromEntries(request.headers);
|
||||
const headers = headersSchema.safeParse(rawHeaders);
|
||||
if (!headers.success) {
|
||||
return json(
|
||||
{ error: "Headers Error", details: fromZodError(headers.error).details },
|
||||
{ status: 400 }
|
||||
);
|
||||
}
|
||||
parsedHeaders = headers.data;
|
||||
}
|
||||
|
||||
let parsedBody: any = undefined;
|
||||
if (bodySchema) {
|
||||
const body = await request.clone().json();
|
||||
const parsed = bodySchema.safeParse(body);
|
||||
if (!parsed.success) {
|
||||
return json(
|
||||
{ error: "Body Error", details: fromZodError(parsed.error).details },
|
||||
{ status: 400 }
|
||||
);
|
||||
}
|
||||
parsedBody = parsed.data;
|
||||
}
|
||||
|
||||
const result = await handler({
|
||||
params: parsedParams,
|
||||
searchParams: parsedSearchParams,
|
||||
authenticatedWorker: authenticationResult,
|
||||
request,
|
||||
body: parsedBody,
|
||||
headers: parsedHeaders,
|
||||
});
|
||||
return result;
|
||||
} catch (error) {
|
||||
console.error("Error in API route:", error);
|
||||
if (error instanceof Response) {
|
||||
return error;
|
||||
}
|
||||
|
||||
logger.error("Error in action", {
|
||||
error:
|
||||
error instanceof Error
|
||||
? {
|
||||
name: error.name,
|
||||
message: error.message,
|
||||
stack: error.stack,
|
||||
}
|
||||
: String(error),
|
||||
url: request.url,
|
||||
});
|
||||
|
||||
return json({ error: "Internal Server Error" }, { status: 500 });
|
||||
}
|
||||
};
|
||||
}
|
||||
|
||||
@@ -54,6 +54,10 @@ import {
|
||||
} from "~/v3/services/cancelDevSessionRuns.server";
|
||||
import { logger } from "./logger.server";
|
||||
import { BatchProcessingOptions, BatchTriggerV3Service } from "~/v3/services/batchTriggerV3.server";
|
||||
import {
|
||||
BatchProcessingOptions as BatchProcessingOptionsV4,
|
||||
BatchTriggerV4Service,
|
||||
} from "~/v3/services/batchTriggerV4.server";
|
||||
|
||||
const workerCatalog = {
|
||||
indexEndpoint: z.object({
|
||||
@@ -194,6 +198,7 @@ const workerCatalog = {
|
||||
}),
|
||||
"v3.cancelDevSessionRuns": CancelDevSessionRunsServiceOptions,
|
||||
"v3.processBatchTaskRun": BatchProcessingOptions,
|
||||
"v3.processBatchTaskRunV3": BatchProcessingOptionsV4,
|
||||
};
|
||||
|
||||
const executionWorkerCatalog = {
|
||||
@@ -717,6 +722,15 @@ function getWorkerQueue() {
|
||||
handler: async (payload, job) => {
|
||||
const service = new BatchTriggerV3Service(payload.strategy);
|
||||
|
||||
await service.processBatchTaskRun(payload);
|
||||
},
|
||||
},
|
||||
"v3.processBatchTaskRunV3": {
|
||||
priority: 0,
|
||||
maxAttempts: 5,
|
||||
handler: async (payload, job) => {
|
||||
const service = new BatchTriggerV4Service(payload.strategy);
|
||||
|
||||
await service.processBatchTaskRun(payload);
|
||||
},
|
||||
},
|
||||
|
||||
@@ -1,3 +1,5 @@
|
||||
import { parseNaturalLanguageDuration } from "@trigger.dev/core/v3/apps";
|
||||
|
||||
export const calculateDurationInMs = (options: {
|
||||
seconds?: number;
|
||||
minutes?: number;
|
||||
@@ -11,3 +13,30 @@ export const calculateDurationInMs = (options: {
|
||||
(options?.days ?? 0) * 24 * 60 * 60 * 1000
|
||||
);
|
||||
};
|
||||
|
||||
export async function parseDelay(value?: string | Date): Promise<Date | undefined> {
|
||||
if (!value) {
|
||||
return;
|
||||
}
|
||||
|
||||
if (value instanceof Date) {
|
||||
return value;
|
||||
}
|
||||
|
||||
try {
|
||||
const date = new Date(value);
|
||||
|
||||
// Check if the date is valid
|
||||
if (isNaN(date.getTime())) {
|
||||
return parseNaturalLanguageDuration(value);
|
||||
}
|
||||
|
||||
if (date.getTime() <= Date.now()) {
|
||||
return;
|
||||
}
|
||||
|
||||
return date;
|
||||
} catch (error) {
|
||||
return parseNaturalLanguageDuration(value);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -22,8 +22,8 @@ export function sse({ request, pingInterval = 1000, updateInterval = 348, run }:
|
||||
return new Response("SSE disabled", { status: 200 });
|
||||
}
|
||||
|
||||
let pinger: NodeJS.Timer | undefined = undefined;
|
||||
let updater: NodeJS.Timer | undefined = undefined;
|
||||
let pinger: NodeJS.Timeout | undefined = undefined;
|
||||
let updater: NodeJS.Timeout | undefined = undefined;
|
||||
let timeout: NodeJS.Timeout | undefined = undefined;
|
||||
|
||||
const abort = () => {
|
||||
|
||||
@@ -0,0 +1,183 @@
|
||||
import { LoaderFunctionArgs } from "@remix-run/node";
|
||||
import { eventStream } from "remix-utils/sse/server";
|
||||
import { setInterval } from "timers/promises";
|
||||
|
||||
type SendFunction = Parameters<Parameters<typeof eventStream>[1]>[0];
|
||||
|
||||
type HandlerParams = {
|
||||
send: SendFunction;
|
||||
};
|
||||
|
||||
type SSEHandlers = {
|
||||
/** Return false to stop */
|
||||
beforeStream?: () => Promise<boolean | void> | boolean | void;
|
||||
/** Return false to stop */
|
||||
initStream?: (params: HandlerParams) => Promise<boolean | void> | boolean | void;
|
||||
/** Return false to stop */
|
||||
iterator?: (params: HandlerParams & { date: Date }) => Promise<boolean | void> | boolean | void;
|
||||
cleanup?: () => void;
|
||||
};
|
||||
|
||||
type SSEContext = {
|
||||
id: string;
|
||||
request: Request;
|
||||
controller: AbortController;
|
||||
debug: (message: string) => void;
|
||||
};
|
||||
|
||||
type SSEOptions = {
|
||||
timeout: number;
|
||||
interval?: number;
|
||||
debug?: boolean;
|
||||
handler: (context: SSEContext) => Promise<SSEHandlers>;
|
||||
};
|
||||
|
||||
// This is used to track the open connections, for debugging
|
||||
const connections: Set<string> = new Set();
|
||||
|
||||
export function createSSELoader(options: SSEOptions) {
|
||||
const { timeout, interval = 500, debug = false, handler } = options;
|
||||
|
||||
return async function loader({ request }: LoaderFunctionArgs) {
|
||||
const id = request.headers.get("x-request-id") || Math.random().toString(36).slice(2, 8);
|
||||
|
||||
const internalController = new AbortController();
|
||||
const timeoutSignal = AbortSignal.timeout(timeout);
|
||||
|
||||
const log = (message: string) => {
|
||||
if (debug) console.log(`SSE: [${id}] ${message} (${connections.size} open connections)`);
|
||||
};
|
||||
|
||||
const context: SSEContext = {
|
||||
id,
|
||||
request,
|
||||
controller: internalController,
|
||||
debug: log,
|
||||
};
|
||||
|
||||
const handlers = await handler(context).catch((error) => {
|
||||
if (error instanceof Response) {
|
||||
throw error;
|
||||
}
|
||||
|
||||
throw new Response("Internal Server Error", { status: 500 });
|
||||
});
|
||||
|
||||
const combinedSignal = AbortSignal.any([
|
||||
request.signal,
|
||||
timeoutSignal,
|
||||
internalController.signal,
|
||||
]);
|
||||
|
||||
log("Start");
|
||||
|
||||
request.signal.addEventListener(
|
||||
"abort",
|
||||
() => {
|
||||
log(`request signal aborted`);
|
||||
internalController.abort("Request aborted");
|
||||
},
|
||||
{ once: true, signal: internalController.signal }
|
||||
);
|
||||
|
||||
combinedSignal.addEventListener(
|
||||
"abort",
|
||||
() => {
|
||||
log(`combinedSignal aborted: ${combinedSignal.reason}`);
|
||||
},
|
||||
{ once: true, signal: internalController.signal }
|
||||
);
|
||||
|
||||
timeoutSignal.addEventListener(
|
||||
"abort",
|
||||
() => {
|
||||
if (internalController.signal.aborted) return;
|
||||
log(`timeoutSignal aborted: ${timeoutSignal.reason}`);
|
||||
internalController.abort("Timeout");
|
||||
},
|
||||
{ once: true, signal: internalController.signal }
|
||||
);
|
||||
|
||||
if (handlers.beforeStream) {
|
||||
const shouldContinue = await handlers.beforeStream();
|
||||
if (shouldContinue === false) {
|
||||
log("beforeStream returned false, so we'll exit before creating the stream");
|
||||
internalController.abort("Init requested stop");
|
||||
return;
|
||||
}
|
||||
}
|
||||
|
||||
return eventStream(combinedSignal, function setup(send) {
|
||||
connections.add(id);
|
||||
|
||||
async function run() {
|
||||
try {
|
||||
log("Initializing");
|
||||
if (handlers.initStream) {
|
||||
const shouldContinue = await handlers.initStream({ send });
|
||||
if (shouldContinue === false) {
|
||||
log("initStream returned false, so we'll stop the stream");
|
||||
internalController.abort("Init requested stop");
|
||||
return;
|
||||
}
|
||||
}
|
||||
|
||||
log("Starting interval");
|
||||
for await (const _ of setInterval(interval, null, {
|
||||
signal: combinedSignal,
|
||||
})) {
|
||||
log("PING");
|
||||
|
||||
const date = new Date();
|
||||
|
||||
if (handlers.iterator) {
|
||||
try {
|
||||
const shouldContinue = await handlers.iterator({ date, send });
|
||||
if (shouldContinue === false) {
|
||||
log("iterator return false, so we'll stop the stream");
|
||||
internalController.abort("Iterator requested stop");
|
||||
break;
|
||||
}
|
||||
} catch (error) {
|
||||
log("iterator threw an error, aborting stream");
|
||||
// Immediately abort to trigger cleanup
|
||||
internalController.abort(error instanceof Error ? error.message : "Iterator error");
|
||||
// No need to re-throw as we're handling it by aborting
|
||||
return; // Exit the run function immediately
|
||||
}
|
||||
}
|
||||
}
|
||||
log("iterator finished all iterations");
|
||||
} catch (error) {
|
||||
if (error instanceof Error) {
|
||||
if (error.name !== "AbortError") {
|
||||
console.error(error);
|
||||
}
|
||||
}
|
||||
} finally {
|
||||
log("iterator finished");
|
||||
}
|
||||
}
|
||||
|
||||
run();
|
||||
|
||||
return () => {
|
||||
connections.delete(id);
|
||||
|
||||
log("Cleanup called");
|
||||
if (handlers.cleanup) {
|
||||
try {
|
||||
handlers.cleanup();
|
||||
} catch (error) {
|
||||
log(
|
||||
`Error in cleanup handler: ${
|
||||
error instanceof Error ? error.message : "Unknown error"
|
||||
}`
|
||||
);
|
||||
console.error("SSE Cleanup Error:", error);
|
||||
}
|
||||
}
|
||||
};
|
||||
});
|
||||
};
|
||||
}
|
||||
@@ -0,0 +1,3 @@
|
||||
export function capitalizeWord(word: string) {
|
||||
return word.charAt(0).toUpperCase() + word.slice(1).toLowerCase();
|
||||
}
|
||||
@@ -65,7 +65,6 @@ export function prepareTrace(events: TaskEvent[]): TraceSummary | undefined {
|
||||
id: event.spanId,
|
||||
parentId: event.parentId ?? undefined,
|
||||
runId: event.runId,
|
||||
idempotencyKey: event.idempotencyKey,
|
||||
data: {
|
||||
message: event.message,
|
||||
style: event.style,
|
||||
@@ -77,8 +76,9 @@ export function prepareTrace(events: TaskEvent[]): TraceSummary | undefined {
|
||||
level: event.level,
|
||||
events: event.events,
|
||||
environmentType: event.environmentType,
|
||||
isDebug: event.isDebug,
|
||||
},
|
||||
};
|
||||
} satisfies SpanSummary;
|
||||
|
||||
spansBySpanId.set(event.spanId, span);
|
||||
|
||||
|
||||
@@ -1,4 +1,8 @@
|
||||
import { clientWebsocketMessages, serverWebsocketMessages } from "@trigger.dev/core/v3";
|
||||
import {
|
||||
clientWebsocketMessages,
|
||||
HeartbeatService,
|
||||
serverWebsocketMessages,
|
||||
} from "@trigger.dev/core/v3";
|
||||
import { ZodMessageHandler, ZodMessageSender } from "@trigger.dev/core/v3/zodMessageHandler";
|
||||
import { Evt } from "evt";
|
||||
import { randomUUID } from "node:crypto";
|
||||
@@ -7,7 +11,6 @@ import { WebSocket } from "ws";
|
||||
import { AuthenticatedEnvironment } from "~/services/apiAuth.server";
|
||||
import { logger } from "~/services/logger.server";
|
||||
import { DevQueueConsumer } from "./marqs/devQueueConsumer.server";
|
||||
import { HeartbeatService } from "./services/heartbeatService.server";
|
||||
|
||||
export class AuthenticatedSocketConnection {
|
||||
public id: string;
|
||||
@@ -86,6 +89,7 @@ export class AuthenticatedSocketConnection {
|
||||
|
||||
ws.ping();
|
||||
},
|
||||
intervalMs: 45_000,
|
||||
});
|
||||
this._pingService.start();
|
||||
|
||||
|
||||
@@ -0,0 +1,66 @@
|
||||
import { RunEngineVersion, RuntimeEnvironmentType } from "@trigger.dev/database";
|
||||
import { AuthenticatedEnvironment } from "~/services/apiAuth.server";
|
||||
import {
|
||||
findCurrentWorkerDeploymentWithoutTasks,
|
||||
findCurrentWorkerFromEnvironment,
|
||||
} from "./models/workerDeployment.server";
|
||||
import { $replica } from "~/db.server";
|
||||
|
||||
export async function determineEngineVersion({
|
||||
environment,
|
||||
workerVersion,
|
||||
engineVersion: version,
|
||||
}: {
|
||||
environment: AuthenticatedEnvironment;
|
||||
workerVersion?: string;
|
||||
engineVersion?: RunEngineVersion;
|
||||
}): Promise<RunEngineVersion> {
|
||||
if (version) {
|
||||
return version;
|
||||
}
|
||||
|
||||
// If the project is V1, then none of the background workers are running V2
|
||||
if (environment.project.engine === RunEngineVersion.V1) {
|
||||
return "V1";
|
||||
}
|
||||
|
||||
/**
|
||||
* The project has V2 enabled so it *could* be V2.
|
||||
*/
|
||||
|
||||
// A specific worker version is requested
|
||||
if (workerVersion) {
|
||||
const worker = await $replica.backgroundWorker.findUnique({
|
||||
select: {
|
||||
engine: true,
|
||||
},
|
||||
where: {
|
||||
projectId_runtimeEnvironmentId_version: {
|
||||
projectId: environment.projectId,
|
||||
runtimeEnvironmentId: environment.id,
|
||||
version: workerVersion,
|
||||
},
|
||||
},
|
||||
});
|
||||
|
||||
if (!worker) {
|
||||
throw new Error(`Worker not found: environment: ${environment.id} version: ${workerVersion}`);
|
||||
}
|
||||
|
||||
return worker.engine;
|
||||
}
|
||||
|
||||
// Dev: use the latest BackgroundWorker
|
||||
if (environment.type === "DEVELOPMENT") {
|
||||
const backgroundWorker = await findCurrentWorkerFromEnvironment(environment);
|
||||
return backgroundWorker?.engine ?? "V1";
|
||||
}
|
||||
|
||||
// Deployed: use the latest deployed BackgroundWorker
|
||||
const currentDeployment = await findCurrentWorkerDeploymentWithoutTasks(environment.id);
|
||||
if (currentDeployment?.type === "V1") {
|
||||
return "V1";
|
||||
}
|
||||
|
||||
return "V2";
|
||||
}
|
||||
@@ -1,4 +1,4 @@
|
||||
import { Attributes, Link, trace, TraceFlags, Tracer } from "@opentelemetry/api";
|
||||
import { Attributes, AttributeValue, Link, trace, TraceFlags, Tracer } from "@opentelemetry/api";
|
||||
import { RandomIdGenerator } from "@opentelemetry/sdk-trace-base";
|
||||
import { SemanticResourceAttributes } from "@opentelemetry/semantic-conventions";
|
||||
import {
|
||||
@@ -10,6 +10,7 @@ import {
|
||||
SpanEvent,
|
||||
SpanEvents,
|
||||
SpanMessagingEvent,
|
||||
TaskEventEnvironment,
|
||||
TaskEventStyle,
|
||||
TaskRunError,
|
||||
correctErrorStackTrace,
|
||||
@@ -26,7 +27,6 @@ import { Gauge } from "prom-client";
|
||||
import { $replica, PrismaClient, PrismaReplicaClient, prisma } from "~/db.server";
|
||||
import { env } from "~/env.server";
|
||||
import { metricsRegister } from "~/metrics.server";
|
||||
import { AuthenticatedEnvironment } from "~/services/apiAuth.server";
|
||||
import { logger } from "~/services/logger.server";
|
||||
import { singleton } from "~/utils/singleton";
|
||||
import { DynamicFlushScheduler } from "./dynamicFlushScheduler.server";
|
||||
@@ -59,6 +59,7 @@ export type TraceAttributes = Partial<
|
||||
| "attemptId"
|
||||
| "isError"
|
||||
| "isCancelled"
|
||||
| "isDebug"
|
||||
| "runId"
|
||||
| "runIsTest"
|
||||
| "output"
|
||||
@@ -84,7 +85,7 @@ export type TraceEventOptions = {
|
||||
parentAsLinkType?: "trigger" | "replay";
|
||||
spanIdSeed?: string;
|
||||
attributes: TraceAttributes;
|
||||
environment: AuthenticatedEnvironment;
|
||||
environment: TaskEventEnvironment;
|
||||
taskSlug: string;
|
||||
startTime?: bigint;
|
||||
endTime?: Date;
|
||||
@@ -125,6 +126,7 @@ export type QueriedEvent = Prisma.TaskEventGetPayload<{
|
||||
isError: true;
|
||||
isPartial: true;
|
||||
isCancelled: true;
|
||||
isDebug: true;
|
||||
level: true;
|
||||
events: true;
|
||||
environmentType: true;
|
||||
@@ -169,6 +171,7 @@ export type SpanSummary = {
|
||||
isError: boolean;
|
||||
isPartial: boolean;
|
||||
isCancelled: boolean;
|
||||
isDebug: boolean;
|
||||
level: NonNullable<CreatableEvent["level"]>;
|
||||
environmentType: CreatableEventEnvironmentType;
|
||||
};
|
||||
@@ -183,6 +186,26 @@ export type UpdateEventOptions = {
|
||||
events?: SpanEvents;
|
||||
};
|
||||
|
||||
type TaskEventSummary = Pick<
|
||||
TaskEvent,
|
||||
| "id"
|
||||
| "spanId"
|
||||
| "parentId"
|
||||
| "runId"
|
||||
| "idempotencyKey"
|
||||
| "message"
|
||||
| "style"
|
||||
| "startTime"
|
||||
| "duration"
|
||||
| "isError"
|
||||
| "isPartial"
|
||||
| "isCancelled"
|
||||
| "level"
|
||||
| "events"
|
||||
| "environmentType"
|
||||
| "isDebug"
|
||||
>;
|
||||
|
||||
export class EventRepository {
|
||||
private readonly _flushScheduler: DynamicFlushScheduler<CreatableEvent>;
|
||||
private _randomIdGenerator = new RandomIdGenerator();
|
||||
@@ -263,7 +286,7 @@ export class EventRepository {
|
||||
eventId: event.id,
|
||||
});
|
||||
|
||||
await this.insert({
|
||||
const completedEvent = {
|
||||
...omit(event, "id"),
|
||||
isPartial: false,
|
||||
isError: options?.attributes.isError ?? false,
|
||||
@@ -283,7 +306,11 @@ export class EventRepository {
|
||||
: "application/json",
|
||||
payload: event.payload as Attributes,
|
||||
payloadType: event.payloadType,
|
||||
});
|
||||
} satisfies CreatableEvent;
|
||||
|
||||
await this.insert(completedEvent);
|
||||
|
||||
return completedEvent;
|
||||
}
|
||||
|
||||
async cancelEvent(event: TaskEventRecord, cancelledAt: Date, reason: string) {
|
||||
@@ -485,6 +512,7 @@ export class EventRepository {
|
||||
isError: event.isError,
|
||||
isPartial: ancestorCancelled ? false : event.isPartial,
|
||||
isCancelled: event.isCancelled === true ? true : event.isPartial && ancestorCancelled,
|
||||
isDebug: event.isDebug,
|
||||
startTime: getDateFromNanoseconds(event.startTime),
|
||||
level: event.level,
|
||||
events: event.events,
|
||||
@@ -541,6 +569,7 @@ export class EventRepository {
|
||||
isError: true,
|
||||
isPartial: true,
|
||||
isCancelled: true,
|
||||
isDebug: true,
|
||||
level: true,
|
||||
events: true,
|
||||
environmentType: true,
|
||||
@@ -633,6 +662,19 @@ export class EventRepository {
|
||||
spanEvent.environmentType === "DEVELOPMENT"
|
||||
);
|
||||
|
||||
const originalRun = rehydrateAttribute<string>(
|
||||
spanEvent.properties,
|
||||
SemanticInternalAttributes.ORIGINAL_RUN_ID
|
||||
);
|
||||
|
||||
const entity = {
|
||||
type: rehydrateAttribute<string>(
|
||||
spanEvent.properties,
|
||||
SemanticInternalAttributes.ENTITY_TYPE
|
||||
),
|
||||
id: rehydrateAttribute<string>(spanEvent.properties, SemanticInternalAttributes.ENTITY_ID),
|
||||
};
|
||||
|
||||
return {
|
||||
...spanEvent,
|
||||
...span.data,
|
||||
@@ -642,6 +684,8 @@ export class EventRepository {
|
||||
events: spanEvents,
|
||||
show,
|
||||
links,
|
||||
originalRun,
|
||||
entity,
|
||||
};
|
||||
});
|
||||
}
|
||||
@@ -788,14 +832,19 @@ export class EventRepository {
|
||||
});
|
||||
}
|
||||
|
||||
public async recordEvent(message: string, options: TraceEventOptions) {
|
||||
public async recordEvent(
|
||||
message: string,
|
||||
options: TraceEventOptions & { duration?: number; parentId?: string }
|
||||
) {
|
||||
const propagatedContext = extractContextFromCarrier(options.context ?? {});
|
||||
|
||||
const startTime = options.startTime ?? getNowInNanoseconds();
|
||||
const duration = options.endTime ? calculateDurationFromStart(startTime, options.endTime) : 100;
|
||||
const duration =
|
||||
options.duration ??
|
||||
(options.endTime ? calculateDurationFromStart(startTime, options.endTime) : 100);
|
||||
|
||||
const traceId = propagatedContext?.traceparent?.traceId ?? this.generateTraceId();
|
||||
const parentId = propagatedContext?.traceparent?.spanId;
|
||||
const parentId = options.parentId ?? propagatedContext?.traceparent?.spanId;
|
||||
const tracestate = propagatedContext?.tracestate;
|
||||
const spanId = options.spanIdSeed
|
||||
? this.#generateDeterministicSpanId(traceId, options.spanIdSeed)
|
||||
@@ -816,8 +865,10 @@ export class EventRepository {
|
||||
...options.attributes.metadata,
|
||||
};
|
||||
|
||||
const isDebug = options.attributes.isDebug;
|
||||
|
||||
const style = {
|
||||
[SemanticInternalAttributes.STYLE_ICON]: "play",
|
||||
[SemanticInternalAttributes.STYLE_ICON]: isDebug ? "warn" : "play",
|
||||
};
|
||||
|
||||
if (!options.attributes.runId) {
|
||||
@@ -832,11 +883,12 @@ export class EventRepository {
|
||||
message: message,
|
||||
serviceName: "api server",
|
||||
serviceNamespace: "trigger.dev",
|
||||
level: "TRACE",
|
||||
level: isDebug ? "WARN" : "TRACE",
|
||||
kind: options.kind,
|
||||
status: "OK",
|
||||
startTime,
|
||||
isPartial: false,
|
||||
isDebug,
|
||||
duration, // convert to nanoseconds
|
||||
environmentId: options.environment.id,
|
||||
environmentType: options.environment.type,
|
||||
@@ -876,7 +928,7 @@ export class EventRepository {
|
||||
|
||||
public async traceEvent<TResult>(
|
||||
message: string,
|
||||
options: TraceEventOptions & { incomplete?: boolean },
|
||||
options: TraceEventOptions & { incomplete?: boolean; isError?: boolean },
|
||||
callback: (
|
||||
e: EventBuilder,
|
||||
traceContext: Record<string, string | undefined>,
|
||||
@@ -1322,7 +1374,7 @@ function excludePartialEventsWithCorrespondingFullEvent(batch: CreatableEvent[])
|
||||
);
|
||||
}
|
||||
|
||||
function extractContextFromCarrier(carrier: Record<string, string | undefined>) {
|
||||
export function extractContextFromCarrier(carrier: Record<string, string | undefined>) {
|
||||
const traceparent = carrier["traceparent"];
|
||||
const tracestate = carrier["tracestate"];
|
||||
|
||||
@@ -1608,3 +1660,139 @@ function rehydrateShow(properties: Prisma.JsonValue): { actions?: boolean } | un
|
||||
|
||||
return;
|
||||
}
|
||||
|
||||
function rehydrateAttribute<T extends AttributeValue>(
|
||||
properties: Prisma.JsonValue,
|
||||
key: string
|
||||
): T | undefined {
|
||||
if (properties === null || properties === undefined) {
|
||||
return;
|
||||
}
|
||||
|
||||
if (typeof properties !== "object") {
|
||||
return;
|
||||
}
|
||||
|
||||
if (Array.isArray(properties)) {
|
||||
return;
|
||||
}
|
||||
|
||||
const value = properties[key];
|
||||
|
||||
if (!value) return;
|
||||
|
||||
return value as T;
|
||||
}
|
||||
|
||||
export async function findRunForEventCreation(runId: string) {
|
||||
return prisma.taskRun.findFirst({
|
||||
where: {
|
||||
id: runId,
|
||||
},
|
||||
select: {
|
||||
friendlyId: true,
|
||||
taskIdentifier: true,
|
||||
traceContext: true,
|
||||
runtimeEnvironment: {
|
||||
select: {
|
||||
id: true,
|
||||
type: true,
|
||||
organizationId: true,
|
||||
projectId: true,
|
||||
project: {
|
||||
select: {
|
||||
externalRef: true,
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
export async function recordRunEvent(
|
||||
runId: string,
|
||||
message: string,
|
||||
options: Omit<TraceEventOptions, "environment" | "taskSlug" | "startTime"> & {
|
||||
duration?: number;
|
||||
parentId?: string;
|
||||
startTime?: Date;
|
||||
}
|
||||
): Promise<
|
||||
| {
|
||||
success: true;
|
||||
}
|
||||
| {
|
||||
success: false;
|
||||
code: "RUN_NOT_FOUND" | "FAILED_TO_RECORD_EVENT";
|
||||
error?: unknown;
|
||||
}
|
||||
> {
|
||||
try {
|
||||
const foundRun = await findRunForEventCreation(runId);
|
||||
|
||||
if (!foundRun) {
|
||||
logger.error("Failed to find run for event creation", { runId });
|
||||
return {
|
||||
success: false,
|
||||
code: "RUN_NOT_FOUND",
|
||||
};
|
||||
}
|
||||
|
||||
const { attributes, startTime, ...optionsRest } = options;
|
||||
|
||||
await eventRepository.recordEvent(message, {
|
||||
environment: foundRun.runtimeEnvironment,
|
||||
taskSlug: foundRun.taskIdentifier,
|
||||
context: foundRun.traceContext as Record<string, string | undefined>,
|
||||
attributes: {
|
||||
runId: foundRun.friendlyId,
|
||||
...attributes,
|
||||
},
|
||||
startTime: BigInt((startTime?.getTime() ?? Date.now()) * 1_000_000),
|
||||
...optionsRest,
|
||||
});
|
||||
|
||||
return {
|
||||
success: true,
|
||||
};
|
||||
} catch (error) {
|
||||
logger.error("Failed to record event for run", {
|
||||
error: error instanceof Error ? error.message : error,
|
||||
runId,
|
||||
});
|
||||
|
||||
return {
|
||||
success: false,
|
||||
code: "FAILED_TO_RECORD_EVENT",
|
||||
error,
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
export async function recordRunDebugLog(
|
||||
runId: string,
|
||||
message: string,
|
||||
options: Omit<TraceEventOptions, "environment" | "taskSlug" | "startTime"> & {
|
||||
duration?: number;
|
||||
parentId?: string;
|
||||
startTime?: Date;
|
||||
}
|
||||
): Promise<
|
||||
| {
|
||||
success: true;
|
||||
}
|
||||
| {
|
||||
success: false;
|
||||
code: "RUN_NOT_FOUND" | "FAILED_TO_RECORD_EVENT";
|
||||
error?: unknown;
|
||||
}
|
||||
> {
|
||||
return recordRunEvent(runId, message, {
|
||||
...options,
|
||||
attributes: {
|
||||
...options?.attributes,
|
||||
isDebug: true,
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
@@ -0,0 +1,54 @@
|
||||
import { z } from "zod";
|
||||
import { prisma, PrismaClientOrTransaction } from "~/db.server";
|
||||
|
||||
const FeatureFlagCatalog = {
|
||||
defaultWorkerInstanceGroupId: z.string(),
|
||||
};
|
||||
|
||||
type FeatureFlagKey = keyof typeof FeatureFlagCatalog;
|
||||
|
||||
export type FlagsOptions = {
|
||||
key: FeatureFlagKey;
|
||||
};
|
||||
|
||||
export function makeFlags(_prisma: PrismaClientOrTransaction = prisma) {
|
||||
return async function flags<T extends FeatureFlagKey>(
|
||||
opts: FlagsOptions
|
||||
): Promise<z.infer<(typeof FeatureFlagCatalog)[T]> | undefined> {
|
||||
const value = await _prisma.featureFlag.findUnique({
|
||||
where: {
|
||||
key: opts.key,
|
||||
},
|
||||
});
|
||||
|
||||
const parsed = FeatureFlagCatalog[opts.key].safeParse(value?.value);
|
||||
|
||||
if (!parsed.success) {
|
||||
return;
|
||||
}
|
||||
|
||||
return parsed.data;
|
||||
};
|
||||
}
|
||||
|
||||
export function makeSetFlags(_prisma: PrismaClientOrTransaction = prisma) {
|
||||
return async function setFlags<T extends FeatureFlagKey>(
|
||||
opts: FlagsOptions & { value: z.infer<(typeof FeatureFlagCatalog)[T]> }
|
||||
): Promise<void> {
|
||||
await _prisma.featureFlag.upsert({
|
||||
where: {
|
||||
key: opts.key,
|
||||
},
|
||||
create: {
|
||||
key: opts.key,
|
||||
value: opts.value,
|
||||
},
|
||||
update: {
|
||||
value: opts.value,
|
||||
},
|
||||
});
|
||||
};
|
||||
}
|
||||
|
||||
export const flags = makeFlags();
|
||||
export const setFlags = makeSetFlags();
|
||||
@@ -1,3 +1,5 @@
|
||||
import { EventBusEventArgs } from "@internal/run-engine";
|
||||
import { createAdapter } from "@socket.io/redis-adapter";
|
||||
import {
|
||||
ClientToSharedQueueMessages,
|
||||
CoordinatorSocketData,
|
||||
@@ -7,24 +9,32 @@ import {
|
||||
ProviderToPlatformMessages,
|
||||
SharedQueueToClientMessages,
|
||||
} from "@trigger.dev/core/v3";
|
||||
import { RunId } from "@trigger.dev/core/v3/apps";
|
||||
import type {
|
||||
WorkerClientToServerEvents,
|
||||
WorkerServerToClientEvents,
|
||||
} from "@trigger.dev/core/v3/workers";
|
||||
import { ZodNamespace } from "@trigger.dev/core/v3/zodNamespace";
|
||||
import { Server } from "socket.io";
|
||||
import { env } from "~/env.server";
|
||||
import { singleton } from "~/utils/singleton";
|
||||
import { SharedSocketConnection } from "./sharedSocketConnection";
|
||||
import { CreateCheckpointService } from "./services/createCheckpoint.server";
|
||||
import { sharedQueueTasks } from "./marqs/sharedQueueConsumer.server";
|
||||
import { CompleteAttemptService } from "./services/completeAttempt.server";
|
||||
import { logger } from "~/services/logger.server";
|
||||
import { findEnvironmentById } from "~/models/runtimeEnvironment.server";
|
||||
import { CreateDeployedBackgroundWorkerService } from "./services/createDeployedBackgroundWorker.server";
|
||||
import { ResumeAttemptService } from "./services/resumeAttempt.server";
|
||||
import { DeploymentIndexFailed } from "./services/deploymentIndexFailed.server";
|
||||
import { Redis } from "ioredis";
|
||||
import { createAdapter } from "@socket.io/redis-adapter";
|
||||
import { Namespace, Server, Socket } from "socket.io";
|
||||
import { env } from "~/env.server";
|
||||
import { findEnvironmentById } from "~/models/runtimeEnvironment.server";
|
||||
import { authenticateApiRequestWithFailure } from "~/services/apiAuth.server";
|
||||
import { logger } from "~/services/logger.server";
|
||||
import { singleton } from "~/utils/singleton";
|
||||
import { recordRunDebugLog } from "./eventRepository.server";
|
||||
import { sharedQueueTasks } from "./marqs/sharedQueueConsumer.server";
|
||||
import { engine } from "./runEngine.server";
|
||||
import { CompleteAttemptService } from "./services/completeAttempt.server";
|
||||
import { CrashTaskRunService } from "./services/crashTaskRun.server";
|
||||
import { CreateCheckpointService } from "./services/createCheckpoint.server";
|
||||
import { CreateDeployedBackgroundWorkerService } from "./services/createDeployedBackgroundWorker.server";
|
||||
import { CreateTaskRunAttemptService } from "./services/createTaskRunAttempt.server";
|
||||
import { DeploymentIndexFailed } from "./services/deploymentIndexFailed.server";
|
||||
import { ResumeAttemptService } from "./services/resumeAttempt.server";
|
||||
import { UpdateFatalRunErrorService } from "./services/updateFatalRunError.server";
|
||||
import { WorkerGroupTokenService } from "./services/worker/workerGroupTokenService.server";
|
||||
import { SharedSocketConnection } from "./sharedSocketConnection";
|
||||
|
||||
export const socketIo = singleton("socketIo", initalizeIoServer);
|
||||
|
||||
@@ -38,12 +48,40 @@ function initalizeIoServer() {
|
||||
const coordinatorNamespace = createCoordinatorNamespace(io);
|
||||
const providerNamespace = createProviderNamespace(io);
|
||||
const sharedQueueConsumerNamespace = createSharedQueueConsumerNamespace(io);
|
||||
const workerNamespace = createWorkerNamespace({
|
||||
io,
|
||||
namespace: "/worker",
|
||||
authenticate: async (request) => {
|
||||
const tokenService = new WorkerGroupTokenService();
|
||||
const authenticatedInstance = await tokenService.authenticate(request);
|
||||
if (!authenticatedInstance) {
|
||||
return false;
|
||||
}
|
||||
return true;
|
||||
},
|
||||
});
|
||||
const devWorkerNamespace = createWorkerNamespace({
|
||||
io,
|
||||
namespace: "/dev-worker",
|
||||
authenticate: async (request) => {
|
||||
const authentication = await authenticateApiRequestWithFailure(request);
|
||||
if (!authentication.ok) {
|
||||
return false;
|
||||
}
|
||||
if (authentication.environment.type !== "DEVELOPMENT") {
|
||||
return false;
|
||||
}
|
||||
return true;
|
||||
},
|
||||
});
|
||||
|
||||
return {
|
||||
io,
|
||||
coordinatorNamespace,
|
||||
providerNamespace,
|
||||
sharedQueueConsumerNamespace,
|
||||
workerNamespace,
|
||||
devWorkerNamespace,
|
||||
};
|
||||
}
|
||||
|
||||
@@ -405,3 +443,230 @@ function createSharedQueueConsumerNamespace(io: Server) {
|
||||
|
||||
return sharedQueue.namespace;
|
||||
}
|
||||
|
||||
function headersFromHandshake(handshake: Socket["handshake"]) {
|
||||
const headers = new Headers();
|
||||
|
||||
for (const [key, value] of Object.entries(handshake.headers)) {
|
||||
if (typeof value !== "string") continue;
|
||||
headers.append(key, value);
|
||||
}
|
||||
|
||||
return headers;
|
||||
}
|
||||
|
||||
function createWorkerNamespace({
|
||||
io,
|
||||
namespace,
|
||||
authenticate,
|
||||
}: {
|
||||
io: Server;
|
||||
namespace: string;
|
||||
authenticate: (request: Request) => Promise<boolean>;
|
||||
}) {
|
||||
const worker: Namespace<WorkerClientToServerEvents, WorkerServerToClientEvents> =
|
||||
io.of(namespace);
|
||||
|
||||
worker.use(async (socket, next) => {
|
||||
try {
|
||||
const headers = headersFromHandshake(socket.handshake);
|
||||
|
||||
logger.debug("Worker authentication", {
|
||||
namespace,
|
||||
socketId: socket.id,
|
||||
headers: Object.fromEntries(headers),
|
||||
});
|
||||
|
||||
const request = new Request("https://example.com", {
|
||||
headers,
|
||||
});
|
||||
|
||||
const success = await authenticate(request);
|
||||
|
||||
if (!success) {
|
||||
throw new Error("unauthorized");
|
||||
}
|
||||
|
||||
next();
|
||||
} catch (error) {
|
||||
logger.error("Worker authentication failed", {
|
||||
namespace,
|
||||
error: error instanceof Error ? error.message : error,
|
||||
});
|
||||
|
||||
socket.disconnect(true);
|
||||
}
|
||||
});
|
||||
|
||||
worker.on("connection", async (socket) => {
|
||||
logger.debug("worker connected", { namespace, socketId: socket.id });
|
||||
|
||||
const rooms = new Set<string>();
|
||||
|
||||
async function onNotification({
|
||||
time,
|
||||
run,
|
||||
snapshot,
|
||||
}: EventBusEventArgs<"workerNotification">[0]) {
|
||||
if (!env.RUN_ENGINE_DEBUG_WORKER_NOTIFICATIONS) {
|
||||
return;
|
||||
}
|
||||
|
||||
logger.debug("[handleSocketIo] Received worker notification", {
|
||||
namespace,
|
||||
time,
|
||||
runId: run.id,
|
||||
snapshot,
|
||||
});
|
||||
|
||||
// Record notification event
|
||||
await recordRunDebugLog(run.id, `run:notify workerNotification event`, {
|
||||
attributes: {
|
||||
properties: {
|
||||
snapshotId: snapshot.id,
|
||||
snapshotStatus: snapshot.executionStatus,
|
||||
rooms: Array.from(rooms),
|
||||
},
|
||||
},
|
||||
startTime: time,
|
||||
});
|
||||
}
|
||||
|
||||
engine.eventBus.on("workerNotification", onNotification);
|
||||
|
||||
const interval = setInterval(() => {
|
||||
logger.debug("Rooms for socket", {
|
||||
namespace,
|
||||
socketId: socket.id,
|
||||
rooms: Array.from(rooms),
|
||||
});
|
||||
}, 5000);
|
||||
|
||||
socket.on("disconnect", (reason, description) => {
|
||||
logger.debug("worker disconnected", {
|
||||
namespace,
|
||||
socketId: socket.id,
|
||||
reason,
|
||||
description,
|
||||
});
|
||||
clearInterval(interval);
|
||||
|
||||
engine.eventBus.off("workerNotification", onNotification);
|
||||
});
|
||||
|
||||
socket.on("disconnecting", (reason, description) => {
|
||||
logger.debug("worker disconnecting", {
|
||||
namespace,
|
||||
socketId: socket.id,
|
||||
reason,
|
||||
description,
|
||||
});
|
||||
clearInterval(interval);
|
||||
});
|
||||
|
||||
socket.on("error", (error) => {
|
||||
logger.error("worker error", {
|
||||
namespace,
|
||||
socketId: socket.id,
|
||||
error: JSON.parse(JSON.stringify(error)),
|
||||
});
|
||||
clearInterval(interval);
|
||||
});
|
||||
|
||||
socket.on("run:subscribe", async ({ version, runFriendlyIds }) => {
|
||||
logger.debug("run:subscribe", { namespace, version, runFriendlyIds });
|
||||
|
||||
const settledResult = await Promise.allSettled(
|
||||
runFriendlyIds.map(async (friendlyId) => {
|
||||
const room = roomFromFriendlyRunId(friendlyId);
|
||||
|
||||
logger.debug("Joining room", { namespace, room });
|
||||
|
||||
socket.join(room);
|
||||
rooms.add(room);
|
||||
|
||||
await recordRunDebugLog(
|
||||
RunId.fromFriendlyId(friendlyId),
|
||||
"run:subscribe received by platform",
|
||||
{
|
||||
attributes: {
|
||||
properties: {
|
||||
friendlyId,
|
||||
runFriendlyIds,
|
||||
room,
|
||||
},
|
||||
},
|
||||
}
|
||||
);
|
||||
})
|
||||
);
|
||||
|
||||
for (const result of settledResult) {
|
||||
if (result.status === "rejected") {
|
||||
logger.error("Error joining room", {
|
||||
namespace,
|
||||
runFriendlyIds,
|
||||
error: result.reason instanceof Error ? result.reason.message : result.reason,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
logger.debug("Rooms for socket after subscribe", {
|
||||
namespace,
|
||||
socketId: socket.id,
|
||||
rooms: Array.from(rooms),
|
||||
});
|
||||
});
|
||||
|
||||
socket.on("run:unsubscribe", async ({ version, runFriendlyIds }) => {
|
||||
logger.debug("run:unsubscribe", { namespace, version, runFriendlyIds });
|
||||
|
||||
const settledResult = await Promise.allSettled(
|
||||
runFriendlyIds.map(async (friendlyId) => {
|
||||
const room = roomFromFriendlyRunId(friendlyId);
|
||||
|
||||
logger.debug("Leaving room", { namespace, room });
|
||||
|
||||
socket.leave(room);
|
||||
rooms.delete(room);
|
||||
|
||||
await recordRunDebugLog(
|
||||
RunId.fromFriendlyId(friendlyId),
|
||||
"run:unsubscribe received by platform",
|
||||
{
|
||||
attributes: {
|
||||
properties: {
|
||||
friendlyId,
|
||||
runFriendlyIds,
|
||||
room,
|
||||
},
|
||||
},
|
||||
}
|
||||
);
|
||||
})
|
||||
);
|
||||
|
||||
for (const result of settledResult) {
|
||||
if (result.status === "rejected") {
|
||||
logger.error("Error leaving room", {
|
||||
namespace,
|
||||
runFriendlyIds,
|
||||
error: result.reason instanceof Error ? result.reason.message : result.reason,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
logger.debug("Rooms for socket after unsubscribe", {
|
||||
namespace,
|
||||
socketId: socket.id,
|
||||
rooms: Array.from(rooms),
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
return worker;
|
||||
}
|
||||
|
||||
export function roomFromFriendlyRunId(id: string) {
|
||||
return `room:${id}`;
|
||||
}
|
||||
|
||||
@@ -51,3 +51,15 @@ function derivePresetNameFromValues(cpu: number, memory: number): MachinePresetN
|
||||
|
||||
return defaultMachine;
|
||||
}
|
||||
|
||||
export function allMachines(): Record<string, MachinePreset> {
|
||||
return Object.fromEntries(
|
||||
Object.entries(machines).map(([name, preset]) => [
|
||||
name,
|
||||
{
|
||||
name: name as MachinePresetName,
|
||||
...preset,
|
||||
},
|
||||
])
|
||||
);
|
||||
}
|
||||
|
||||
@@ -19,7 +19,7 @@ import { FailedTaskRunService } from "../failedTaskRun.server";
|
||||
import { CancelDevSessionRunsService } from "../services/cancelDevSessionRuns.server";
|
||||
import { CompleteAttemptService } from "../services/completeAttempt.server";
|
||||
import { attributesFromAuthenticatedEnv, tracer } from "../tracer.server";
|
||||
import { getMaxDuration } from "../utils/maxDuration";
|
||||
import { getMaxDuration } from "@trigger.dev/core/v3/apps";
|
||||
import { DevSubscriber, devPubSub } from "./devPubSub.server";
|
||||
import { findQueueInEnvironment, sanitizeQueueName } from "~/models/taskQueue.server";
|
||||
import { createRedisClient, RedisClient } from "~/redis.server";
|
||||
|
||||
@@ -1,6 +1,9 @@
|
||||
import type { Prettify } from "@trigger.dev/core";
|
||||
import { BackgroundWorker } from "@trigger.dev/database";
|
||||
import { CURRENT_DEPLOYMENT_LABEL } from "~/consts";
|
||||
import { BackgroundWorker, WorkerDeployment } from "@trigger.dev/database";
|
||||
import {
|
||||
CURRENT_DEPLOYMENT_LABEL,
|
||||
CURRENT_UNMANAGED_DEPLOYMENT_LABEL,
|
||||
} from "@trigger.dev/core/v3/apps";
|
||||
import { Prisma, prisma } from "~/db.server";
|
||||
import { AuthenticatedEnvironment } from "~/services/apiAuth.server";
|
||||
|
||||
@@ -34,6 +37,7 @@ type WorkerDeploymentWithWorkerTasks = Prisma.WorkerDeploymentGetPayload<{
|
||||
sdkVersion: true;
|
||||
cliVersion: true;
|
||||
supportsLazyAttempts: true;
|
||||
engine: true;
|
||||
tasks: {
|
||||
select: {
|
||||
id: true;
|
||||
@@ -53,12 +57,13 @@ type WorkerDeploymentWithWorkerTasks = Prisma.WorkerDeploymentGetPayload<{
|
||||
}>;
|
||||
|
||||
export async function findCurrentWorkerDeployment(
|
||||
environmentId: string
|
||||
environmentId: string,
|
||||
label = CURRENT_DEPLOYMENT_LABEL
|
||||
): Promise<WorkerDeploymentWithWorkerTasks | undefined> {
|
||||
const promotion = await prisma.workerDeploymentPromotion.findFirst({
|
||||
where: {
|
||||
environmentId,
|
||||
label: CURRENT_DEPLOYMENT_LABEL,
|
||||
label,
|
||||
},
|
||||
select: {
|
||||
deployment: {
|
||||
@@ -75,6 +80,7 @@ export async function findCurrentWorkerDeployment(
|
||||
cliVersion: true,
|
||||
supportsLazyAttempts: true,
|
||||
tasks: true,
|
||||
engine: true,
|
||||
},
|
||||
},
|
||||
},
|
||||
@@ -85,11 +91,37 @@ export async function findCurrentWorkerDeployment(
|
||||
return promotion?.deployment;
|
||||
}
|
||||
|
||||
export async function findCurrentWorkerDeploymentWithoutTasks(
|
||||
environmentId: string,
|
||||
label = CURRENT_DEPLOYMENT_LABEL
|
||||
): Promise<WorkerDeployment | undefined> {
|
||||
const promotion = await prisma.workerDeploymentPromotion.findUnique({
|
||||
where: {
|
||||
environmentId_label: {
|
||||
environmentId,
|
||||
label,
|
||||
},
|
||||
},
|
||||
include: {
|
||||
deployment: true,
|
||||
},
|
||||
});
|
||||
|
||||
return promotion?.deployment;
|
||||
}
|
||||
|
||||
export async function findCurrentUnmanagedWorkerDeployment(
|
||||
environmentId: string
|
||||
): Promise<WorkerDeploymentWithWorkerTasks | undefined> {
|
||||
return await findCurrentWorkerDeployment(environmentId, CURRENT_UNMANAGED_DEPLOYMENT_LABEL);
|
||||
}
|
||||
|
||||
export async function findCurrentWorkerFromEnvironment(
|
||||
environment: Pick<AuthenticatedEnvironment, "id" | "type">
|
||||
environment: Pick<AuthenticatedEnvironment, "id" | "type">,
|
||||
label = CURRENT_DEPLOYMENT_LABEL
|
||||
): Promise<Pick<
|
||||
BackgroundWorker,
|
||||
"id" | "friendlyId" | "version" | "sdkVersion" | "cliVersion" | "supportsLazyAttempts"
|
||||
"id" | "friendlyId" | "version" | "sdkVersion" | "cliVersion" | "supportsLazyAttempts" | "engine"
|
||||
> | null> {
|
||||
if (environment.type === "DEVELOPMENT") {
|
||||
const latestDevWorker = await prisma.backgroundWorker.findFirst({
|
||||
@@ -102,11 +134,24 @@ export async function findCurrentWorkerFromEnvironment(
|
||||
});
|
||||
return latestDevWorker;
|
||||
} else {
|
||||
const deployment = await findCurrentWorkerDeployment(environment.id);
|
||||
const deployment = await findCurrentWorkerDeployment(environment.id, label);
|
||||
return deployment?.worker ?? null;
|
||||
}
|
||||
}
|
||||
|
||||
export async function findCurrentUnmanagedWorkerFromEnvironment(
|
||||
environment: Pick<AuthenticatedEnvironment, "id" | "type">
|
||||
): Promise<Pick<
|
||||
BackgroundWorker,
|
||||
"id" | "friendlyId" | "version" | "sdkVersion" | "cliVersion" | "supportsLazyAttempts"
|
||||
> | null> {
|
||||
if (environment.type === "DEVELOPMENT") {
|
||||
return null;
|
||||
}
|
||||
|
||||
return await findCurrentWorkerFromEnvironment(environment, CURRENT_UNMANAGED_DEPLOYMENT_LABEL);
|
||||
}
|
||||
|
||||
export async function getWorkerDeploymentFromWorker(
|
||||
workerId: string
|
||||
): Promise<WorkerDeploymentWithWorkerTasks | undefined> {
|
||||
|
||||
@@ -13,6 +13,7 @@ import { mkdtemp } from "fs/promises";
|
||||
import { createReadStream, createWriteStream } from "node:fs";
|
||||
import { pipeline } from "node:stream/promises";
|
||||
import { unlinkSync } from "fs";
|
||||
import { parseDockerImageReference, rebuildDockerImageReference } from "@trigger.dev/core/v3";
|
||||
|
||||
const TokenResponseBody = z.object({
|
||||
token: z.string(),
|
||||
@@ -466,70 +467,3 @@ async function streamRequestBodyToTempFile(request: IncomingMessage): Promise<st
|
||||
return;
|
||||
}
|
||||
}
|
||||
|
||||
type DockerImageParts = {
|
||||
registry?: string;
|
||||
repo: string;
|
||||
tag?: string;
|
||||
digest?: string;
|
||||
};
|
||||
|
||||
function parseDockerImageReference(imageReference: string): DockerImageParts {
|
||||
const parts: DockerImageParts = { repo: "" }; // Initialize with an empty repo
|
||||
|
||||
// Splitting by '@' to separate the digest (if exists)
|
||||
const atSplit = imageReference.split("@");
|
||||
if (atSplit.length > 1) {
|
||||
parts.digest = atSplit[1];
|
||||
imageReference = atSplit[0];
|
||||
}
|
||||
|
||||
// Splitting by ':' to separate the tag (if exists) and to ensure it's not part of a port
|
||||
let colonSplit = imageReference.split(":");
|
||||
if (colonSplit.length > 2 || (colonSplit.length === 2 && !colonSplit[1].includes("/"))) {
|
||||
// It's a tag if there's no '/' in the second part (after colon), or there are more than 2 parts (implying a port number in registry)
|
||||
parts.tag = colonSplit.pop(); // The last part is the tag
|
||||
imageReference = colonSplit.join(":"); // Join back in case it was a port number
|
||||
}
|
||||
|
||||
// Check for registry
|
||||
let slashIndex = imageReference.indexOf("/");
|
||||
if (slashIndex !== -1) {
|
||||
let potentialRegistry = imageReference.substring(0, slashIndex);
|
||||
// Validate if the first part is a valid hostname-like string (registry), otherwise treat the entire string as the repo
|
||||
if (
|
||||
potentialRegistry.includes(".") ||
|
||||
potentialRegistry === "localhost" ||
|
||||
potentialRegistry.includes(":")
|
||||
) {
|
||||
parts.registry = potentialRegistry;
|
||||
parts.repo = imageReference.substring(slashIndex + 1);
|
||||
} else {
|
||||
parts.repo = imageReference; // No valid registry found, treat as repo
|
||||
}
|
||||
} else {
|
||||
parts.repo = imageReference; // Only repo is present
|
||||
}
|
||||
|
||||
return parts;
|
||||
}
|
||||
|
||||
function rebuildDockerImageReference(parts: DockerImageParts): string {
|
||||
let imageReference = "";
|
||||
|
||||
if (parts.registry) {
|
||||
imageReference += `${parts.registry}/`;
|
||||
}
|
||||
|
||||
imageReference += parts.repo; // Repo is now guaranteed to be defined
|
||||
|
||||
if (parts.tag) {
|
||||
imageReference += `:${parts.tag}`;
|
||||
}
|
||||
|
||||
if (parts.digest) {
|
||||
imageReference += `@${parts.digest}`;
|
||||
}
|
||||
|
||||
return imageReference;
|
||||
}
|
||||
|
||||
@@ -0,0 +1,68 @@
|
||||
import { RunEngine } from "@internal/run-engine";
|
||||
import { prisma } from "~/db.server";
|
||||
import { env } from "~/env.server";
|
||||
import { tracer } from "./tracer.server";
|
||||
import { singleton } from "~/utils/singleton";
|
||||
import { defaultMachine, machines } from "@trigger.dev/platform/v3";
|
||||
import { allMachines } from "./machinePresets.server";
|
||||
|
||||
export const engine = singleton("RunEngine", createRunEngine);
|
||||
|
||||
export type { RunEngine };
|
||||
|
||||
function createRunEngine() {
|
||||
const engine = new RunEngine({
|
||||
prisma,
|
||||
worker: {
|
||||
workers: env.RUN_ENGINE_WORKER_COUNT,
|
||||
tasksPerWorker: env.RUN_ENGINE_TASKS_PER_WORKER,
|
||||
pollIntervalMs: env.RUN_ENGINE_WORKER_POLL_INTERVAL,
|
||||
redis: {
|
||||
keyPrefix: "engine:",
|
||||
port: env.RUN_ENGINE_WORKER_REDIS_PORT ?? undefined,
|
||||
host: env.RUN_ENGINE_WORKER_REDIS_HOST ?? undefined,
|
||||
username: env.RUN_ENGINE_WORKER_REDIS_USERNAME ?? undefined,
|
||||
password: env.RUN_ENGINE_WORKER_REDIS_PASSWORD ?? undefined,
|
||||
enableAutoPipelining: true,
|
||||
...(env.RUN_ENGINE_WORKER_REDIS_TLS_DISABLED === "true" ? {} : { tls: {} }),
|
||||
},
|
||||
},
|
||||
machines: {
|
||||
defaultMachine,
|
||||
machines: allMachines(),
|
||||
baseCostInCents: env.CENTS_PER_RUN,
|
||||
},
|
||||
queue: {
|
||||
defaultEnvConcurrency: env.DEFAULT_ENV_EXECUTION_CONCURRENCY_LIMIT,
|
||||
redis: {
|
||||
keyPrefix: "engine:",
|
||||
port: env.RUN_ENGINE_RUN_QUEUE_REDIS_PORT ?? undefined,
|
||||
host: env.RUN_ENGINE_RUN_QUEUE_REDIS_HOST ?? undefined,
|
||||
username: env.RUN_ENGINE_RUN_QUEUE_REDIS_USERNAME ?? undefined,
|
||||
password: env.RUN_ENGINE_RUN_QUEUE_REDIS_PASSWORD ?? undefined,
|
||||
enableAutoPipelining: true,
|
||||
...(env.RUN_ENGINE_RUN_QUEUE_REDIS_TLS_DISABLED === "true" ? {} : { tls: {} }),
|
||||
},
|
||||
},
|
||||
runLock: {
|
||||
redis: {
|
||||
keyPrefix: "engine:",
|
||||
port: env.RUN_ENGINE_RUN_LOCK_REDIS_PORT ?? undefined,
|
||||
host: env.RUN_ENGINE_RUN_LOCK_REDIS_HOST ?? undefined,
|
||||
username: env.RUN_ENGINE_RUN_LOCK_REDIS_USERNAME ?? undefined,
|
||||
password: env.RUN_ENGINE_RUN_LOCK_REDIS_PASSWORD ?? undefined,
|
||||
enableAutoPipelining: true,
|
||||
...(env.RUN_ENGINE_RUN_LOCK_REDIS_TLS_DISABLED === "true" ? {} : { tls: {} }),
|
||||
},
|
||||
},
|
||||
tracer,
|
||||
heartbeatTimeoutsMs: {
|
||||
PENDING_EXECUTING: env.RUN_ENGINE_TIMEOUT_PENDING_EXECUTING,
|
||||
PENDING_CANCEL: env.RUN_ENGINE_TIMEOUT_PENDING_CANCEL,
|
||||
EXECUTING: env.RUN_ENGINE_TIMEOUT_EXECUTING,
|
||||
EXECUTING_WITH_WAITPOINTS: env.RUN_ENGINE_TIMEOUT_EXECUTING_WITH_WAITPOINTS,
|
||||
},
|
||||
});
|
||||
|
||||
return engine;
|
||||
}
|
||||
@@ -0,0 +1,521 @@
|
||||
import { $replica, prisma } from "~/db.server";
|
||||
import {
|
||||
createExceptionPropertiesFromError,
|
||||
eventRepository,
|
||||
recordRunDebugLog,
|
||||
} from "./eventRepository.server";
|
||||
import { createJsonErrorObject, sanitizeError } from "@trigger.dev/core/v3";
|
||||
import { logger } from "~/services/logger.server";
|
||||
import { safeJsonParse } from "~/utils/json";
|
||||
import type { Attributes } from "@opentelemetry/api";
|
||||
import { reportInvocationUsage } from "~/services/platform.v3.server";
|
||||
import { roomFromFriendlyRunId, socketIo } from "./handleSocketIo.server";
|
||||
import { engine } from "./runEngine.server";
|
||||
import { PerformTaskRunAlertsService } from "./services/alerts/performTaskRunAlerts.server";
|
||||
import { RunId } from "@trigger.dev/core/v3/apps";
|
||||
import { updateMetadataService } from "~/services/metadata/updateMetadata.server";
|
||||
import { findEnvironmentFromRun } from "~/models/runtimeEnvironment.server";
|
||||
import { env } from "~/env.server";
|
||||
import { getTaskEventStoreTableForRun } from "./taskEventStore.server";
|
||||
|
||||
export function registerRunEngineEventBusHandlers() {
|
||||
engine.eventBus.on("runSucceeded", async ({ time, run }) => {
|
||||
try {
|
||||
const completedEvent = await eventRepository.completeEvent(
|
||||
getTaskEventStoreTableForRun(run),
|
||||
run.spanId,
|
||||
run.createdAt,
|
||||
run.completedAt ?? undefined,
|
||||
{
|
||||
endTime: time,
|
||||
attributes: {
|
||||
isError: false,
|
||||
output:
|
||||
run.outputType === "application/store" || run.outputType === "text/plain"
|
||||
? run.output
|
||||
: run.output
|
||||
? (safeJsonParse(run.output) as Attributes)
|
||||
: undefined,
|
||||
outputType: run.outputType,
|
||||
},
|
||||
}
|
||||
);
|
||||
|
||||
if (!completedEvent) {
|
||||
logger.error("[runSucceeded] Failed to complete event for unknown reason", {
|
||||
runId: run.id,
|
||||
spanId: run.spanId,
|
||||
});
|
||||
return;
|
||||
}
|
||||
} catch (error) {
|
||||
logger.error("[runSucceeded] Failed to complete event", {
|
||||
error: error instanceof Error ? error.message : error,
|
||||
runId: run.id,
|
||||
spanId: run.spanId,
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
// Handle alerts
|
||||
engine.eventBus.on("runFailed", async ({ time, run }) => {
|
||||
try {
|
||||
await PerformTaskRunAlertsService.enqueue(run.id);
|
||||
} catch (error) {
|
||||
logger.error("[runFailed] Failed to enqueue alerts", {
|
||||
error: error instanceof Error ? error.message : error,
|
||||
runId: run.id,
|
||||
spanId: run.spanId,
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
// Handle events
|
||||
engine.eventBus.on("runFailed", async ({ time, run }) => {
|
||||
try {
|
||||
const sanitizedError = sanitizeError(run.error);
|
||||
const exception = createExceptionPropertiesFromError(sanitizedError);
|
||||
|
||||
const eventStore = getTaskEventStoreTableForRun(run);
|
||||
|
||||
const completedEvent = await eventRepository.completeEvent(
|
||||
eventStore,
|
||||
run.spanId,
|
||||
run.createdAt,
|
||||
run.completedAt ?? undefined,
|
||||
{
|
||||
endTime: time,
|
||||
attributes: {
|
||||
isError: true,
|
||||
},
|
||||
events: [
|
||||
{
|
||||
name: "exception",
|
||||
time,
|
||||
properties: {
|
||||
exception,
|
||||
},
|
||||
},
|
||||
],
|
||||
}
|
||||
);
|
||||
|
||||
if (!completedEvent) {
|
||||
logger.error("[runFailed] Failed to complete event for unknown reason", {
|
||||
runId: run.id,
|
||||
spanId: run.spanId,
|
||||
});
|
||||
return;
|
||||
}
|
||||
|
||||
const inProgressEvents = await eventRepository.queryIncompleteEvents(
|
||||
eventStore,
|
||||
{
|
||||
runId: completedEvent?.runId,
|
||||
},
|
||||
run.createdAt,
|
||||
run.completedAt ?? undefined
|
||||
);
|
||||
|
||||
await Promise.all(
|
||||
inProgressEvents.map((event) => {
|
||||
try {
|
||||
const completedEvent = eventRepository.completeEvent(
|
||||
eventStore,
|
||||
run.spanId,
|
||||
run.createdAt,
|
||||
run.completedAt ?? undefined,
|
||||
{
|
||||
endTime: time,
|
||||
attributes: {
|
||||
isError: true,
|
||||
},
|
||||
events: [
|
||||
{
|
||||
name: "exception",
|
||||
time,
|
||||
properties: {
|
||||
exception,
|
||||
},
|
||||
},
|
||||
],
|
||||
}
|
||||
);
|
||||
|
||||
if (!completedEvent) {
|
||||
logger.error("[runFailed] Failed to complete in-progress event for unknown reason", {
|
||||
runId: run.id,
|
||||
spanId: run.spanId,
|
||||
eventId: event.id,
|
||||
});
|
||||
return;
|
||||
}
|
||||
} catch (error) {
|
||||
logger.error("[runFailed] Failed to complete in-progress event", {
|
||||
error: error instanceof Error ? error.message : error,
|
||||
runId: run.id,
|
||||
spanId: run.spanId,
|
||||
eventId: event.id,
|
||||
});
|
||||
}
|
||||
})
|
||||
);
|
||||
} catch (error) {
|
||||
logger.error("[runFailed] Failed to complete event", {
|
||||
error: error instanceof Error ? error.message : error,
|
||||
runId: run.id,
|
||||
spanId: run.spanId,
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
engine.eventBus.on("runAttemptFailed", async ({ time, run }) => {
|
||||
try {
|
||||
const sanitizedError = sanitizeError(run.error);
|
||||
const exception = createExceptionPropertiesFromError(sanitizedError);
|
||||
const eventStore = getTaskEventStoreTableForRun(run);
|
||||
|
||||
const inProgressEvents = await eventRepository.queryIncompleteEvents(
|
||||
eventStore,
|
||||
{
|
||||
runId: RunId.toFriendlyId(run.id),
|
||||
spanId: {
|
||||
not: run.spanId,
|
||||
},
|
||||
},
|
||||
run.createdAt,
|
||||
run.completedAt ?? undefined
|
||||
);
|
||||
|
||||
await Promise.all(
|
||||
inProgressEvents.map((event) => {
|
||||
return eventRepository.crashEvent({
|
||||
event: event,
|
||||
crashedAt: time,
|
||||
exception,
|
||||
});
|
||||
})
|
||||
);
|
||||
} catch (error) {
|
||||
logger.error("[runAttemptFailed] Failed to complete event", {
|
||||
error: error instanceof Error ? error.message : error,
|
||||
runId: run.id,
|
||||
spanId: run.spanId,
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
engine.eventBus.on("cachedRunCompleted", async ({ time, span, blockedRunId, hasError }) => {
|
||||
try {
|
||||
const blockedRun = await $replica.taskRun.findFirst({
|
||||
select: {
|
||||
taskEventStore: true,
|
||||
},
|
||||
where: {
|
||||
id: blockedRunId,
|
||||
},
|
||||
});
|
||||
|
||||
if (!blockedRun) {
|
||||
logger.error("[cachedRunCompleted] Blocked run not found", {
|
||||
blockedRunId,
|
||||
});
|
||||
return;
|
||||
}
|
||||
|
||||
const eventStore = getTaskEventStoreTableForRun(blockedRun);
|
||||
|
||||
const completedEvent = await eventRepository.completeEvent(
|
||||
eventStore,
|
||||
span.id,
|
||||
span.createdAt,
|
||||
time,
|
||||
{
|
||||
endTime: time,
|
||||
attributes: {
|
||||
isError: hasError,
|
||||
},
|
||||
}
|
||||
);
|
||||
|
||||
if (!completedEvent) {
|
||||
logger.error("[cachedRunCompleted] Failed to complete event for unknown reason", {
|
||||
span,
|
||||
});
|
||||
return;
|
||||
}
|
||||
} catch (error) {
|
||||
logger.error("[cachedRunCompleted] Failed to complete event for unknown reason", {
|
||||
error: error instanceof Error ? error.message : error,
|
||||
span,
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
engine.eventBus.on("runExpired", async ({ time, run }) => {
|
||||
try {
|
||||
const eventStore = getTaskEventStoreTableForRun(run);
|
||||
|
||||
const completedEvent = await eventRepository.completeEvent(
|
||||
eventStore,
|
||||
run.spanId,
|
||||
run.createdAt,
|
||||
run.completedAt ?? undefined,
|
||||
{
|
||||
endTime: time,
|
||||
attributes: {
|
||||
isError: true,
|
||||
},
|
||||
events: [
|
||||
{
|
||||
name: "exception",
|
||||
time,
|
||||
properties: {
|
||||
exception: {
|
||||
message: `Run expired because the TTL (${run.ttl}) was reached`,
|
||||
},
|
||||
},
|
||||
},
|
||||
],
|
||||
}
|
||||
);
|
||||
|
||||
if (!completedEvent) {
|
||||
logger.error("[runFailed] Failed to complete event for unknown reason", {
|
||||
runId: run.id,
|
||||
spanId: run.spanId,
|
||||
});
|
||||
return;
|
||||
}
|
||||
} catch (error) {
|
||||
logger.error("[runExpired] Failed to complete event", {
|
||||
error: error instanceof Error ? error.message : error,
|
||||
runId: run.id,
|
||||
spanId: run.spanId,
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
engine.eventBus.on("runCancelled", async ({ time, run }) => {
|
||||
try {
|
||||
const eventStore = getTaskEventStoreTableForRun(run);
|
||||
|
||||
const inProgressEvents = await eventRepository.queryIncompleteEvents(
|
||||
eventStore,
|
||||
{
|
||||
runId: run.friendlyId,
|
||||
},
|
||||
run.createdAt,
|
||||
run.completedAt ?? undefined
|
||||
);
|
||||
|
||||
await Promise.all(
|
||||
inProgressEvents.map((event) => {
|
||||
const error = createJsonErrorObject(run.error);
|
||||
return eventRepository.cancelEvent(event, time, error.message);
|
||||
})
|
||||
);
|
||||
} catch (error) {
|
||||
logger.error("[runCancelled] Failed to cancel event", {
|
||||
error: error instanceof Error ? error.message : error,
|
||||
runId: run.id,
|
||||
spanId: run.spanId,
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
engine.eventBus.on("runRetryScheduled", async ({ time, run, environment, retryAt }) => {
|
||||
try {
|
||||
await eventRepository.recordEvent(`Retry #${run.attemptNumber} delay`, {
|
||||
taskSlug: run.taskIdentifier,
|
||||
environment,
|
||||
attributes: {
|
||||
properties: {
|
||||
retryAt: retryAt.toISOString(),
|
||||
},
|
||||
runId: run.friendlyId,
|
||||
style: {
|
||||
icon: "schedule-attempt",
|
||||
},
|
||||
queueName: run.queue,
|
||||
},
|
||||
context: run.traceContext as Record<string, string | undefined>,
|
||||
spanIdSeed: `retry-${run.attemptNumber + 1}`,
|
||||
endTime: retryAt,
|
||||
});
|
||||
} catch (error) {
|
||||
logger.error("[runRetryScheduled] Failed to record retry event", {
|
||||
error: error instanceof Error ? error.message : error,
|
||||
runId: run.id,
|
||||
spanId: run.spanId,
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
engine.eventBus.on("runAttemptStarted", async ({ time, run, organization }) => {
|
||||
try {
|
||||
if (run.attemptNumber === 1 && run.baseCostInCents > 0) {
|
||||
await reportInvocationUsage(organization.id, run.baseCostInCents, { runId: run.id });
|
||||
}
|
||||
} catch (error) {
|
||||
logger.error("[runAttemptStarted] Failed to report invocation usage", {
|
||||
error: error instanceof Error ? error.message : error,
|
||||
runId: run.id,
|
||||
orgId: organization.id,
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
engine.eventBus.on("runMetadataUpdated", async ({ time, run }) => {
|
||||
const env = await findEnvironmentFromRun(run.id);
|
||||
|
||||
if (!env) {
|
||||
logger.error("[runMetadataUpdated] Failed to find environment", { runId: run.id });
|
||||
return;
|
||||
}
|
||||
|
||||
try {
|
||||
await updateMetadataService.call(run.id, run.metadata, env);
|
||||
} catch (e) {
|
||||
logger.error("[runMetadataUpdated] Failed to update metadata", {
|
||||
taskRun: run.id,
|
||||
error:
|
||||
e instanceof Error
|
||||
? {
|
||||
name: e.name,
|
||||
message: e.message,
|
||||
stack: e.stack,
|
||||
}
|
||||
: e,
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
engine.eventBus.on("executionSnapshotCreated", async ({ time, run, snapshot }) => {
|
||||
const eventResult = await recordRunDebugLog(
|
||||
run.id,
|
||||
`${snapshot.executionStatus} - ${snapshot.description}`,
|
||||
{
|
||||
attributes: {
|
||||
properties: {
|
||||
snapshotId: snapshot.id,
|
||||
snapshotDescription: snapshot.description,
|
||||
snapshotStatus: snapshot.executionStatus,
|
||||
workerId: snapshot.workerId ?? undefined,
|
||||
runnerId: snapshot.runnerId ?? undefined,
|
||||
},
|
||||
},
|
||||
startTime: time,
|
||||
}
|
||||
);
|
||||
|
||||
if (!eventResult.success) {
|
||||
logger.error("[executionSnapshotCreated] Failed to record event", {
|
||||
runId: run.id,
|
||||
snapshot,
|
||||
error: eventResult.error,
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
engine.eventBus.on("workerNotification", async ({ time, run, snapshot }) => {
|
||||
logger.debug("[workerNotification] Notifying worker", { time, runId: run.id, snapshot });
|
||||
|
||||
// Notify the worker
|
||||
try {
|
||||
const runFriendlyId = RunId.toFriendlyId(run.id);
|
||||
const room = roomFromFriendlyRunId(runFriendlyId);
|
||||
|
||||
//send the notification to connected workers
|
||||
socketIo.workerNamespace
|
||||
.to(room)
|
||||
.emit("run:notify", { version: "1", run: { friendlyId: runFriendlyId } });
|
||||
|
||||
//send the notification to connected dev workers
|
||||
socketIo.devWorkerNamespace
|
||||
.to(room)
|
||||
.emit("run:notify", { version: "1", run: { friendlyId: runFriendlyId } });
|
||||
|
||||
if (!env.RUN_ENGINE_DEBUG_WORKER_NOTIFICATIONS) {
|
||||
return;
|
||||
}
|
||||
|
||||
// Record notification event
|
||||
const eventResult = await recordRunDebugLog(
|
||||
run.id,
|
||||
`run:notify platform -> supervisor: ${snapshot.executionStatus}`,
|
||||
{
|
||||
attributes: {
|
||||
properties: {
|
||||
snapshotId: snapshot.id,
|
||||
snapshotStatus: snapshot.executionStatus,
|
||||
},
|
||||
},
|
||||
startTime: time,
|
||||
}
|
||||
);
|
||||
|
||||
if (!eventResult.success) {
|
||||
logger.error("[workerNotification] Failed to record event", {
|
||||
runId: run.id,
|
||||
snapshot,
|
||||
error: eventResult.error,
|
||||
});
|
||||
}
|
||||
} catch (error) {
|
||||
logger.error("[workerNotification] Failed to notify worker", {
|
||||
error: error instanceof Error ? error.message : error,
|
||||
runId: run.id,
|
||||
snapshot,
|
||||
});
|
||||
|
||||
// Record notification event
|
||||
const eventResult = await recordRunDebugLog(
|
||||
run.id,
|
||||
`run:notify ERROR platform -> supervisor: ${snapshot.executionStatus}`,
|
||||
{
|
||||
attributes: {
|
||||
properties: {
|
||||
snapshotId: snapshot.id,
|
||||
snapshotStatus: snapshot.executionStatus,
|
||||
error: error instanceof Error ? error.message : String(error),
|
||||
},
|
||||
},
|
||||
startTime: time,
|
||||
}
|
||||
);
|
||||
|
||||
if (!eventResult.success) {
|
||||
logger.error("[workerNotification] Failed to record event", {
|
||||
runId: run.id,
|
||||
snapshot,
|
||||
error: eventResult.error,
|
||||
});
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
engine.eventBus.on("incomingCheckpointDiscarded", async ({ time, run, snapshot, checkpoint }) => {
|
||||
const eventResult = await recordRunDebugLog(
|
||||
run.id,
|
||||
`Checkpoint discarded: ${checkpoint.discardReason}`,
|
||||
{
|
||||
attributes: {
|
||||
properties: {
|
||||
snapshotId: snapshot.id,
|
||||
...checkpoint.metadata,
|
||||
},
|
||||
},
|
||||
startTime: time,
|
||||
}
|
||||
);
|
||||
|
||||
if (!eventResult.success) {
|
||||
logger.error("[incomingCheckpointDiscarded] Failed to record event", {
|
||||
runId: run.id,
|
||||
snapshot,
|
||||
error: eventResult.error,
|
||||
});
|
||||
}
|
||||
});
|
||||
}
|
||||
@@ -0,0 +1,36 @@
|
||||
import { AuthenticatedEnvironment } from "~/services/apiAuth.server";
|
||||
import { marqs } from "./marqs/index.server";
|
||||
import { engine } from "./runEngine.server";
|
||||
|
||||
//This allows us to update MARQS and the RunQueue
|
||||
|
||||
/** Updates MARQS and the RunQueue limits */
|
||||
export async function updateEnvConcurrencyLimits(environment: AuthenticatedEnvironment) {
|
||||
await Promise.allSettled([
|
||||
marqs?.updateEnvConcurrencyLimits(environment),
|
||||
engine.runQueue.updateEnvConcurrencyLimits(environment),
|
||||
]);
|
||||
}
|
||||
|
||||
/** Updates MARQS and the RunQueue limits for a queue */
|
||||
export async function updateQueueConcurrencyLimits(
|
||||
environment: AuthenticatedEnvironment,
|
||||
queueName: string,
|
||||
concurrency: number
|
||||
) {
|
||||
await Promise.allSettled([
|
||||
marqs?.updateQueueConcurrencyLimits(environment, queueName, concurrency),
|
||||
engine.runQueue.updateQueueConcurrencyLimits(environment, queueName, concurrency),
|
||||
]);
|
||||
}
|
||||
|
||||
/** Removes MARQS and the RunQueue limits for a queue */
|
||||
export async function removeQueueConcurrencyLimits(
|
||||
environment: AuthenticatedEnvironment,
|
||||
queueName: string
|
||||
) {
|
||||
await Promise.allSettled([
|
||||
marqs?.removeQueueConcurrencyLimits(environment, queueName),
|
||||
engine.runQueue.removeQueueConcurrencyLimits(environment, queueName),
|
||||
]);
|
||||
}
|
||||
@@ -2,6 +2,7 @@ import { Span, SpanKind } from "@opentelemetry/api";
|
||||
import { PrismaClientOrTransaction, prisma } from "~/db.server";
|
||||
import { AuthenticatedEnvironment } from "~/services/apiAuth.server";
|
||||
import { attributesFromAuthenticatedEnv, tracer } from "../tracer.server";
|
||||
import { engine, RunEngine } from "../runEngine.server";
|
||||
|
||||
export abstract class BaseService {
|
||||
constructor(protected readonly _prisma: PrismaClientOrTransaction = prisma) {}
|
||||
@@ -37,6 +38,20 @@ export abstract class BaseService {
|
||||
}
|
||||
}
|
||||
|
||||
export type WithRunEngineOptions<T> = T & {
|
||||
prisma?: PrismaClientOrTransaction;
|
||||
engine?: RunEngine;
|
||||
};
|
||||
|
||||
export class WithRunEngine extends BaseService {
|
||||
protected readonly _engine: RunEngine;
|
||||
|
||||
constructor(opts: { prisma?: PrismaClientOrTransaction; engine?: RunEngine } = {}) {
|
||||
super(opts.prisma);
|
||||
this._engine = opts.engine ?? engine;
|
||||
}
|
||||
}
|
||||
|
||||
export class ServiceValidationError extends Error {
|
||||
constructor(message: string, public status?: number) {
|
||||
super(message);
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user