feat(webapp): hosted webhook ingress, delivery pipeline, and dashboard (#4344)
## Summary The server half of hosted webhooks: the public ingress endpoint, signature verification, the delivery pipeline (Postgres partitioned storage + ClickHouse for ordering), the in-app partition manager, the HTTP API, and the dashboard (Deliveries, Endpoints, and the in-app test console). The public SDK and docs half is #4537. That PR carries the user-facing API (`webhook()`, `chat.event` / `chat.channels`, the `@trigger.dev/slack` connector) and builds on the shared `@trigger.dev/core` schemas that ship here. ## Shipping behind a flag A `WEBHOOK_ENABLED` env var (default off) gates the public ingress route and the engine worker plus partition cron, so merging and deploying this changes nothing in production until it is flipped on per environment. The dashboard is separately gated per org by the `hasWebhooksAccess` feature flag. ## Note on packages This PR includes the `@trigger.dev/core` schema additions the server compiles against, but carries no changeset. Core is not consumed independently of the SDK, so it is released together with the SDK via #4537. Keeping its changeset off `main` means no release cut from `main` publishes it early.
This commit is contained in:
Eric Allam
+221
@@ -0,0 +1,221 @@
|
||||
# Onboarding: taking over the hosted webhooks PR (#4344) for design
|
||||
|
||||
You are picking up **PR #4344 "hosted webhooks, agent channels, and human-in-the-loop"** to own the UX and front-end. This doc gets you from a clean machine to a running dashboard with realistic webhook data you can screenshot, restyle, and iterate on.
|
||||
|
||||
The feature is built and green (all backend plumbing, all four dashboard surfaces, the in-app test console). Your job is the visual and interaction design of the dashboard surfaces, not the backend. Everything below is oriented around that.
|
||||
|
||||
---
|
||||
|
||||
## 1. What you are designing
|
||||
|
||||
Hosted webhooks let a Trigger.dev user receive and verify a provider's webhooks (Stripe, GitHub, and so on) as a task, with no ingress or verification code of their own. A `webhook()` handler in their project gets a hosted URL; deliveries to that URL are verified, recorded, and routed to their `onEvent` handler.
|
||||
|
||||
The dashboard has **four surfaces you own**, all under the "Webhooks" nav section (teal icon):
|
||||
|
||||
| Surface | Route (under `/orgs/:org/projects/:project/env/:env`) | What it shows |
|
||||
| --- | --- | --- |
|
||||
| **Deliveries list** | `/webhooks` | Every delivery across all endpoints in the environment. Runs-style filter bar (Status, Webhook, Created, plus a More-filters menu for Delivery ID / Run ID), applied-filter pills, a Webhook column linking to the handler. This is the main screen. |
|
||||
| **Delivery detail** | `/webhooks/deliveries/:deliveryParam` | One delivery. Main panel is a tabbed view (Event payload / Request headers) rendered as JSON. Sidebar property table (status badge, webhook + run links, external delivery id, idempotency key, timestamps, computed duration, error). Also has a friendly "not available / retained for N days" empty state for expired or bogus links. |
|
||||
| **Handler detail + Console** | `/webhooks/:webhookParam` | The handler (the `webhook()` in the user's code). Tabs: Deliveries, Runs, Endpoints. This page also hosts the **Webhook Console / Composer** (see section 5), the tool you will lean on for data. |
|
||||
| **Endpoint detail** | `/webhooks/endpoints/:endpointParam` | One endpoint. Left: scoped deliveries. Right: a **Connect** card (webhook URL, signing secret set/rotate/generate, provider setup rendered from the verifier config), Routing, Scope, Metadata. |
|
||||
|
||||
The status vocabulary, badges, and colors live in `components/webhookDeliveries/v1/DeliveryStatus.tsx` and `components/webhookEndpoints/v1/EndpointStatus.tsx`. The nav accent color is a Tailwind token `--color-webhooks` (teal), used via `text-webhooks`.
|
||||
|
||||
---
|
||||
|
||||
## 2. Get the code
|
||||
|
||||
You need the PR branch, `feat/hosted-webhook-ingress`.
|
||||
|
||||
```bash
|
||||
git clone https://github.com/triggerdotdev/trigger.dev.git
|
||||
cd trigger.dev
|
||||
gh pr checkout 4344 # lands you on feat/hosted-webhook-ingress
|
||||
```
|
||||
|
||||
If you plan to push design changes back to this branch, coordinate with Eric first: the branch is rebased and force-pushed periodically, so agree on timing or work on a child branch and open a follow-up.
|
||||
|
||||
Toolchain: pnpm 10.33.2 via corepack, Node 22+. Use `corepack pnpm` (a bare `pnpm` can be an old global that wipes `node_modules`).
|
||||
|
||||
```bash
|
||||
corepack enable
|
||||
corepack pnpm install
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 3. Bring the stack up
|
||||
|
||||
Four services and the webapp. Run from the repo root.
|
||||
|
||||
```bash
|
||||
# 1. Core dev services: Postgres, Redis, Electric, MinIO, ClickHouse, s2-lite
|
||||
corepack pnpm run docker
|
||||
|
||||
# 2. Config
|
||||
cp .env.example .env
|
||||
```
|
||||
|
||||
Now edit `.env` and add the two webhook-delivery replication lines (they are NOT in `.env.example`, and without them the Deliveries list looks empty even after you send webhooks, see section 5):
|
||||
|
||||
```bash
|
||||
# webhook deliveries replication (required for the Deliveries list/detail to populate)
|
||||
WEBHOOK_DELIVERIES_REPLICATION_CLICKHOUSE_URL=http://default:password@localhost:8123
|
||||
WEBHOOK_DELIVERIES_REPLICATION_ENABLED=1
|
||||
```
|
||||
|
||||
Then migrate, seed, build, and run:
|
||||
|
||||
```bash
|
||||
corepack pnpm run db:migrate
|
||||
corepack pnpm run db:seed # creates the References org + hello-world project
|
||||
|
||||
# Build the pieces you will run (do these sequentially, not with db:seed running)
|
||||
corepack pnpm run build --filter webapp --filter trigger.dev --filter "@trigger.dev/sdk"
|
||||
|
||||
# Run the webapp (http://localhost:3030)
|
||||
corepack pnpm run dev --filter webapp
|
||||
curl -s http://localhost:3030/healthcheck # verify
|
||||
```
|
||||
|
||||
**Log in (dev):** open http://localhost:3030, submit the email `local@trigger.dev`. Dev auto-verifies the magic link (watch the webapp log for `/magic?token=`). That seeded user is an org admin, which matters for the next step.
|
||||
|
||||
---
|
||||
|
||||
## 4. Turn the feature on
|
||||
|
||||
The dashboard is gated by a feature flag, `hasWebhooksAccess` (default off).
|
||||
|
||||
- The seeded dev user `local@trigger.dev` is an **admin**, and admins bypass the flag, so on a fresh seed the Webhooks nav section is already visible to you. Nothing to do.
|
||||
- If you use a non-admin user, set `featureFlags.hasWebhooksAccess = true` on the `Organization` row to reveal the nav section. (A global `FeatureFlag` row with key `hasWebhooksAccess` makes the pages reachable by URL, but the left nav reads only org-level flags, so the section stays hidden for non-admins.)
|
||||
|
||||
If the "Webhooks" section is missing from the left nav, this flag is why.
|
||||
|
||||
---
|
||||
|
||||
## 5. Get nice data (the part that matters)
|
||||
|
||||
Delivery rows are what make these screens interesting: a spread of providers, statuses, payloads, timestamps. Here is how the data flows and how to produce it.
|
||||
|
||||
### The pipeline (why an empty list is usually a setup issue, not a bug)
|
||||
|
||||
`ingest -> engine (verify, filter, route) -> Postgres WebhookDelivery rows -> replication -> ClickHouse`. The Deliveries **list orders and paginates from ClickHouse**, then hydrates every visible field from Postgres. So if replication is off (section 3), you can create deliveries and still see an empty list. Enable the two replication env vars and restart the webapp.
|
||||
|
||||
One caveat baked into the design: replication starts streaming from the moment it is enabled, so deliveries written **before** you turned it on will not appear. Turn replication on first, then generate data.
|
||||
|
||||
### Fastest path: the seed script
|
||||
|
||||
There is a seed script that inserts a full, stable dataset directly into both stores (Postgres and ClickHouse), so you get realistic screens on a fresh DB with no workers, no `trigger dev`, and no signing secrets to set:
|
||||
|
||||
```bash
|
||||
corepack pnpm --filter webapp run db:seed:webhooks
|
||||
# optional: deliveries per endpoint (default 45)
|
||||
corepack pnpm --filter webapp run db:seed:webhooks -- 60
|
||||
```
|
||||
|
||||
It creates six endpoints across different providers and verifier schemes (Stripe, GitHub, Slack, Svix, Discord, and a custom shared-secret one, with a mix of active/inactive and secret-set/not-set), then a spread of deliveries over the last two weeks covering **every** delivery status (SUCCEEDED, FAILED, FILTERED, PENDING, PROCESSING), realistic per-provider payloads and headers, and a mix of test and live. It attaches to the first DEVELOPMENT environment your local user can see (set `WEBHOOK_SEED_PROJECT="<project name>"` to target a specific one), and prints the exact Deliveries URL when it finishes. Re-running clears and reseeds that environment, so you always get the same clean dataset. The script is `apps/webapp/seed-webhook-deliveries.ts`; edit the `ENDPOINTS` array or the status weights to shape the data to whatever you are designing.
|
||||
|
||||
Because it writes the ClickHouse rows directly, seeded data shows up **without** the replication setup in section 3. That replication env is only needed for the live and Composer paths below. (The seed uses `WEBHOOK_DELIVERIES_REPLICATION_CLICKHOUSE_URL` if set, otherwise `CLICKHOUSE_URL`, which is already in `.env.example`.)
|
||||
|
||||
This is the recommended way to get data. The interactive paths below are for exercising the live pipeline (real verification, real routed runs) or the in-app test console.
|
||||
|
||||
### Interactive: create an endpoint (one-time)
|
||||
|
||||
The Composer sends to an endpoint, and endpoints only exist once a Trigger project that declares a `webhook()` has been dev-run or deployed. Quickest path: a tiny demo project.
|
||||
|
||||
```ts
|
||||
// demo/src/trigger/demo-webhook.ts
|
||||
import { webhook, webhooks } from "@trigger.dev/sdk";
|
||||
|
||||
export const demoWebhook = webhook({
|
||||
id: "demo-webhook",
|
||||
source: webhooks.custom<{ message: string }>({ /* generic HMAC */ }),
|
||||
onEvent: async ({ event, headers, ctx }) => {
|
||||
// event is the parsed body, headers is a Web Headers object
|
||||
},
|
||||
});
|
||||
|
||||
// A real provider, for realistic payloads:
|
||||
export const stripeWebhook = webhook({
|
||||
id: "stripe-webhook",
|
||||
source: webhooks.stripe(),
|
||||
onEvent: async ({ event }) => {},
|
||||
});
|
||||
```
|
||||
|
||||
Link that demo project to your local build and run `trigger dev` (see `AGENTS.md` "Testing with the hello-world Reference Project" for linking; the `triggerdotdev/references` repo has ready-made projects). Running `trigger dev` registers the `webhook()` handlers, which creates their endpoints. Set each endpoint's signing secret from the **endpoint detail Connect card** (Generate or paste).
|
||||
|
||||
### Interactive: fire deliveries with the Webhook Console
|
||||
|
||||
Open the handler detail page (`/webhooks/:webhookParam`). It hosts the **Composer** (`components/webhookConsole/WebhookComposer.tsx`). It has four source tabs and four signature modes, and it injects the delivery straight through the engine in-process, so it is fast and does not consume any real rate budget:
|
||||
|
||||
- **Sample tab**: pick a real provider event from the built-in catalog (`@internal/webhook-sources`, six first-class providers plus a large sample manifest). This is the fastest way to get realistic Stripe / GitHub / Svix / Square / Discord payloads with correct-looking headers.
|
||||
- **Body tab**: hand-write any JSON.
|
||||
- **Replay tab**: re-send a prior delivery.
|
||||
- **AI tab**: generate a payload with a prompt.
|
||||
- **Signature modes** `signed | unsigned | tampered | simulate`: this is how you produce a **spread of delivery statuses**. `signed` (with a secret set) verifies and routes to a SUCCEEDED delivery; `unsigned` and `tampered` produce failed/rejected deliveries. Send a mix to populate every status badge you need to design.
|
||||
|
||||
To get SUCCEEDED deliveries whose **runs** also complete (nicest end-to-end data), keep the demo project's `trigger dev` running so the routed task actually executes.
|
||||
|
||||
### Interactive: a real provider (most realistic)
|
||||
|
||||
For genuine payloads and headers, point the Stripe CLI at an endpoint: `stripe listen --forward-to http://localhost:3030/webhooks/v1/ingest/<opaqueId>`, set that endpoint's `whsec` via the Connect card, then `stripe trigger payment_intent.succeeded`.
|
||||
|
||||
---
|
||||
|
||||
## 6. Where the front-end code lives
|
||||
|
||||
| Area | Path |
|
||||
| --- | --- |
|
||||
| Routes (pages) | `apps/webapp/app/routes/_app.orgs.$organizationSlug.projects.$projectParam.env.$envParam.webhooks*` |
|
||||
| Deliveries list / detail components | `apps/webapp/app/components/webhookDeliveries/v1/` (`DeliveriesTable`, `DeliveryStatus`, `WebhookDeliveryFilters`, `DeliveryTimeline`, `useDeliveriesLiveReload`) |
|
||||
| Endpoint components | `apps/webapp/app/components/webhookEndpoints/v1/` (`EndpointsTable`, `EndpointStatus`) |
|
||||
| Console / Composer | `apps/webapp/app/components/webhookConsole/` (`WebhookComposer`, `SampleSourcePicker`, `ReplaySourcePicker`) |
|
||||
| Data (presenters, read-only from your side) | `apps/webapp/app/presenters/v3/WebhookDeliveriesListPresenter.server.ts`, `WebhookDeliveryDetailPresenter.server.ts`, `WebhookDetailPresenter.server.ts`, `webhookComposerEndpoints.server.ts` |
|
||||
| Nav entry | `apps/webapp/app/components/navigation/SideMenu.tsx` (the `staticSections` "webhooks" push) |
|
||||
| Path builders | `apps/webapp/app/utils/pathBuilder.ts` (`v3WebhooksPath`, `v3WebhookDeliveryPath`, `v3WebhookEndpointPath`, `v3WebhookTaskPath`) |
|
||||
| Accent color token | `apps/webapp/app/tailwind.css` (`--color-webhooks`, used as `text-webhooks`) |
|
||||
| Data seed script | `apps/webapp/seed-webhook-deliveries.ts` (run via `db:seed:webhooks`) |
|
||||
|
||||
**Styling:** the webapp is on Tailwind v4 (CSS-first `@theme` in `apps/webapp/app/tailwind.css`, there is no `tailwind.config.js`). Add or change design tokens there.
|
||||
|
||||
**Design language to match:** these screens deliberately reuse the Runs page primitives (the filter bar is built from `RunFilters` / `SharedFilters`, the tables mirror the Runs table cells). Match the Runs and Sessions pages, not a new visual system.
|
||||
|
||||
---
|
||||
|
||||
## 7. Iterating
|
||||
|
||||
- **HMR vs restart:** editing a component (`.tsx`) hot-reloads. Editing a `.server.ts` file makes the Remix dev server restart the app (a brief connection refused, then it comes back). Editing Tailwind tokens hot-reloads.
|
||||
- **Screenshots:** capture from the running dashboard at http://localhost:3030. Save shots outside the repo or to a scratch folder so they do not get committed.
|
||||
- **Typecheck after non-trivial changes:** `corepack pnpm run typecheck --filter webapp` (about 1 to 2 minutes). For small style tweaks, trust it and let CI catch anything.
|
||||
- **One boundary gotcha that the dev server will NOT catch:** route files must not leak server-only imports into the client bundle. The dev server tolerates it, but the production build fails. If you touch a route file and import anything server-only, run `corepack pnpm --filter webapp run build:remix` before pushing. Pure component and style edits are unaffected.
|
||||
|
||||
---
|
||||
|
||||
## 8. Shipping your changes
|
||||
|
||||
Follow the repo PR workflow:
|
||||
|
||||
- Format and lint before committing: `corepack pnpm run format` (oxfmt) and `corepack pnpm run lint:fix` (oxlint). CI enforces both.
|
||||
- Commit style is Conventional Commits, for example `feat(webapp): redesign webhook deliveries table`. No emoji, no attribution footer.
|
||||
- The PR is a **draft** awaiting an AI review pass, then a human review, before it flips to ready. Do not flip it to ready yourself; push your commits and let Eric coordinate the review and any rebase onto `main`.
|
||||
- CI to expect: `code-quality` (oxfmt + oxlint), `typecheck`, webapp unit shards, and the Playwright `e2e-webapp` job. Style-only changes usually only risk `code-quality`.
|
||||
|
||||
---
|
||||
|
||||
## 9. Quick reference
|
||||
|
||||
- **Webapp:** http://localhost:3030 (port comes from `REMIX_APP_PORT`, falling back to `PORT`/3030).
|
||||
- **Default docker services:** Postgres 5432, Redis 6379, ClickHouse HTTP 8123 (`default:password`), MinIO, Electric, s2-lite.
|
||||
- **Feature flag:** `hasWebhooksAccess` (admins bypass).
|
||||
- **Seed data:** `corepack pnpm --filter webapp run db:seed:webhooks` (append `-- <n>` for deliveries per endpoint).
|
||||
- **Must-set env for data to show:** `WEBHOOK_DELIVERIES_REPLICATION_ENABLED=1` and `WEBHOOK_DELIVERIES_REPLICATION_CLICKHOUSE_URL=http://default:password@localhost:8123`.
|
||||
- **Login:** `local@trigger.dev`, magic link auto-verifies in dev.
|
||||
- **Feature docs:** `docs/webhooks/` (overview, sources, connect, deliveries, channels, human-in-the-loop). Read `overview.mdx` and `deliveries.mdx` first for the mental model behind the screens.
|
||||
- **PR:** https://github.com/triggerdotdev/trigger.dev/pull/4344
|
||||
|
||||
---
|
||||
|
||||
## 10. Mental model in one paragraph
|
||||
|
||||
A user writes a `webhook()` in their project. On deploy (or `trigger dev`) that handler gets one or more hosted endpoints, each with a signing secret. A provider POSTs to the endpoint's URL; the engine verifies the signature, optionally filters, records a `WebhookDelivery`, and triggers the routed task run. The dashboard reads those deliveries: the list orders them out of ClickHouse and hydrates the rest from Postgres, the detail page reads Postgres directly (it holds the only copy of the event payload and headers). Everything you design sits on top of that delivery record and the endpoint that produced it.
|
||||
@@ -1,11 +1,15 @@
|
||||
export function WebhookIcon({ className }: { className?: string }) {
|
||||
return (
|
||||
<svg className={className} viewBox="0 0 190 180" fill="none" xmlns="http://www.w3.org/2000/svg">
|
||||
<svg className={className} viewBox="0 0 24 24" fill="none" xmlns="http://www.w3.org/2000/svg">
|
||||
<circle cx="12" cy="7" r="1.75" fill="currentColor" />
|
||||
<circle cx="7" cy="16" r="1.75" fill="currentColor" />
|
||||
<circle cx="17" cy="16" r="1.75" fill="currentColor" />
|
||||
<path
|
||||
fill="currentColor"
|
||||
fillRule="evenodd"
|
||||
clipRule="evenodd"
|
||||
d="M97.1775 62.8673C96.7903 62.892 96.4025 62.9043 96.0145 62.9042C86.0767 62.9042 78.008 54.8167 78.008 44.8556C78.008 34.8946 86.0767 26.807 96.0145 26.807C105.952 26.807 114.021 34.8946 114.021 44.8556C114.027 48.9969 112.605 53.0129 109.996 56.2245L129.9 92.8984C134.805 91.1113 139.984 90.1972 145.203 90.1972C169.928 90.1972 190 110.317 190 135.099C190 159.881 169.928 180 145.203 180C136.454 179.997 127.896 177.435 120.577 172.628C119.792 172.117 119.128 171.439 118.632 170.642C118.136 169.846 117.82 168.95 117.706 168.019C117.593 167.087 117.684 166.141 117.974 165.249C118.263 164.356 118.745 163.538 119.384 162.851C119.397 162.837 119.411 162.822 119.437 162.819C120.511 161.665 121.955 160.925 123.518 160.727C125.08 160.53 126.662 160.888 127.989 161.738C133.116 165.067 139.095 166.838 145.203 166.838C162.68 166.838 176.868 152.616 176.868 135.099C176.868 117.581 162.68 103.36 145.203 103.36C139.151 103.36 130.562 106.013 125.398 109.298C125.132 109.498 124.826 109.639 124.502 109.71C124.177 109.781 123.841 109.781 123.516 109.711C123.191 109.64 122.885 109.5 122.619 109.301C122.352 109.101 122.132 108.846 121.973 108.554L97.1775 62.8673ZM88.9103 140.885C88.0018 146.036 86.2022 150.988 83.5926 155.517C71.2305 176.98 43.8113 184.345 22.3993 171.954C0.987294 159.563 -6.35938 132.079 6.0019 110.616C10.3798 103.023 16.8727 96.8744 24.6848 92.9248C25.5195 92.4994 26.4369 92.2623 27.3727 92.2302C28.3085 92.1981 29.24 92.3718 30.1017 92.739C30.9639 93.1059 31.7354 93.6575 32.3621 94.3552C32.9888 95.0529 33.4555 95.8797 33.7294 96.7775C33.7346 96.7969 33.7399 96.8153 33.7294 96.8391C34.1899 98.3489 34.1078 99.9727 33.4973 101.428C32.8868 102.883 31.7866 104.078 30.3881 104.804C24.9482 107.59 20.4286 111.894 17.3741 117.197C8.63613 132.367 13.829 151.795 28.9642 160.554C44.0994 169.313 63.4824 164.107 72.2204 148.936C75.2464 143.683 77.2482 134.9 76.9926 128.775C76.9519 128.444 76.9829 128.108 77.0835 127.79C77.1841 127.473 77.3519 127.18 77.5754 126.934C77.799 126.687 78.073 126.491 78.3788 126.36C78.6846 126.229 79.015 126.166 79.3475 126.175L131.219 127.494C131.39 127.148 131.573 126.805 131.769 126.466C136.737 117.84 147.759 114.879 156.366 119.859C164.974 124.841 167.926 135.888 162.957 144.515C157.988 153.141 146.966 156.101 138.359 151.12C134.778 149.055 132.019 145.813 130.548 141.943L88.9103 140.885ZM67.1689 79.2862C63.1721 75.9215 59.7926 71.8828 57.1827 67.3521C44.8205 45.8901 52.1672 18.4061 73.5792 6.01594C94.9912 -6.3742 122.411 0.98963 134.773 22.4507C139.144 30.0472 141.21 38.757 140.717 47.5127C140.667 48.4501 140.414 49.3652 139.973 50.1937C139.533 51.0222 138.917 51.7442 138.169 52.3088C137.421 52.8734 136.559 53.2671 135.643 53.4622C134.727 53.6572 133.779 53.649 132.866 53.438C132.847 53.4336 132.828 53.4283 132.812 53.4072C131.278 53.0521 129.916 52.169 128.964 50.9115C128.012 49.654 127.53 48.1018 127.602 46.5249C127.915 40.4094 126.456 34.3338 123.4 29.031C114.662 13.8614 95.2802 8.65544 80.145 17.4139C65.0107 26.1731 59.8161 45.6005 68.5549 60.771C71.5809 66.0244 78.1687 72.1539 83.5882 74.9942C83.8943 75.1245 84.1687 75.3194 84.3928 75.5657C84.617 75.8119 84.7855 76.1036 84.8871 76.421C84.9886 76.7384 85.0207 77.074 84.9812 77.405C84.9417 77.7359 84.8315 78.0544 84.6581 78.3389L57.5815 122.706C57.7958 123.027 58.0005 123.357 58.1963 123.696C63.1644 132.324 60.2113 143.371 51.6042 148.352C42.9979 153.332 31.9762 150.372 27.0072 141.745C22.0383 133.119 24.9914 122.071 33.5976 117.091C37.173 115.015 41.3538 114.241 45.4328 114.9L67.1689 79.2862Z"
|
||||
d="M16 7C16 4.79086 14.2091 3 12 3C9.79086 3 8 4.79086 8 7C8 8.14562 8.48161 9.17875 9.25341 9.90798C9.65459 10.287 9.83991 10.8882 9.57187 11.3706L8.94292 12.5027L7 16M12 7L13.9429 10.4973L14.571 11.6278C14.8394 12.1109 15.4487 12.2704 15.9833 12.1304C16.3079 12.0453 16.6487 12 17 12C19.2091 12 21 13.7909 21 16C21 18.2091 19.2091 20 17 20C16.2949 20 15.6323 19.8175 15.0571 19.4973M17 16H12C11.4477 16 11.0128 16.4547 10.8766 16.9899C10.4361 18.7202 8.86748 20 7 20C4.79086 20 3 18.2091 3 16C3 14.496 3.83007 13.1859 5.05708 12.5027"
|
||||
stroke="currentColor"
|
||||
strokeWidth="2"
|
||||
strokeLinecap="round"
|
||||
strokeLinejoin="round"
|
||||
/>
|
||||
</svg>
|
||||
);
|
||||
|
||||
@@ -14,13 +14,31 @@ type List = {
|
||||
export const DirectionSchema = z.union([z.literal("forward"), z.literal("backward")]);
|
||||
export type Direction = z.infer<typeof DirectionSchema>;
|
||||
|
||||
export function ListPagination({ list, className }: { list: List; className?: string }) {
|
||||
export function ListPagination({
|
||||
list,
|
||||
className,
|
||||
cursorParam = "cursor",
|
||||
directionParam = "direction",
|
||||
}: {
|
||||
list: List;
|
||||
className?: string;
|
||||
cursorParam?: string;
|
||||
directionParam?: string;
|
||||
}) {
|
||||
const bothDisabled = !list.pagination.previous && !list.pagination.next;
|
||||
|
||||
return (
|
||||
<div className={cn("flex items-center", className)}>
|
||||
<PreviousButton cursor={list.pagination.previous} />
|
||||
<NextButton cursor={list.pagination.next} />
|
||||
<PreviousButton
|
||||
cursor={list.pagination.previous}
|
||||
cursorParam={cursorParam}
|
||||
directionParam={directionParam}
|
||||
/>
|
||||
<NextButton
|
||||
cursor={list.pagination.next}
|
||||
cursorParam={cursorParam}
|
||||
directionParam={directionParam}
|
||||
/>
|
||||
<div
|
||||
className={cn(
|
||||
"order-2 h-6 w-px bg-surface-control transition-colors peer-hover/next:bg-surface-control-hover peer-hover/prev:bg-surface-control-hover",
|
||||
@@ -31,8 +49,16 @@ export function ListPagination({ list, className }: { list: List; className?: st
|
||||
);
|
||||
}
|
||||
|
||||
function PreviousButton({ cursor }: { cursor?: string }) {
|
||||
const path = useCursorPath(cursor, "backward");
|
||||
function PreviousButton({
|
||||
cursor,
|
||||
cursorParam,
|
||||
directionParam,
|
||||
}: {
|
||||
cursor?: string;
|
||||
cursorParam: string;
|
||||
directionParam: string;
|
||||
}) {
|
||||
const path = useCursorPath(cursor, "backward", cursorParam, directionParam);
|
||||
|
||||
return (
|
||||
<div className={cn("peer/prev order-1", !path && "pointer-events-none")}>
|
||||
@@ -53,8 +79,16 @@ function PreviousButton({ cursor }: { cursor?: string }) {
|
||||
);
|
||||
}
|
||||
|
||||
function NextButton({ cursor }: { cursor?: string }) {
|
||||
const path = useCursorPath(cursor, "forward");
|
||||
function NextButton({
|
||||
cursor,
|
||||
cursorParam,
|
||||
directionParam,
|
||||
}: {
|
||||
cursor?: string;
|
||||
cursorParam: string;
|
||||
directionParam: string;
|
||||
}) {
|
||||
const path = useCursorPath(cursor, "forward", cursorParam, directionParam);
|
||||
|
||||
return (
|
||||
<div className={cn("peer/next order-3", !path && "pointer-events-none")}>
|
||||
@@ -75,7 +109,12 @@ function NextButton({ cursor }: { cursor?: string }) {
|
||||
);
|
||||
}
|
||||
|
||||
function useCursorPath(cursor: string | undefined, direction: Direction) {
|
||||
function useCursorPath(
|
||||
cursor: string | undefined,
|
||||
direction: Direction,
|
||||
cursorParam: string,
|
||||
directionParam: string
|
||||
) {
|
||||
const location = useLocation();
|
||||
|
||||
if (!cursor) {
|
||||
@@ -83,7 +122,7 @@ function useCursorPath(cursor: string | undefined, direction: Direction) {
|
||||
}
|
||||
|
||||
const search = new URLSearchParams(location.search);
|
||||
search.set("cursor", cursor);
|
||||
search.set("direction", direction);
|
||||
search.set(cursorParam, cursor);
|
||||
search.set(directionParam, direction);
|
||||
return location.pathname + "?" + search.toString();
|
||||
}
|
||||
|
||||
@@ -12,6 +12,7 @@ import {
|
||||
useRevalidator,
|
||||
useSubmit,
|
||||
} from "@remix-run/react";
|
||||
import { WebhookIcon } from "~/assets/icons/WebhookIcon";
|
||||
import { LayoutGroup, motion } from "framer-motion";
|
||||
import {
|
||||
type CSSProperties,
|
||||
@@ -131,6 +132,7 @@ import {
|
||||
v3SessionsPath,
|
||||
v3UsagePath,
|
||||
v3WaitpointTokensPath,
|
||||
v3WebhooksPath,
|
||||
} from "~/utils/pathBuilder";
|
||||
import { FreePlanUsage } from "../billing/FreePlanUsage";
|
||||
import { ConnectionIcon, DevPresencePanel, useDevPresence } from "../DevPresence";
|
||||
@@ -1201,6 +1203,19 @@ export function SideMenu({
|
||||
isCollapsed={isCollapsed}
|
||||
yieldActiveToFavorite
|
||||
/>
|
||||
{(user.admin || user.isImpersonating || featureFlags.hasWebhooksAccess) && (
|
||||
<SideMenuItem
|
||||
name="Webhooks"
|
||||
icon={WebhookIcon}
|
||||
activeIconColor="text-webhooks"
|
||||
inactiveIconColor="text-text-dimmed"
|
||||
to={v3WebhooksPath(organization, project, environment)}
|
||||
data-action="webhooks"
|
||||
badge={<NewBadge />}
|
||||
isCollapsed={isCollapsed}
|
||||
yieldActiveToFavorite
|
||||
/>
|
||||
)}
|
||||
</div>
|
||||
|
||||
{orderedSectionIds.map((sectionId) => {
|
||||
|
||||
@@ -40,6 +40,7 @@ import { TasksIcon } from "~/assets/icons/TasksIcon";
|
||||
import { UsageIcon } from "~/assets/icons/UsageIcon";
|
||||
import { UserGroupIcon } from "~/assets/icons/UserGroupIcon";
|
||||
import { WaitpointTokenIcon } from "~/assets/icons/WaitpointTokenIcon";
|
||||
import { WebhookIcon } from "~/assets/icons/WebhookIcon";
|
||||
import { VercelLogo } from "~/components/integrations/VercelLogo";
|
||||
import { useOptionalUser } from "~/hooks/useUser";
|
||||
import { type FavoritePage } from "~/services/dashboardPreferences.server";
|
||||
@@ -71,6 +72,7 @@ const FAVORITE_PAGE_ICONS: Record<
|
||||
"task-agent": { icon: CubeSparkleIcon, activeColor: "text-agents" },
|
||||
runs: { icon: RunsIcon, activeColor: "text-runs" },
|
||||
sessions: { icon: AIChatIcon, activeColor: "text-sessions" },
|
||||
webhooks: { icon: WebhookIcon, activeColor: "text-webhooks" },
|
||||
prompts: { icon: AIPenIcon, activeColor: "text-aiPrompts" },
|
||||
models: { icon: Box3DIcon, activeColor: "text-models" },
|
||||
logs: { icon: LogsIcon, activeColor: "text-logs" },
|
||||
@@ -213,6 +215,7 @@ const ENV_PAGE_META: Record<string, PageMeta> = {
|
||||
"": { icon: "tasks", name: "Tasks", singular: "Task" },
|
||||
runs: { icon: "runs", name: "Runs", singular: "Run" },
|
||||
sessions: { icon: "sessions", name: "Sessions", singular: "Session" },
|
||||
webhooks: { icon: "webhooks", name: "Webhook deliveries" },
|
||||
prompts: { icon: "prompts", name: "Prompts", singular: "Prompt" },
|
||||
models: { icon: "models", name: "Models", singular: "Model" },
|
||||
logs: { icon: "logs", name: "Logs" },
|
||||
|
||||
@@ -12,6 +12,7 @@ export function CopyableText({
|
||||
asChild,
|
||||
variant,
|
||||
hideTooltip,
|
||||
truncate,
|
||||
}: {
|
||||
value: string;
|
||||
copyValue?: string;
|
||||
@@ -24,6 +25,12 @@ export function CopyableText({
|
||||
* fire Radix's global "one tooltip open at a time" close and dismiss the parent.
|
||||
*/
|
||||
hideTooltip?: boolean;
|
||||
/**
|
||||
* Ellipsise the value rather than letting it overflow its column. For unbreakable strings
|
||||
* (hashes, opaque ids) that offer no wrap opportunity. The copy button moves into a reserved
|
||||
* right gutter so it stays visible instead of sitting outside the column.
|
||||
*/
|
||||
truncate?: boolean;
|
||||
}) {
|
||||
const [isHovered, setIsHovered] = useState(false);
|
||||
const { copy, copied } = useCopy(copyValue ?? value);
|
||||
@@ -51,15 +58,26 @@ export function CopyableText({
|
||||
|
||||
return (
|
||||
<span
|
||||
className={cn("group relative inline-flex h-6 items-center", className)}
|
||||
className={cn(
|
||||
"group relative inline-flex h-6 items-center",
|
||||
truncate && "max-w-full pr-7",
|
||||
className
|
||||
)}
|
||||
onMouseLeave={() => setIsHovered(false)}
|
||||
>
|
||||
<span onMouseEnter={() => setIsHovered(true)}>{value}</span>
|
||||
<span
|
||||
className={cn(truncate && "min-w-0 truncate")}
|
||||
onMouseEnter={() => setIsHovered(true)}
|
||||
>
|
||||
{value}
|
||||
</span>
|
||||
<span
|
||||
onClick={copy}
|
||||
onMouseDown={(e) => e.stopPropagation()}
|
||||
className={cn(
|
||||
"absolute -right-6 top-0 z-10 size-6 font-sans",
|
||||
"absolute top-0 z-10 size-6 font-sans",
|
||||
// Truncated values reserve a right gutter, so the button sits inside it
|
||||
truncate ? "right-0" : "-right-6",
|
||||
isHovered ? "flex" : "hidden"
|
||||
)}
|
||||
>
|
||||
|
||||
@@ -13,11 +13,11 @@ import { LiveTimer } from "../runs/v3/LiveTimer";
|
||||
// Types for the RunTimeline component
|
||||
export type TimelineEventState = "complete" | "error" | "inprogress" | "delayed";
|
||||
|
||||
type TimelineLineVariant = "light" | "normal";
|
||||
export type TimelineLineVariant = "light" | "normal";
|
||||
|
||||
type TimelineStyle = "normal" | "diminished";
|
||||
|
||||
type TimelineEventVariant =
|
||||
export type TimelineEventVariant =
|
||||
| "start-cap"
|
||||
| "dot-hollow"
|
||||
| "dot-solid"
|
||||
@@ -323,7 +323,7 @@ function buildTimelineItems(run: TimelineSpanRun): TimelineItem[] {
|
||||
export type RunTimelineEventProps = {
|
||||
title: ReactNode;
|
||||
subtitle?: ReactNode;
|
||||
state?: "complete" | "error" | "inprogress";
|
||||
state?: TimelineEventState;
|
||||
variant?: TimelineEventVariant;
|
||||
helpText?: string;
|
||||
style?: TimelineStyle;
|
||||
@@ -483,6 +483,12 @@ export type RunTimelineLineProps = {
|
||||
state?: TimelineEventState;
|
||||
variant?: TimelineLineVariant;
|
||||
style?: TimelineStyle;
|
||||
/**
|
||||
* Round the top of a thick ("normal") line. Needed when the line itself starts the thick bar,
|
||||
* as in the delivery timeline, where nothing above it supplies a `start-cap-thick`. The run
|
||||
* timeline always precedes its thick line with that cap, so it leaves this off.
|
||||
*/
|
||||
roundedTop?: boolean;
|
||||
};
|
||||
|
||||
export function RunTimelineLine({
|
||||
@@ -490,11 +496,12 @@ export function RunTimelineLine({
|
||||
state,
|
||||
variant = "normal",
|
||||
style = "normal",
|
||||
roundedTop = false,
|
||||
}: RunTimelineLineProps) {
|
||||
return (
|
||||
<div className="grid h-6 grid-cols-[1.125rem_1fr] gap-1 text-xs">
|
||||
<div className="flex items-stretch justify-center">
|
||||
<LineMarker state={state} variant={variant} style={style} />
|
||||
<LineMarker state={state} variant={variant} style={style} roundedTop={roundedTop} />
|
||||
</div>
|
||||
<div className="flex items-center justify-between gap-3">
|
||||
<span className="text-text-dimmed">{title}</span>
|
||||
@@ -507,10 +514,12 @@ function LineMarker({
|
||||
state,
|
||||
variant,
|
||||
style,
|
||||
roundedTop = false,
|
||||
}: {
|
||||
state?: TimelineEventState;
|
||||
variant: TimelineLineVariant;
|
||||
style?: TimelineStyle;
|
||||
roundedTop?: boolean;
|
||||
}) {
|
||||
let containerClass = "bg-text-dimmed";
|
||||
switch (state) {
|
||||
@@ -532,7 +541,7 @@ function LineMarker({
|
||||
switch (variant) {
|
||||
case "normal":
|
||||
return (
|
||||
<div className={cn("relative w-1.75", containerClass)}>
|
||||
<div className={cn("relative w-1.75", roundedTop && "rounded-t-xs", containerClass)}>
|
||||
{state === "inprogress" && (
|
||||
<div
|
||||
className="absolute inset-0 h-full w-full animate-tile-scroll opacity-30"
|
||||
|
||||
@@ -200,7 +200,7 @@ export const TaskRunListSearchFilters = z.object({
|
||||
),
|
||||
errorId: z.string().optional().describe("Error ID to filter runs by (e.g. error_abc123)"),
|
||||
sources: StringOrStringArray.describe(
|
||||
"Task trigger sources to filter by (STANDARD, SCHEDULED, AGENT)"
|
||||
"Task trigger sources to filter by (STANDARD, SCHEDULED, AGENT, WEBHOOK)"
|
||||
),
|
||||
});
|
||||
|
||||
@@ -1918,6 +1918,7 @@ const sourceOptions: { value: TaskTriggerSource; title: string }[] = [
|
||||
{ value: "STANDARD", title: "Standard" },
|
||||
{ value: "SCHEDULED", title: "Scheduled" },
|
||||
{ value: "AGENT", title: "Agent" },
|
||||
{ value: "WEBHOOK", title: "Webhook" },
|
||||
];
|
||||
|
||||
function SourceDropdown({
|
||||
|
||||
@@ -358,6 +358,8 @@ export interface TimeFilterProps {
|
||||
maxPeriodDays?: number;
|
||||
/** Optional className override for the value text in the filter pill */
|
||||
valueClassName?: string;
|
||||
/** Extra URL params to clear when the range changes, in addition to the default cursor/direction (e.g. a page's namespaced pagination params). */
|
||||
clearParams?: string[];
|
||||
}
|
||||
|
||||
export function TimeFilter({
|
||||
@@ -372,6 +374,7 @@ export function TimeFilter({
|
||||
onValueChange,
|
||||
maxPeriodDays,
|
||||
valueClassName,
|
||||
clearParams,
|
||||
}: TimeFilterProps = {}) {
|
||||
const { value } = useSearchParams();
|
||||
// In controlled mode (onValueChange provided) the caller owns all three values via local
|
||||
@@ -441,6 +444,7 @@ export function TimeFilter({
|
||||
applyShortcut={applyShortcut}
|
||||
onValueChange={onValueChange}
|
||||
maxPeriodDays={maxPeriodDays}
|
||||
clearParams={clearParams}
|
||||
/>
|
||||
)}
|
||||
</FilterMenuProvider>
|
||||
@@ -471,6 +475,7 @@ export function TimeDropdown({
|
||||
onApply,
|
||||
onValueChange,
|
||||
maxPeriodDays,
|
||||
clearParams,
|
||||
}: {
|
||||
trigger: ReactNode;
|
||||
period?: string;
|
||||
@@ -484,10 +489,13 @@ export function TimeDropdown({
|
||||
onValueChange?: (values: TimeFilterApplyValues) => void;
|
||||
/** When set an upgrade message will be shown if you select a period further back than this number of days */
|
||||
maxPeriodDays?: number;
|
||||
/** Extra URL params to clear on apply, alongside the default cursor/direction. */
|
||||
clearParams?: string[];
|
||||
}) {
|
||||
const organization = useOptionalOrganization();
|
||||
const [open, setOpen] = useState<boolean | undefined>();
|
||||
const { replace } = useSearchParams();
|
||||
const extraCleared = Object.fromEntries((clearParams ?? []).map((key) => [key, undefined]));
|
||||
const [fromValue, setFromValue] = useState(from);
|
||||
const [toValue, setToValue] = useState(to);
|
||||
|
||||
@@ -561,6 +569,7 @@ export function TimeDropdown({
|
||||
onValueChange(values);
|
||||
} else {
|
||||
replace({
|
||||
...extraCleared,
|
||||
period: periodToApply,
|
||||
cursor: undefined,
|
||||
direction: undefined,
|
||||
@@ -620,6 +629,7 @@ export function TimeDropdown({
|
||||
} else {
|
||||
// URL mode - navigate
|
||||
replace({
|
||||
...extraCleared,
|
||||
period: undefined,
|
||||
cursor: undefined,
|
||||
direction: undefined,
|
||||
|
||||
@@ -2,6 +2,7 @@ import type { TaskTriggerSource } from "@trigger.dev/database";
|
||||
import { ClockIcon } from "~/assets/icons/ClockIcon";
|
||||
import { CubeSparkleIcon } from "~/assets/icons/CubeSparkleIcon";
|
||||
import { TaskIconSmall } from "~/assets/icons/TaskIcon";
|
||||
import { WebhookIcon } from "~/assets/icons/WebhookIcon";
|
||||
import { cn } from "~/utils/cn";
|
||||
|
||||
export function TaskTriggerSourceIcon({
|
||||
@@ -21,6 +22,11 @@ export function TaskTriggerSourceIcon({
|
||||
case "AGENT": {
|
||||
return <CubeSparkleIcon className={cn("size-4.5 min-w-4.5 text-agents", className)} />;
|
||||
}
|
||||
case "WEBHOOK": {
|
||||
return (
|
||||
<WebhookIcon className={cn("size-[1.125rem] min-w-[1.125rem] text-webhooks", className)} />
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -35,5 +41,8 @@ export function taskTriggerSourceDescription(source: TaskTriggerSource) {
|
||||
case "AGENT": {
|
||||
return "Agent task";
|
||||
}
|
||||
case "WEBHOOK": {
|
||||
return "Webhook task";
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,91 @@
|
||||
import { type WebhookDeliveryStatus } from "@trigger.dev/database";
|
||||
import { useEffect, useState } from "react";
|
||||
import { useTypedFetcher } from "remix-typedjson";
|
||||
import { DateTime } from "~/components/primitives/DateTime";
|
||||
import { Spinner } from "~/components/primitives/Spinner";
|
||||
import { DeliveryStatusBadge } from "~/components/webhookDeliveries/v1/DeliveryStatus";
|
||||
import { type loader as replaySourceLoader } from "~/routes/resources.orgs.$organizationSlug.projects.$projectParam.env.$envParam.webhooks.endpoints.$endpointParam.replay-source";
|
||||
|
||||
export function ReplaySourcePicker({
|
||||
replaySourcePath,
|
||||
onLoad,
|
||||
}: {
|
||||
replaySourcePath: string;
|
||||
onLoad: (body: string, headers: Record<string, string>) => void;
|
||||
}) {
|
||||
const listFetcher = useTypedFetcher<typeof replaySourceLoader>();
|
||||
const payloadFetcher = useTypedFetcher<typeof replaySourceLoader>();
|
||||
|
||||
useEffect(() => {
|
||||
if (listFetcher.state === "idle" && listFetcher.data === undefined) {
|
||||
listFetcher.load(replaySourcePath);
|
||||
}
|
||||
}, [listFetcher, replaySourcePath]);
|
||||
|
||||
useEffect(() => {
|
||||
const data = payloadFetcher.data;
|
||||
if (data?.kind === "payload") {
|
||||
onLoad(data.body, data.headers);
|
||||
}
|
||||
}, [payloadFetcher.data, onLoad]);
|
||||
|
||||
const listLoading =
|
||||
listFetcher.state === "loading" ||
|
||||
(listFetcher.data === undefined && listFetcher.state !== "idle");
|
||||
const list = listFetcher.data?.kind === "list" ? listFetcher.data.deliveries : [];
|
||||
const [loadingDeliveryId, setLoadingDeliveryId] = useState<string | undefined>(undefined);
|
||||
|
||||
function selectDelivery(friendlyId: string) {
|
||||
setLoadingDeliveryId(friendlyId);
|
||||
payloadFetcher.load(`${replaySourcePath}?deliveryId=${encodeURIComponent(friendlyId)}`);
|
||||
}
|
||||
|
||||
return (
|
||||
<div className="flex h-full flex-col overflow-hidden">
|
||||
<div className="flex items-center gap-2 border-b border-grid-dimmed px-3 py-2">
|
||||
<span className="text-xs font-medium text-text-dimmed">
|
||||
Load a past delivery's payload into the composer
|
||||
</span>
|
||||
{listLoading ? <Spinner className="size-3.5" /> : null}
|
||||
</div>
|
||||
<div className="flex-1 overflow-y-auto scrollbar-thin scrollbar-track-transparent scrollbar-thumb-charcoal-600">
|
||||
{!listLoading && list.length === 0 ? (
|
||||
<p className="px-3 py-8 text-center text-sm text-text-dimmed">
|
||||
No past deliveries to replay yet.
|
||||
</p>
|
||||
) : (
|
||||
list.map((delivery) => (
|
||||
<button
|
||||
key={delivery.friendlyId}
|
||||
type="button"
|
||||
onClick={() => selectDelivery(delivery.friendlyId)}
|
||||
disabled={payloadFetcher.state !== "idle"}
|
||||
className="flex w-full items-center justify-between gap-2 border-b border-grid-dimmed px-3 py-2 text-left hover:bg-charcoal-800 disabled:opacity-60"
|
||||
>
|
||||
<span className="flex min-w-0 items-center gap-1.5">
|
||||
<span className="truncate font-mono text-xs text-text-bright">
|
||||
{delivery.friendlyId}
|
||||
</span>
|
||||
{delivery.isTest ? (
|
||||
<span className="shrink-0 rounded-sm bg-charcoal-700 px-1 py-0.5 text-xxs font-semibold uppercase tracking-wide text-text-dimmed">
|
||||
Test
|
||||
</span>
|
||||
) : null}
|
||||
</span>
|
||||
<span className="flex shrink-0 items-center gap-2">
|
||||
<span className="text-xxs text-text-dimmed">
|
||||
<DateTime date={new Date(delivery.createdAt)} />
|
||||
</span>
|
||||
{payloadFetcher.state !== "idle" && loadingDeliveryId === delivery.friendlyId ? (
|
||||
<Spinner className="size-3.5" />
|
||||
) : (
|
||||
<DeliveryStatusBadge status={delivery.status as WebhookDeliveryStatus} />
|
||||
)}
|
||||
</span>
|
||||
</button>
|
||||
))
|
||||
)}
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,197 @@
|
||||
import { MagnifyingGlassIcon } from "@heroicons/react/20/solid";
|
||||
import { useEffect, useMemo, useState } from "react";
|
||||
import { useTypedFetcher } from "remix-typedjson";
|
||||
import { Input } from "~/components/primitives/Input";
|
||||
import { Spinner } from "~/components/primitives/Spinner";
|
||||
import { cn } from "~/utils/cn";
|
||||
import {
|
||||
type WebhookProviderMeta,
|
||||
type WebhookSampleMeta,
|
||||
type loader as samplesLoader,
|
||||
} from "~/routes/resources.orgs.$organizationSlug.projects.$projectParam.env.$envParam.webhooks.samples";
|
||||
|
||||
export function SampleSourcePicker({
|
||||
samplesPath,
|
||||
endpointSource,
|
||||
onLoad,
|
||||
}: {
|
||||
samplesPath: string;
|
||||
/** The endpoint's provider (e.g. "github"); pre-selected in the producer list. */
|
||||
endpointSource?: string;
|
||||
onLoad: (body: string, headers: Record<string, string>) => void;
|
||||
}) {
|
||||
const listFetcher = useTypedFetcher<typeof samplesLoader>();
|
||||
const bodyFetcher = useTypedFetcher<typeof samplesLoader>();
|
||||
const [selectedProvider, setSelectedProvider] = useState<string | null>(null);
|
||||
const [producerQuery, setProducerQuery] = useState("");
|
||||
const [topicQuery, setTopicQuery] = useState("");
|
||||
|
||||
useEffect(() => {
|
||||
if (listFetcher.state === "idle" && listFetcher.data === undefined) {
|
||||
listFetcher.load(samplesPath);
|
||||
}
|
||||
}, [listFetcher, samplesPath]);
|
||||
|
||||
useEffect(() => {
|
||||
const data = bodyFetcher.data;
|
||||
if (data?.kind === "body") {
|
||||
onLoad(data.body, data.extraHeaders ?? {});
|
||||
}
|
||||
}, [bodyFetcher.data, onLoad]);
|
||||
|
||||
const manifest = listFetcher.data?.kind === "manifest" ? listFetcher.data : undefined;
|
||||
const providers = manifest?.providers ?? [];
|
||||
const samples = manifest?.samples ?? [];
|
||||
const listLoading = listFetcher.data === undefined;
|
||||
|
||||
useEffect(() => {
|
||||
if (providers.length === 0) return;
|
||||
setSelectedProvider((current) => {
|
||||
if (current && providers.some((p) => p.id === current)) return current;
|
||||
if (endpointSource && providers.some((p) => p.id === endpointSource)) return endpointSource;
|
||||
return null;
|
||||
});
|
||||
}, [providers, endpointSource]);
|
||||
|
||||
const filteredProviders = useMemo(() => {
|
||||
const query = producerQuery.trim().toLowerCase();
|
||||
if (!query) return providers;
|
||||
return providers.filter(
|
||||
(p) => p.label.toLowerCase().includes(query) || (p.category ?? "").includes(query)
|
||||
);
|
||||
}, [providers, producerQuery]);
|
||||
|
||||
const groupedProviders = useMemo(() => {
|
||||
const groups = new Map<string, WebhookProviderMeta[]>();
|
||||
for (const provider of filteredProviders) {
|
||||
const key = provider.category ?? "other";
|
||||
const group = groups.get(key) ?? [];
|
||||
group.push(provider);
|
||||
groups.set(key, group);
|
||||
}
|
||||
return [...groups.entries()];
|
||||
}, [filteredProviders]);
|
||||
|
||||
const events = samples
|
||||
.filter((item) => item.provider === selectedProvider)
|
||||
.filter((item) => {
|
||||
const query = topicQuery.trim().toLowerCase();
|
||||
return !query || item.eventType.toLowerCase().includes(query);
|
||||
});
|
||||
|
||||
const [loadingEventType, setLoadingEventType] = useState<string | undefined>(undefined);
|
||||
|
||||
function selectEvent(item: WebhookSampleMeta) {
|
||||
setLoadingEventType(item.eventType);
|
||||
const query = new URLSearchParams({ provider: item.provider, eventType: item.eventType });
|
||||
bodyFetcher.load(`${samplesPath}?${query.toString()}`);
|
||||
}
|
||||
|
||||
return (
|
||||
<div className="flex h-full flex-col overflow-hidden bg-charcoal-900">
|
||||
<div className="grid min-h-0 flex-1 grid-cols-[minmax(0,0.45fr)_minmax(0,0.55fr)]">
|
||||
<div className="flex min-h-0 flex-col border-r border-grid-dimmed">
|
||||
<SearchInput
|
||||
placeholder="Webhook producer"
|
||||
value={producerQuery}
|
||||
onChange={setProducerQuery}
|
||||
/>
|
||||
<div className="flex-1 overflow-y-auto scrollbar-thin scrollbar-track-transparent scrollbar-thumb-charcoal-600">
|
||||
{listLoading ? (
|
||||
<div className="flex items-center justify-center py-8">
|
||||
<Spinner className="size-4" />
|
||||
</div>
|
||||
) : filteredProviders.length === 0 ? (
|
||||
<p className="px-3 py-6 text-center text-xs text-text-dimmed">No providers</p>
|
||||
) : (
|
||||
groupedProviders.map(([category, group]) => (
|
||||
<div key={category}>
|
||||
<p className="sticky top-0 z-10 bg-charcoal-900 px-3 pb-1 pt-3 text-xs font-medium text-text-dimmed">
|
||||
{group[0]?.categoryLabel ?? category}
|
||||
</p>
|
||||
{group.map((provider) => (
|
||||
<button
|
||||
key={provider.id}
|
||||
type="button"
|
||||
onClick={() => setSelectedProvider(provider.id)}
|
||||
className={cn(
|
||||
"flex w-full items-center gap-2 px-3 py-2 text-left text-sm transition-colors",
|
||||
provider.id === selectedProvider
|
||||
? "bg-indigo-500/15 text-indigo-400"
|
||||
: "text-text-bright hover:bg-charcoal-800"
|
||||
)}
|
||||
>
|
||||
<span className="flex-1 truncate">{provider.label}</span>
|
||||
<span className="shrink-0 text-xxs tabular-nums text-text-dimmed">
|
||||
{provider.eventCount}
|
||||
</span>
|
||||
</button>
|
||||
))}
|
||||
</div>
|
||||
))
|
||||
)}
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div className="flex min-h-0 flex-col">
|
||||
<SearchInput
|
||||
placeholder="Webhook type or topic"
|
||||
value={topicQuery}
|
||||
onChange={setTopicQuery}
|
||||
/>
|
||||
<div className="flex-1 overflow-y-auto scrollbar-thin scrollbar-track-transparent scrollbar-thumb-charcoal-600">
|
||||
{!selectedProvider ? (
|
||||
<p className="px-3 py-6 text-center text-xs text-text-dimmed">
|
||||
Select a producer to see its events.
|
||||
</p>
|
||||
) : events.length === 0 ? (
|
||||
<p className="px-3 py-6 text-center text-xs text-text-dimmed">No matching events</p>
|
||||
) : (
|
||||
events.map((item) => (
|
||||
<button
|
||||
key={`${item.provider}:${item.eventType}`}
|
||||
type="button"
|
||||
onClick={() => selectEvent(item)}
|
||||
disabled={bodyFetcher.state !== "idle"}
|
||||
className="flex w-full items-center justify-between gap-2 px-3 py-2 text-left transition-colors hover:bg-charcoal-800 disabled:opacity-60"
|
||||
>
|
||||
<span className="truncate font-mono text-sm text-text-bright">
|
||||
{item.eventType}
|
||||
</span>
|
||||
{bodyFetcher.state !== "idle" && loadingEventType === item.eventType ? (
|
||||
<Spinner className="size-3.5 shrink-0" />
|
||||
) : null}
|
||||
</button>
|
||||
))
|
||||
)}
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div className="shrink-0 border-t border-grid-dimmed px-3 py-2 text-xxs text-text-dimmed">
|
||||
Curated event payloads. Signed with this endpoint's config at send time.
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
function SearchInput({
|
||||
placeholder,
|
||||
value,
|
||||
onChange,
|
||||
}: {
|
||||
placeholder: string;
|
||||
value: string;
|
||||
onChange: (value: string) => void;
|
||||
}) {
|
||||
return (
|
||||
<div className="border-b border-grid-dimmed p-2">
|
||||
<Input
|
||||
variant="small"
|
||||
placeholder={placeholder}
|
||||
value={value}
|
||||
icon={MagnifyingGlassIcon}
|
||||
onChange={(event) => onChange(event.target.value)}
|
||||
/>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,595 @@
|
||||
import { PlusIcon, XMarkIcon } from "@heroicons/react/20/solid";
|
||||
import { useFetcher } from "@remix-run/react";
|
||||
import { useCallback, useMemo, useRef, useState } from "react";
|
||||
import { CodeBlock } from "~/components/code/CodeBlock";
|
||||
import { JSONEditor } from "~/components/code/JSONEditor";
|
||||
import { Button, LinkButton } from "~/components/primitives/Buttons";
|
||||
import { Callout } from "~/components/primitives/Callout";
|
||||
import { ClipboardField } from "~/components/primitives/ClipboardField";
|
||||
import { Dialog, DialogContent, DialogHeader, DialogTrigger } from "~/components/primitives/Dialog";
|
||||
import { Hint } from "~/components/primitives/Hint";
|
||||
import { Input } from "~/components/primitives/Input";
|
||||
import { InputGroup } from "~/components/primitives/InputGroup";
|
||||
import { Label } from "~/components/primitives/Label";
|
||||
import {
|
||||
ResizableHandle,
|
||||
ResizablePanel,
|
||||
ResizablePanelGroup,
|
||||
} from "~/components/primitives/Resizable";
|
||||
import { Select, SelectItem } from "~/components/primitives/Select";
|
||||
import { TabButton, TabContainer } from "~/components/primitives/Tabs";
|
||||
import { cn } from "~/utils/cn";
|
||||
import type { WebhookSendResult } from "~/routes/resources.orgs.$organizationSlug.projects.$projectParam.env.$envParam.webhooks.endpoints.$endpointParam.send";
|
||||
import { AIPayloadTabContent } from "~/routes/_app.orgs.$organizationSlug.projects.$projectParam.env.$envParam.test.tasks.$taskParam/AIPayloadTabContent";
|
||||
import { ReplaySourcePicker } from "./ReplaySourcePicker";
|
||||
import { SampleSourcePicker } from "./SampleSourcePicker";
|
||||
|
||||
type SourceTab = "body" | "sample" | "replay" | "ai";
|
||||
|
||||
export type WebhookComposerEndpoint = {
|
||||
friendlyId: string;
|
||||
label: string;
|
||||
source: string;
|
||||
ingressUrl: string;
|
||||
scheme: "hmac" | "shared-secret" | "url-secret" | "asymmetric";
|
||||
hasSigningSecret: boolean;
|
||||
handshake: { matchPath: string; matchValue: string; respondPath: string } | null;
|
||||
};
|
||||
|
||||
export type WebhookComposerProps = {
|
||||
endpoints: WebhookComposerEndpoint[];
|
||||
organizationSlug: string;
|
||||
projectSlug: string;
|
||||
environmentSlug: string;
|
||||
isDevEnvironment: boolean;
|
||||
environmentLabel: string;
|
||||
defaultBody?: string;
|
||||
/** When false, a successful send stays put and shows the inline result strip instead of
|
||||
* redirecting to the delivery detail page (the console tab keeps its live feed alongside). */
|
||||
redirectOnSuccess?: boolean;
|
||||
};
|
||||
|
||||
type SignatureMode = "signed" | "unsigned" | "tampered" | "simulate";
|
||||
|
||||
type HeaderRow = { id: string; key: string; value: string };
|
||||
|
||||
const DEFAULT_BODY = JSON.stringify({ message: "hello from the webhook console" }, null, 2);
|
||||
|
||||
export function WebhookComposer({
|
||||
endpoints,
|
||||
organizationSlug,
|
||||
projectSlug,
|
||||
environmentSlug,
|
||||
isDevEnvironment,
|
||||
environmentLabel,
|
||||
defaultBody,
|
||||
redirectOnSuccess = true,
|
||||
}: WebhookComposerProps) {
|
||||
const fetcher = useFetcher<WebhookSendResult>();
|
||||
const isSending = fetcher.state !== "idle";
|
||||
|
||||
const [endpointId, setEndpointId] = useState(endpoints[0]?.friendlyId ?? "");
|
||||
const [sourceTab, setSourceTab] = useState<SourceTab>("body");
|
||||
const [bodyDefault, setBodyDefault] = useState(defaultBody ?? DEFAULT_BODY);
|
||||
const bodyRef = useRef(bodyDefault);
|
||||
const [payloadReloadKey, setPayloadReloadKey] = useState(0);
|
||||
const [headerRows, setHeaderRows] = useState<HeaderRow[]>([]);
|
||||
const headerIdRef = useRef(0);
|
||||
|
||||
const newHeaderRow = useCallback(
|
||||
(key = "", value = ""): HeaderRow => ({ id: String(headerIdRef.current++), key, value }),
|
||||
[]
|
||||
);
|
||||
|
||||
const endpoint = useMemo(
|
||||
() => endpoints.find((e) => e.friendlyId === endpointId) ?? endpoints[0],
|
||||
[endpoints, endpointId]
|
||||
);
|
||||
|
||||
const applyPayload = useCallback(
|
||||
(body: string, headers: Record<string, string>) => {
|
||||
setBodyDefault(body);
|
||||
bodyRef.current = body;
|
||||
setPayloadReloadKey((key) => key + 1);
|
||||
const entries = Object.entries(headers);
|
||||
setHeaderRows(entries.map(([key, value]) => newHeaderRow(key, value)));
|
||||
setSourceTab("body");
|
||||
},
|
||||
[newHeaderRow]
|
||||
);
|
||||
|
||||
const signedAvailable = Boolean(
|
||||
endpoint && endpoint.scheme !== "asymmetric" && endpoint.hasSigningSecret
|
||||
);
|
||||
const signedDisabledReason = !endpoint
|
||||
? undefined
|
||||
: endpoint.scheme === "asymmetric"
|
||||
? "This endpoint uses asymmetric signatures, which cannot be produced here."
|
||||
: !endpoint.hasSigningSecret
|
||||
? "Set a signing secret on this endpoint first."
|
||||
: undefined;
|
||||
|
||||
const [signatureMode, setSignatureMode] = useState<SignatureMode>(
|
||||
signedAvailable ? "signed" : "simulate"
|
||||
);
|
||||
|
||||
const endpointBasePath = endpoint
|
||||
? `/resources/orgs/${organizationSlug}/projects/${projectSlug}/env/${environmentSlug}/webhooks/endpoints/${endpoint.friendlyId}`
|
||||
: "";
|
||||
const sendPath = endpointBasePath ? `${endpointBasePath}/send` : "";
|
||||
const replaySourcePath = endpointBasePath ? `${endpointBasePath}/replay-source` : "";
|
||||
const samplesPath = `/resources/orgs/${organizationSlug}/projects/${projectSlug}/env/${environmentSlug}/webhooks/samples`;
|
||||
|
||||
function submit(override?: {
|
||||
body?: string;
|
||||
signatureMode?: SignatureMode;
|
||||
headers?: Record<string, string>;
|
||||
}) {
|
||||
if (!endpoint) return;
|
||||
let headers = override?.headers;
|
||||
if (!headers) {
|
||||
headers = {};
|
||||
for (const row of headerRows) {
|
||||
const key = row.key.trim();
|
||||
if (key) headers[key] = row.value;
|
||||
}
|
||||
}
|
||||
fetcher.submit(
|
||||
{
|
||||
body: override?.body ?? bodyRef.current,
|
||||
headers,
|
||||
signatureMode: override?.signatureMode ?? signatureMode,
|
||||
redirect: redirectOnSuccess,
|
||||
},
|
||||
{ method: "post", action: sendPath, encType: "application/json" }
|
||||
);
|
||||
}
|
||||
|
||||
function sendHandshake() {
|
||||
if (!endpoint?.handshake) return;
|
||||
const challenge = `chal_${Math.random().toString(36).slice(2, 10)}`;
|
||||
const body = JSON.stringify(buildHandshakeBody(endpoint.handshake, challenge), null, 2);
|
||||
applyPayload(body, {});
|
||||
submit({ body, signatureMode: "signed", headers: {} });
|
||||
}
|
||||
|
||||
const result = fetcher.data;
|
||||
const deliveryPath =
|
||||
result?.success && result.deliveryId?.startsWith("whd_")
|
||||
? `/orgs/${organizationSlug}/projects/${projectSlug}/env/${environmentSlug}/webhooks/deliveries/${result.deliveryId}`
|
||||
: undefined;
|
||||
|
||||
return (
|
||||
<div className="flex h-full max-h-full flex-col">
|
||||
<ResizablePanelGroup orientation="horizontal" className="grow">
|
||||
<ResizablePanel id="webhook-composer-editor" min="360px">
|
||||
<div className="flex h-full flex-col overflow-hidden bg-charcoal-900">
|
||||
<div className="flex h-9 items-center border-b border-grid-dimmed bg-background-bright px-3">
|
||||
<TabContainer className="-mb-px">
|
||||
<TabButton
|
||||
isActive={sourceTab === "body"}
|
||||
layoutId="webhook-composer-source"
|
||||
onClick={() => setSourceTab("body")}
|
||||
>
|
||||
Body
|
||||
</TabButton>
|
||||
<TabButton
|
||||
isActive={sourceTab === "sample"}
|
||||
layoutId="webhook-composer-source"
|
||||
onClick={() => setSourceTab("sample")}
|
||||
>
|
||||
Library
|
||||
</TabButton>
|
||||
<TabButton
|
||||
isActive={sourceTab === "replay"}
|
||||
layoutId="webhook-composer-source"
|
||||
onClick={() => setSourceTab("replay")}
|
||||
>
|
||||
Replay
|
||||
</TabButton>
|
||||
<TabButton
|
||||
isActive={sourceTab === "ai"}
|
||||
layoutId="webhook-composer-source"
|
||||
onClick={() => setSourceTab("ai")}
|
||||
>
|
||||
AI
|
||||
</TabButton>
|
||||
</TabContainer>
|
||||
</div>
|
||||
<div className="relative flex-1 overflow-hidden">
|
||||
<div className={cn("h-full", sourceTab !== "body" && "hidden")}>
|
||||
<JSONEditor
|
||||
key={payloadReloadKey}
|
||||
defaultValue={bodyDefault}
|
||||
readOnly={false}
|
||||
basicSetup
|
||||
autoFocus
|
||||
onChange={(v) => {
|
||||
bodyRef.current = v;
|
||||
}}
|
||||
height="100%"
|
||||
className="h-full overflow-auto"
|
||||
showClearButton={false}
|
||||
additionalActions={
|
||||
<span className="text-xs font-medium text-text-dimmed">Event body</span>
|
||||
}
|
||||
/>
|
||||
</div>
|
||||
{sourceTab === "sample" ? (
|
||||
<div className="absolute inset-0">
|
||||
<SampleSourcePicker
|
||||
samplesPath={samplesPath}
|
||||
endpointSource={endpoint?.source}
|
||||
onLoad={applyPayload}
|
||||
/>
|
||||
</div>
|
||||
) : null}
|
||||
{sourceTab === "replay" ? (
|
||||
<div className="absolute inset-0">
|
||||
<ReplaySourcePicker replaySourcePath={replaySourcePath} onLoad={applyPayload} />
|
||||
</div>
|
||||
) : null}
|
||||
{sourceTab === "ai" ? (
|
||||
<div className="absolute inset-0 overflow-y-auto bg-charcoal-900 p-3 scrollbar-thin scrollbar-track-transparent scrollbar-thumb-charcoal-600">
|
||||
<AIPayloadTabContent
|
||||
onPayloadGenerated={(payload) => applyPayload(payload, {})}
|
||||
taskIdentifier={endpoint?.source ?? "webhook"}
|
||||
payloadKind="webhook"
|
||||
providerSource={endpoint?.source}
|
||||
generateButtonLabel="Generate event"
|
||||
placeholder="e.g. a payment succeeded event with a $42.00 charge"
|
||||
/>
|
||||
</div>
|
||||
) : null}
|
||||
</div>
|
||||
{result ? (
|
||||
<div className="max-h-72 shrink-0 overflow-y-auto border-t border-grid-dimmed bg-background-dimmed p-3">
|
||||
<ResultStrip result={result} deliveryPath={deliveryPath} />
|
||||
</div>
|
||||
) : null}
|
||||
</div>
|
||||
</ResizablePanel>
|
||||
|
||||
<ResizableHandle id="webhook-composer-handle" />
|
||||
|
||||
<ResizablePanel
|
||||
id="webhook-composer-options"
|
||||
min="280px"
|
||||
default="360px"
|
||||
max="480px"
|
||||
isStaticAtRest
|
||||
>
|
||||
<div className="flex h-full flex-col gap-4 overflow-y-auto bg-background-bright p-3 scrollbar-thin scrollbar-track-transparent scrollbar-thumb-charcoal-600">
|
||||
{!isDevEnvironment ? (
|
||||
<Callout variant="warning">
|
||||
Sends a real delivery through the {environmentLabel} endpoint and triggers a real
|
||||
run.
|
||||
</Callout>
|
||||
) : null}
|
||||
|
||||
{endpoints.length > 1 ? (
|
||||
<InputGroup>
|
||||
<Label variant="small">Endpoint</Label>
|
||||
<Select
|
||||
variant="tertiary/small"
|
||||
dropdownIcon
|
||||
value={endpointId}
|
||||
setValue={(v) => {
|
||||
if (Array.isArray(v)) return;
|
||||
setEndpointId(v);
|
||||
}}
|
||||
items={endpoints.map((e) => e.friendlyId)}
|
||||
>
|
||||
{endpoints.map((e) => (
|
||||
<SelectItem key={e.friendlyId} value={e.friendlyId}>
|
||||
{e.label}
|
||||
</SelectItem>
|
||||
))}
|
||||
</Select>
|
||||
</InputGroup>
|
||||
) : null}
|
||||
|
||||
<InputGroup>
|
||||
<Label variant="small">Signature</Label>
|
||||
<Select
|
||||
variant="tertiary/small"
|
||||
dropdownIcon
|
||||
value={signatureMode}
|
||||
setValue={(v) => {
|
||||
if (Array.isArray(v)) return;
|
||||
setSignatureMode(v as SignatureMode);
|
||||
}}
|
||||
items={["signed", "simulate", "unsigned", "tampered"]}
|
||||
>
|
||||
<SelectItem value="signed" disabled={!signedAvailable}>
|
||||
Signed (valid)
|
||||
</SelectItem>
|
||||
<SelectItem value="simulate">Simulate (skip verification)</SelectItem>
|
||||
<SelectItem value="unsigned">Unsigned (expect 400)</SelectItem>
|
||||
<SelectItem value="tampered">Tampered (expect 400)</SelectItem>
|
||||
</Select>
|
||||
{signatureMode === "signed" && signedDisabledReason ? (
|
||||
<Hint>{signedDisabledReason}</Hint>
|
||||
) : signatureMode === "signed" ? (
|
||||
<Hint>Signed server-side with the endpoint's stored secret.</Hint>
|
||||
) : signatureMode === "simulate" ? (
|
||||
<Hint>
|
||||
Injected via the engine, skipping signature verification. Filter, startOn,
|
||||
routing, and the run all still execute.
|
||||
</Hint>
|
||||
) : (
|
||||
<Hint>The delivery is rejected fail-closed; no delivery row is written.</Hint>
|
||||
)}
|
||||
</InputGroup>
|
||||
|
||||
<InputGroup>
|
||||
<Label variant="small">Headers</Label>
|
||||
<HeadersEditor
|
||||
rows={headerRows}
|
||||
onChange={setHeaderRows}
|
||||
onAdd={() => setHeaderRows((rows) => [...rows, newHeaderRow()])}
|
||||
/>
|
||||
<Hint>
|
||||
Optional provider routing headers (e.g. x-github-event). The signature header is
|
||||
added automatically.
|
||||
</Hint>
|
||||
</InputGroup>
|
||||
|
||||
{endpoint ? (
|
||||
<InputGroup>
|
||||
<Label variant="small">Webhook URL</Label>
|
||||
<ClipboardField
|
||||
value={endpoint.ingressUrl}
|
||||
variant="secondary/small"
|
||||
className="font-mono"
|
||||
icon={
|
||||
<span className="pl-1 font-mono text-xxs font-semibold uppercase text-text-dimmed">
|
||||
POST
|
||||
</span>
|
||||
}
|
||||
/>
|
||||
<Hint>
|
||||
The public URL providers POST to. Test sends run the same pipeline in-process.
|
||||
</Hint>
|
||||
</InputGroup>
|
||||
) : null}
|
||||
</div>
|
||||
</ResizablePanel>
|
||||
</ResizablePanelGroup>
|
||||
|
||||
<div className="flex items-center justify-between gap-3 border-t border-grid-bright bg-background-dimmed p-2">
|
||||
<span className="text-xs text-text-dimmed">
|
||||
{isDevEnvironment
|
||||
? "Signs with the endpoint's secret and runs the full delivery pipeline."
|
||||
: `Sends through the ${environmentLabel} endpoint.`}
|
||||
</span>
|
||||
<div className="flex items-center gap-2">
|
||||
{endpoint?.handshake && signedAvailable ? (
|
||||
<Button
|
||||
variant="tertiary/small"
|
||||
onClick={() => sendHandshake()}
|
||||
disabled={isSending || !endpoint}
|
||||
tooltip="Send a signed handshake and assert the endpoint echoes the challenge"
|
||||
>
|
||||
Send handshake
|
||||
</Button>
|
||||
) : null}
|
||||
{isDevEnvironment ? (
|
||||
<Button
|
||||
variant="primary/small"
|
||||
onClick={() => submit()}
|
||||
disabled={isSending || !endpoint}
|
||||
>
|
||||
{isSending ? "Sending…" : "Send event"}
|
||||
</Button>
|
||||
) : (
|
||||
<ConfirmSendDialog
|
||||
environmentLabel={environmentLabel}
|
||||
disabled={isSending || !endpoint}
|
||||
isSending={isSending}
|
||||
onConfirm={() => submit()}
|
||||
/>
|
||||
)}
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
function HeadersEditor({
|
||||
rows,
|
||||
onChange,
|
||||
onAdd,
|
||||
}: {
|
||||
rows: HeaderRow[];
|
||||
onChange: (rows: HeaderRow[]) => void;
|
||||
onAdd: () => void;
|
||||
}) {
|
||||
return (
|
||||
<div className="flex flex-col gap-1.5">
|
||||
{rows.map((row) => (
|
||||
<div key={row.id} className="flex items-center gap-1.5">
|
||||
<div className="min-w-0 flex-1">
|
||||
<Input
|
||||
variant="small"
|
||||
placeholder="Header"
|
||||
value={row.key}
|
||||
spellCheck={false}
|
||||
className="font-mono"
|
||||
onChange={(event) =>
|
||||
onChange(rows.map((r) => (r.id === row.id ? { ...r, key: event.target.value } : r)))
|
||||
}
|
||||
/>
|
||||
</div>
|
||||
<div className="min-w-0 flex-1">
|
||||
<Input
|
||||
variant="small"
|
||||
placeholder="Value"
|
||||
value={row.value}
|
||||
spellCheck={false}
|
||||
className="font-mono"
|
||||
onChange={(event) =>
|
||||
onChange(
|
||||
rows.map((r) => (r.id === row.id ? { ...r, value: event.target.value } : r))
|
||||
)
|
||||
}
|
||||
/>
|
||||
</div>
|
||||
<button
|
||||
type="button"
|
||||
aria-label="Remove header"
|
||||
onClick={() => onChange(rows.filter((r) => r.id !== row.id))}
|
||||
className="shrink-0 rounded p-1 text-text-dimmed transition-colors hover:bg-charcoal-700 hover:text-text-bright"
|
||||
>
|
||||
<XMarkIcon className="size-3.5" />
|
||||
</button>
|
||||
</div>
|
||||
))}
|
||||
<button
|
||||
type="button"
|
||||
onClick={onAdd}
|
||||
className="flex w-fit items-center gap-1 rounded py-0.5 pr-1.5 text-xs text-text-dimmed transition-colors hover:text-text-bright"
|
||||
>
|
||||
<PlusIcon className="size-3.5" />
|
||||
Add header
|
||||
</button>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
const UNSAFE_PATH_KEYS = new Set(["__proto__", "constructor", "prototype"]);
|
||||
|
||||
function setPath(target: Record<string, unknown>, path: string, value: unknown) {
|
||||
const parts = path.split(".");
|
||||
if (parts.some((part) => UNSAFE_PATH_KEYS.has(part))) {
|
||||
return;
|
||||
}
|
||||
let cursor = target;
|
||||
for (let i = 0; i < parts.length - 1; i++) {
|
||||
const key = parts[i];
|
||||
const next = cursor[key];
|
||||
if (typeof next !== "object" || next === null) {
|
||||
cursor[key] = {};
|
||||
}
|
||||
cursor = cursor[key] as Record<string, unknown>;
|
||||
}
|
||||
cursor[parts[parts.length - 1]] = value;
|
||||
}
|
||||
|
||||
function buildHandshakeBody(
|
||||
handshake: { matchPath: string; matchValue: string; respondPath: string },
|
||||
challenge: string
|
||||
): Record<string, unknown> {
|
||||
const body: Record<string, unknown> = {};
|
||||
setPath(body, handshake.matchPath, handshake.matchValue);
|
||||
setPath(body, handshake.respondPath, challenge);
|
||||
return body;
|
||||
}
|
||||
|
||||
function ConfirmSendDialog({
|
||||
environmentLabel,
|
||||
disabled,
|
||||
isSending,
|
||||
onConfirm,
|
||||
}: {
|
||||
environmentLabel: string;
|
||||
disabled: boolean;
|
||||
isSending: boolean;
|
||||
onConfirm: () => void;
|
||||
}) {
|
||||
const [open, setOpen] = useState(false);
|
||||
return (
|
||||
<Dialog open={open} onOpenChange={setOpen}>
|
||||
<DialogTrigger asChild>
|
||||
<Button variant="primary/small" disabled={disabled}>
|
||||
{isSending ? "Sending…" : "Send event"}
|
||||
</Button>
|
||||
</DialogTrigger>
|
||||
<DialogContent>
|
||||
<DialogHeader>Send to {environmentLabel}?</DialogHeader>
|
||||
<div className="flex flex-col gap-3 pt-2">
|
||||
<p className="text-sm text-text-dimmed">
|
||||
This delivers a real event to a non-development endpoint and triggers a real run.
|
||||
</p>
|
||||
<div className="flex justify-end gap-2">
|
||||
<Button type="button" variant="tertiary/small" onClick={() => setOpen(false)}>
|
||||
Cancel
|
||||
</Button>
|
||||
<Button
|
||||
type="button"
|
||||
variant="primary/small"
|
||||
onClick={() => {
|
||||
setOpen(false);
|
||||
onConfirm();
|
||||
}}
|
||||
>
|
||||
Send to {environmentLabel}
|
||||
</Button>
|
||||
</div>
|
||||
</div>
|
||||
</DialogContent>
|
||||
</Dialog>
|
||||
);
|
||||
}
|
||||
|
||||
function ResultStrip({
|
||||
result,
|
||||
deliveryPath,
|
||||
}: {
|
||||
result: WebhookSendResult;
|
||||
deliveryPath?: string;
|
||||
}) {
|
||||
const status = result.success ? result.httpStatus : undefined;
|
||||
const handshake = result.success && result.handshake;
|
||||
const deduplicated = result.success && result.deduplicated;
|
||||
const ok = result.success && status === 200 && !deduplicated;
|
||||
return (
|
||||
<div className="flex flex-col gap-2">
|
||||
<div className="flex items-center gap-2">
|
||||
<span
|
||||
className={cn(
|
||||
"rounded px-1.5 py-0.5 text-xs font-medium",
|
||||
handshake
|
||||
? "bg-blue-500/20 text-blue-400"
|
||||
: deduplicated
|
||||
? "bg-amber-500/20 text-amber-400"
|
||||
: ok
|
||||
? "bg-success/20 text-success"
|
||||
: "bg-error/20 text-error"
|
||||
)}
|
||||
>
|
||||
{handshake
|
||||
? "Handshake"
|
||||
: deduplicated
|
||||
? "Deduplicated"
|
||||
: result.success
|
||||
? `HTTP ${result.httpStatus}`
|
||||
: "Failed"}
|
||||
</span>
|
||||
{result.success && result.deliveryId ? (
|
||||
<span className="font-mono text-xs text-text-dimmed">{result.deliveryId}</span>
|
||||
) : null}
|
||||
{deliveryPath ? (
|
||||
<LinkButton variant="minimal/small" to={deliveryPath} className="ml-auto">
|
||||
{deduplicated ? "View original →" : "View delivery →"}
|
||||
</LinkButton>
|
||||
) : null}
|
||||
</div>
|
||||
{handshake ? (
|
||||
<Hint>
|
||||
Challenge echoed by the endpoint. Handshakes are answered inline; no delivery is recorded.
|
||||
</Hint>
|
||||
) : deduplicated ? (
|
||||
<Hint>
|
||||
Identical payload was deduplicated to the original delivery. Vary it to send a new one.
|
||||
</Hint>
|
||||
) : null}
|
||||
<CodeBlock
|
||||
code={result.success ? result.responseBody : result.error}
|
||||
language="json"
|
||||
showLineNumbers={false}
|
||||
maxLines={8}
|
||||
/>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,268 @@
|
||||
import { ArrowRightIcon } from "@heroicons/react/20/solid";
|
||||
import { useLocation, useNavigation } from "@remix-run/react";
|
||||
import { AIChatIcon } from "~/assets/icons/AIChatIcon";
|
||||
import { RunsIcon } from "~/assets/icons/RunsIcon";
|
||||
import { WebhookIcon } from "~/assets/icons/WebhookIcon";
|
||||
import { Badge } from "~/components/primitives/Badge";
|
||||
import { DateTime } from "~/components/primitives/DateTime";
|
||||
import { MiddleTruncate } from "~/components/primitives/MiddleTruncate";
|
||||
import { Paragraph } from "~/components/primitives/Paragraph";
|
||||
import { PopoverMenuItem } from "~/components/primitives/Popover";
|
||||
import { Spinner } from "~/components/primitives/Spinner";
|
||||
import {
|
||||
Table,
|
||||
TableBlankRow,
|
||||
TableBody,
|
||||
TableCell,
|
||||
TableCellMenu,
|
||||
TableHeader,
|
||||
TableHeaderCell,
|
||||
TableRow,
|
||||
} from "~/components/primitives/Table";
|
||||
import { SimpleTooltip } from "~/components/primitives/Tooltip";
|
||||
import { useEnvironment } from "~/hooks/useEnvironment";
|
||||
import { useOrganization } from "~/hooks/useOrganizations";
|
||||
import { useProject } from "~/hooks/useProject";
|
||||
import { type WebhookDeliveryListItem } from "~/presenters/v3/WebhookDetailPresenter.server";
|
||||
import {
|
||||
v3RunPath,
|
||||
v3SessionPath,
|
||||
v3WebhookDeliveryPath,
|
||||
v3WebhookTaskPath,
|
||||
} from "~/utils/pathBuilder";
|
||||
import { cn } from "~/utils/cn";
|
||||
import { DeliveryStatusBadge } from "./DeliveryStatus";
|
||||
|
||||
export function DeliveriesTable({
|
||||
deliveries,
|
||||
hasFilters,
|
||||
showTopBorder = true,
|
||||
stickyHeader = false,
|
||||
showWebhook = false,
|
||||
}: {
|
||||
deliveries: WebhookDeliveryListItem[];
|
||||
hasFilters?: boolean;
|
||||
showTopBorder?: boolean;
|
||||
stickyHeader?: boolean;
|
||||
// The top-level (cross-endpoint) deliveries page shows which webhook each delivery
|
||||
// belongs to; the per-webhook detail page leaves this off.
|
||||
showWebhook?: boolean;
|
||||
}) {
|
||||
const navigation = useNavigation();
|
||||
const location = useLocation();
|
||||
const isLoading =
|
||||
navigation.state !== "idle" && navigation.location?.pathname === location.pathname;
|
||||
|
||||
const organization = useOrganization();
|
||||
const project = useProject();
|
||||
const environment = useEnvironment();
|
||||
|
||||
return (
|
||||
<Table
|
||||
className="max-h-full overflow-y-auto"
|
||||
showTopBorder={showTopBorder}
|
||||
stickyHeader={stickyHeader}
|
||||
>
|
||||
<TableHeader>
|
||||
<TableRow>
|
||||
{showWebhook && <TableHeaderCell>Webhook</TableHeaderCell>}
|
||||
<TableHeaderCell>Delivery</TableHeaderCell>
|
||||
<TableHeaderCell>Status</TableHeaderCell>
|
||||
<TableHeaderCell>External delivery ID</TableHeaderCell>
|
||||
<TableHeaderCell>Target</TableHeaderCell>
|
||||
<TableHeaderCell>Created</TableHeaderCell>
|
||||
<TableHeaderCell>Processed</TableHeaderCell>
|
||||
<TableHeaderCell>Error</TableHeaderCell>
|
||||
<TableHeaderCell>
|
||||
<span className="sr-only">Actions</span>
|
||||
</TableHeaderCell>
|
||||
</TableRow>
|
||||
</TableHeader>
|
||||
<TableBody>
|
||||
{deliveries.length === 0 ? (
|
||||
<TableBlankRow colSpan={showWebhook ? 9 : 8}>
|
||||
<div className="flex items-center justify-center">
|
||||
<Paragraph className="w-auto">
|
||||
{hasFilters
|
||||
? "No deliveries match these filters"
|
||||
: "No deliveries for this webhook yet"}
|
||||
</Paragraph>
|
||||
</div>
|
||||
</TableBlankRow>
|
||||
) : (
|
||||
deliveries.map((delivery) => {
|
||||
const runPath = delivery.run
|
||||
? v3RunPath(organization, project, environment, {
|
||||
friendlyId: delivery.run.friendlyId,
|
||||
})
|
||||
: undefined;
|
||||
|
||||
// Session deliveries route to a session (their run is just its current turn), so the
|
||||
// session is the meaningful target; task deliveries fall back to the run.
|
||||
const sessionPath = delivery.session
|
||||
? v3SessionPath(organization, project, environment, {
|
||||
friendlyId: delivery.session.friendlyId,
|
||||
})
|
||||
: undefined;
|
||||
|
||||
const webhookPath = delivery.webhook
|
||||
? v3WebhookTaskPath(organization, project, environment, delivery.webhook.slug)
|
||||
: undefined;
|
||||
|
||||
const deliveryPath = v3WebhookDeliveryPath(
|
||||
organization,
|
||||
project,
|
||||
environment,
|
||||
delivery.friendlyId
|
||||
);
|
||||
|
||||
return (
|
||||
<TableRow key={delivery.id}>
|
||||
{showWebhook && (
|
||||
<TableCell to={webhookPath}>
|
||||
{delivery.webhook ? (
|
||||
<span className="flex items-center gap-x-1">
|
||||
<WebhookIcon className="size-4.5 min-w-4.5 text-webhooks" />
|
||||
{delivery.webhook.slug}
|
||||
</span>
|
||||
) : (
|
||||
<span className="text-text-dimmed group-hover/table-row:text-text-bright">
|
||||
Unknown
|
||||
</span>
|
||||
)}
|
||||
</TableCell>
|
||||
)}
|
||||
<TableCell to={deliveryPath}>
|
||||
<span className="flex items-center gap-1.5">
|
||||
<span className="font-mono text-xs">{delivery.friendlyId}</span>
|
||||
{delivery.isTest ? <Badge variant="extra-small">Test</Badge> : null}
|
||||
</span>
|
||||
</TableCell>
|
||||
<TableCell to={deliveryPath}>
|
||||
<DeliveryStatusBadge status={delivery.status} />
|
||||
</TableCell>
|
||||
<TableCell to={deliveryPath}>
|
||||
{delivery.externalDeliveryId ? (
|
||||
<div className="w-[24ch]">
|
||||
<MiddleTruncate
|
||||
text={delivery.externalDeliveryId}
|
||||
className="font-mono text-xs"
|
||||
/>
|
||||
</div>
|
||||
) : (
|
||||
<span className="text-text-dimmed group-hover/table-row:text-text-bright">
|
||||
None
|
||||
</span>
|
||||
)}
|
||||
</TableCell>
|
||||
{/* Falls back to the delivery so the whole row stays clickable when there is no target */}
|
||||
<TableCell to={sessionPath ?? runPath ?? deliveryPath}>
|
||||
{delivery.session ? (
|
||||
<span className="flex items-center gap-x-1">
|
||||
<AIChatIcon className="size-4 text-sessions" />
|
||||
<span className="font-mono text-xs">{delivery.session.friendlyId}</span>
|
||||
</span>
|
||||
) : delivery.run ? (
|
||||
<span className="flex items-center gap-x-1">
|
||||
<RunsIcon className="size-4 text-runs" />
|
||||
<span className="font-mono text-xs">{delivery.run.friendlyId}</span>
|
||||
</span>
|
||||
) : (
|
||||
<span className="text-text-dimmed group-hover/table-row:text-text-bright">
|
||||
None
|
||||
</span>
|
||||
)}
|
||||
</TableCell>
|
||||
<TableCell to={deliveryPath}>
|
||||
<DateTime date={delivery.createdAt} />
|
||||
</TableCell>
|
||||
<TableCell to={deliveryPath}>
|
||||
{delivery.processedAt ? (
|
||||
<DateTime date={delivery.processedAt} />
|
||||
) : (
|
||||
<span className="text-text-dimmed group-hover/table-row:text-text-bright">
|
||||
None
|
||||
</span>
|
||||
)}
|
||||
</TableCell>
|
||||
<TableCell to={deliveryPath}>
|
||||
{delivery.status === "FAILED" && delivery.errorMessage ? (
|
||||
<SimpleTooltip
|
||||
content={delivery.errorMessage}
|
||||
button={
|
||||
<span className="block max-w-[32ch] truncate text-xs text-error">
|
||||
{delivery.errorMessage}
|
||||
</span>
|
||||
}
|
||||
/>
|
||||
) : (
|
||||
<span className="text-text-dimmed group-hover/table-row:text-text-bright">
|
||||
None
|
||||
</span>
|
||||
)}
|
||||
</TableCell>
|
||||
<DeliveryActionsCell
|
||||
deliveryPath={deliveryPath}
|
||||
runPath={runPath}
|
||||
sessionPath={sessionPath}
|
||||
/>
|
||||
</TableRow>
|
||||
);
|
||||
})
|
||||
)}
|
||||
{isLoading && (
|
||||
<TableBlankRow
|
||||
colSpan={showWebhook ? 9 : 8}
|
||||
className={cn(
|
||||
"absolute left-0 top-0 flex h-full w-full items-center justify-center gap-2 bg-charcoal-900/90"
|
||||
)}
|
||||
>
|
||||
<Spinner /> <span className="text-text-dimmed">Loading…</span>
|
||||
</TableBlankRow>
|
||||
)}
|
||||
</TableBody>
|
||||
</Table>
|
||||
);
|
||||
}
|
||||
|
||||
function DeliveryActionsCell({
|
||||
deliveryPath,
|
||||
runPath,
|
||||
sessionPath,
|
||||
}: {
|
||||
deliveryPath: string;
|
||||
runPath?: string;
|
||||
sessionPath?: string;
|
||||
}) {
|
||||
return (
|
||||
<TableCellMenu
|
||||
isSticky
|
||||
popoverContent={
|
||||
<>
|
||||
<PopoverMenuItem
|
||||
to={deliveryPath}
|
||||
icon={ArrowRightIcon}
|
||||
leadingIconClassName="text-webhooks"
|
||||
title="View delivery"
|
||||
/>
|
||||
{sessionPath ? (
|
||||
<PopoverMenuItem
|
||||
to={sessionPath}
|
||||
icon={ArrowRightIcon}
|
||||
leadingIconClassName="text-runs"
|
||||
title="View session"
|
||||
/>
|
||||
) : null}
|
||||
{runPath ? (
|
||||
<PopoverMenuItem
|
||||
to={runPath}
|
||||
icon={ArrowRightIcon}
|
||||
leadingIconClassName="text-runs"
|
||||
title="View run"
|
||||
/>
|
||||
) : null}
|
||||
</>
|
||||
}
|
||||
/>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,38 @@
|
||||
import { type WebhookDeliveryStatus } from "@trigger.dev/database";
|
||||
import { cn } from "~/utils/cn";
|
||||
|
||||
// Reuse the run-status hex palette for the four delivery statuses (matches the
|
||||
// detail page activity chart and the task-list status bars). No invented colors.
|
||||
export const DELIVERY_STATUS_COLOR: Record<WebhookDeliveryStatus, string> = {
|
||||
SUCCEEDED: "#28BF5C",
|
||||
FAILED: "#E11D48",
|
||||
PROCESSING: "#3B82F6",
|
||||
PENDING: "#878C99",
|
||||
FILTERED: "#64748B", // received + verified, intentionally not routed; neutral, not a failure
|
||||
};
|
||||
|
||||
export const DELIVERY_STATUS_LABEL: Record<WebhookDeliveryStatus, string> = {
|
||||
SUCCEEDED: "Succeeded",
|
||||
FAILED: "Failed",
|
||||
PROCESSING: "Processing",
|
||||
PENDING: "Pending",
|
||||
FILTERED: "Filtered",
|
||||
};
|
||||
|
||||
export function DeliveryStatusBadge({
|
||||
status,
|
||||
className,
|
||||
}: {
|
||||
status: WebhookDeliveryStatus;
|
||||
className?: string;
|
||||
}) {
|
||||
return (
|
||||
<span className={cn("flex items-center gap-1.5", className)}>
|
||||
<span
|
||||
className="size-2 shrink-0 rounded-full"
|
||||
style={{ backgroundColor: DELIVERY_STATUS_COLOR[status] }}
|
||||
/>
|
||||
<span>{DELIVERY_STATUS_LABEL[status]}</span>
|
||||
</span>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,114 @@
|
||||
import { formatDuration } from "@trigger.dev/core/v3/utils/durations";
|
||||
import { Fragment } from "react";
|
||||
import { AIChatIcon } from "~/assets/icons/AIChatIcon";
|
||||
import { RunsIcon } from "~/assets/icons/RunsIcon";
|
||||
import { DateTimeAccurate } from "~/components/primitives/DateTime";
|
||||
import { TextLink } from "~/components/primitives/TextLink";
|
||||
import { RunTimelineEvent, RunTimelineLine } from "~/components/run/RunTimeline";
|
||||
import { LiveTimer } from "~/components/runs/v3/LiveTimer";
|
||||
import {
|
||||
buildDeliveryTimelineItems,
|
||||
type BuildDeliveryTimelineInput,
|
||||
type DeliveryTimelineEventItem,
|
||||
} from "./buildDeliveryTimelineItems";
|
||||
|
||||
type DeliveryTimelineProps = {
|
||||
delivery: BuildDeliveryTimelineInput;
|
||||
runPath?: string;
|
||||
sessionPath?: string;
|
||||
};
|
||||
|
||||
export function DeliveryTimeline({ delivery, runPath, sessionPath }: DeliveryTimelineProps) {
|
||||
const items = buildDeliveryTimelineItems(delivery);
|
||||
|
||||
return (
|
||||
<div className="mb-4 min-w-fit max-w-full border-b border-grid-dimmed pb-4">
|
||||
{items.map((item) => {
|
||||
if (item.type === "line") {
|
||||
return (
|
||||
<RunTimelineLine
|
||||
key={item.id}
|
||||
state={item.state}
|
||||
variant={item.variant}
|
||||
// "Received" is a thin start-cap, so a thick line here begins the bar and has to
|
||||
// round its own top. Thin ("light") lines, as in the FILTERED case, need nothing.
|
||||
roundedTop={item.variant === "normal"}
|
||||
title={
|
||||
<span className="flex items-center gap-1.5">
|
||||
{item.to ? (
|
||||
formatDuration(item.from, item.to)
|
||||
) : (
|
||||
<LiveTimer startTime={item.from} />
|
||||
)}
|
||||
{item.label ? (
|
||||
<span className="text-text-dimmed/60">({item.label.toLowerCase()})</span>
|
||||
) : null}
|
||||
</span>
|
||||
}
|
||||
/>
|
||||
);
|
||||
}
|
||||
|
||||
return (
|
||||
<Fragment key={item.id}>
|
||||
<RunTimelineEvent
|
||||
title={item.title}
|
||||
state={item.state}
|
||||
variant={item.variant}
|
||||
subtitle={
|
||||
item.date ? (
|
||||
<DateTimeAccurate date={item.date} previousDate={item.previousDate} />
|
||||
) : null
|
||||
}
|
||||
/>
|
||||
<TimelineEventExtras item={item} runPath={runPath} sessionPath={sessionPath} />
|
||||
</Fragment>
|
||||
);
|
||||
})}
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
function TimelineEventExtras({
|
||||
item,
|
||||
runPath,
|
||||
sessionPath,
|
||||
}: {
|
||||
item: DeliveryTimelineEventItem;
|
||||
runPath?: string;
|
||||
sessionPath?: string;
|
||||
}) {
|
||||
const showSession = Boolean(item.target?.session && sessionPath);
|
||||
const showRun = Boolean(item.target?.run && runPath);
|
||||
const hasTarget = showSession || showRun;
|
||||
|
||||
if (!item.note && !hasTarget) {
|
||||
return null;
|
||||
}
|
||||
|
||||
return (
|
||||
<div className="grid grid-cols-[1.125rem_1fr] gap-1">
|
||||
<div />
|
||||
<div className="flex flex-col gap-0.5 pb-1">
|
||||
{item.note ? (
|
||||
<span
|
||||
className={item.state === "error" ? "text-xs text-error" : "text-xs text-text-dimmed"}
|
||||
>
|
||||
{item.note}
|
||||
</span>
|
||||
) : null}
|
||||
{showSession && item.target?.session ? (
|
||||
<TextLink to={sessionPath!} className="inline-flex items-center gap-1 font-mono text-xs">
|
||||
<AIChatIcon className="size-3.5 text-sessions" />
|
||||
{item.target.session.friendlyId}
|
||||
</TextLink>
|
||||
) : showRun && item.target?.run ? (
|
||||
<TextLink to={runPath!} className="inline-flex items-center gap-1 font-mono text-xs">
|
||||
<RunsIcon className="size-3.5 text-runs" />
|
||||
{item.target.run.friendlyId}
|
||||
</TextLink>
|
||||
) : null}
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,638 @@
|
||||
import * as Ariakit from "@ariakit/react";
|
||||
import { BeakerIcon, FingerPrintIcon, PlusIcon, XMarkIcon } from "@heroicons/react/20/solid";
|
||||
import { Form } from "@remix-run/react";
|
||||
import { type WebhookDeliveryStatus } from "@trigger.dev/database";
|
||||
import { type ReactNode, useMemo, useRef, useState } from "react";
|
||||
import { StatusIcon } from "~/assets/icons/StatusIcon";
|
||||
import { WebhookIcon } from "~/assets/icons/WebhookIcon";
|
||||
import { AppliedFilter } from "~/components/primitives/AppliedFilter";
|
||||
import {
|
||||
ComboBox,
|
||||
SelectButtonItem,
|
||||
SelectItem,
|
||||
SelectList,
|
||||
SelectPopover,
|
||||
SelectProvider,
|
||||
SelectTrigger,
|
||||
shortcutFromIndex,
|
||||
} from "~/components/primitives/Select";
|
||||
import { ShortcutKey } from "~/components/primitives/ShortcutKey";
|
||||
import { useOptimisticLocation } from "~/hooks/useOptimisticLocation";
|
||||
import { useSearchParams } from "~/hooks/useSearchParam";
|
||||
import { useShortcutKeys } from "~/hooks/useShortcutKeys";
|
||||
import { Button } from "../../primitives/Buttons";
|
||||
import {
|
||||
appliedSummary,
|
||||
FilterMenuProvider,
|
||||
IdFilterDropdown,
|
||||
type IdFilterDropdownProps,
|
||||
TimeFilter,
|
||||
} from "~/components/runs/v3/SharedFilters";
|
||||
|
||||
// Match DeliveriesTable's DELIVERY_STATUS_COLOR / DELIVERY_STATUS_LABEL.
|
||||
const deliveryStatuses: { value: WebhookDeliveryStatus; title: string; color: string }[] = [
|
||||
{ value: "PENDING", title: "Pending", color: "#878C99" },
|
||||
{ value: "PROCESSING", title: "Processing", color: "#3B82F6" },
|
||||
{ value: "SUCCEEDED", title: "Succeeded", color: "#28BF5C" },
|
||||
{ value: "FAILED", title: "Failed", color: "#E11D48" },
|
||||
{ value: "FILTERED", title: "Filtered", color: "#64748B" },
|
||||
];
|
||||
|
||||
const statusTitleByValue = new Map(deliveryStatuses.map((s) => [s.value, s.title]));
|
||||
|
||||
function StatusDot({ color }: { color: string }) {
|
||||
return <span className="size-2 rounded-full" style={{ backgroundColor: color }} />;
|
||||
}
|
||||
|
||||
export type PossibleWebhook = { slug: string; source: string };
|
||||
|
||||
type WebhookDeliveryFiltersProps = {
|
||||
possibleWebhooks: PossibleWebhook[];
|
||||
/** Custom default period for the time filter (e.g., "1h", "7d") */
|
||||
defaultPeriod?: string;
|
||||
};
|
||||
|
||||
export function WebhookDeliveryFilters(props: WebhookDeliveryFiltersProps) {
|
||||
const location = useOptimisticLocation();
|
||||
const searchParams = new URLSearchParams(location.search);
|
||||
const hasFilters =
|
||||
searchParams.has("statuses") ||
|
||||
searchParams.has("webhooks") ||
|
||||
searchParams.has("deliveryId") ||
|
||||
searchParams.has("runId") ||
|
||||
searchParams.has("test");
|
||||
|
||||
return (
|
||||
<div className="flex flex-row flex-wrap items-center gap-1.5">
|
||||
<PermanentStatusFilter />
|
||||
<PermanentWebhookFilter possibleWebhooks={props.possibleWebhooks} />
|
||||
<PermanentTestFilter />
|
||||
<TimeFilter defaultPeriod={props.defaultPeriod} shortcut={{ key: "d" }} />
|
||||
<AppliedFilters />
|
||||
<FilterMenu />
|
||||
{hasFilters && (
|
||||
<Form className="-ml-1 h-6">
|
||||
<Button
|
||||
variant="minimal/small"
|
||||
LeadingIcon={XMarkIcon}
|
||||
tooltip="Clear all filters"
|
||||
className="group-hover/button:bg-transparent"
|
||||
leadingIconClassName="group-hover/button:text-text-bright"
|
||||
/>
|
||||
</Form>
|
||||
)}
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
const filterTypes = [
|
||||
{ name: "deliveryId", title: "Delivery ID", icon: <FingerPrintIcon className="size-4" /> },
|
||||
{ name: "runId", title: "Run ID", icon: <FingerPrintIcon className="size-4" /> },
|
||||
] as const;
|
||||
|
||||
type FilterType = (typeof filterTypes)[number]["name"];
|
||||
|
||||
const moreFiltersShortcut = { key: "f" };
|
||||
|
||||
function FilterMenu() {
|
||||
const [filterType, setFilterType] = useState<FilterType | undefined>();
|
||||
|
||||
const filterTrigger = (
|
||||
<SelectTrigger
|
||||
icon={
|
||||
<div className="flex size-4 items-center justify-center">
|
||||
<PlusIcon className="size-3.5" />
|
||||
</div>
|
||||
}
|
||||
variant={"secondary/small"}
|
||||
shortcut={moreFiltersShortcut}
|
||||
tooltipTitle={"More filters"}
|
||||
className="pl-1 pr-2"
|
||||
>
|
||||
More filters
|
||||
</SelectTrigger>
|
||||
);
|
||||
|
||||
return (
|
||||
<FilterMenuProvider onClose={() => setFilterType(undefined)}>
|
||||
{(search, setSearch) => (
|
||||
<Menu
|
||||
searchValue={search}
|
||||
clearSearchValue={() => setSearch("")}
|
||||
trigger={filterTrigger}
|
||||
filterType={filterType}
|
||||
setFilterType={setFilterType}
|
||||
/>
|
||||
)}
|
||||
</FilterMenuProvider>
|
||||
);
|
||||
}
|
||||
|
||||
function AppliedFilters() {
|
||||
return (
|
||||
<>
|
||||
<AppliedDeliveryIdFilter />
|
||||
<AppliedRunIdFilter />
|
||||
</>
|
||||
);
|
||||
}
|
||||
|
||||
type MenuProps = {
|
||||
searchValue: string;
|
||||
clearSearchValue: () => void;
|
||||
trigger: ReactNode;
|
||||
filterType: FilterType | undefined;
|
||||
setFilterType: (filterType: FilterType | undefined) => void;
|
||||
};
|
||||
|
||||
function Menu(props: MenuProps) {
|
||||
switch (props.filterType) {
|
||||
case undefined:
|
||||
return <MainMenu {...props} />;
|
||||
case "deliveryId":
|
||||
return <DeliveryIdDropdown onClose={() => props.setFilterType(undefined)} {...props} />;
|
||||
case "runId":
|
||||
return <RunIdDropdown onClose={() => props.setFilterType(undefined)} {...props} />;
|
||||
}
|
||||
}
|
||||
|
||||
function MainMenu({ trigger, clearSearchValue, setFilterType }: MenuProps) {
|
||||
return (
|
||||
<SelectProvider virtualFocus={true}>
|
||||
{trigger}
|
||||
<SelectPopover>
|
||||
<SelectList>
|
||||
{filterTypes.map((type, index) => (
|
||||
<SelectButtonItem
|
||||
key={type.name}
|
||||
onClick={() => {
|
||||
clearSearchValue();
|
||||
setFilterType(type.name);
|
||||
}}
|
||||
icon={type.icon}
|
||||
shortcut={shortcutFromIndex(index, { shortcutsEnabled: true })}
|
||||
>
|
||||
<span className="text-text-bright">{type.title}</span>
|
||||
</SelectButtonItem>
|
||||
))}
|
||||
</SelectList>
|
||||
</SelectPopover>
|
||||
</SelectProvider>
|
||||
);
|
||||
}
|
||||
|
||||
function StatusDropdown({
|
||||
trigger,
|
||||
clearSearchValue,
|
||||
onClose,
|
||||
}: {
|
||||
trigger: ReactNode;
|
||||
clearSearchValue: () => void;
|
||||
onClose?: () => void;
|
||||
}) {
|
||||
const { values, replace } = useSearchParams();
|
||||
|
||||
const handleChange = (values: string[]) => {
|
||||
clearSearchValue();
|
||||
replace({ statuses: values, cursor: undefined, direction: undefined });
|
||||
};
|
||||
|
||||
return (
|
||||
<SelectProvider value={values("statuses")} setValue={handleChange} virtualFocus={true}>
|
||||
{trigger}
|
||||
<SelectPopover
|
||||
className="min-w-0 max-w-[min(240px,var(--popover-available-width))]"
|
||||
hideOnEscape={() => {
|
||||
if (onClose) {
|
||||
onClose();
|
||||
return false;
|
||||
}
|
||||
|
||||
return true;
|
||||
}}
|
||||
>
|
||||
<SelectList>
|
||||
{deliveryStatuses.map((item, index) => (
|
||||
<SelectItem
|
||||
key={item.value}
|
||||
value={item.value}
|
||||
shortcut={shortcutFromIndex(index, { shortcutsEnabled: true })}
|
||||
>
|
||||
<span className="flex items-center gap-1.5 text-text-bright">
|
||||
<StatusDot color={item.color} />
|
||||
<span>{item.title}</span>
|
||||
</span>
|
||||
</SelectItem>
|
||||
))}
|
||||
</SelectList>
|
||||
</SelectPopover>
|
||||
</SelectProvider>
|
||||
);
|
||||
}
|
||||
|
||||
const statusShortcut = { key: "s" };
|
||||
|
||||
function PermanentStatusFilter() {
|
||||
const { values, del } = useSearchParams();
|
||||
const statuses = values("statuses");
|
||||
const hasStatuses = statuses.length > 0 && !statuses.every((v) => v === "");
|
||||
const triggerRef = useRef<HTMLButtonElement>(null);
|
||||
|
||||
useShortcutKeys({
|
||||
shortcut: statusShortcut,
|
||||
action: (e) => {
|
||||
e.preventDefault();
|
||||
e.stopPropagation();
|
||||
triggerRef.current?.click();
|
||||
},
|
||||
});
|
||||
|
||||
return (
|
||||
<FilterMenuProvider>
|
||||
{(_search, setSearch) => (
|
||||
<StatusDropdown
|
||||
trigger={
|
||||
<Ariakit.TooltipProvider timeout={200}>
|
||||
<Ariakit.TooltipAnchor
|
||||
render={
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any
|
||||
<Ariakit.Select
|
||||
ref={triggerRef as any}
|
||||
render={<div className="group cursor-pointer focus-custom" />}
|
||||
/>
|
||||
}
|
||||
>
|
||||
{hasStatuses ? (
|
||||
<AppliedFilter
|
||||
label="Status"
|
||||
icon={<StatusIcon className="size-4 border-text-bright" />}
|
||||
value={appliedSummary(
|
||||
statuses.map((v) => statusTitleByValue.get(v as WebhookDeliveryStatus) ?? v)
|
||||
)}
|
||||
onRemove={() => del(["statuses", "cursor", "direction"])}
|
||||
variant="secondary/small"
|
||||
className="pl-1"
|
||||
/>
|
||||
) : (
|
||||
<div className="flex h-6 items-center gap-1 rounded border border-charcoal-600 bg-secondary pl-1 pr-2 text-xs text-text-bright transition group-hover:border-charcoal-550 group-hover:bg-charcoal-600">
|
||||
<div className="grid size-4 place-items-center">
|
||||
<div className="size-[75%] rounded-full border-2 border-text-bright" />
|
||||
</div>
|
||||
<span>Status</span>
|
||||
</div>
|
||||
)}
|
||||
</Ariakit.TooltipAnchor>
|
||||
<Ariakit.Tooltip className="z-40 cursor-default rounded border border-charcoal-700 bg-background-bright px-2 py-1.5 text-xs">
|
||||
<div className="flex items-center gap-2">
|
||||
<span>Filter by status</span>
|
||||
<ShortcutKey
|
||||
className="size-4 flex-none"
|
||||
shortcut={statusShortcut}
|
||||
variant="small"
|
||||
/>
|
||||
</div>
|
||||
</Ariakit.Tooltip>
|
||||
</Ariakit.TooltipProvider>
|
||||
}
|
||||
clearSearchValue={() => setSearch("")}
|
||||
/>
|
||||
)}
|
||||
</FilterMenuProvider>
|
||||
);
|
||||
}
|
||||
|
||||
function WebhookDropdown({
|
||||
trigger,
|
||||
clearSearchValue,
|
||||
searchValue,
|
||||
onClose,
|
||||
possibleWebhooks,
|
||||
}: {
|
||||
trigger: ReactNode;
|
||||
clearSearchValue: () => void;
|
||||
searchValue: string;
|
||||
onClose?: () => void;
|
||||
possibleWebhooks: PossibleWebhook[];
|
||||
}) {
|
||||
const { values, replace } = useSearchParams();
|
||||
|
||||
const handleChange = (newValues: string[]) => {
|
||||
clearSearchValue();
|
||||
replace({
|
||||
webhooks: newValues.length > 0 ? newValues : undefined,
|
||||
cursor: undefined,
|
||||
direction: undefined,
|
||||
});
|
||||
};
|
||||
|
||||
const filtered = useMemo(() => {
|
||||
return possibleWebhooks.filter((item) =>
|
||||
item.slug.toLowerCase().includes(searchValue.toLowerCase())
|
||||
);
|
||||
}, [searchValue, possibleWebhooks]);
|
||||
|
||||
return (
|
||||
<SelectProvider value={values("webhooks")} setValue={handleChange} virtualFocus={true}>
|
||||
{trigger}
|
||||
<SelectPopover
|
||||
className="min-w-0 max-w-[min(360px,var(--popover-available-width))]"
|
||||
hideOnEscape={() => {
|
||||
if (onClose) {
|
||||
onClose();
|
||||
return false;
|
||||
}
|
||||
|
||||
return true;
|
||||
}}
|
||||
>
|
||||
<ComboBox placeholder={"Filter by webhook..."} value={searchValue} />
|
||||
<SelectList>
|
||||
{filtered.length > 0 ? (
|
||||
filtered.map((item) => (
|
||||
<SelectItem
|
||||
key={item.slug}
|
||||
value={item.slug}
|
||||
icon={<WebhookIcon className="size-4 flex-none text-webhooks" />}
|
||||
className="text-text-bright"
|
||||
>
|
||||
{item.slug}
|
||||
</SelectItem>
|
||||
))
|
||||
) : (
|
||||
<SelectItem disabled>No webhooks found</SelectItem>
|
||||
)}
|
||||
</SelectList>
|
||||
</SelectPopover>
|
||||
</SelectProvider>
|
||||
);
|
||||
}
|
||||
|
||||
const webhookShortcut = { key: "w" };
|
||||
|
||||
function PermanentWebhookFilter({ possibleWebhooks }: { possibleWebhooks: PossibleWebhook[] }) {
|
||||
const { values, del } = useSearchParams();
|
||||
const webhooks = values("webhooks");
|
||||
const hasWebhooks = webhooks.length > 0 && !webhooks.every((v) => v === "");
|
||||
const triggerRef = useRef<HTMLButtonElement>(null);
|
||||
|
||||
useShortcutKeys({
|
||||
shortcut: webhookShortcut,
|
||||
action: (e) => {
|
||||
e.preventDefault();
|
||||
e.stopPropagation();
|
||||
triggerRef.current?.click();
|
||||
},
|
||||
});
|
||||
|
||||
return (
|
||||
<FilterMenuProvider>
|
||||
{(search, setSearch) => (
|
||||
<WebhookDropdown
|
||||
trigger={
|
||||
<Ariakit.TooltipProvider timeout={200}>
|
||||
<Ariakit.TooltipAnchor
|
||||
render={
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any
|
||||
<Ariakit.Select
|
||||
ref={triggerRef as any}
|
||||
render={<div className="group cursor-pointer focus-custom" />}
|
||||
/>
|
||||
}
|
||||
>
|
||||
{hasWebhooks ? (
|
||||
<AppliedFilter
|
||||
label="Webhook"
|
||||
icon={<WebhookIcon className="size-4 text-webhooks" />}
|
||||
value={appliedSummary(webhooks)}
|
||||
onRemove={() => del(["webhooks", "cursor", "direction"])}
|
||||
variant="secondary/small"
|
||||
className="pl-1"
|
||||
/>
|
||||
) : (
|
||||
<div className="flex h-6 items-center gap-1.5 rounded border border-charcoal-600 bg-secondary pl-1 pr-2 text-xs text-text-bright transition group-hover:border-charcoal-550 group-hover:bg-charcoal-600">
|
||||
<WebhookIcon className="size-4 text-webhooks" />
|
||||
<span>Webhook</span>
|
||||
</div>
|
||||
)}
|
||||
</Ariakit.TooltipAnchor>
|
||||
<Ariakit.Tooltip className="z-40 cursor-default rounded border border-charcoal-700 bg-background-bright px-2 py-1.5 text-xs">
|
||||
<div className="flex items-center gap-2">
|
||||
<span>Filter by webhook</span>
|
||||
<ShortcutKey
|
||||
className="size-4 flex-none"
|
||||
shortcut={webhookShortcut}
|
||||
variant="small"
|
||||
/>
|
||||
</div>
|
||||
</Ariakit.Tooltip>
|
||||
</Ariakit.TooltipProvider>
|
||||
}
|
||||
searchValue={search}
|
||||
clearSearchValue={() => setSearch("")}
|
||||
possibleWebhooks={possibleWebhooks}
|
||||
/>
|
||||
)}
|
||||
</FilterMenuProvider>
|
||||
);
|
||||
}
|
||||
|
||||
const testModes = [
|
||||
{ value: "all", title: "All deliveries" },
|
||||
{ value: "hide", title: "Hide test sends" },
|
||||
{ value: "only", title: "Test sends only" },
|
||||
] as const;
|
||||
|
||||
const testShortcut = { key: "t" };
|
||||
|
||||
function TestDropdown({ trigger, onClose }: { trigger: ReactNode; onClose?: () => void }) {
|
||||
const { value, replace } = useSearchParams();
|
||||
const current = value("test") ?? "all";
|
||||
|
||||
const handleChange = (next: string) => {
|
||||
replace({ test: next === "all" ? undefined : next, cursor: undefined, direction: undefined });
|
||||
};
|
||||
|
||||
return (
|
||||
<SelectProvider value={current} setValue={handleChange} virtualFocus={true}>
|
||||
{trigger}
|
||||
<SelectPopover
|
||||
className="min-w-0 max-w-[min(240px,var(--popover-available-width))]"
|
||||
hideOnEscape={() => {
|
||||
if (onClose) {
|
||||
onClose();
|
||||
return false;
|
||||
}
|
||||
return true;
|
||||
}}
|
||||
>
|
||||
<SelectList>
|
||||
{testModes.map((mode, index) => (
|
||||
<SelectItem
|
||||
key={mode.value}
|
||||
value={mode.value}
|
||||
shortcut={shortcutFromIndex(index, { shortcutsEnabled: true })}
|
||||
>
|
||||
<span className="text-text-bright">{mode.title}</span>
|
||||
</SelectItem>
|
||||
))}
|
||||
</SelectList>
|
||||
</SelectPopover>
|
||||
</SelectProvider>
|
||||
);
|
||||
}
|
||||
|
||||
function PermanentTestFilter() {
|
||||
const { value, del } = useSearchParams();
|
||||
const current = value("test");
|
||||
const active = current === "hide" || current === "only";
|
||||
const triggerRef = useRef<HTMLButtonElement>(null);
|
||||
|
||||
useShortcutKeys({
|
||||
shortcut: testShortcut,
|
||||
action: (e) => {
|
||||
e.preventDefault();
|
||||
e.stopPropagation();
|
||||
triggerRef.current?.click();
|
||||
},
|
||||
});
|
||||
|
||||
return (
|
||||
<TestDropdown
|
||||
trigger={
|
||||
<Ariakit.TooltipProvider timeout={200}>
|
||||
<Ariakit.TooltipAnchor
|
||||
render={
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any
|
||||
<Ariakit.Select
|
||||
ref={triggerRef as any}
|
||||
render={<div className="group cursor-pointer focus-custom" />}
|
||||
/>
|
||||
}
|
||||
>
|
||||
{active ? (
|
||||
<AppliedFilter
|
||||
label="Test"
|
||||
icon={<BeakerIcon className="size-4" />}
|
||||
value={current === "only" ? "Test sends only" : "Hidden"}
|
||||
onRemove={() => del(["test", "cursor", "direction"])}
|
||||
variant="secondary/small"
|
||||
className="pl-1"
|
||||
/>
|
||||
) : (
|
||||
<div className="flex h-6 items-center gap-1.5 rounded border border-charcoal-600 bg-secondary pl-1 pr-2 text-xs text-text-bright transition group-hover:border-charcoal-550 group-hover:bg-charcoal-600">
|
||||
<BeakerIcon className="size-4 text-text-bright" />
|
||||
<span>Test</span>
|
||||
</div>
|
||||
)}
|
||||
</Ariakit.TooltipAnchor>
|
||||
<Ariakit.Tooltip className="z-40 cursor-default rounded border border-charcoal-700 bg-background-bright px-2 py-1.5 text-xs">
|
||||
<div className="flex items-center gap-2">
|
||||
<span>Filter test sends</span>
|
||||
<ShortcutKey className="size-4 flex-none" shortcut={testShortcut} variant="small" />
|
||||
</div>
|
||||
</Ariakit.Tooltip>
|
||||
</Ariakit.TooltipProvider>
|
||||
}
|
||||
/>
|
||||
);
|
||||
}
|
||||
|
||||
function DeliveryIdDropdown(
|
||||
props: Omit<IdFilterDropdownProps, "label" | "placeholder" | "paramKey">
|
||||
) {
|
||||
return (
|
||||
<IdFilterDropdown
|
||||
{...props}
|
||||
label="Delivery ID"
|
||||
placeholder="whd_ or external id"
|
||||
paramKey="deliveryId"
|
||||
/>
|
||||
);
|
||||
}
|
||||
|
||||
function AppliedDeliveryIdFilter() {
|
||||
const { value, del } = useSearchParams();
|
||||
|
||||
if (value("deliveryId") === undefined) {
|
||||
return null;
|
||||
}
|
||||
|
||||
const deliveryId = value("deliveryId");
|
||||
|
||||
return (
|
||||
<FilterMenuProvider>
|
||||
{(search, setSearch) => (
|
||||
<DeliveryIdDropdown
|
||||
trigger={
|
||||
<Ariakit.Select render={<div className="group cursor-pointer focus-custom" />}>
|
||||
<AppliedFilter
|
||||
label="Delivery ID"
|
||||
icon={<FingerPrintIcon className="size-4" />}
|
||||
value={deliveryId}
|
||||
onRemove={() => del(["deliveryId", "cursor", "direction"])}
|
||||
variant="secondary/small"
|
||||
/>
|
||||
</Ariakit.Select>
|
||||
}
|
||||
searchValue={search}
|
||||
clearSearchValue={() => setSearch("")}
|
||||
/>
|
||||
)}
|
||||
</FilterMenuProvider>
|
||||
);
|
||||
}
|
||||
|
||||
function validateRunId(value: string): string | undefined {
|
||||
if (!value.startsWith("run_")) return "Run IDs start with 'run_'";
|
||||
if (value.length !== 25 && value.length !== 29) return "Run IDs are 25 or 29 characters long";
|
||||
}
|
||||
|
||||
function RunIdDropdown(
|
||||
props: Omit<
|
||||
IdFilterDropdownProps,
|
||||
"label" | "placeholder" | "paramKey" | "validate" | "inputWidth"
|
||||
>
|
||||
) {
|
||||
return (
|
||||
<IdFilterDropdown
|
||||
{...props}
|
||||
label="Run ID"
|
||||
placeholder="run_"
|
||||
paramKey="runId"
|
||||
validate={validateRunId}
|
||||
inputWidth="w-[27ch]"
|
||||
/>
|
||||
);
|
||||
}
|
||||
|
||||
function AppliedRunIdFilter() {
|
||||
const { value, del } = useSearchParams();
|
||||
|
||||
if (value("runId") === undefined) {
|
||||
return null;
|
||||
}
|
||||
|
||||
const runId = value("runId");
|
||||
|
||||
return (
|
||||
<FilterMenuProvider>
|
||||
{(search, setSearch) => (
|
||||
<RunIdDropdown
|
||||
trigger={
|
||||
<Ariakit.Select render={<div className="group cursor-pointer focus-custom" />}>
|
||||
<AppliedFilter
|
||||
label="Run ID"
|
||||
icon={<FingerPrintIcon className="size-4" />}
|
||||
value={runId}
|
||||
onRemove={() => del(["runId", "cursor", "direction"])}
|
||||
variant="secondary/small"
|
||||
/>
|
||||
</Ariakit.Select>
|
||||
}
|
||||
searchValue={search}
|
||||
clearSearchValue={() => setSearch("")}
|
||||
/>
|
||||
)}
|
||||
</FilterMenuProvider>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,125 @@
|
||||
import { type WebhookDeliveryStatus } from "@trigger.dev/database";
|
||||
import {
|
||||
type TimelineEventState,
|
||||
type TimelineEventVariant,
|
||||
type TimelineLineVariant,
|
||||
} from "~/components/run/RunTimeline";
|
||||
|
||||
export type DeliveryRunTarget = {
|
||||
run: { friendlyId: string } | null;
|
||||
session: { friendlyId: string; externalId: string | null } | null;
|
||||
};
|
||||
|
||||
export type DeliveryTimelineEventItem = {
|
||||
type: "event";
|
||||
id: string;
|
||||
title: string;
|
||||
date: Date | null;
|
||||
previousDate?: Date;
|
||||
state: TimelineEventState;
|
||||
variant: TimelineEventVariant;
|
||||
note?: string | null;
|
||||
target?: DeliveryRunTarget;
|
||||
};
|
||||
|
||||
export type DeliveryTimelineLineItem = {
|
||||
type: "line";
|
||||
id: string;
|
||||
from: Date;
|
||||
to: Date | null;
|
||||
state: TimelineEventState;
|
||||
variant: TimelineLineVariant;
|
||||
label?: string;
|
||||
};
|
||||
|
||||
export type DeliveryTimelineItem = DeliveryTimelineEventItem | DeliveryTimelineLineItem;
|
||||
|
||||
export type BuildDeliveryTimelineInput = {
|
||||
status: WebhookDeliveryStatus;
|
||||
createdAt: Date;
|
||||
processedAt: Date | null;
|
||||
errorMessage: string | null;
|
||||
filterReason: string | null;
|
||||
run: { friendlyId: string } | null;
|
||||
session: { friendlyId: string; externalId: string | null } | null;
|
||||
};
|
||||
|
||||
export function buildDeliveryTimelineItems(
|
||||
delivery: BuildDeliveryTimelineInput
|
||||
): DeliveryTimelineItem[] {
|
||||
const { status, createdAt, processedAt } = delivery;
|
||||
const inFlight = status === "PENDING" || status === "PROCESSING";
|
||||
|
||||
const items: DeliveryTimelineItem[] = [
|
||||
{
|
||||
type: "event",
|
||||
id: "received",
|
||||
title: "Received",
|
||||
date: createdAt,
|
||||
state: "complete",
|
||||
variant: "start-cap",
|
||||
},
|
||||
];
|
||||
|
||||
if (status === "FILTERED") {
|
||||
items.push({
|
||||
type: "line",
|
||||
id: "routing",
|
||||
from: createdAt,
|
||||
to: processedAt ?? createdAt,
|
||||
state: "delayed",
|
||||
variant: "light",
|
||||
});
|
||||
items.push({
|
||||
type: "event",
|
||||
id: "filtered",
|
||||
title: "Filtered",
|
||||
date: processedAt ?? createdAt,
|
||||
previousDate: createdAt,
|
||||
state: "delayed",
|
||||
variant: "dot-solid",
|
||||
note: delivery.filterReason,
|
||||
});
|
||||
return items;
|
||||
}
|
||||
|
||||
items.push({
|
||||
type: "line",
|
||||
id: "routing",
|
||||
from: createdAt,
|
||||
to: inFlight ? null : processedAt,
|
||||
state: inFlight ? "inprogress" : status === "FAILED" ? "error" : "complete",
|
||||
variant: "normal",
|
||||
label: "Routing",
|
||||
});
|
||||
|
||||
if (inFlight) {
|
||||
return items;
|
||||
}
|
||||
|
||||
if (status === "SUCCEEDED") {
|
||||
items.push({
|
||||
type: "event",
|
||||
id: "delivered",
|
||||
title: "Delivered",
|
||||
date: processedAt,
|
||||
previousDate: createdAt,
|
||||
state: "complete",
|
||||
variant: "end-cap-thick",
|
||||
target: { run: delivery.run, session: delivery.session },
|
||||
});
|
||||
} else if (status === "FAILED") {
|
||||
items.push({
|
||||
type: "event",
|
||||
id: "failed",
|
||||
title: "Failed",
|
||||
date: processedAt,
|
||||
previousDate: createdAt,
|
||||
state: "error",
|
||||
variant: "end-cap-thick",
|
||||
note: delivery.errorMessage,
|
||||
});
|
||||
}
|
||||
|
||||
return items;
|
||||
}
|
||||
@@ -0,0 +1,240 @@
|
||||
import { useLocation } from "@remix-run/react";
|
||||
import { useCallback, useEffect, useMemo, useRef, useState } from "react";
|
||||
import { useTypedFetcher } from "remix-typedjson";
|
||||
import { useInterval } from "~/hooks/useInterval";
|
||||
import { type WebhookDeliveryListItem } from "~/presenters/v3/WebhookDetailPresenter.server";
|
||||
import {
|
||||
type LiveDeliveryFields,
|
||||
type loader as liveDeliveriesLoader,
|
||||
} from "~/routes/resources.orgs.$organizationSlug.projects.$projectParam.env.$envParam.webhooks.deliveries.live";
|
||||
|
||||
const DELIVERIES_POLL_INTERVAL_MS = 3000;
|
||||
const NEW_DELIVERIES_EVERY_N_POLL_TICKS = 2;
|
||||
|
||||
const IN_FLIGHT_STATUSES = new Set(["PENDING", "PROCESSING"]);
|
||||
|
||||
type ListedDelivery = WebhookDeliveryListItem;
|
||||
type LivePollFetcherData =
|
||||
| { deliveries: LiveDeliveryFields[] }
|
||||
| { deliveries: LiveDeliveryFields[]; count: number; since: number }
|
||||
| undefined;
|
||||
|
||||
function hasNewDeliveriesCountFields(
|
||||
data: LivePollFetcherData
|
||||
): data is NonNullable<LivePollFetcherData> & { count: number; since: number } {
|
||||
return data !== undefined && "count" in data && "since" in data;
|
||||
}
|
||||
|
||||
function maxCreatedAtMs(deliveries: ListedDelivery[]): number | undefined {
|
||||
if (deliveries.length === 0) return undefined;
|
||||
|
||||
return deliveries.reduce<number>((maxTimestamp, delivery) => {
|
||||
return Math.max(maxTimestamp, new Date(delivery.createdAt).getTime());
|
||||
}, 0);
|
||||
}
|
||||
|
||||
function patchVisibleDeliveriesWithLiveUpdates(
|
||||
currentDeliveries: ListedDelivery[],
|
||||
liveDeliveries: LiveDeliveryFields[]
|
||||
) {
|
||||
const updatesById = new Map(liveDeliveries.map((delivery) => [delivery.friendlyId, delivery]));
|
||||
|
||||
return currentDeliveries.map((delivery) => {
|
||||
const update = updatesById.get(delivery.friendlyId);
|
||||
if (!update) return delivery;
|
||||
|
||||
return {
|
||||
...delivery,
|
||||
status: update.status,
|
||||
runId: update.runId,
|
||||
run: update.run,
|
||||
session: update.session,
|
||||
errorMessage: update.errorMessage,
|
||||
processedAt: update.processedAt,
|
||||
};
|
||||
});
|
||||
}
|
||||
|
||||
function isNewDeliveriesCheckTick(tick: number) {
|
||||
return tick === 1 || tick % NEW_DELIVERIES_EVERY_N_POLL_TICKS === 0;
|
||||
}
|
||||
|
||||
function useNewDeliveriesDetection({
|
||||
deliveries,
|
||||
isLoading,
|
||||
}: {
|
||||
deliveries: ListedDelivery[];
|
||||
isLoading: boolean;
|
||||
}) {
|
||||
const pollTickRef = useRef(0);
|
||||
const [knownNewestDeliveryMs, setKnownNewestDeliveryMs] = useState(
|
||||
() => maxCreatedAtMs(deliveries) ?? Date.now()
|
||||
);
|
||||
const [newDeliveriesCount, setNewDeliveriesCount] = useState(0);
|
||||
|
||||
const shouldPollForNewDeliveries = !isLoading && newDeliveriesCount < 100;
|
||||
|
||||
const resetNewDeliveriesTracking = useCallback(() => {
|
||||
setKnownNewestDeliveryMs(maxCreatedAtMs(deliveries) ?? Date.now());
|
||||
setNewDeliveriesCount(0);
|
||||
pollTickRef.current = 0;
|
||||
}, [deliveries]);
|
||||
|
||||
const dismissNewDeliveries = useCallback(() => {
|
||||
setNewDeliveriesCount(0);
|
||||
setKnownNewestDeliveryMs(Date.now());
|
||||
pollTickRef.current = 0;
|
||||
}, []);
|
||||
|
||||
const checkNewDeliveriesOnTick = useCallback(() => {
|
||||
pollTickRef.current += 1;
|
||||
return shouldPollForNewDeliveries && isNewDeliveriesCheckTick(pollTickRef.current);
|
||||
}, [shouldPollForNewDeliveries]);
|
||||
|
||||
const showNewDeliveriesBanner = newDeliveriesCount > 0;
|
||||
|
||||
return {
|
||||
knownNewestDeliveryMs,
|
||||
newDeliveriesCount,
|
||||
setNewDeliveriesCount,
|
||||
shouldPollForNewDeliveries,
|
||||
showNewDeliveriesBanner,
|
||||
dismissNewDeliveries,
|
||||
checkNewDeliveriesOnTick,
|
||||
resetNewDeliveriesTracking,
|
||||
};
|
||||
}
|
||||
|
||||
export function useDeliveriesLiveReload({
|
||||
deliveries,
|
||||
isLoading,
|
||||
webhookEndpointId,
|
||||
organizationSlug,
|
||||
projectSlug,
|
||||
environmentSlug,
|
||||
}: {
|
||||
deliveries: ListedDelivery[];
|
||||
isLoading: boolean;
|
||||
/** Omit to poll across every endpoint in the environment (the cross-endpoint deliveries list). */
|
||||
webhookEndpointId?: string;
|
||||
organizationSlug: string;
|
||||
projectSlug: string;
|
||||
environmentSlug: string;
|
||||
}) {
|
||||
const location = useLocation();
|
||||
const deliveriesPollFetcher = useTypedFetcher<typeof liveDeliveriesLoader>();
|
||||
const deliveriesPollFetcherStateRef = useRef(deliveriesPollFetcher.state);
|
||||
deliveriesPollFetcherStateRef.current = deliveriesPollFetcher.state;
|
||||
|
||||
const [visibleDeliveries, setVisibleDeliveries] = useState(deliveries);
|
||||
|
||||
const {
|
||||
knownNewestDeliveryMs,
|
||||
newDeliveriesCount,
|
||||
setNewDeliveriesCount,
|
||||
shouldPollForNewDeliveries,
|
||||
showNewDeliveriesBanner,
|
||||
dismissNewDeliveries,
|
||||
checkNewDeliveriesOnTick,
|
||||
resetNewDeliveriesTracking,
|
||||
} = useNewDeliveriesDetection({ deliveries, isLoading });
|
||||
|
||||
useEffect(() => {
|
||||
setVisibleDeliveries(deliveries);
|
||||
resetNewDeliveriesTracking();
|
||||
}, [deliveries, location.search, resetNewDeliveriesTracking]);
|
||||
|
||||
useEffect(() => {
|
||||
const data = deliveriesPollFetcher.data;
|
||||
if (!data?.deliveries.length) return;
|
||||
|
||||
setVisibleDeliveries((current) =>
|
||||
patchVisibleDeliveriesWithLiveUpdates(current, data.deliveries)
|
||||
);
|
||||
}, [deliveriesPollFetcher.data]);
|
||||
|
||||
useEffect(() => {
|
||||
const data = deliveriesPollFetcher.data;
|
||||
if (!hasNewDeliveriesCountFields(data)) return;
|
||||
|
||||
if (data.since === knownNewestDeliveryMs) {
|
||||
setNewDeliveriesCount(data.count);
|
||||
}
|
||||
}, [deliveriesPollFetcher.data, knownNewestDeliveryMs, setNewDeliveriesCount]);
|
||||
|
||||
const activeDeliveryIdsParam = useMemo(
|
||||
() =>
|
||||
visibleDeliveries
|
||||
.filter((delivery) => IN_FLIGHT_STATUSES.has(delivery.status))
|
||||
.map((delivery) => delivery.friendlyId)
|
||||
.join(","),
|
||||
[visibleDeliveries]
|
||||
);
|
||||
const hasActiveDeliveries = activeDeliveryIdsParam.length > 0;
|
||||
|
||||
const deliveriesResourcesBasePath = useMemo(
|
||||
() =>
|
||||
`/resources/orgs/${organizationSlug}/projects/${projectSlug}/env/${environmentSlug}/webhooks/deliveries`,
|
||||
[organizationSlug, projectSlug, environmentSlug]
|
||||
);
|
||||
|
||||
const loadDeliveriesPoll = useCallback(
|
||||
(checkForNewDeliveries: boolean) => {
|
||||
if (deliveriesPollFetcherStateRef.current !== "idle") return;
|
||||
|
||||
if (!hasActiveDeliveries && !checkForNewDeliveries) return;
|
||||
|
||||
const searchParams = new URLSearchParams();
|
||||
if (webhookEndpointId) {
|
||||
searchParams.set("webhookEndpointId", webhookEndpointId);
|
||||
}
|
||||
if (hasActiveDeliveries) {
|
||||
searchParams.set("deliveryIds", activeDeliveryIdsParam);
|
||||
}
|
||||
|
||||
if (checkForNewDeliveries) {
|
||||
searchParams.set("includeNewDeliveries", "true");
|
||||
searchParams.set("since", String(knownNewestDeliveryMs));
|
||||
const current = new URLSearchParams(location.search);
|
||||
const to = current.get("to");
|
||||
if (to) searchParams.set("to", to);
|
||||
for (const status of current.getAll("statuses")) searchParams.append("statuses", status);
|
||||
for (const webhook of current.getAll("webhooks")) searchParams.append("webhooks", webhook);
|
||||
for (const key of ["deliveryId", "runId", "test"] as const) {
|
||||
const value = current.get(key);
|
||||
if (value) searchParams.set(key, value);
|
||||
}
|
||||
}
|
||||
|
||||
deliveriesPollFetcher.load(`${deliveriesResourcesBasePath}/live?${searchParams.toString()}`);
|
||||
},
|
||||
[
|
||||
activeDeliveryIdsParam,
|
||||
hasActiveDeliveries,
|
||||
location.search,
|
||||
knownNewestDeliveryMs,
|
||||
webhookEndpointId,
|
||||
deliveriesPollFetcher,
|
||||
deliveriesResourcesBasePath,
|
||||
]
|
||||
);
|
||||
|
||||
const shouldPoll = !isLoading && (hasActiveDeliveries || shouldPollForNewDeliveries);
|
||||
|
||||
useInterval({
|
||||
interval: DELIVERIES_POLL_INTERVAL_MS,
|
||||
onLoad: true,
|
||||
pauseWhenHidden: true,
|
||||
disabled: !shouldPoll,
|
||||
callback: () => {
|
||||
loadDeliveriesPoll(checkNewDeliveriesOnTick());
|
||||
},
|
||||
});
|
||||
|
||||
return {
|
||||
visibleDeliveries,
|
||||
showNewDeliveriesBanner,
|
||||
newDeliveriesCount,
|
||||
dismissNewDeliveries,
|
||||
};
|
||||
}
|
||||
@@ -0,0 +1,25 @@
|
||||
import { type WebhookEndpointStatus } from "@trigger.dev/database";
|
||||
|
||||
const ENDPOINT_STATUS_COLOR: Record<WebhookEndpointStatus, string> = {
|
||||
ACTIVE: "#28BF5C",
|
||||
INACTIVE: "#878C99",
|
||||
DELETING: "#F59E0B",
|
||||
};
|
||||
|
||||
const ENDPOINT_STATUS_LABEL: Record<WebhookEndpointStatus, string> = {
|
||||
ACTIVE: "Active",
|
||||
INACTIVE: "Inactive",
|
||||
DELETING: "Deleting",
|
||||
};
|
||||
|
||||
export function EndpointStatusBadge({ status }: { status: WebhookEndpointStatus }) {
|
||||
return (
|
||||
<span className="flex items-center gap-1.5">
|
||||
<span
|
||||
className="size-2 rounded-full"
|
||||
style={{ backgroundColor: ENDPOINT_STATUS_COLOR[status] }}
|
||||
/>
|
||||
<span>{ENDPOINT_STATUS_LABEL[status]}</span>
|
||||
</span>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,117 @@
|
||||
import { ArrowRightIcon } from "@heroicons/react/20/solid";
|
||||
import { Badge } from "~/components/primitives/Badge";
|
||||
import { Paragraph } from "~/components/primitives/Paragraph";
|
||||
import { PopoverMenuItem } from "~/components/primitives/Popover";
|
||||
import {
|
||||
Table,
|
||||
TableBlankRow,
|
||||
TableBody,
|
||||
TableCell,
|
||||
TableCellMenu,
|
||||
TableHeader,
|
||||
TableHeaderCell,
|
||||
TableRow,
|
||||
} from "~/components/primitives/Table";
|
||||
import { useEnvironment } from "~/hooks/useEnvironment";
|
||||
import { useOrganization } from "~/hooks/useOrganizations";
|
||||
import { useProject } from "~/hooks/useProject";
|
||||
import { type WebhookEndpointListItem } from "~/presenters/v3/WebhookDetailPresenter.server";
|
||||
import { v3WebhookEndpointPath } from "~/utils/pathBuilder";
|
||||
import { EndpointStatusBadge } from "./EndpointStatus";
|
||||
|
||||
export function EndpointsTable({
|
||||
endpoints,
|
||||
stickyHeader = false,
|
||||
showTopBorder = true,
|
||||
}: {
|
||||
endpoints: WebhookEndpointListItem[];
|
||||
stickyHeader?: boolean;
|
||||
showTopBorder?: boolean;
|
||||
}) {
|
||||
const organization = useOrganization();
|
||||
const project = useProject();
|
||||
const environment = useEnvironment();
|
||||
|
||||
return (
|
||||
<Table showTopBorder={showTopBorder} stickyHeader={stickyHeader}>
|
||||
<TableHeader>
|
||||
<TableRow>
|
||||
<TableHeaderCell>Endpoint</TableHeaderCell>
|
||||
<TableHeaderCell>Tenant</TableHeaderCell>
|
||||
<TableHeaderCell>External ref</TableHeaderCell>
|
||||
<TableHeaderCell>Status</TableHeaderCell>
|
||||
<TableHeaderCell>Secret</TableHeaderCell>
|
||||
<TableHeaderCell alignment="right">Deliveries (7d)</TableHeaderCell>
|
||||
<TableHeaderCell>
|
||||
<span className="sr-only">Actions</span>
|
||||
</TableHeaderCell>
|
||||
</TableRow>
|
||||
</TableHeader>
|
||||
<TableBody>
|
||||
{endpoints.length === 0 ? (
|
||||
<TableBlankRow colSpan={7}>
|
||||
<div className="flex items-center justify-center">
|
||||
<Paragraph className="w-auto">No endpoints for this webhook yet</Paragraph>
|
||||
</div>
|
||||
</TableBlankRow>
|
||||
) : (
|
||||
endpoints.map((endpoint) => {
|
||||
const endpointPath = v3WebhookEndpointPath(
|
||||
organization,
|
||||
project,
|
||||
environment,
|
||||
endpoint.friendlyId
|
||||
);
|
||||
|
||||
return (
|
||||
<TableRow key={endpoint.friendlyId}>
|
||||
<TableCell to={endpointPath}>
|
||||
<span className="flex items-center gap-x-2">
|
||||
<span className="font-mono text-xs">{endpoint.friendlyId}</span>
|
||||
{endpoint.isDefault ? <Badge variant="extra-small">default</Badge> : null}
|
||||
</span>
|
||||
</TableCell>
|
||||
<TableCell to={endpointPath}>
|
||||
{endpoint.tenantId ?? <span className="text-text-dimmed">default</span>}
|
||||
</TableCell>
|
||||
<TableCell to={endpointPath}>
|
||||
{endpoint.externalRef ?? <span className="text-text-dimmed">None</span>}
|
||||
</TableCell>
|
||||
<TableCell to={endpointPath}>
|
||||
<EndpointStatusBadge status={endpoint.status} />
|
||||
</TableCell>
|
||||
<TableCell to={endpointPath}>
|
||||
{endpoint.hasSigningSecret ? (
|
||||
<span className="flex items-center gap-1.5">
|
||||
<span className="size-2 rounded-full bg-success" />
|
||||
<span>Set</span>
|
||||
</span>
|
||||
) : (
|
||||
<span className="flex items-center gap-1.5">
|
||||
<span className="size-2 rounded-full bg-charcoal-600" />
|
||||
<span className="text-text-dimmed">Not set</span>
|
||||
</span>
|
||||
)}
|
||||
</TableCell>
|
||||
<TableCell to={endpointPath} alignment="right">
|
||||
{endpoint.deliveryCount.toLocaleString()}
|
||||
</TableCell>
|
||||
<TableCellMenu
|
||||
isSticky
|
||||
popoverContent={
|
||||
<PopoverMenuItem
|
||||
to={endpointPath}
|
||||
icon={ArrowRightIcon}
|
||||
leadingIconClassName="text-webhooks"
|
||||
title="View endpoint"
|
||||
/>
|
||||
}
|
||||
/>
|
||||
</TableRow>
|
||||
);
|
||||
})
|
||||
)}
|
||||
</TableBody>
|
||||
</Table>
|
||||
);
|
||||
}
|
||||
@@ -7,6 +7,8 @@ import {
|
||||
type PrismaReplicaClient,
|
||||
type PrismaTransactionClient,
|
||||
type PrismaTransactionOptions,
|
||||
type WebhookDatabase,
|
||||
type WebhookReplicaDatabase,
|
||||
} from "@trigger.dev/database";
|
||||
import { RunOpsPrismaClient } from "@internal/run-ops-database";
|
||||
import { markReadReplicaClient } from "@internal/run-store";
|
||||
@@ -48,6 +50,8 @@ export type {
|
||||
PrismaClientOrTransaction,
|
||||
PrismaTransactionOptions,
|
||||
PrismaReplicaClient,
|
||||
WebhookDatabase,
|
||||
WebhookReplicaDatabase,
|
||||
};
|
||||
|
||||
// Boundary logger for transac(): skips an error the client extension already
|
||||
@@ -159,7 +163,9 @@ type DatasourceLabel =
|
||||
| "legacy-run-ops-writer"
|
||||
| "legacy-run-ops-replica"
|
||||
| "run-ops-writer"
|
||||
| "run-ops-replica";
|
||||
| "run-ops-replica"
|
||||
| "webhook-writer"
|
||||
| "webhook-replica";
|
||||
|
||||
function tagDatasource<T extends PrismaClient>(datasource: DatasourceLabel, client: T): T {
|
||||
return client.$extends({
|
||||
@@ -222,6 +228,52 @@ export const $replica: PrismaReplicaClient = singleton("replica", () => {
|
||||
: prisma;
|
||||
});
|
||||
|
||||
/**
|
||||
* Webhook feature data-plane seam. The whole webhook feature (WebhookEndpoint + WebhookDelivery)
|
||||
* can run on a dedicated Postgres via WEBHOOK_DATABASE_URL; unset reuses the main prisma instance,
|
||||
* so single-DB installs open no extra pool.
|
||||
*/
|
||||
export const webhookPrisma: WebhookDatabase = singleton("webhookPrisma", () => {
|
||||
if (!env.WEBHOOK_DATABASE_URL) {
|
||||
return prisma;
|
||||
}
|
||||
return captureInfrastructureErrors(
|
||||
tagDatasource(
|
||||
"webhook-writer",
|
||||
buildWriterClient({
|
||||
url: env.WEBHOOK_DATABASE_URL,
|
||||
clientType: "webhook-writer",
|
||||
connectionLimit: env.WEBHOOK_DATABASE_CONNECTION_LIMIT ?? env.DATABASE_CONNECTION_LIMIT,
|
||||
})
|
||||
)
|
||||
);
|
||||
});
|
||||
|
||||
/**
|
||||
* Webhook reader chain: an explicit webhook replica, else the webhook writer once split (no
|
||||
* separate replica yet), else the main $replica when the feature is not split.
|
||||
*/
|
||||
export const webhookReplica: WebhookReplicaDatabase = singleton("webhookReplica", () => {
|
||||
if (env.WEBHOOK_DATABASE_READ_REPLICA_URL) {
|
||||
return markReadReplicaClient(
|
||||
captureInfrastructureErrors(
|
||||
tagDatasource(
|
||||
"webhook-replica",
|
||||
buildReplicaClient({
|
||||
url: env.WEBHOOK_DATABASE_READ_REPLICA_URL,
|
||||
clientType: "webhook-reader",
|
||||
connectionLimit: env.WEBHOOK_DATABASE_CONNECTION_LIMIT ?? env.DATABASE_CONNECTION_LIMIT,
|
||||
})
|
||||
)
|
||||
)
|
||||
);
|
||||
}
|
||||
if (env.WEBHOOK_DATABASE_URL) {
|
||||
return webhookPrisma;
|
||||
}
|
||||
return $replica;
|
||||
});
|
||||
|
||||
export type RunOpsClients = { writer: PrismaClient; replica: PrismaReplicaClient };
|
||||
export type NewRunOpsClients = { writer: RunOpsPrismaClient; replica: RunOpsPrismaClient };
|
||||
export type RunOpsTopology = {
|
||||
@@ -527,18 +579,20 @@ function buildDriverAdapterPool(
|
||||
export function buildWriterClient({
|
||||
url,
|
||||
clientType,
|
||||
connectionLimit = env.DATABASE_CONNECTION_LIMIT,
|
||||
poolTimeout,
|
||||
connectTimeout,
|
||||
useDriverAdapter = false,
|
||||
}: {
|
||||
url: string;
|
||||
clientType: string;
|
||||
connectionLimit?: number;
|
||||
poolTimeout?: number;
|
||||
connectTimeout?: number;
|
||||
useDriverAdapter?: boolean;
|
||||
}): PrismaClient {
|
||||
const databaseUrl = buildPrismaConnectionUrl(url, {
|
||||
connectionLimit: env.DATABASE_CONNECTION_LIMIT.toString(),
|
||||
connectionLimit: connectionLimit.toString(),
|
||||
poolTimeout: (poolTimeout ?? env.DATABASE_POOL_TIMEOUT).toString(),
|
||||
connectTimeout: (connectTimeout ?? env.DATABASE_CONNECTION_TIMEOUT).toString(),
|
||||
applicationName: env.SERVICE_NAME,
|
||||
@@ -711,18 +765,20 @@ function getReplicaClient() {
|
||||
export function buildReplicaClient({
|
||||
url,
|
||||
clientType,
|
||||
connectionLimit = env.DATABASE_CONNECTION_LIMIT,
|
||||
poolTimeout,
|
||||
connectTimeout,
|
||||
useDriverAdapter = false,
|
||||
}: {
|
||||
url: string;
|
||||
clientType: string;
|
||||
connectionLimit?: number;
|
||||
poolTimeout?: number;
|
||||
connectTimeout?: number;
|
||||
useDriverAdapter?: boolean;
|
||||
}): PrismaClient {
|
||||
const replicaUrl = buildPrismaConnectionUrl(url, {
|
||||
connectionLimit: env.DATABASE_CONNECTION_LIMIT.toString(),
|
||||
connectionLimit: connectionLimit.toString(),
|
||||
poolTimeout: (poolTimeout ?? env.DATABASE_POOL_TIMEOUT).toString(),
|
||||
connectTimeout: (connectTimeout ?? env.DATABASE_CONNECTION_TIMEOUT).toString(),
|
||||
applicationName: env.SERVICE_NAME,
|
||||
|
||||
@@ -41,6 +41,18 @@ import { registerRunChangeNotifierHandlers } from "./services/realtime/runChange
|
||||
// TRI-9864 for the incident write-up.
|
||||
import { sessionsReplicationInstance } from "./services/sessionsReplicationInstance.server";
|
||||
(globalThis as Record<string, unknown>).__sessionsReplicationInstance = sessionsReplicationInstance;
|
||||
// Touch the webhook deliveries replication singleton at entry so it boots
|
||||
// deterministically alongside the sessions replicator. Same `sideEffects: false`
|
||||
// tree-shaking constraint applies — assign to globalThis, do NOT use `void`.
|
||||
import { webhookDeliveriesReplicationInstance } from "./services/webhookDeliveriesReplicationInstance.server";
|
||||
(globalThis as Record<string, unknown>).__webhookDeliveriesReplicationInstance =
|
||||
webhookDeliveriesReplicationInstance;
|
||||
// Touch the webhook engine singleton at entry so its redis-worker boots
|
||||
// deterministically on webapp startup (the constructor calls worker.start()).
|
||||
// Same `sideEffects: false` tree-shaking constraint applies: assign to
|
||||
// globalThis, do NOT use `void`.
|
||||
import { webhookEngine } from "./v3/webhookEngine.server";
|
||||
(globalThis as Record<string, unknown>).__webhookEngine = webhookEngine;
|
||||
import { globalFlagsRegistry } from "./v3/globalFlagsRegistry.server";
|
||||
(globalThis as Record<string, unknown>).__globalFlagsRegistry = globalFlagsRegistry;
|
||||
import { workerRegionRegistry } from "./v3/workerRegions.server";
|
||||
@@ -338,6 +350,7 @@ export { engineRateLimiter } from "./services/engineRateLimit.server";
|
||||
export { otlpRateLimiter } from "./services/otlpRateLimit.server";
|
||||
export { runWithHttpContext } from "./services/httpAsyncStorage.server";
|
||||
export { tenantContextMiddleware } from "./services/tenantContextResolver.server";
|
||||
export { webhookIngressIpRateLimiter } from "./services/webhookIngressIpRateLimit.server";
|
||||
export { socketIo } from "./v3/handleSocketIo.server";
|
||||
export { wss } from "./v3/handleWebsockets.server";
|
||||
|
||||
|
||||
@@ -290,6 +290,11 @@ const EnvironmentSchema = z
|
||||
// Control-plane cache relax knobs. Unset -> defaults (DEFAULT_CP_CACHE_TTL_MS / _MAX_ENTRIES).
|
||||
CONTROL_PLANE_CACHE_TTL_MS: z.coerce.number().int().optional(),
|
||||
CONTROL_PLANE_CACHE_MAX_ENTRIES: z.coerce.number().int().optional(),
|
||||
// Webhook feature data-plane DB (WebhookEndpoint + WebhookDelivery). Unset -> the webhook
|
||||
// clients reuse the main prisma / $replica, so this is connection-neutral until you split.
|
||||
WEBHOOK_DATABASE_URL: z.string().optional(),
|
||||
WEBHOOK_DATABASE_READ_REPLICA_URL: z.string().optional(),
|
||||
WEBHOOK_DATABASE_CONNECTION_LIMIT: z.coerce.number().int().optional(),
|
||||
SESSION_SECRET: z.string().min(1).refine(isNotInsecureSecret, INSECURE_SECRET_MESSAGE),
|
||||
MAGIC_LINK_SECRET: z.string().min(1).refine(isNotInsecureSecret, INSECURE_SECRET_MESSAGE),
|
||||
ENCRYPTION_KEY: z
|
||||
@@ -1766,6 +1771,65 @@ const EnvironmentSchema = z
|
||||
.default(process.env.REDIS_TLS_DISABLED ?? "false"),
|
||||
SCHEDULE_WORKER_REDIS_CLUSTER_MODE_ENABLED: z.string().default("0"),
|
||||
|
||||
WEBHOOK_ENGINE_LOG_LEVEL: z.enum(["log", "error", "warn", "info", "debug"]).default("info"),
|
||||
WEBHOOK_WORKER_ENABLED: z.string().default(process.env.WORKER_ENABLED ?? "true"),
|
||||
WEBHOOK_WORKER_CONCURRENCY_LIMIT: z.coerce.number().int().default(50),
|
||||
WEBHOOK_WORKER_CONCURRENCY_WORKERS: z.coerce.number().int().default(2),
|
||||
WEBHOOK_WORKER_CONCURRENCY_TASKS_PER_WORKER: z.coerce.number().int().default(10),
|
||||
WEBHOOK_WORKER_POLL_INTERVAL: z.coerce.number().int().default(1000),
|
||||
WEBHOOK_WORKER_SHUTDOWN_TIMEOUT_MS: z.coerce.number().int().default(30_000),
|
||||
WEBHOOK_ENABLED: z.string().default("0"),
|
||||
|
||||
WEBHOOK_WORKER_REDIS_HOST: z
|
||||
.string()
|
||||
.optional()
|
||||
.transform((v) => v ?? process.env.REDIS_HOST),
|
||||
WEBHOOK_WORKER_REDIS_PORT: z.coerce
|
||||
.number()
|
||||
.optional()
|
||||
.transform(
|
||||
(v) => v ?? (process.env.REDIS_PORT ? parseInt(process.env.REDIS_PORT) : undefined)
|
||||
),
|
||||
WEBHOOK_WORKER_REDIS_USERNAME: z
|
||||
.string()
|
||||
.optional()
|
||||
.transform((v) => v ?? process.env.REDIS_USERNAME),
|
||||
WEBHOOK_WORKER_REDIS_PASSWORD: z
|
||||
.string()
|
||||
.optional()
|
||||
.transform((v) => v ?? process.env.REDIS_PASSWORD),
|
||||
WEBHOOK_WORKER_REDIS_TLS_DISABLED: z
|
||||
.string()
|
||||
.default(process.env.REDIS_TLS_DISABLED ?? "false"),
|
||||
|
||||
WEBHOOK_PARTITION_ENSURE_SCHEDULE: z.string().optional(),
|
||||
WEBHOOK_PARTITION_ENSURE_JITTER_MS: z.coerce.number().int().optional(),
|
||||
WEBHOOK_PARTITION_LOOKAHEAD_DAYS: z.coerce.number().int().default(10),
|
||||
WEBHOOK_PARTITION_RETENTION_DAYS: z.coerce.number().int().default(60),
|
||||
|
||||
// Ingest hot-path cache for the endpoint + resolved signing secret (keyed by opaqueId). 0 disables.
|
||||
WEBHOOK_ENDPOINT_CACHE_TTL_MS: z.coerce.number().int().default(30_000),
|
||||
WEBHOOK_ENDPOINT_CACHE_MAX_SIZE: z.coerce.number().int().default(10_000),
|
||||
|
||||
WEBHOOK_INGRESS_ENABLED: z.string().default("1"),
|
||||
|
||||
// Public origin for the webhook ingress URL shown to users / returned by the API. Defaults to the
|
||||
// API origin; set to a dedicated host (e.g. https://webhook.trigger.dev) when one is fronted.
|
||||
WEBHOOK_INGRESS_ORIGIN: z.string().optional(),
|
||||
WEBHOOK_INGRESS_BODY_SIZE_LIMIT_MB: z.coerce.number().int().default(1),
|
||||
WEBHOOK_INGRESS_RATE_LIMIT_WINDOW: z.string().default("10s"),
|
||||
WEBHOOK_INGRESS_RATE_LIMIT_TOKENS: z.coerce.number().int().default(100),
|
||||
WEBHOOK_INGRESS_IP_RATE_LIMIT_WINDOW: z.string().default("10s"),
|
||||
WEBHOOK_INGRESS_IP_RATE_LIMIT_TOKENS: z.coerce.number().int().default(300),
|
||||
WEBHOOK_FRONT_GATE_DEFAULT_TTL_SECONDS: z.coerce
|
||||
.number()
|
||||
.int()
|
||||
.default(6 * 60 * 60),
|
||||
WEBHOOK_FRONT_GATE_MAX_TTL_SECONDS: z.coerce
|
||||
.number()
|
||||
.int()
|
||||
.default(6 * 60 * 60),
|
||||
|
||||
TASK_EVENT_PARTITIONING_ENABLED: z.string().default("0"),
|
||||
TASK_EVENT_PARTITIONED_WINDOW_IN_SECONDS: z.coerce.number().int().default(60), // 1 minute
|
||||
|
||||
@@ -1923,6 +1987,47 @@ const EnvironmentSchema = z
|
||||
SESSION_REPLICATION_INSERT_BASE_DELAY_MS: z.coerce.number().int().default(100),
|
||||
SESSION_REPLICATION_INSERT_MAX_DELAY_MS: z.coerce.number().int().default(2000),
|
||||
|
||||
// Webhook deliveries replication (Postgres → ClickHouse webhook_deliveries_v1).
|
||||
// Shares Redis with the runs replicator for leader locking but has its own
|
||||
// slot and publication so the two consume independently. The source table is
|
||||
// a partitioned parent, so the publication is created with
|
||||
// publish_via_partition_root.
|
||||
WEBHOOK_DELIVERIES_REPLICATION_CLICKHOUSE_URL: z.string().optional(),
|
||||
WEBHOOK_DELIVERIES_REPLICATION_ENABLED: z.string().default("0"),
|
||||
WEBHOOK_DELIVERIES_REPLICATION_SLOT_NAME: z
|
||||
.string()
|
||||
.default("webhook_deliveries_to_clickhouse_v1"),
|
||||
WEBHOOK_DELIVERIES_REPLICATION_PUBLICATION_NAME: z
|
||||
.string()
|
||||
.default("webhook_deliveries_to_clickhouse_v1_publication"),
|
||||
WEBHOOK_DELIVERIES_REPLICATION_MAX_FLUSH_CONCURRENCY: z.coerce.number().int().default(1),
|
||||
WEBHOOK_DELIVERIES_REPLICATION_FLUSH_INTERVAL_MS: z.coerce.number().int().default(1000),
|
||||
WEBHOOK_DELIVERIES_REPLICATION_FLUSH_BATCH_SIZE: z.coerce.number().int().default(100),
|
||||
WEBHOOK_DELIVERIES_REPLICATION_LEADER_LOCK_TIMEOUT_MS: z.coerce.number().int().default(30_000),
|
||||
WEBHOOK_DELIVERIES_REPLICATION_LEADER_LOCK_EXTEND_INTERVAL_MS: z.coerce
|
||||
.number()
|
||||
.int()
|
||||
.default(10_000),
|
||||
WEBHOOK_DELIVERIES_REPLICATION_LEADER_LOCK_ADDITIONAL_TIME_MS: z.coerce
|
||||
.number()
|
||||
.int()
|
||||
.default(10_000),
|
||||
WEBHOOK_DELIVERIES_REPLICATION_LEADER_LOCK_RETRY_INTERVAL_MS: z.coerce
|
||||
.number()
|
||||
.int()
|
||||
.default(500),
|
||||
WEBHOOK_DELIVERIES_REPLICATION_ACK_INTERVAL_SECONDS: z.coerce.number().int().default(10),
|
||||
WEBHOOK_DELIVERIES_REPLICATION_LOG_LEVEL: z
|
||||
.enum(["log", "error", "warn", "info", "debug"])
|
||||
.default("info"),
|
||||
WEBHOOK_DELIVERIES_REPLICATION_WAIT_FOR_ASYNC_INSERT: z.string().default("0"),
|
||||
WEBHOOK_DELIVERIES_REPLICATION_INSERT_STRATEGY: z
|
||||
.enum(["insert", "insert_async"])
|
||||
.default("insert"),
|
||||
WEBHOOK_DELIVERIES_REPLICATION_INSERT_MAX_RETRIES: z.coerce.number().int().default(3),
|
||||
WEBHOOK_DELIVERIES_REPLICATION_INSERT_BASE_DELAY_MS: z.coerce.number().int().default(100),
|
||||
WEBHOOK_DELIVERIES_REPLICATION_INSERT_MAX_DELAY_MS: z.coerce.number().int().default(2000),
|
||||
|
||||
// Clickhouse
|
||||
CLICKHOUSE_URL: z.string(),
|
||||
// Optional read replica endpoint. Read-only clients (logs, query, admin, runsList,
|
||||
|
||||
@@ -0,0 +1,162 @@
|
||||
import {
|
||||
type WebhookDeliveryListItem as ApiWebhookDeliveryListItem,
|
||||
type WebhookDeliveryObject,
|
||||
} from "@trigger.dev/core/v3";
|
||||
import { type WebhookDeliveryStatus } from "@trigger.dev/database";
|
||||
import { z } from "zod";
|
||||
import { clickhouseFactory } from "~/services/clickhouse/clickhouseFactoryInstance.server";
|
||||
import { type ApiAuthenticationResultSuccess } from "~/services/apiAuth.server";
|
||||
import { CoercedDate } from "~/utils/zod";
|
||||
import { BasePresenter } from "./basePresenter.server";
|
||||
import { WebhookDeliveriesListPresenter } from "./WebhookDeliveriesListPresenter.server";
|
||||
import { WebhookDeliveryDetailPresenter } from "./WebhookDeliveryDetailPresenter.server";
|
||||
import { type WebhookDeliveryListItem } from "./WebhookDetailPresenter.server";
|
||||
|
||||
const DB_STATUS_TO_API: Record<WebhookDeliveryStatus, ApiWebhookDeliveryListItem["status"]> = {
|
||||
PENDING: "pending",
|
||||
PROCESSING: "processing",
|
||||
SUCCEEDED: "succeeded",
|
||||
FAILED: "failed",
|
||||
FILTERED: "filtered",
|
||||
};
|
||||
|
||||
// API status -> DB status (for the filter).
|
||||
const API_STATUS_TO_DB: Record<string, WebhookDeliveryStatus> = {
|
||||
pending: "PENDING",
|
||||
processing: "PROCESSING",
|
||||
succeeded: "SUCCEEDED",
|
||||
failed: "FAILED",
|
||||
filtered: "FILTERED",
|
||||
};
|
||||
|
||||
function toApiListItem(d: WebhookDeliveryListItem): ApiWebhookDeliveryListItem {
|
||||
return {
|
||||
id: d.friendlyId,
|
||||
webhook: d.webhook?.slug ?? null,
|
||||
status: DB_STATUS_TO_API[d.status],
|
||||
externalDeliveryId: d.externalDeliveryId,
|
||||
runId: d.run?.friendlyId ?? null,
|
||||
createdAt: d.createdAt,
|
||||
processedAt: d.processedAt,
|
||||
};
|
||||
}
|
||||
|
||||
export const ApiWebhookDeliveryListSearchParams = z.object({
|
||||
"page[size]": z.coerce.number().int().positive().min(1).max(100).optional(),
|
||||
"page[after]": z.string().optional(),
|
||||
"page[before]": z.string().optional(),
|
||||
"filter[webhook]": z
|
||||
.string()
|
||||
.optional()
|
||||
.transform((value) => (value ? value.split(",") : undefined)),
|
||||
"filter[status]": z
|
||||
.string()
|
||||
.optional()
|
||||
.transform((value, ctx) => {
|
||||
if (!value) return undefined;
|
||||
const statuses = value.split(",");
|
||||
const invalid = statuses.filter(
|
||||
(s) => !Object.prototype.hasOwnProperty.call(API_STATUS_TO_DB, s)
|
||||
);
|
||||
if (invalid.length > 0) {
|
||||
ctx.addIssue({
|
||||
code: z.ZodIssueCode.custom,
|
||||
message: `Invalid status values: ${invalid.join(
|
||||
", "
|
||||
)}. Allowed: pending, processing, succeeded, failed.`,
|
||||
});
|
||||
return z.NEVER;
|
||||
}
|
||||
return Array.from(new Set(statuses.map((s) => API_STATUS_TO_DB[s])));
|
||||
}),
|
||||
"filter[period]": z.string().optional(),
|
||||
"filter[from]": CoercedDate,
|
||||
"filter[to]": CoercedDate,
|
||||
});
|
||||
export type ApiWebhookDeliveryListSearchParams = z.infer<typeof ApiWebhookDeliveryListSearchParams>;
|
||||
|
||||
export class ApiWebhookDeliveryListPresenter extends BasePresenter {
|
||||
public async call(
|
||||
environment: { id: string; projectId: string; organizationId: string },
|
||||
searchParams: ApiWebhookDeliveryListSearchParams
|
||||
): Promise<{
|
||||
data: ApiWebhookDeliveryListItem[];
|
||||
pagination: { next?: string; previous?: string };
|
||||
}> {
|
||||
return this.trace("call", async () => {
|
||||
const clickhouse = await clickhouseFactory.getClickhouseForOrganization(
|
||||
environment.organizationId,
|
||||
"standard"
|
||||
);
|
||||
|
||||
const presenter = new WebhookDeliveriesListPresenter(this._replica, clickhouse);
|
||||
const result = await presenter.call({
|
||||
organizationId: environment.organizationId,
|
||||
projectId: environment.projectId,
|
||||
environmentId: environment.id,
|
||||
webhooks: searchParams["filter[webhook]"],
|
||||
statuses: searchParams["filter[status]"],
|
||||
period: searchParams["filter[period]"],
|
||||
from: searchParams["filter[from]"]?.getTime(),
|
||||
to: searchParams["filter[to]"]?.getTime(),
|
||||
cursor: searchParams["page[after]"] ?? searchParams["page[before]"],
|
||||
direction: searchParams["page[before]"] ? "backward" : "forward",
|
||||
pageSize: searchParams["page[size]"],
|
||||
});
|
||||
|
||||
return { data: result.deliveries.map(toApiListItem), pagination: result.pagination };
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
export class ApiWebhookDeliveryPresenter extends BasePresenter {
|
||||
public async call(
|
||||
environment: { id: string; projectId: string; organizationId: string },
|
||||
deliveryFriendlyId: string
|
||||
): Promise<WebhookDeliveryObject | undefined> {
|
||||
return this.trace("call", async () => {
|
||||
const clickhouse = await clickhouseFactory.getClickhouseForOrganization(
|
||||
environment.organizationId,
|
||||
"standard"
|
||||
);
|
||||
|
||||
const presenter = new WebhookDeliveryDetailPresenter(this._replica, clickhouse);
|
||||
const d = await presenter.call({
|
||||
organizationId: environment.organizationId,
|
||||
projectId: environment.projectId,
|
||||
environmentId: environment.id,
|
||||
deliveryFriendlyId,
|
||||
});
|
||||
|
||||
if (!d) return undefined;
|
||||
|
||||
return {
|
||||
id: d.friendlyId,
|
||||
webhook: d.webhook?.slug ?? null,
|
||||
status: DB_STATUS_TO_API[d.status],
|
||||
externalDeliveryId: d.externalDeliveryId,
|
||||
runId: d.run?.friendlyId ?? null,
|
||||
createdAt: d.createdAt,
|
||||
processedAt: d.processedAt,
|
||||
idempotencyKey: d.idempotencyKey,
|
||||
event: d.parsedEvent ?? null,
|
||||
headers: (d.headers as Record<string, string> | null) ?? null,
|
||||
rawBodyHash: d.rawBodyHash,
|
||||
error: d.errorMessage,
|
||||
filterReason: d.filterReason,
|
||||
updatedAt: d.updatedAt,
|
||||
};
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
export function findWebhookDeliveryResource(
|
||||
authentication: ApiAuthenticationResultSuccess,
|
||||
deliveryId: string
|
||||
): Promise<WebhookDeliveryObject | undefined> {
|
||||
const env = authentication.environment;
|
||||
return new ApiWebhookDeliveryPresenter().call(
|
||||
{ id: env.id, projectId: env.projectId, organizationId: env.organizationId },
|
||||
deliveryId
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,105 @@
|
||||
import { type WebhookEndpointObject } from "@trigger.dev/core/v3";
|
||||
import {
|
||||
type Prisma,
|
||||
type RuntimeEnvironment,
|
||||
type WebhookEndpointStatus,
|
||||
} from "@trigger.dev/database";
|
||||
import { z } from "zod";
|
||||
import { boundedIn, webhookReplica } from "~/db.server";
|
||||
import { type ApiAuthenticationResultSuccess } from "~/services/apiAuth.server";
|
||||
import { webhookIngressUrl } from "~/utils/webhookIngressUrl.server";
|
||||
import { BasePresenter } from "./basePresenter.server";
|
||||
|
||||
const DB_STATUS_TO_API: Record<WebhookEndpointStatus, WebhookEndpointObject["status"]> = {
|
||||
ACTIVE: "active",
|
||||
INACTIVE: "inactive",
|
||||
DELETING: "deleting",
|
||||
};
|
||||
|
||||
// The columns needed to build the public API object.
|
||||
const endpointSelect = {
|
||||
friendlyId: true,
|
||||
opaqueId: true,
|
||||
handlerWebhookId: true,
|
||||
source: true,
|
||||
status: true,
|
||||
secretProvisioning: true,
|
||||
signingSecretKey: true,
|
||||
endpointTenantId: true,
|
||||
endpointExternalRef: true,
|
||||
createdAt: true,
|
||||
updatedAt: true,
|
||||
} satisfies Prisma.WebhookEndpointSelect;
|
||||
|
||||
type EndpointRow = Prisma.WebhookEndpointGetPayload<{ select: typeof endpointSelect }>;
|
||||
|
||||
function toApiEndpoint(endpoint: EndpointRow): WebhookEndpointObject {
|
||||
return {
|
||||
id: endpoint.friendlyId,
|
||||
webhook: endpoint.handlerWebhookId,
|
||||
source: endpoint.source,
|
||||
status: DB_STATUS_TO_API[endpoint.status],
|
||||
secretProvisioning:
|
||||
(endpoint.secretProvisioning as WebhookEndpointObject["secretProvisioning"]) ?? "either",
|
||||
secretSet: endpoint.signingSecretKey != null && endpoint.signingSecretKey !== "",
|
||||
tenantId: endpoint.endpointTenantId === "" ? null : endpoint.endpointTenantId,
|
||||
externalRef: endpoint.endpointExternalRef === "" ? null : endpoint.endpointExternalRef,
|
||||
url: webhookIngressUrl(endpoint.opaqueId),
|
||||
createdAt: endpoint.createdAt,
|
||||
updatedAt: endpoint.updatedAt,
|
||||
};
|
||||
}
|
||||
|
||||
export const ApiWebhookEndpointListSearchParams = z.object({
|
||||
"filter[webhook]": z
|
||||
.string()
|
||||
.optional()
|
||||
.transform((value) => (value ? value.split(",") : undefined)),
|
||||
});
|
||||
export type ApiWebhookEndpointListSearchParams = z.infer<typeof ApiWebhookEndpointListSearchParams>;
|
||||
|
||||
export class ApiWebhookEndpointListPresenter extends BasePresenter {
|
||||
public async call(
|
||||
environment: Pick<RuntimeEnvironment, "id">,
|
||||
searchParams: ApiWebhookEndpointListSearchParams
|
||||
): Promise<{ data: WebhookEndpointObject[] }> {
|
||||
return this.trace("call", async () => {
|
||||
const endpoints = await webhookReplica.webhookEndpoint.findMany({
|
||||
where: {
|
||||
runtimeEnvironmentId: environment.id,
|
||||
...(searchParams["filter[webhook]"]
|
||||
? { handlerWebhookId: { in: boundedIn(searchParams["filter[webhook]"]) } }
|
||||
: {}),
|
||||
},
|
||||
select: endpointSelect,
|
||||
orderBy: [{ handlerWebhookId: "asc" }, { createdAt: "desc" }],
|
||||
});
|
||||
|
||||
return { data: endpoints.map(toApiEndpoint) };
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
export class ApiWebhookEndpointPresenter extends BasePresenter {
|
||||
public async call(
|
||||
environmentId: string,
|
||||
endpointFriendlyId: string
|
||||
): Promise<WebhookEndpointObject | undefined> {
|
||||
return this.trace("call", async () => {
|
||||
const endpoint = await webhookReplica.webhookEndpoint.findFirst({
|
||||
// friendlyId is globally unique; scope to the env so a foreign id 404s.
|
||||
where: { friendlyId: endpointFriendlyId, runtimeEnvironmentId: environmentId },
|
||||
select: endpointSelect,
|
||||
});
|
||||
|
||||
return endpoint ? toApiEndpoint(endpoint) : undefined;
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
export function findWebhookEndpointResource(
|
||||
authentication: ApiAuthenticationResultSuccess,
|
||||
endpointId: string
|
||||
): Promise<WebhookEndpointObject | undefined> {
|
||||
return new ApiWebhookEndpointPresenter().call(authentication.environment.id, endpointId);
|
||||
}
|
||||
@@ -52,11 +52,13 @@ export class TestPresenter extends BasePresenter {
|
||||
SELECT bwt.id, version, slug, "filePath", bwt."friendlyId", bwt."triggerSource"
|
||||
FROM latest_workers
|
||||
JOIN ${sqlDatabaseSchema}."BackgroundWorkerTask" bwt ON bwt."workerId" = latest_workers.id
|
||||
WHERE bwt."triggerSource" != 'AGENT'
|
||||
WHERE bwt."triggerSource" NOT IN ('AGENT', 'WEBHOOK')
|
||||
ORDER BY slug ASC;`;
|
||||
} else {
|
||||
const currentDeployment = await findCurrentWorkerDeployment({ environmentId: envId });
|
||||
return (currentDeployment?.worker?.tasks ?? []).filter((t) => t.triggerSource !== "AGENT");
|
||||
return (currentDeployment?.worker?.tasks ?? []).filter(
|
||||
(t) => t.triggerSource !== "AGENT" && t.triggerSource !== "WEBHOOK"
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -108,6 +108,10 @@ export type TestTaskResult =
|
||||
allowArbitraryQueues: boolean;
|
||||
taskRunTemplates: TaskRunTemplate[];
|
||||
}
|
||||
| {
|
||||
foundTask: true;
|
||||
triggerSource: "WEBHOOK";
|
||||
}
|
||||
| {
|
||||
foundTask: false;
|
||||
};
|
||||
@@ -120,7 +124,6 @@ export type ScheduledTaskResult = Extract<
|
||||
TestTaskResult,
|
||||
{ foundTask: true; triggerSource: "SCHEDULED" }
|
||||
>;
|
||||
|
||||
type RawRun = {
|
||||
id: string;
|
||||
queue: string;
|
||||
@@ -386,6 +389,9 @@ export class TestTaskPresenter {
|
||||
// AGENT tasks are filtered out by TestPresenter and shouldn't reach here
|
||||
return { foundTask: false };
|
||||
}
|
||||
case "WEBHOOK": {
|
||||
return { foundTask: true, triggerSource: "WEBHOOK" };
|
||||
}
|
||||
default: {
|
||||
return task.triggerSource satisfies never;
|
||||
}
|
||||
|
||||
@@ -13,7 +13,7 @@ import { findCurrentWorkerFromEnvironment } from "~/v3/models/workerDeployment.s
|
||||
import { agentListPresenter, type AgentActiveState } from "./AgentListPresenter.server";
|
||||
import { taskListPresenter, type TaskListItem } from "./TaskListPresenter.server";
|
||||
|
||||
export type UnifiedTaskKind = "STANDARD" | "SCHEDULED" | "AGENT";
|
||||
export type UnifiedTaskKind = "STANDARD" | "SCHEDULED" | "AGENT" | "WEBHOOK";
|
||||
|
||||
export type UnifiedTaskListItem = {
|
||||
kind: UnifiedTaskKind;
|
||||
@@ -215,7 +215,12 @@ function toUnifiedItems(
|
||||
|
||||
for (const task of tasks) {
|
||||
items.push({
|
||||
kind: task.triggerSource === "SCHEDULED" ? "SCHEDULED" : "STANDARD",
|
||||
kind:
|
||||
task.triggerSource === "SCHEDULED"
|
||||
? "SCHEDULED"
|
||||
: task.triggerSource === "WEBHOOK"
|
||||
? "WEBHOOK"
|
||||
: "STANDARD",
|
||||
slug: task.slug,
|
||||
filePath: task.filePath,
|
||||
triggerSource: task.triggerSource,
|
||||
|
||||
@@ -0,0 +1,236 @@
|
||||
import { type ClickHouse } from "@internal/clickhouse";
|
||||
import { type PrismaClientOrTransaction, type WebhookDeliveryStatus } from "@trigger.dev/database";
|
||||
import parseDuration from "parse-duration";
|
||||
import { boundedIn, webhookReplica } from "~/db.server";
|
||||
import { runStore } from "~/v3/runStore.server";
|
||||
import { webhookDeliveriesRepository } from "~/services/webhookDeliveriesRepository/webhookDeliveriesRepository.server";
|
||||
import {
|
||||
resolveDeliveryRunTargets,
|
||||
type WebhookDeliveryListItem,
|
||||
} from "./WebhookDetailPresenter.server";
|
||||
|
||||
const DELIVERIES_PAGE_SIZE = 60;
|
||||
type Direction = "forward" | "backward";
|
||||
|
||||
export type WebhookDeliveriesListResult = {
|
||||
deliveries: WebhookDeliveryListItem[];
|
||||
pagination: { next?: string; previous?: string };
|
||||
};
|
||||
|
||||
/**
|
||||
* The top-level (cross-endpoint) deliveries list: every delivery in the environment,
|
||||
* across all webhook endpoints/handlers. Mirrors WebhookDetailPresenter.listDeliveries but
|
||||
* with no webhookEndpointId filter, plus it hydrates which webhook each delivery belongs to
|
||||
* for the table's Webhook column.
|
||||
*/
|
||||
export class WebhookDeliveriesListPresenter {
|
||||
constructor(
|
||||
private readonly replica: PrismaClientOrTransaction,
|
||||
private readonly clickhouse: ClickHouse
|
||||
) {}
|
||||
|
||||
async call({
|
||||
organizationId,
|
||||
projectId,
|
||||
environmentId,
|
||||
webhooks,
|
||||
statuses,
|
||||
deliveryId,
|
||||
runId,
|
||||
isTest,
|
||||
period,
|
||||
from,
|
||||
to,
|
||||
cursor,
|
||||
direction,
|
||||
pageSize,
|
||||
}: {
|
||||
organizationId: string;
|
||||
projectId: string;
|
||||
environmentId: string;
|
||||
webhooks?: string[];
|
||||
statuses?: WebhookDeliveryStatus[];
|
||||
deliveryId?: string;
|
||||
runId?: string;
|
||||
isTest?: boolean;
|
||||
period?: string;
|
||||
from?: number;
|
||||
to?: number;
|
||||
cursor?: string;
|
||||
direction?: Direction;
|
||||
pageSize?: number;
|
||||
}): Promise<WebhookDeliveriesListResult> {
|
||||
const periodMs = period ? (parseDuration(period) ?? undefined) : undefined;
|
||||
|
||||
const { webhookEndpointIds, internalRunId } = await this.#resolveFilterScope(
|
||||
environmentId,
|
||||
webhooks,
|
||||
runId
|
||||
);
|
||||
|
||||
// Built per request (factory, NOT a singleton), matching every RunsRepository consumer.
|
||||
const repository = webhookDeliveriesRepository({
|
||||
clickhouse: this.clickhouse,
|
||||
prisma: webhookReplica,
|
||||
});
|
||||
|
||||
// No webhookEndpointId: all endpoints in the environment.
|
||||
const { deliveries, pagination } = await repository.listDeliveries({
|
||||
organizationId,
|
||||
projectId,
|
||||
environmentId,
|
||||
webhookEndpointIds,
|
||||
deliveryId,
|
||||
runId: internalRunId,
|
||||
statuses,
|
||||
isTest,
|
||||
period: periodMs,
|
||||
from,
|
||||
to,
|
||||
page: { size: pageSize ?? DELIVERIES_PAGE_SIZE, cursor, direction },
|
||||
});
|
||||
|
||||
// Resolve run friendlyIds (the runId is the INTERNAL id; the table links by friendlyId) and, for
|
||||
// session deliveries, the session the run belongs to.
|
||||
const { runFriendlyIdById, sessionByRunId } = await resolveDeliveryRunTargets(
|
||||
this.replica,
|
||||
deliveries
|
||||
);
|
||||
|
||||
// Resolve which webhook (handler) each delivery belongs to for the Webhook column.
|
||||
const endpointIds = Array.from(new Set(deliveries.map((d) => d.webhookEndpointId)));
|
||||
const endpointById = new Map<string, { slug: string; source: string }>();
|
||||
if (endpointIds.length > 0) {
|
||||
const endpoints = await webhookReplica.webhookEndpoint.findMany({
|
||||
where: { id: { in: boundedIn(endpointIds) } },
|
||||
select: { id: true, handlerWebhookId: true, source: true },
|
||||
});
|
||||
for (const e of endpoints) {
|
||||
endpointById.set(e.id, { slug: e.handlerWebhookId, source: e.source });
|
||||
}
|
||||
}
|
||||
|
||||
const items: WebhookDeliveryListItem[] = deliveries.map((d) => {
|
||||
const friendlyId = d.runId ? runFriendlyIdById.get(d.runId) : undefined;
|
||||
return {
|
||||
id: d.id,
|
||||
friendlyId: d.friendlyId,
|
||||
externalDeliveryId: d.externalDeliveryId,
|
||||
status: d.status,
|
||||
isTest: d.isTest,
|
||||
runId: d.runId,
|
||||
run: friendlyId ? { friendlyId } : null,
|
||||
session: d.runId ? (sessionByRunId.get(d.runId) ?? null) : null,
|
||||
errorMessage: d.errorMessage,
|
||||
createdAt: d.createdAt,
|
||||
processedAt: d.processedAt,
|
||||
webhook: endpointById.get(d.webhookEndpointId) ?? null,
|
||||
};
|
||||
});
|
||||
|
||||
return {
|
||||
deliveries: items,
|
||||
pagination: {
|
||||
next: pagination.nextCursor ?? undefined,
|
||||
previous: pagination.previousCursor ?? undefined,
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Count deliveries newer than `since` that match the same filters the list is showing, for the
|
||||
* live "N new deliveries" badge. Applies the same filter resolution as {@link call} so the badge
|
||||
* never counts events the filtered list would exclude. `webhookEndpointId` scopes to a single
|
||||
* endpoint (the per-webhook page); `webhooks` is the cross-endpoint handler filter.
|
||||
*/
|
||||
async countNewDeliveries({
|
||||
organizationId,
|
||||
projectId,
|
||||
environmentId,
|
||||
webhookEndpointId,
|
||||
webhooks,
|
||||
statuses,
|
||||
deliveryId,
|
||||
runId,
|
||||
isTest,
|
||||
since,
|
||||
to,
|
||||
}: {
|
||||
organizationId: string;
|
||||
projectId: string;
|
||||
environmentId: string;
|
||||
webhookEndpointId?: string;
|
||||
webhooks?: string[];
|
||||
statuses?: WebhookDeliveryStatus[];
|
||||
deliveryId?: string;
|
||||
runId?: string;
|
||||
isTest?: boolean;
|
||||
since: number;
|
||||
to?: number;
|
||||
}): Promise<number> {
|
||||
if (to !== undefined && to <= since) return 0;
|
||||
|
||||
const { webhookEndpointIds, internalRunId } = await this.#resolveFilterScope(
|
||||
environmentId,
|
||||
webhooks,
|
||||
runId
|
||||
);
|
||||
|
||||
const repository = webhookDeliveriesRepository({
|
||||
clickhouse: this.clickhouse,
|
||||
prisma: webhookReplica,
|
||||
});
|
||||
|
||||
const { deliveryIds } = await repository.listDeliveryIds({
|
||||
organizationId,
|
||||
projectId,
|
||||
environmentId,
|
||||
webhookEndpointId,
|
||||
webhookEndpointIds,
|
||||
deliveryId,
|
||||
runId: internalRunId,
|
||||
statuses,
|
||||
isTest,
|
||||
from: since + 1,
|
||||
to,
|
||||
page: { size: 100 },
|
||||
});
|
||||
|
||||
return deliveryIds.length;
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve the handler-slug webhook filter to endpoint ids and the friendly runId to the internal
|
||||
* id. A non-empty filter that matches nothing resolves to a sentinel that can never match, so the
|
||||
* filter returns nothing rather than being dropped.
|
||||
*/
|
||||
async #resolveFilterScope(
|
||||
environmentId: string,
|
||||
webhooks?: string[],
|
||||
runId?: string
|
||||
): Promise<{ webhookEndpointIds?: string[]; internalRunId?: string }> {
|
||||
let webhookEndpointIds: string[] | undefined;
|
||||
if (webhooks && webhooks.length > 0) {
|
||||
const endpoints = await webhookReplica.webhookEndpoint.findMany({
|
||||
where: {
|
||||
runtimeEnvironmentId: environmentId,
|
||||
handlerWebhookId: { in: boundedIn(webhooks) },
|
||||
},
|
||||
select: { id: true },
|
||||
});
|
||||
webhookEndpointIds = endpoints.length > 0 ? endpoints.map((e) => e.id) : ["__none__"];
|
||||
}
|
||||
|
||||
let internalRunId: string | undefined;
|
||||
if (runId) {
|
||||
const run = await runStore.findRun(
|
||||
{ friendlyId: runId },
|
||||
{ select: { id: true } },
|
||||
this.replica
|
||||
);
|
||||
internalRunId = run?.id ?? "__none__";
|
||||
}
|
||||
|
||||
return { webhookEndpointIds, internalRunId };
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,120 @@
|
||||
import { type ClickHouse } from "@internal/clickhouse";
|
||||
import {
|
||||
type Prisma,
|
||||
type PrismaClientOrTransaction,
|
||||
type RuntimeEnvironmentType,
|
||||
type WebhookDeliveryStatus,
|
||||
} from "@trigger.dev/database";
|
||||
import { webhookReplica } from "~/db.server";
|
||||
import { runStore } from "~/v3/runStore.server";
|
||||
import { webhookDeliveriesRepository } from "~/services/webhookDeliveriesRepository/webhookDeliveriesRepository.server";
|
||||
|
||||
export type WebhookDeliveryDetail = {
|
||||
id: string;
|
||||
friendlyId: string;
|
||||
status: WebhookDeliveryStatus;
|
||||
externalDeliveryId: string;
|
||||
idempotencyKey: string;
|
||||
rawBodyHash: string | null;
|
||||
parsedEvent: Prisma.JsonValue | null;
|
||||
headers: Prisma.JsonValue | null;
|
||||
errorMessage: string | null;
|
||||
filterReason: string | null;
|
||||
environmentType: RuntimeEnvironmentType;
|
||||
createdAt: Date;
|
||||
updatedAt: Date;
|
||||
processedAt: Date | null;
|
||||
// Resolved from the run id (deliveries store the INTERNAL id; the UI links by friendlyId).
|
||||
run: { friendlyId: string } | null;
|
||||
// Set when the run belongs to a chat.agent session (the delivery routed to a session); the session
|
||||
// is the meaningful target, so the UI links it instead of the incidental run.
|
||||
session: { friendlyId: string; externalId: string | null } | null;
|
||||
// The handler webhook this delivery routed to.
|
||||
webhook: { slug: string; source: string } | null;
|
||||
};
|
||||
|
||||
/**
|
||||
* A single webhook delivery, by its friendlyId. ClickHouse resolves the friendlyId
|
||||
* to (id, createdAt) for a partition-pruned Postgres point lookup, then this hydrates
|
||||
* the run friendlyId and the handler webhook for the detail view.
|
||||
*/
|
||||
export class WebhookDeliveryDetailPresenter {
|
||||
constructor(
|
||||
private readonly replica: PrismaClientOrTransaction,
|
||||
private readonly clickhouse: ClickHouse
|
||||
) {}
|
||||
|
||||
async call({
|
||||
organizationId,
|
||||
projectId,
|
||||
environmentId,
|
||||
deliveryFriendlyId,
|
||||
}: {
|
||||
organizationId: string;
|
||||
projectId: string;
|
||||
environmentId: string;
|
||||
deliveryFriendlyId: string;
|
||||
}): Promise<WebhookDeliveryDetail | null> {
|
||||
const repository = webhookDeliveriesRepository({
|
||||
clickhouse: this.clickhouse,
|
||||
prisma: webhookReplica,
|
||||
});
|
||||
|
||||
const delivery = await repository.getDelivery({
|
||||
organizationId,
|
||||
projectId,
|
||||
environmentId,
|
||||
friendlyId: deliveryFriendlyId,
|
||||
});
|
||||
|
||||
if (!delivery) {
|
||||
return null;
|
||||
}
|
||||
|
||||
// Defense in depth: getDelivery is already env-scoped via ClickHouse, but the
|
||||
// Postgres row is the source of truth, so confirm it belongs to this environment.
|
||||
if (delivery.runtimeEnvironmentId !== environmentId) {
|
||||
return null;
|
||||
}
|
||||
|
||||
let run: { friendlyId: string } | null = null;
|
||||
let session: { friendlyId: string; externalId: string | null } | null = null;
|
||||
if (delivery.runId) {
|
||||
const [taskRun, sessionRun] = await Promise.all([
|
||||
runStore.findRun({ id: delivery.runId }, { select: { friendlyId: true } }, this.replica),
|
||||
this.replica.sessionRun.findFirst({
|
||||
where: { runId: delivery.runId },
|
||||
select: { session: { select: { friendlyId: true, externalId: true } } },
|
||||
}),
|
||||
]);
|
||||
run = taskRun ? { friendlyId: taskRun.friendlyId } : null;
|
||||
session = sessionRun?.session ?? null;
|
||||
}
|
||||
|
||||
const endpoint = await webhookReplica.webhookEndpoint.findFirst({
|
||||
where: { id: delivery.webhookEndpointId },
|
||||
select: { handlerWebhookId: true, source: true },
|
||||
});
|
||||
const webhook = endpoint ? { slug: endpoint.handlerWebhookId, source: endpoint.source } : null;
|
||||
|
||||
return {
|
||||
id: delivery.id,
|
||||
friendlyId: delivery.friendlyId,
|
||||
status: delivery.status,
|
||||
externalDeliveryId: delivery.externalDeliveryId,
|
||||
idempotencyKey: delivery.idempotencyKey,
|
||||
rawBodyHash: delivery.rawBodyHash,
|
||||
parsedEvent: delivery.parsedEvent,
|
||||
headers: delivery.headers,
|
||||
errorMessage: delivery.errorMessage,
|
||||
filterReason: delivery.filterReason,
|
||||
environmentType: delivery.environmentType,
|
||||
createdAt: delivery.createdAt,
|
||||
updatedAt: delivery.updatedAt,
|
||||
processedAt: delivery.processedAt,
|
||||
run,
|
||||
session,
|
||||
webhook,
|
||||
};
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,648 @@
|
||||
import { type ClickHouse } from "@internal/clickhouse";
|
||||
import {
|
||||
type Prisma,
|
||||
type PrismaClientOrTransaction,
|
||||
type RuntimeEnvironmentType,
|
||||
type WebhookDeliveryStatus,
|
||||
type WebhookEndpointStatus,
|
||||
} from "@trigger.dev/database";
|
||||
import parseDuration from "parse-duration";
|
||||
import { z } from "zod";
|
||||
import { type Direction } from "~/components/ListPagination";
|
||||
import { boundedIn, webhookReplica } from "~/db.server";
|
||||
import { runStore } from "~/v3/runStore.server";
|
||||
import { findCurrentWorkerFromEnvironment } from "~/v3/models/workerDeployment.server";
|
||||
import { webhookDeliveriesRepository } from "~/services/webhookDeliveriesRepository/webhookDeliveriesRepository.server";
|
||||
import {
|
||||
buildWebhookComposerEndpoints,
|
||||
type WebhookComposerEndpointData,
|
||||
} from "./webhookComposerEndpoints.server";
|
||||
|
||||
export type WebhookEndpointSummary = {
|
||||
id: string;
|
||||
opaqueId: string;
|
||||
status: string;
|
||||
hasSigningSecret: boolean;
|
||||
};
|
||||
|
||||
export type WebhookDetail = {
|
||||
slug: string;
|
||||
filePath: string;
|
||||
triggerSource: "WEBHOOK";
|
||||
source: string;
|
||||
metadata: unknown;
|
||||
createdAt: Date;
|
||||
endpoint: WebhookEndpointSummary;
|
||||
};
|
||||
|
||||
export type WebhookActivityPoint = {
|
||||
bucket: number; // epoch ms
|
||||
} & Record<string, number>;
|
||||
|
||||
export type WebhookActivity = {
|
||||
data: WebhookActivityPoint[];
|
||||
statuses: string[];
|
||||
};
|
||||
|
||||
export type WebhookDeliveryListItem = {
|
||||
id: string;
|
||||
friendlyId: string;
|
||||
externalDeliveryId: string;
|
||||
status: WebhookDeliveryStatus;
|
||||
isTest: boolean;
|
||||
runId: string | null;
|
||||
run: { friendlyId: string } | null;
|
||||
// Set when the delivery routed to a chat.agent session (the run belongs to a session). The session
|
||||
// is the meaningful target here, so the table links it instead of the incidental run.
|
||||
session: { friendlyId: string; externalId: string | null } | null;
|
||||
errorMessage: string | null;
|
||||
createdAt: Date;
|
||||
processedAt: Date | null;
|
||||
// Only populated by the cross-endpoint (top-level) deliveries list, where the
|
||||
// table shows which webhook each delivery belongs to. Undefined on the scoped
|
||||
// per-webhook detail page.
|
||||
webhook?: { slug: string; source: string } | null;
|
||||
};
|
||||
|
||||
export type WebhookDeliveriesList = {
|
||||
deliveries: WebhookDeliveryListItem[];
|
||||
pagination: { next?: string; previous?: string };
|
||||
filters: { from?: number; to?: number };
|
||||
hasFilters: boolean;
|
||||
};
|
||||
|
||||
/**
|
||||
* Resolve delivery run ids (INTERNAL, no FK) to their run friendlyId and, when the run belongs to a
|
||||
* chat.agent session, the session it targeted. Shared by the per-endpoint and cross-endpoint lists.
|
||||
*/
|
||||
export async function resolveDeliveryRunTargets(
|
||||
replica: PrismaClientOrTransaction,
|
||||
deliveries: { runId: string | null }[]
|
||||
): Promise<{
|
||||
runFriendlyIdById: Map<string, string>;
|
||||
sessionByRunId: Map<string, { friendlyId: string; externalId: string | null }>;
|
||||
}> {
|
||||
const runIds = Array.from(
|
||||
new Set(deliveries.map((d) => d.runId).filter((id): id is string => Boolean(id)))
|
||||
);
|
||||
const runFriendlyIdById = new Map<string, string>();
|
||||
const sessionByRunId = new Map<string, { friendlyId: string; externalId: string | null }>();
|
||||
if (runIds.length === 0) return { runFriendlyIdById, sessionByRunId };
|
||||
|
||||
const [runs, sessionRuns] = await Promise.all([
|
||||
runStore.findRuns(
|
||||
{ where: { id: { in: boundedIn(runIds) } }, select: { id: true, friendlyId: true } },
|
||||
replica
|
||||
),
|
||||
replica.sessionRun.findMany({
|
||||
where: { runId: { in: boundedIn(runIds) } },
|
||||
select: { runId: true, session: { select: { friendlyId: true, externalId: true } } },
|
||||
}),
|
||||
]);
|
||||
for (const run of runs) runFriendlyIdById.set(run.id, run.friendlyId);
|
||||
for (const sr of sessionRuns) sessionByRunId.set(sr.runId, sr.session);
|
||||
return { runFriendlyIdById, sessionByRunId };
|
||||
}
|
||||
|
||||
export type WebhookEndpointListItem = {
|
||||
friendlyId: string;
|
||||
// The declared default endpoint (no tenant/externalRef scope).
|
||||
isDefault: boolean;
|
||||
tenantId: string | null;
|
||||
externalRef: string | null;
|
||||
status: WebhookEndpointStatus;
|
||||
hasSigningSecret: boolean;
|
||||
deliveryCount: number;
|
||||
};
|
||||
|
||||
export type WebhookEndpointDetail = {
|
||||
id: string;
|
||||
friendlyId: string;
|
||||
opaqueId: string;
|
||||
handlerWebhookId: string;
|
||||
source: string;
|
||||
status: WebhookEndpointStatus;
|
||||
isDefault: boolean;
|
||||
tenantId: string;
|
||||
externalRef: string;
|
||||
hasSigningSecret: boolean;
|
||||
// "provider" | "integrator" | "either" — drives the Connect UI (paste vs generate).
|
||||
secretProvisioning: string;
|
||||
// Tagged-union JSON parsed by the route with the @trigger.dev/core schemas.
|
||||
routingTarget: Prisma.JsonValue;
|
||||
verifierArtifact: Prisma.JsonValue;
|
||||
metadata: Prisma.JsonValue;
|
||||
createdAt: Date;
|
||||
updatedAt: Date;
|
||||
};
|
||||
|
||||
// 7-day rolling window for the per-endpoint delivery counts on the Endpoints tab.
|
||||
const ENDPOINT_DELIVERY_COUNT_WINDOW_MS = 7 * 24 * 60 * 60 * 1000;
|
||||
|
||||
// Run-status group order, shared with getRunActivity. Mirrors AgentDetailPresenter.
|
||||
const TERMINAL_GROUPS = {
|
||||
COMPLETED: ["COMPLETED_SUCCESSFULLY"],
|
||||
FAILED: ["COMPLETED_WITH_ERRORS", "SYSTEM_FAILURE", "CRASHED", "INTERRUPTED", "TIMED_OUT"],
|
||||
CANCELED: ["CANCELED", "EXPIRED"],
|
||||
RUNNING: [
|
||||
"EXECUTING",
|
||||
"DEQUEUED",
|
||||
"PENDING_EXECUTING",
|
||||
"WAITING_TO_RESUME",
|
||||
"QUEUED_EXECUTING",
|
||||
"PENDING",
|
||||
"PENDING_VERSION",
|
||||
"DELAYED",
|
||||
"WAITING_FOR_DEPLOY",
|
||||
],
|
||||
} as const;
|
||||
|
||||
const GROUP_LABEL = ["COMPLETED", "FAILED", "CANCELED", "RUNNING"] as const;
|
||||
type GroupLabel = (typeof GROUP_LABEL)[number];
|
||||
|
||||
function groupForStatus(status: string): GroupLabel | undefined {
|
||||
for (const label of GROUP_LABEL) {
|
||||
if ((TERMINAL_GROUPS[label] as readonly string[]).includes(status)) return label;
|
||||
}
|
||||
return undefined;
|
||||
}
|
||||
|
||||
// Stable legend order for the deliveries activity chart.
|
||||
const DELIVERY_STATUSES = ["PENDING", "PROCESSING", "SUCCEEDED", "FAILED", "FILTERED"] as const;
|
||||
|
||||
const DELIVERIES_PAGE_SIZE = 25;
|
||||
|
||||
export class WebhookDetailPresenter {
|
||||
constructor(
|
||||
private readonly replica: PrismaClientOrTransaction,
|
||||
private readonly clickhouse: ClickHouse
|
||||
) {}
|
||||
|
||||
async findWebhook({
|
||||
environmentId,
|
||||
environmentType,
|
||||
webhookSlug,
|
||||
}: {
|
||||
environmentId: string;
|
||||
environmentType: RuntimeEnvironmentType;
|
||||
webhookSlug: string;
|
||||
}): Promise<WebhookDetail | null> {
|
||||
const currentWorker = await findCurrentWorkerFromEnvironment(
|
||||
{ id: environmentId, type: environmentType },
|
||||
this.replica
|
||||
);
|
||||
|
||||
if (!currentWorker) return null;
|
||||
|
||||
const task = await this.replica.backgroundWorkerTask.findFirst({
|
||||
where: {
|
||||
workerId: currentWorker.id,
|
||||
slug: webhookSlug,
|
||||
triggerSource: "WEBHOOK",
|
||||
},
|
||||
select: {
|
||||
slug: true,
|
||||
filePath: true,
|
||||
triggerSource: true,
|
||||
createdAt: true,
|
||||
},
|
||||
});
|
||||
|
||||
if (!task) return null;
|
||||
|
||||
const endpoint = await webhookReplica.webhookEndpoint.findFirst({
|
||||
where: {
|
||||
runtimeEnvironmentId: environmentId,
|
||||
handlerWebhookId: webhookSlug,
|
||||
endpointTenantId: "",
|
||||
endpointExternalRef: "",
|
||||
},
|
||||
select: {
|
||||
id: true,
|
||||
opaqueId: true,
|
||||
status: true,
|
||||
source: true,
|
||||
metadata: true,
|
||||
// signingSecretKey is selected ONLY to derive hasSigningSecret below.
|
||||
// The secret value never leaves this method.
|
||||
signingSecretKey: true,
|
||||
},
|
||||
});
|
||||
|
||||
if (!endpoint) return null;
|
||||
|
||||
return {
|
||||
slug: task.slug,
|
||||
filePath: task.filePath,
|
||||
triggerSource: "WEBHOOK",
|
||||
source: endpoint.source,
|
||||
metadata: endpoint.metadata,
|
||||
createdAt: task.createdAt,
|
||||
endpoint: {
|
||||
id: endpoint.id,
|
||||
opaqueId: endpoint.opaqueId,
|
||||
status: endpoint.status,
|
||||
hasSigningSecret: endpoint.signingSecretKey != null && endpoint.signingSecretKey !== "",
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
async listEndpoints({
|
||||
organizationId,
|
||||
projectId,
|
||||
environmentId,
|
||||
handlerWebhookId,
|
||||
}: {
|
||||
organizationId: string;
|
||||
projectId: string;
|
||||
environmentId: string;
|
||||
handlerWebhookId: string;
|
||||
}): Promise<WebhookEndpointListItem[]> {
|
||||
const endpoints = await webhookReplica.webhookEndpoint.findMany({
|
||||
where: { runtimeEnvironmentId: environmentId, handlerWebhookId },
|
||||
select: {
|
||||
id: true,
|
||||
friendlyId: true,
|
||||
endpointTenantId: true,
|
||||
endpointExternalRef: true,
|
||||
status: true,
|
||||
signingSecretKey: true,
|
||||
createdAt: true,
|
||||
},
|
||||
// Default endpoint (empty scope) first, then most recent.
|
||||
orderBy: [{ endpointTenantId: "asc" }, { createdAt: "desc" }],
|
||||
});
|
||||
|
||||
const repository = webhookDeliveriesRepository({
|
||||
clickhouse: this.clickhouse,
|
||||
prisma: webhookReplica,
|
||||
});
|
||||
|
||||
// Per-endpoint 7d delivery counts in ONE grouped CH query (not an N+1 of count queries).
|
||||
// Degrade to empty (0 per endpoint) on error rather than failing the whole tab.
|
||||
const deliveryCounts = await repository
|
||||
.countDeliveriesByEndpoint({
|
||||
organizationId,
|
||||
projectId,
|
||||
environmentId,
|
||||
webhookEndpointIds: endpoints.map((endpoint) => endpoint.id),
|
||||
period: ENDPOINT_DELIVERY_COUNT_WINDOW_MS,
|
||||
})
|
||||
.catch(() => new Map<string, number>());
|
||||
|
||||
return endpoints.map((endpoint) => {
|
||||
const isDefault = endpoint.endpointTenantId === "" && endpoint.endpointExternalRef === "";
|
||||
|
||||
return {
|
||||
friendlyId: endpoint.friendlyId,
|
||||
isDefault,
|
||||
tenantId: endpoint.endpointTenantId === "" ? null : endpoint.endpointTenantId,
|
||||
externalRef: endpoint.endpointExternalRef === "" ? null : endpoint.endpointExternalRef,
|
||||
status: endpoint.status,
|
||||
hasSigningSecret: endpoint.signingSecretKey != null && endpoint.signingSecretKey !== "",
|
||||
deliveryCount: deliveryCounts.get(endpoint.id) ?? 0,
|
||||
} satisfies WebhookEndpointListItem;
|
||||
});
|
||||
}
|
||||
|
||||
async listComposerEndpoints({
|
||||
environmentId,
|
||||
handlerWebhookId,
|
||||
}: {
|
||||
environmentId: string;
|
||||
handlerWebhookId: string;
|
||||
}): Promise<WebhookComposerEndpointData[]> {
|
||||
const endpoints = await webhookReplica.webhookEndpoint.findMany({
|
||||
where: { runtimeEnvironmentId: environmentId, handlerWebhookId },
|
||||
select: {
|
||||
friendlyId: true,
|
||||
opaqueId: true,
|
||||
source: true,
|
||||
endpointTenantId: true,
|
||||
endpointExternalRef: true,
|
||||
verifierArtifact: true,
|
||||
signingSecretKey: true,
|
||||
},
|
||||
orderBy: [{ endpointTenantId: "asc" }, { createdAt: "desc" }],
|
||||
});
|
||||
return buildWebhookComposerEndpoints(endpoints);
|
||||
}
|
||||
|
||||
async findEndpoint({
|
||||
environmentId,
|
||||
endpointFriendlyId,
|
||||
}: {
|
||||
environmentId: string;
|
||||
endpointFriendlyId: string;
|
||||
}): Promise<WebhookEndpointDetail | null> {
|
||||
const endpoint = await webhookReplica.webhookEndpoint.findFirst({
|
||||
// friendlyId is globally unique, but scope to the env so a foreign id 404s.
|
||||
where: { friendlyId: endpointFriendlyId, runtimeEnvironmentId: environmentId },
|
||||
select: {
|
||||
id: true,
|
||||
friendlyId: true,
|
||||
opaqueId: true,
|
||||
handlerWebhookId: true,
|
||||
source: true,
|
||||
status: true,
|
||||
endpointTenantId: true,
|
||||
endpointExternalRef: true,
|
||||
signingSecretKey: true,
|
||||
secretProvisioning: true,
|
||||
routingTarget: true,
|
||||
verifierArtifact: true,
|
||||
metadata: true,
|
||||
createdAt: true,
|
||||
updatedAt: true,
|
||||
},
|
||||
});
|
||||
|
||||
if (!endpoint) return null;
|
||||
|
||||
return {
|
||||
id: endpoint.id,
|
||||
friendlyId: endpoint.friendlyId,
|
||||
opaqueId: endpoint.opaqueId,
|
||||
handlerWebhookId: endpoint.handlerWebhookId,
|
||||
source: endpoint.source,
|
||||
status: endpoint.status,
|
||||
isDefault: endpoint.endpointTenantId === "" && endpoint.endpointExternalRef === "",
|
||||
tenantId: endpoint.endpointTenantId,
|
||||
externalRef: endpoint.endpointExternalRef,
|
||||
hasSigningSecret: endpoint.signingSecretKey != null && endpoint.signingSecretKey !== "",
|
||||
secretProvisioning: endpoint.secretProvisioning,
|
||||
routingTarget: endpoint.routingTarget,
|
||||
verifierArtifact: endpoint.verifierArtifact,
|
||||
metadata: endpoint.metadata,
|
||||
createdAt: endpoint.createdAt,
|
||||
updatedAt: endpoint.updatedAt,
|
||||
};
|
||||
}
|
||||
|
||||
async getRunActivity({
|
||||
organizationId,
|
||||
projectId,
|
||||
environmentId,
|
||||
webhookSlug,
|
||||
from,
|
||||
to,
|
||||
}: {
|
||||
organizationId: string;
|
||||
projectId: string;
|
||||
environmentId: string;
|
||||
webhookSlug: string;
|
||||
from: Date;
|
||||
to: Date;
|
||||
}): Promise<WebhookActivity> {
|
||||
const rangeMs = Math.max(1, to.getTime() - from.getTime());
|
||||
const oneHour = 60 * 60 * 1000;
|
||||
const oneDay = 24 * oneHour;
|
||||
|
||||
const bucketSeconds =
|
||||
rangeMs <= oneDay ? 60 * 60 : rangeMs <= 7 * oneDay ? 6 * 60 * 60 : 24 * 60 * 60;
|
||||
|
||||
// FINAL + _is_deleted = 0 because task_runs_v2 is a ReplacingMergeTree;
|
||||
// org/project filters engage the sort-key prefix for partition pruning.
|
||||
const queryFn = this.clickhouse.reader.query({
|
||||
name: "webhookRunStatusActivity",
|
||||
query: `SELECT
|
||||
toUnixTimestamp(toStartOfInterval(created_at, INTERVAL {bucketSeconds: UInt32} SECOND)) AS bucket,
|
||||
status,
|
||||
count() AS val
|
||||
FROM trigger_dev.task_runs_v2 FINAL
|
||||
WHERE organization_id = {organizationId: String}
|
||||
AND project_id = {projectId: String}
|
||||
AND environment_id = {environmentId: String}
|
||||
AND task_identifier = {webhookSlug: String}
|
||||
AND created_at >= {fromTime: DateTime64(3, 'UTC')}
|
||||
AND created_at < {toTime: DateTime64(3, 'UTC')}
|
||||
AND _is_deleted = 0
|
||||
GROUP BY bucket, status
|
||||
ORDER BY bucket`,
|
||||
params: z.object({
|
||||
organizationId: z.string(),
|
||||
projectId: z.string(),
|
||||
environmentId: z.string(),
|
||||
webhookSlug: z.string(),
|
||||
bucketSeconds: z.number(),
|
||||
fromTime: z.string(),
|
||||
toTime: z.string(),
|
||||
}),
|
||||
schema: z.object({
|
||||
bucket: z.coerce.number(),
|
||||
status: z.string(),
|
||||
val: z.coerce.number(),
|
||||
}),
|
||||
});
|
||||
|
||||
const [error, rows] = await queryFn({
|
||||
organizationId,
|
||||
projectId,
|
||||
environmentId,
|
||||
webhookSlug,
|
||||
bucketSeconds,
|
||||
// ClickHouse's DateTime64(3, 'UTC') parser rejects the trailing `Z` from
|
||||
// JS toISOString(). Strip it.
|
||||
fromTime: from.toISOString().slice(0, -1),
|
||||
toTime: to.toISOString().slice(0, -1),
|
||||
});
|
||||
|
||||
if (error) {
|
||||
console.error("Webhook run activity query failed:", error);
|
||||
return { data: [], statuses: [] };
|
||||
}
|
||||
|
||||
const bucketMap = new Map<number, Record<string, number>>();
|
||||
for (const row of rows) {
|
||||
const group = groupForStatus(row.status) ?? "RUNNING";
|
||||
const ts = row.bucket * 1000;
|
||||
const existing = bucketMap.get(ts) ?? {};
|
||||
existing[group] = (existing[group] ?? 0) + row.val;
|
||||
bucketMap.set(ts, existing);
|
||||
}
|
||||
|
||||
const bucketMs = bucketSeconds * 1000;
|
||||
const start = Math.floor(from.getTime() / bucketMs) * bucketMs;
|
||||
const end = Math.ceil(to.getTime() / bucketMs) * bucketMs;
|
||||
const points: WebhookActivityPoint[] = [];
|
||||
const orderedStatuses = [...GROUP_LABEL];
|
||||
for (let ts = start; ts < end; ts += bucketMs) {
|
||||
const existing = bucketMap.get(ts) ?? {};
|
||||
const point: WebhookActivityPoint = { bucket: ts };
|
||||
for (const g of orderedStatuses) {
|
||||
point[g] = existing[g] ?? 0;
|
||||
}
|
||||
points.push(point);
|
||||
}
|
||||
|
||||
return { data: points, statuses: orderedStatuses };
|
||||
}
|
||||
|
||||
async getDeliveryActivity({
|
||||
organizationId,
|
||||
projectId,
|
||||
environmentId,
|
||||
webhookEndpointId,
|
||||
from,
|
||||
to,
|
||||
}: {
|
||||
organizationId: string;
|
||||
projectId: string;
|
||||
environmentId: string;
|
||||
webhookEndpointId: string;
|
||||
from: Date;
|
||||
to: Date;
|
||||
}): Promise<WebhookActivity> {
|
||||
const rangeMs = Math.max(1, to.getTime() - from.getTime());
|
||||
const oneHour = 60 * 60 * 1000;
|
||||
const oneDay = 24 * oneHour;
|
||||
const bucketSeconds = rangeMs <= oneDay ? 3600 : rangeMs <= 7 * oneDay ? 6 * 3600 : 24 * 3600;
|
||||
|
||||
const queryFn = this.clickhouse.reader.query({
|
||||
name: "webhookDeliveryStatusActivity",
|
||||
query: `SELECT
|
||||
toUnixTimestamp(toStartOfInterval(created_at, INTERVAL {bucketSeconds: UInt32} SECOND)) AS bucket,
|
||||
status, count() AS val
|
||||
FROM trigger_dev.webhook_deliveries_v1 FINAL
|
||||
WHERE organization_id = {organizationId: String}
|
||||
AND project_id = {projectId: String}
|
||||
AND environment_id = {environmentId: String}
|
||||
AND webhook_endpoint_id = {webhookEndpointId: String}
|
||||
AND created_at >= {fromTime: DateTime64(3, 'UTC')}
|
||||
AND created_at < {toTime: DateTime64(3, 'UTC')}
|
||||
AND _is_deleted = 0
|
||||
GROUP BY bucket, status
|
||||
ORDER BY bucket`,
|
||||
params: z.object({
|
||||
organizationId: z.string(),
|
||||
projectId: z.string(),
|
||||
environmentId: z.string(),
|
||||
webhookEndpointId: z.string(),
|
||||
bucketSeconds: z.number(),
|
||||
fromTime: z.string(),
|
||||
toTime: z.string(),
|
||||
}),
|
||||
schema: z.object({
|
||||
bucket: z.coerce.number(),
|
||||
status: z.string(),
|
||||
val: z.coerce.number(),
|
||||
}),
|
||||
});
|
||||
|
||||
const [error, rows] = await queryFn({
|
||||
organizationId,
|
||||
projectId,
|
||||
environmentId,
|
||||
webhookEndpointId,
|
||||
bucketSeconds,
|
||||
fromTime: from.toISOString().slice(0, -1),
|
||||
toTime: to.toISOString().slice(0, -1),
|
||||
});
|
||||
|
||||
if (error) {
|
||||
console.error("Webhook delivery activity query failed:", error);
|
||||
return { data: [], statuses: [] };
|
||||
}
|
||||
|
||||
const bucketMap = new Map<number, Record<string, number>>();
|
||||
for (const row of rows) {
|
||||
const ts = row.bucket * 1000;
|
||||
const existing = bucketMap.get(ts) ?? {};
|
||||
existing[row.status] = (existing[row.status] ?? 0) + row.val;
|
||||
bucketMap.set(ts, existing);
|
||||
}
|
||||
|
||||
const bucketMs = bucketSeconds * 1000;
|
||||
const start = Math.floor(from.getTime() / bucketMs) * bucketMs;
|
||||
const end = Math.ceil(to.getTime() / bucketMs) * bucketMs;
|
||||
const points: WebhookActivityPoint[] = [];
|
||||
const orderedStatuses = [...DELIVERY_STATUSES];
|
||||
for (let ts = start; ts < end; ts += bucketMs) {
|
||||
const existing = bucketMap.get(ts) ?? {};
|
||||
const point: WebhookActivityPoint = { bucket: ts };
|
||||
for (const s of orderedStatuses) {
|
||||
point[s] = existing[s] ?? 0;
|
||||
}
|
||||
points.push(point);
|
||||
}
|
||||
|
||||
return { data: points, statuses: orderedStatuses };
|
||||
}
|
||||
|
||||
async listDeliveries({
|
||||
organizationId,
|
||||
projectId,
|
||||
environmentId,
|
||||
webhookEndpointId,
|
||||
period,
|
||||
from,
|
||||
to,
|
||||
hasExplicitWindow,
|
||||
cursor,
|
||||
direction,
|
||||
}: {
|
||||
organizationId: string;
|
||||
projectId: string;
|
||||
environmentId: string;
|
||||
webhookEndpointId: string;
|
||||
period?: string;
|
||||
from?: number;
|
||||
to?: number;
|
||||
hasExplicitWindow?: boolean;
|
||||
cursor?: string;
|
||||
direction?: Direction;
|
||||
}): Promise<WebhookDeliveriesList> {
|
||||
const periodMs = period ? (parseDuration(period) ?? undefined) : undefined;
|
||||
|
||||
// Built per request (factory, NOT a singleton), matching every RunsRepository consumer.
|
||||
const repository = webhookDeliveriesRepository({
|
||||
clickhouse: this.clickhouse,
|
||||
prisma: webhookReplica,
|
||||
});
|
||||
|
||||
const { deliveries, pagination } = await repository.listDeliveries({
|
||||
organizationId,
|
||||
projectId,
|
||||
environmentId,
|
||||
webhookEndpointId,
|
||||
period: periodMs,
|
||||
from,
|
||||
to,
|
||||
page: { size: DELIVERIES_PAGE_SIZE, cursor, direction },
|
||||
});
|
||||
|
||||
// A delivery's runId is the INTERNAL run id (no FK); resolve friendlyIds and, for session
|
||||
// deliveries, the session the run belongs to, with a small keyed lookup.
|
||||
const { runFriendlyIdById, sessionByRunId } = await resolveDeliveryRunTargets(
|
||||
this.replica,
|
||||
deliveries
|
||||
);
|
||||
|
||||
const items: WebhookDeliveryListItem[] = deliveries.map((d) => {
|
||||
const friendlyId = d.runId ? runFriendlyIdById.get(d.runId) : undefined;
|
||||
return {
|
||||
id: d.id,
|
||||
friendlyId: d.friendlyId,
|
||||
externalDeliveryId: d.externalDeliveryId,
|
||||
status: d.status,
|
||||
isTest: d.isTest,
|
||||
runId: d.runId,
|
||||
run: friendlyId ? { friendlyId } : null,
|
||||
session: d.runId ? (sessionByRunId.get(d.runId) ?? null) : null,
|
||||
errorMessage: d.errorMessage,
|
||||
createdAt: d.createdAt,
|
||||
processedAt: d.processedAt,
|
||||
};
|
||||
});
|
||||
|
||||
return {
|
||||
deliveries: items,
|
||||
pagination: {
|
||||
next: pagination.nextCursor ?? undefined,
|
||||
previous: pagination.previousCursor ?? undefined,
|
||||
},
|
||||
filters: { from, to },
|
||||
hasFilters: hasExplicitWindow ?? Boolean(from || to),
|
||||
};
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,53 @@
|
||||
import { type WebhookHandshakeConfig, WebhookVerifierArtifact } from "@trigger.dev/core/v3";
|
||||
import { webhookIngressUrl } from "~/utils/webhookIngressUrl.server";
|
||||
|
||||
export type WebhookComposerEndpointData = {
|
||||
friendlyId: string;
|
||||
label: string;
|
||||
source: string;
|
||||
ingressUrl: string;
|
||||
scheme: "hmac" | "shared-secret" | "url-secret" | "asymmetric";
|
||||
hasSigningSecret: boolean;
|
||||
/** Present only when the endpoint declares a provider handshake (unlocks the handshake test). */
|
||||
handshake: WebhookHandshakeConfig | null;
|
||||
};
|
||||
|
||||
type EndpointRow = {
|
||||
friendlyId: string;
|
||||
opaqueId: string;
|
||||
source: string;
|
||||
endpointTenantId: string;
|
||||
endpointExternalRef: string;
|
||||
verifierArtifact: unknown;
|
||||
signingSecretKey: string | null;
|
||||
};
|
||||
|
||||
/**
|
||||
* Map raw WebhookEndpoint rows to the shape the composer consumes (label + scheme + ingress URL +
|
||||
* handshake). Shared by the /test WEBHOOK arm and the console tab so the two stay in lockstep.
|
||||
*/
|
||||
export function buildWebhookComposerEndpoints(rows: EndpointRow[]): WebhookComposerEndpointData[] {
|
||||
return rows.map((endpoint) => {
|
||||
const parsed = WebhookVerifierArtifact.safeParse(endpoint.verifierArtifact);
|
||||
const scheme =
|
||||
parsed.success && parsed.data.kind !== "bundle" ? parsed.data.config.scheme : "hmac";
|
||||
const handshake =
|
||||
parsed.success && parsed.data.kind !== "bundle" ? (parsed.data.handshake ?? null) : null;
|
||||
|
||||
const isDefault = endpoint.endpointTenantId === "" && endpoint.endpointExternalRef === "";
|
||||
|
||||
return {
|
||||
friendlyId: endpoint.friendlyId,
|
||||
label: isDefault
|
||||
? "default"
|
||||
: endpoint.endpointExternalRef
|
||||
? `${endpoint.endpointTenantId}: ${endpoint.endpointExternalRef}`
|
||||
: endpoint.endpointTenantId,
|
||||
source: endpoint.source,
|
||||
ingressUrl: webhookIngressUrl(endpoint.opaqueId),
|
||||
scheme,
|
||||
hasSigningSecret: endpoint.signingSecretKey != null && endpoint.signingSecretKey !== "",
|
||||
handshake,
|
||||
} satisfies WebhookComposerEndpointData;
|
||||
});
|
||||
}
|
||||
+49
-19
@@ -17,6 +17,7 @@ import { PlusIcon } from "~/assets/icons/PlusIcon";
|
||||
import { QuestionMarkIcon } from "~/assets/icons/QuestionMarkIcon";
|
||||
import { RunsIcon } from "~/assets/icons/RunsIcon";
|
||||
import { TaskIcon } from "~/assets/icons/TaskIcon";
|
||||
import { WebhookIcon } from "~/assets/icons/WebhookIcon";
|
||||
import { AdminDebugTooltip } from "~/components/admin/debugTooltip";
|
||||
import { CodeBlock } from "~/components/code/CodeBlock";
|
||||
import { InlineCode } from "~/components/code/InlineCode";
|
||||
@@ -101,6 +102,7 @@ import {
|
||||
v3StandardTaskPath,
|
||||
v3TasksStreamingPath,
|
||||
v3TestTaskPath,
|
||||
v3WebhookTaskPath,
|
||||
} from "~/utils/pathBuilder";
|
||||
import { sectionAgentPageContext } from "~/components/dashboard-agent/suggested-prompts";
|
||||
import { WhenAgentUnavailable } from "~/components/dashboard-agent/WhenAgentUnavailable";
|
||||
@@ -178,6 +180,7 @@ const KIND_OPTIONS: { value: UnifiedTaskKind; label: string }[] = [
|
||||
{ value: "AGENT", label: "Agent" },
|
||||
{ value: "STANDARD", label: "Standard" },
|
||||
{ value: "SCHEDULED", label: "Scheduled" },
|
||||
{ value: "WEBHOOK", label: "Webhook" },
|
||||
];
|
||||
|
||||
const VALID_KINDS = new Set<UnifiedTaskKind>(KIND_OPTIONS.map((o) => o.value));
|
||||
@@ -207,6 +210,7 @@ const TASK_TYPE_SEGMENTS: {
|
||||
{ value: "AGENT", tooltip: "Agent tasks", source: "AGENT" },
|
||||
{ value: "STANDARD", tooltip: "Standard tasks", source: "STANDARD" },
|
||||
{ value: "SCHEDULED", tooltip: "Scheduled tasks", source: "SCHEDULED" },
|
||||
{ value: "WEBHOOK", tooltip: "Webhook tasks", source: "WEBHOOK" },
|
||||
];
|
||||
|
||||
const PAGE_SIZE = 25;
|
||||
@@ -437,12 +441,18 @@ function TaskRow({
|
||||
? v3AgentTaskPath(organization, project, environment, item.slug)
|
||||
: item.kind === "SCHEDULED"
|
||||
? v3ScheduledTaskPath(organization, project, environment, item.slug)
|
||||
: v3StandardTaskPath(organization, project, environment, item.slug);
|
||||
: item.kind === "WEBHOOK"
|
||||
? v3WebhookTaskPath(organization, project, environment, item.slug)
|
||||
: v3StandardTaskPath(organization, project, environment, item.slug);
|
||||
|
||||
// A webhook task runs from a verified inbound event, not a hand-built
|
||||
// payload, so it has no Test page.
|
||||
const testPath =
|
||||
item.kind === "AGENT"
|
||||
? v3PlaygroundAgentPath(organization, project, environment, item.slug)
|
||||
: v3TestTaskPath(organization, project, environment, { taskIdentifier: item.slug });
|
||||
item.kind === "WEBHOOK"
|
||||
? undefined
|
||||
: item.kind === "AGENT"
|
||||
? v3PlaygroundAgentPath(organization, project, environment, item.slug)
|
||||
: v3TestTaskPath(organization, project, environment, { taskIdentifier: item.slug });
|
||||
|
||||
const runsPath = v3RunsPath(organization, project, environment, { tasks: [item.slug] });
|
||||
|
||||
@@ -461,7 +471,13 @@ function TaskRow({
|
||||
<TableCell to={rowPath}>
|
||||
<div className="flex items-center gap-2">
|
||||
<span>
|
||||
{item.kind === "AGENT" ? "Agent" : item.kind === "SCHEDULED" ? "Scheduled" : "Standard"}
|
||||
{item.kind === "AGENT"
|
||||
? "Agent"
|
||||
: item.kind === "SCHEDULED"
|
||||
? "Scheduled"
|
||||
: item.kind === "WEBHOOK"
|
||||
? "Webhook"
|
||||
: "Standard"}
|
||||
</span>
|
||||
{item.kind === "AGENT" && item.agentType && (
|
||||
<Badge variant="extra-small">{formatAgentType(item.agentType)}</Badge>
|
||||
@@ -516,23 +532,27 @@ function TaskRow({
|
||||
title="View runs"
|
||||
leadingIconClassName="-mx-1 text-runs"
|
||||
/>
|
||||
<PopoverMenuItem
|
||||
icon={BeakerIcon}
|
||||
to={testPath}
|
||||
title="Test"
|
||||
leadingIconClassName="-mx-1 text-tests"
|
||||
/>
|
||||
{testPath && (
|
||||
<PopoverMenuItem
|
||||
icon={BeakerIcon}
|
||||
to={testPath}
|
||||
title="Test"
|
||||
leadingIconClassName="-mx-1 text-tests"
|
||||
/>
|
||||
)}
|
||||
</>
|
||||
}
|
||||
hiddenButtons={
|
||||
<LinkButton
|
||||
variant="minimal/small"
|
||||
LeadingIcon={BeakerIcon}
|
||||
leadingIconClassName="-mx-2.5 text-tests"
|
||||
to={testPath}
|
||||
>
|
||||
<span className="text-text-bright">Test</span>
|
||||
</LinkButton>
|
||||
testPath ? (
|
||||
<LinkButton
|
||||
variant="minimal/small"
|
||||
LeadingIcon={BeakerIcon}
|
||||
leadingIconClassName="-mx-2.5 text-tests"
|
||||
to={testPath}
|
||||
>
|
||||
<span className="text-text-bright">Test</span>
|
||||
</LinkButton>
|
||||
) : undefined
|
||||
}
|
||||
/>
|
||||
</TableRow>
|
||||
@@ -777,6 +797,16 @@ function TaskTypeBreakdown() {
|
||||
you need.
|
||||
</Paragraph>
|
||||
</div>
|
||||
<div>
|
||||
<div className="flex items-center gap-1.5">
|
||||
<WebhookIcon className="size-4.5 shrink-0 text-webhooks" />
|
||||
<Paragraph variant="small/bright">Webhook task</Paragraph>
|
||||
</div>
|
||||
<Paragraph variant="small" className="mt-1">
|
||||
Runs from a verified inbound event sent to a hosted endpoint. Each delivery is logged, and
|
||||
a successful one starts a run.
|
||||
</Paragraph>
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
+34
-10
@@ -28,6 +28,8 @@ export function AIPayloadTabContent({
|
||||
placeholder,
|
||||
examplePromptsOverride,
|
||||
isAgent = false,
|
||||
payloadKind,
|
||||
providerSource,
|
||||
showExamplePromptsHeader = true,
|
||||
}: {
|
||||
onPayloadGenerated: (payload: string) => void;
|
||||
@@ -38,6 +40,8 @@ export function AIPayloadTabContent({
|
||||
placeholder?: string;
|
||||
examplePromptsOverride?: string[];
|
||||
isAgent?: boolean;
|
||||
payloadKind?: "standard" | "agent" | "webhook";
|
||||
providerSource?: string;
|
||||
showExamplePromptsHeader?: boolean;
|
||||
}) {
|
||||
const [prompt, setPrompt] = useState("");
|
||||
@@ -79,6 +83,12 @@ export function AIPayloadTabContent({
|
||||
formData.append("prompt", queryPrompt);
|
||||
formData.append("taskIdentifier", taskIdentifier);
|
||||
formData.append("isAgent", isAgent ? "true" : "false");
|
||||
if (payloadKind) {
|
||||
formData.append("payloadKind", payloadKind);
|
||||
}
|
||||
if (providerSource) {
|
||||
formData.append("providerSource", providerSource);
|
||||
}
|
||||
if (payloadSchema) {
|
||||
formData.append("payloadSchema", JSON.stringify(payloadSchema));
|
||||
}
|
||||
@@ -150,7 +160,15 @@ export function AIPayloadTabContent({
|
||||
setIsLoading(false);
|
||||
}
|
||||
},
|
||||
[resourcePath, taskIdentifier, payloadSchema, getCurrentPayload, isAgent]
|
||||
[
|
||||
resourcePath,
|
||||
taskIdentifier,
|
||||
payloadSchema,
|
||||
getCurrentPayload,
|
||||
isAgent,
|
||||
payloadKind,
|
||||
providerSource,
|
||||
]
|
||||
);
|
||||
|
||||
const processStreamEvent = useCallback(
|
||||
@@ -200,17 +218,23 @@ export function AIPayloadTabContent({
|
||||
|
||||
const examplePrompts =
|
||||
examplePromptsOverride ??
|
||||
(payloadSchema
|
||||
(payloadKind === "webhook"
|
||||
? [
|
||||
"Generate a valid payload",
|
||||
"Generate a payload with edge cases",
|
||||
"Generate a minimal payload with only required fields",
|
||||
"Generate a realistic event body",
|
||||
"Generate a failure/refund style event",
|
||||
"Generate an event with nested data",
|
||||
]
|
||||
: [
|
||||
"Generate a simple JSON payload",
|
||||
"Generate a payload with nested objects",
|
||||
"Generate a payload with an array of items",
|
||||
]);
|
||||
: payloadSchema
|
||||
? [
|
||||
"Generate a valid payload",
|
||||
"Generate a payload with edge cases",
|
||||
"Generate a minimal payload with only required fields",
|
||||
]
|
||||
: [
|
||||
"Generate a simple JSON payload",
|
||||
"Generate a payload with nested objects",
|
||||
"Generate a payload with an array of items",
|
||||
]);
|
||||
|
||||
return (
|
||||
<div className="space-y-0">
|
||||
|
||||
+15
-2
@@ -8,7 +8,12 @@ import {
|
||||
} from "@heroicons/react/20/solid";
|
||||
import { DialogClose, DialogDescription } from "@radix-ui/react-dialog";
|
||||
import { Form, useActionData, useFetcher, useParams, useSubmit } from "@remix-run/react";
|
||||
import { type ActionFunction, type LoaderFunctionArgs, json } from "@remix-run/server-runtime";
|
||||
import {
|
||||
type ActionFunction,
|
||||
type LoaderFunctionArgs,
|
||||
json,
|
||||
redirect,
|
||||
} from "@remix-run/server-runtime";
|
||||
import { MachinePresetName } from "@trigger.dev/core/v3";
|
||||
import { AnimatePresence, motion } from "framer-motion";
|
||||
import { useCallback, useEffect, useMemo, useRef, useState } from "react";
|
||||
@@ -132,8 +137,16 @@ export const loader = async ({ request, params }: LoaderFunctionArgs) => {
|
||||
}),
|
||||
]);
|
||||
|
||||
if (result.foundTask && result.triggerSource === "WEBHOOK") {
|
||||
throw redirect(
|
||||
`/orgs/${organizationSlug}/projects/${projectParam}/env/${envParam}/webhooks/${taskParam}?tab=console`
|
||||
);
|
||||
}
|
||||
|
||||
return typedjson({ ...result, regions: regionsResult.regions });
|
||||
} catch (error) {
|
||||
if (error instanceof Response) throw error;
|
||||
|
||||
logger.error("Failed to load test page", {
|
||||
taskParam,
|
||||
error: error instanceof Error ? error.message : error,
|
||||
@@ -303,7 +316,7 @@ export default function Page() {
|
||||
}
|
||||
}, [params.organizationSlug, params.projectParam, params.envParam]);
|
||||
|
||||
const defaultTaskQueue = result.queue;
|
||||
const defaultTaskQueue = "queue" in result ? result.queue : undefined;
|
||||
const queues = useMemo(() => {
|
||||
const customQueues = queueFetcher.data?.queues ?? [];
|
||||
|
||||
|
||||
+874
@@ -0,0 +1,874 @@
|
||||
import { BookOpenIcon } from "@heroicons/react/24/solid";
|
||||
import {
|
||||
Link,
|
||||
type MetaFunction,
|
||||
useNavigation,
|
||||
useRevalidator,
|
||||
useSearchParams,
|
||||
} from "@remix-run/react";
|
||||
import { type LoaderFunctionArgs } from "@remix-run/server-runtime";
|
||||
import { type ReactNode, Suspense, useMemo, useState } from "react";
|
||||
import { TypedAwait, typeddefer, useTypedLoaderData } from "remix-typedjson";
|
||||
import { z } from "zod";
|
||||
import { WebhookIcon } from "~/assets/icons/WebhookIcon";
|
||||
import { PageBody } from "~/components/layout/AppLayout";
|
||||
import { DirectionSchema, ListPagination } from "~/components/ListPagination";
|
||||
import { Button, LinkButton } from "~/components/primitives/Buttons";
|
||||
import { Card } from "~/components/primitives/charts/Card";
|
||||
import { Chart, type ChartConfig } from "~/components/primitives/charts/ChartCompound";
|
||||
import { CopyableText } from "~/components/primitives/CopyableText";
|
||||
import { DateTime } from "~/components/primitives/DateTime";
|
||||
import { Header2 } from "~/components/primitives/Headers";
|
||||
import { NavBar, PageAccessories, PageTitle } from "~/components/primitives/PageHeader";
|
||||
import * as Property from "~/components/primitives/PropertyTable";
|
||||
import {
|
||||
ResizableHandle,
|
||||
ResizablePanel,
|
||||
ResizablePanelGroup,
|
||||
} from "~/components/primitives/Resizable";
|
||||
import { PulsingDot } from "~/components/primitives/PulsingDot";
|
||||
import { Spinner } from "~/components/primitives/Spinner";
|
||||
import { TabButton, TabContainer } from "~/components/primitives/Tabs";
|
||||
import { TimeFilter, timeFilterFromTo } from "~/components/runs/v3/SharedFilters";
|
||||
import { TaskRunsTable } from "~/components/runs/v3/TaskRunsTable";
|
||||
import { DeliveriesTable } from "~/components/webhookDeliveries/v1/DeliveriesTable";
|
||||
import { DeliveryStatusBadge } from "~/components/webhookDeliveries/v1/DeliveryStatus";
|
||||
import { EndpointsTable } from "~/components/webhookEndpoints/v1/EndpointsTable";
|
||||
import { WebhookComposer } from "~/components/webhookConsole/WebhookComposer";
|
||||
import { AIChatIcon } from "~/assets/icons/AIChatIcon";
|
||||
import { RunsIcon } from "~/assets/icons/RunsIcon";
|
||||
import { $replica } from "~/db.server";
|
||||
import { useEnvironment } from "~/hooks/useEnvironment";
|
||||
import { useOrganization } from "~/hooks/useOrganizations";
|
||||
import { useProject } from "~/hooks/useProject";
|
||||
import { findProjectBySlug } from "~/models/project.server";
|
||||
import { findEnvironmentBySlug } from "~/models/runtimeEnvironment.server";
|
||||
import { NextRunListPresenter } from "~/presenters/v3/NextRunListPresenter.server";
|
||||
import {
|
||||
WebhookDetailPresenter,
|
||||
type WebhookActivity,
|
||||
type WebhookDeliveriesList,
|
||||
type WebhookDetail,
|
||||
} from "~/presenters/v3/WebhookDetailPresenter.server";
|
||||
import { clickhouseFactory } from "~/services/clickhouse/clickhouseFactoryInstance.server";
|
||||
import { requireUser } from "~/services/session.server";
|
||||
import { FEATURE_FLAG } from "~/v3/featureFlags";
|
||||
import { flag } from "~/v3/featureFlags.server";
|
||||
import {
|
||||
docsPath,
|
||||
EnvironmentParamSchema,
|
||||
v3EnvironmentPath,
|
||||
v3WebhookDeliveryPath,
|
||||
} from "~/utils/pathBuilder";
|
||||
import { parseFiniteInt } from "~/utils/searchParams";
|
||||
import { useDeliveriesLiveReload } from "~/components/webhookDeliveries/v1/useDeliveriesLiveReload";
|
||||
|
||||
export const meta: MetaFunction<typeof loader> = ({ data }) => {
|
||||
const slug = (data as { webhook?: WebhookDetail | null } | undefined)?.webhook?.slug;
|
||||
return [{ title: slug ? `${slug} | Webhooks | Trigger.dev` : "Webhook | Trigger.dev" }];
|
||||
};
|
||||
|
||||
const WebhookParamSchema = EnvironmentParamSchema.extend({
|
||||
webhookParam: z.string(),
|
||||
});
|
||||
|
||||
export const loader = async ({ request, params }: LoaderFunctionArgs) => {
|
||||
const user = await requireUser(request);
|
||||
const userId = user.id;
|
||||
const { organizationSlug, projectParam, envParam, webhookParam } =
|
||||
WebhookParamSchema.parse(params);
|
||||
|
||||
const project = await findProjectBySlug(organizationSlug, projectParam, userId);
|
||||
if (!project) {
|
||||
throw new Response("Project not found", { status: 404 });
|
||||
}
|
||||
|
||||
const environment = await findEnvironmentBySlug(project.id, envParam, userId);
|
||||
if (!environment) {
|
||||
throw new Response("Environment not found", { status: 404 });
|
||||
}
|
||||
|
||||
if (!user.admin && !user.isImpersonating) {
|
||||
const org = await $replica.organization.findFirst({
|
||||
where: { id: project.organizationId },
|
||||
select: { featureFlags: true },
|
||||
});
|
||||
const enabled = await flag({
|
||||
key: FEATURE_FLAG.hasWebhooksAccess,
|
||||
defaultValue: false,
|
||||
overrides: (org?.featureFlags as Record<string, unknown>) ?? {},
|
||||
});
|
||||
if (!enabled) throw new Response("Not found", { status: 404 });
|
||||
}
|
||||
|
||||
const url = new URL(request.url);
|
||||
const periodParam = url.searchParams.get("period") ?? undefined;
|
||||
const from = parseFiniteInt(url.searchParams.get("from"));
|
||||
const to = parseFiniteInt(url.searchParams.get("to"));
|
||||
const hasExplicitWindow = Boolean(periodParam || from || to);
|
||||
const period = periodParam ?? (hasExplicitWindow ? undefined : "7d");
|
||||
const deliveriesCursor = url.searchParams.get("deliveriesCursor") ?? undefined;
|
||||
const deliveriesDirectionRaw = url.searchParams.get("deliveriesDirection") ?? undefined;
|
||||
const deliveriesDirection = deliveriesDirectionRaw
|
||||
? DirectionSchema.parse(deliveriesDirectionRaw)
|
||||
: undefined;
|
||||
const runsCursor = url.searchParams.get("runsCursor") ?? undefined;
|
||||
const runsDirectionRaw = url.searchParams.get("runsDirection") ?? undefined;
|
||||
const runsDirection = runsDirectionRaw ? DirectionSchema.parse(runsDirectionRaw) : undefined;
|
||||
|
||||
const clickhouse = await clickhouseFactory.getClickhouseForOrganization(
|
||||
project.organizationId,
|
||||
"standard"
|
||||
);
|
||||
|
||||
const presenter = new WebhookDetailPresenter($replica, clickhouse);
|
||||
const webhook = await presenter.findWebhook({
|
||||
environmentId: environment.id,
|
||||
environmentType: environment.type,
|
||||
webhookSlug: webhookParam,
|
||||
});
|
||||
|
||||
if (!webhook) {
|
||||
throw new Response("Webhook not found", { status: 404 });
|
||||
}
|
||||
|
||||
const time = timeFilterFromTo({ period, from, to, defaultPeriod: "7d" });
|
||||
|
||||
const runActivity = presenter
|
||||
.getRunActivity({
|
||||
organizationId: project.organizationId,
|
||||
projectId: project.id,
|
||||
environmentId: environment.id,
|
||||
webhookSlug: webhook.slug,
|
||||
from: time.from,
|
||||
to: time.to,
|
||||
})
|
||||
.catch(() => ({ data: [], statuses: [] }) satisfies WebhookActivity);
|
||||
|
||||
const deliveryActivity = presenter
|
||||
.getDeliveryActivity({
|
||||
organizationId: project.organizationId,
|
||||
projectId: project.id,
|
||||
environmentId: environment.id,
|
||||
webhookEndpointId: webhook.endpoint.id,
|
||||
from: time.from,
|
||||
to: time.to,
|
||||
})
|
||||
.catch(() => ({ data: [], statuses: [] }) satisfies WebhookActivity);
|
||||
|
||||
const runList = new NextRunListPresenter($replica, clickhouse)
|
||||
.call(project.organizationId, environment.id, {
|
||||
userId,
|
||||
projectId: project.id,
|
||||
tasks: [webhook.slug],
|
||||
period,
|
||||
from,
|
||||
to,
|
||||
cursor: runsCursor,
|
||||
direction: runsDirection,
|
||||
})
|
||||
.catch(() => null);
|
||||
|
||||
const deliveriesList = presenter
|
||||
.listDeliveries({
|
||||
organizationId: project.organizationId,
|
||||
projectId: project.id,
|
||||
environmentId: environment.id,
|
||||
webhookEndpointId: webhook.endpoint.id,
|
||||
period,
|
||||
from,
|
||||
to,
|
||||
hasExplicitWindow,
|
||||
cursor: deliveriesCursor,
|
||||
direction: deliveriesDirection,
|
||||
})
|
||||
.catch(() => null);
|
||||
|
||||
const endpointsList = presenter
|
||||
.listEndpoints({
|
||||
organizationId: project.organizationId,
|
||||
projectId: project.id,
|
||||
environmentId: environment.id,
|
||||
handlerWebhookId: webhook.slug,
|
||||
})
|
||||
.catch(() => [] as Awaited<ReturnType<typeof presenter.listEndpoints>>);
|
||||
|
||||
const composerEndpoints = presenter
|
||||
.listComposerEndpoints({ environmentId: environment.id, handlerWebhookId: webhook.slug })
|
||||
.catch(() => [] as Awaited<ReturnType<typeof presenter.listComposerEndpoints>>);
|
||||
|
||||
return typeddefer({
|
||||
webhook,
|
||||
runActivity,
|
||||
deliveryActivity,
|
||||
runList,
|
||||
deliveriesList,
|
||||
endpointsList,
|
||||
composerEndpoints,
|
||||
});
|
||||
};
|
||||
|
||||
type WebhookTab = "runs" | "deliveries" | "endpoints" | "console";
|
||||
|
||||
export default function Page() {
|
||||
const {
|
||||
webhook,
|
||||
runActivity,
|
||||
deliveryActivity,
|
||||
runList,
|
||||
deliveriesList,
|
||||
endpointsList,
|
||||
composerEndpoints,
|
||||
} = useTypedLoaderData<typeof loader>();
|
||||
const organization = useOrganization();
|
||||
const project = useProject();
|
||||
const environment = useEnvironment();
|
||||
|
||||
const tasksPath = v3EnvironmentPath(organization, project, environment);
|
||||
|
||||
const [searchParams] = useSearchParams();
|
||||
const [tab, setTab] = useState<WebhookTab>(() => {
|
||||
const requested = searchParams.get("tab");
|
||||
return requested === "console" || requested === "runs" || requested === "endpoints"
|
||||
? requested
|
||||
: "deliveries";
|
||||
});
|
||||
const tabLabel =
|
||||
tab === "deliveries"
|
||||
? "Deliveries"
|
||||
: tab === "runs"
|
||||
? "Runs"
|
||||
: tab === "console"
|
||||
? "Console"
|
||||
: "Endpoints";
|
||||
|
||||
return (
|
||||
<>
|
||||
<NavBar>
|
||||
<PageTitle
|
||||
backButton={{ to: tasksPath, text: "Tasks" }}
|
||||
title={
|
||||
<span className="flex items-center gap-1">
|
||||
<WebhookIcon className="size-4.5 text-webhooks" />
|
||||
<span>{webhook.slug}</span>
|
||||
</span>
|
||||
}
|
||||
/>
|
||||
<PageAccessories>
|
||||
<LinkButton
|
||||
variant="docs/small"
|
||||
LeadingIcon={BookOpenIcon}
|
||||
to={docsPath("webhooks/overview")}
|
||||
>
|
||||
Webhooks docs
|
||||
</LinkButton>
|
||||
</PageAccessories>
|
||||
</NavBar>
|
||||
<PageBody scrollable={false}>
|
||||
<ResizablePanelGroup orientation="horizontal" className="max-h-full">
|
||||
<ResizablePanel id="webhook-main" min="300px">
|
||||
<div className="grid h-full grid-rows-[auto_1fr] overflow-hidden">
|
||||
{/* Top bar: tabs on the left, TimeFilter + pagination on the right.
|
||||
h-10 matches the right-hand sidebar header height. */}
|
||||
<div className="flex h-10 items-end border-b border-grid-dimmed bg-background-bright pl-3 pr-2">
|
||||
<TabContainer className="-mb-px">
|
||||
<TabButton
|
||||
isActive={tab === "deliveries"}
|
||||
layoutId="webhook-page-tabs"
|
||||
onClick={() => setTab("deliveries")}
|
||||
>
|
||||
Deliveries
|
||||
</TabButton>
|
||||
<TabButton
|
||||
isActive={tab === "runs"}
|
||||
layoutId="webhook-page-tabs"
|
||||
onClick={() => setTab("runs")}
|
||||
>
|
||||
Runs
|
||||
</TabButton>
|
||||
<TabButton
|
||||
isActive={tab === "endpoints"}
|
||||
layoutId="webhook-page-tabs"
|
||||
onClick={() => setTab("endpoints")}
|
||||
>
|
||||
Endpoints
|
||||
</TabButton>
|
||||
<TabButton
|
||||
isActive={tab === "console"}
|
||||
layoutId="webhook-page-tabs"
|
||||
onClick={() => setTab("console")}
|
||||
>
|
||||
Console
|
||||
</TabButton>
|
||||
</TabContainer>
|
||||
{tab !== "endpoints" && tab !== "console" && (
|
||||
<div className="ml-auto flex items-center gap-2 self-center">
|
||||
<TimeFilter
|
||||
defaultPeriod="7d"
|
||||
labelName={tabLabel}
|
||||
clearParams={[
|
||||
"deliveriesCursor",
|
||||
"deliveriesDirection",
|
||||
"runsCursor",
|
||||
"runsDirection",
|
||||
]}
|
||||
/>
|
||||
{tab === "deliveries" ? (
|
||||
<Suspense fallback={null}>
|
||||
<TypedAwait resolve={deliveriesList} errorElement={null}>
|
||||
{(list) =>
|
||||
list ? (
|
||||
<ListPagination
|
||||
list={list}
|
||||
cursorParam="deliveriesCursor"
|
||||
directionParam="deliveriesDirection"
|
||||
/>
|
||||
) : null
|
||||
}
|
||||
</TypedAwait>
|
||||
</Suspense>
|
||||
) : (
|
||||
<Suspense fallback={null}>
|
||||
<TypedAwait resolve={runList} errorElement={null}>
|
||||
{(list) =>
|
||||
list ? (
|
||||
<ListPagination
|
||||
list={list}
|
||||
cursorParam="runsCursor"
|
||||
directionParam="runsDirection"
|
||||
/>
|
||||
) : null
|
||||
}
|
||||
</TypedAwait>
|
||||
</Suspense>
|
||||
)}
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
|
||||
{tab === "endpoints" ? (
|
||||
// Endpoints aren't a time series, so no activity chart or time filter.
|
||||
<div className="h-full overflow-y-auto scrollbar-thin scrollbar-track-transparent scrollbar-thumb-charcoal-600">
|
||||
<Suspense fallback={<TableLoading />}>
|
||||
<TypedAwait resolve={endpointsList} errorElement={<TableLoading />}>
|
||||
{(endpoints) => (
|
||||
<EndpointsTable endpoints={endpoints} showTopBorder={false} stickyHeader />
|
||||
)}
|
||||
</TypedAwait>
|
||||
</Suspense>
|
||||
</div>
|
||||
) : tab === "console" ? (
|
||||
<div className="h-full overflow-hidden">
|
||||
<Suspense fallback={<TableLoading />}>
|
||||
<TypedAwait resolve={composerEndpoints} errorElement={<TableLoading />}>
|
||||
{(endpoints) =>
|
||||
endpoints.length === 0 ? (
|
||||
<div className="flex h-full items-center justify-center p-4 text-center text-sm text-text-dimmed">
|
||||
This webhook has no synced endpoints to send to yet.
|
||||
</div>
|
||||
) : (
|
||||
<WebhookComposer
|
||||
endpoints={endpoints}
|
||||
organizationSlug={organization.slug}
|
||||
projectSlug={project.slug}
|
||||
environmentSlug={environment.slug}
|
||||
isDevEnvironment={environment.type === "DEVELOPMENT"}
|
||||
environmentLabel={
|
||||
environment.type.charAt(0) + environment.type.slice(1).toLowerCase()
|
||||
}
|
||||
redirectOnSuccess={false}
|
||||
/>
|
||||
)
|
||||
}
|
||||
</TypedAwait>
|
||||
</Suspense>
|
||||
</div>
|
||||
) : (
|
||||
<ResizablePanelGroup orientation="vertical" className="max-h-full">
|
||||
{/* Activity chart (one status-bucket chart per tab). */}
|
||||
<ResizablePanel id="webhook-activity" min="220px" default="320px">
|
||||
<div className="flex h-full flex-col overflow-hidden bg-background p-2">
|
||||
<div className="grid min-h-0 flex-1 grid-cols-1 gap-2">
|
||||
<ChartCard title={tabLabel}>
|
||||
{tab === "deliveries" ? (
|
||||
<Suspense fallback={<ActivityChartSkeleton />}>
|
||||
<TypedAwait
|
||||
resolve={deliveryActivity}
|
||||
errorElement={<ActivityChartSkeleton />}
|
||||
>
|
||||
{(result) => <ActivityChart activity={result} />}
|
||||
</TypedAwait>
|
||||
</Suspense>
|
||||
) : (
|
||||
<Suspense fallback={<ActivityChartSkeleton />}>
|
||||
<TypedAwait
|
||||
resolve={runActivity}
|
||||
errorElement={<ActivityChartSkeleton />}
|
||||
>
|
||||
{(result) => <ActivityChart activity={result} />}
|
||||
</TypedAwait>
|
||||
</Suspense>
|
||||
)}
|
||||
</ChartCard>
|
||||
</div>
|
||||
</div>
|
||||
</ResizablePanel>
|
||||
|
||||
<ResizableHandle id="webhook-activity-handle" />
|
||||
|
||||
{/* Table */}
|
||||
<ResizablePanel id="webhook-content" min="160px">
|
||||
<WebhookContentArea
|
||||
tab={tab}
|
||||
deliveriesList={deliveriesList}
|
||||
runList={runList}
|
||||
webhookEndpointId={webhook.endpoint.id}
|
||||
/>
|
||||
</ResizablePanel>
|
||||
</ResizablePanelGroup>
|
||||
)}
|
||||
</div>
|
||||
</ResizablePanel>
|
||||
|
||||
<ResizableHandle id="webhook-detail-handle" />
|
||||
<ResizablePanel
|
||||
id="webhook-detail"
|
||||
min="280px"
|
||||
default="380px"
|
||||
max="500px"
|
||||
isStaticAtRest
|
||||
>
|
||||
{tab === "console" ? (
|
||||
<ConsoleLiveFeed
|
||||
deliveriesList={deliveriesList}
|
||||
webhookEndpointId={webhook.endpoint.id}
|
||||
/>
|
||||
) : (
|
||||
<WebhookDetailSidebar webhook={webhook} onViewEndpoints={() => setTab("endpoints")} />
|
||||
)}
|
||||
</ResizablePanel>
|
||||
</ResizablePanelGroup>
|
||||
</PageBody>
|
||||
</>
|
||||
);
|
||||
}
|
||||
|
||||
type LoaderData = ReturnType<typeof useTypedLoaderData<typeof loader>>;
|
||||
|
||||
function WebhookContentArea({
|
||||
tab,
|
||||
deliveriesList,
|
||||
runList,
|
||||
webhookEndpointId,
|
||||
}: {
|
||||
tab: WebhookTab;
|
||||
webhookEndpointId: string;
|
||||
} & Pick<LoaderData, "deliveriesList" | "runList">) {
|
||||
return (
|
||||
<div className="h-full overflow-hidden">
|
||||
{tab === "deliveries" ? (
|
||||
<Suspense fallback={<TableLoading />}>
|
||||
<TypedAwait resolve={deliveriesList} errorElement={<TableLoading />}>
|
||||
{(list) =>
|
||||
list ? (
|
||||
<div className="h-full overflow-y-auto scrollbar-thin scrollbar-track-transparent scrollbar-thumb-charcoal-600">
|
||||
<LiveDeliveriesTable list={list} webhookEndpointId={webhookEndpointId} />
|
||||
</div>
|
||||
) : (
|
||||
<TableLoading />
|
||||
)
|
||||
}
|
||||
</TypedAwait>
|
||||
</Suspense>
|
||||
) : (
|
||||
<Suspense fallback={<TableLoading />}>
|
||||
<TypedAwait resolve={runList} errorElement={<TableLoading />}>
|
||||
{(list) =>
|
||||
list ? (
|
||||
<div className="h-full overflow-y-auto scrollbar-thin scrollbar-track-transparent scrollbar-thumb-charcoal-600">
|
||||
<TaskRunsTable
|
||||
total={list.runs.length}
|
||||
hasFilters={list.hasFilters}
|
||||
filters={list.filters}
|
||||
runs={list.runs}
|
||||
variant="dimmed"
|
||||
showTopBorder={false}
|
||||
stickyHeader
|
||||
/>
|
||||
</div>
|
||||
) : (
|
||||
<TableLoading />
|
||||
)
|
||||
}
|
||||
</TypedAwait>
|
||||
</Suspense>
|
||||
)}
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
function LiveDeliveriesTable({
|
||||
list,
|
||||
webhookEndpointId,
|
||||
}: {
|
||||
list: WebhookDeliveriesList;
|
||||
webhookEndpointId: string;
|
||||
}) {
|
||||
const organization = useOrganization();
|
||||
const project = useProject();
|
||||
const environment = useEnvironment();
|
||||
const navigation = useNavigation();
|
||||
const revalidator = useRevalidator();
|
||||
const [searchParams, setSearchParams] = useSearchParams();
|
||||
|
||||
const { visibleDeliveries, showNewDeliveriesBanner, newDeliveriesCount, dismissNewDeliveries } =
|
||||
useDeliveriesLiveReload({
|
||||
deliveries: list.deliveries,
|
||||
isLoading: navigation.state !== "idle",
|
||||
webhookEndpointId,
|
||||
organizationSlug: organization.slug,
|
||||
projectSlug: project.slug,
|
||||
environmentSlug: environment.slug,
|
||||
});
|
||||
|
||||
const onClickShowNewDeliveries = () => {
|
||||
dismissNewDeliveries();
|
||||
if (searchParams.has("deliveriesCursor") || searchParams.has("deliveriesDirection")) {
|
||||
setSearchParams((prev) => {
|
||||
prev.delete("deliveriesCursor");
|
||||
prev.delete("deliveriesDirection");
|
||||
return prev;
|
||||
});
|
||||
return;
|
||||
}
|
||||
revalidator.revalidate();
|
||||
};
|
||||
|
||||
return (
|
||||
<>
|
||||
{showNewDeliveriesBanner ? (
|
||||
<div className="flex justify-end px-2 py-1.5">
|
||||
<span className="flex duration-150 animate-in fade-in-0">
|
||||
<Button
|
||||
variant="secondary/small"
|
||||
className="text-text-bright"
|
||||
onClick={onClickShowNewDeliveries}
|
||||
LeadingIcon={<PulsingDot className="h-2 w-2" />}
|
||||
tooltip="Refresh to see new deliveries"
|
||||
aria-label="New deliveries received. Refresh to see them."
|
||||
>
|
||||
{newDeliveriesCount >= 100
|
||||
? "99+ new deliveries"
|
||||
: `${newDeliveriesCount} new ${
|
||||
newDeliveriesCount === 1 ? "delivery" : "deliveries"
|
||||
}`}
|
||||
</Button>
|
||||
</span>
|
||||
</div>
|
||||
) : null}
|
||||
<DeliveriesTable
|
||||
deliveries={visibleDeliveries}
|
||||
hasFilters={list.hasFilters}
|
||||
showTopBorder={false}
|
||||
stickyHeader
|
||||
/>
|
||||
</>
|
||||
);
|
||||
}
|
||||
|
||||
function ConsoleLiveFeed({
|
||||
deliveriesList,
|
||||
webhookEndpointId,
|
||||
}: {
|
||||
webhookEndpointId: string;
|
||||
} & Pick<LoaderData, "deliveriesList">) {
|
||||
return (
|
||||
<div className="grid h-full grid-rows-[auto_1fr] overflow-hidden bg-background-bright">
|
||||
<div className="flex h-10 items-center justify-between gap-2 border-b border-grid-dimmed pl-3 pr-2">
|
||||
<Header2 className="truncate">Live deliveries</Header2>
|
||||
<span className="flex items-center gap-1.5 text-xs text-text-dimmed">
|
||||
<PulsingDot className="h-2 w-2" />
|
||||
Live
|
||||
</span>
|
||||
</div>
|
||||
<div className="overflow-y-auto scrollbar-thin scrollbar-track-transparent scrollbar-thumb-charcoal-600">
|
||||
<Suspense fallback={<TableLoading />}>
|
||||
<TypedAwait resolve={deliveriesList} errorElement={<TableLoading />}>
|
||||
{(list) =>
|
||||
list ? (
|
||||
<ConsoleFeedList list={list} webhookEndpointId={webhookEndpointId} />
|
||||
) : (
|
||||
<TableLoading />
|
||||
)
|
||||
}
|
||||
</TypedAwait>
|
||||
</Suspense>
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
function ConsoleFeedList({
|
||||
list,
|
||||
webhookEndpointId,
|
||||
}: {
|
||||
list: WebhookDeliveriesList;
|
||||
webhookEndpointId: string;
|
||||
}) {
|
||||
const organization = useOrganization();
|
||||
const project = useProject();
|
||||
const environment = useEnvironment();
|
||||
const navigation = useNavigation();
|
||||
const revalidator = useRevalidator();
|
||||
|
||||
const { visibleDeliveries, showNewDeliveriesBanner, newDeliveriesCount, dismissNewDeliveries } =
|
||||
useDeliveriesLiveReload({
|
||||
deliveries: list.deliveries,
|
||||
isLoading: navigation.state !== "idle",
|
||||
webhookEndpointId,
|
||||
organizationSlug: organization.slug,
|
||||
projectSlug: project.slug,
|
||||
environmentSlug: environment.slug,
|
||||
});
|
||||
|
||||
const onShowNew = () => {
|
||||
dismissNewDeliveries();
|
||||
revalidator.revalidate();
|
||||
};
|
||||
|
||||
if (visibleDeliveries.length === 0) {
|
||||
return (
|
||||
<p className="px-3 py-8 text-center text-sm text-text-dimmed">
|
||||
No deliveries yet. Send an event to watch it arrive here.
|
||||
</p>
|
||||
);
|
||||
}
|
||||
|
||||
return (
|
||||
<div className="flex flex-col">
|
||||
{showNewDeliveriesBanner ? (
|
||||
<button
|
||||
type="button"
|
||||
onClick={onShowNew}
|
||||
className="flex items-center justify-center gap-1.5 border-b border-grid-dimmed bg-charcoal-800 py-1.5 text-xs text-text-bright hover:bg-charcoal-700"
|
||||
>
|
||||
<PulsingDot className="h-2 w-2" />
|
||||
{newDeliveriesCount >= 100 ? "99+" : newDeliveriesCount} new
|
||||
</button>
|
||||
) : null}
|
||||
{visibleDeliveries.map((delivery) => (
|
||||
<Link
|
||||
key={delivery.id}
|
||||
to={v3WebhookDeliveryPath(organization, project, environment, delivery.friendlyId)}
|
||||
className="flex flex-col gap-1 border-b border-grid-dimmed px-3 py-2 hover:bg-charcoal-800"
|
||||
>
|
||||
<div className="flex items-center justify-between gap-2">
|
||||
<span className="flex items-center gap-1.5">
|
||||
<span className="font-mono text-xs text-text-bright">{delivery.friendlyId}</span>
|
||||
{delivery.isTest ? (
|
||||
<span className="rounded-sm bg-charcoal-700 px-1 py-0.5 text-xxs font-semibold uppercase tracking-wide text-text-dimmed">
|
||||
Test
|
||||
</span>
|
||||
) : null}
|
||||
</span>
|
||||
<DeliveryStatusBadge
|
||||
status={delivery.status}
|
||||
className="shrink-0 text-xs text-text-dimmed"
|
||||
/>
|
||||
</div>
|
||||
{delivery.session ? (
|
||||
<span className="flex items-center gap-1 text-xxs text-text-dimmed">
|
||||
<AIChatIcon className="size-3.5 text-sessions" />
|
||||
<span className="font-mono">{delivery.session.friendlyId}</span>
|
||||
</span>
|
||||
) : delivery.run ? (
|
||||
<span className="flex items-center gap-1 text-xxs text-text-dimmed">
|
||||
<RunsIcon className="size-3.5 text-runs" />
|
||||
<span className="font-mono">{delivery.run.friendlyId}</span>
|
||||
</span>
|
||||
) : null}
|
||||
</Link>
|
||||
))}
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
function WebhookDetailSidebar({
|
||||
webhook,
|
||||
onViewEndpoints,
|
||||
}: {
|
||||
webhook: WebhookDetail;
|
||||
onViewEndpoints: () => void;
|
||||
}) {
|
||||
return (
|
||||
<div className="grid h-full grid-rows-[auto_1fr] overflow-hidden bg-background-bright">
|
||||
<div className="flex items-center gap-2 border-b border-grid-dimmed py-2 pl-3 pr-2">
|
||||
<Header2 className="flex min-w-0 flex-1 items-center gap-1.5">
|
||||
<WebhookIcon className="size-4.5 shrink-0 text-webhooks" />
|
||||
<span className="truncate">{webhook.slug}</span>
|
||||
</Header2>
|
||||
</div>
|
||||
<div className="overflow-y-auto px-3 py-3 scrollbar-thin scrollbar-track-transparent scrollbar-thumb-charcoal-600">
|
||||
<Property.Table>
|
||||
<Property.Item>
|
||||
<Property.Label>Source</Property.Label>
|
||||
<Property.Value>
|
||||
<span className="font-mono text-sm">{webhook.source}</span>
|
||||
</Property.Value>
|
||||
</Property.Item>
|
||||
<Property.Item>
|
||||
<Property.Label>Endpoints</Property.Label>
|
||||
<Property.Value>
|
||||
{/* The connect flow (ingress URL, secret, provider setup) lives on each
|
||||
endpoint, so this handler view points there instead of holding it. */}
|
||||
<Button variant="secondary/small" onClick={onViewEndpoints}>
|
||||
View endpoints
|
||||
</Button>
|
||||
</Property.Value>
|
||||
</Property.Item>
|
||||
<Property.Item>
|
||||
<Property.Label>File path</Property.Label>
|
||||
<Property.Value>
|
||||
<CopyableText value={webhook.filePath} />
|
||||
</Property.Value>
|
||||
</Property.Item>
|
||||
<Property.Item>
|
||||
<Property.Label>Created</Property.Label>
|
||||
<Property.Value>
|
||||
<DateTime date={webhook.createdAt} />
|
||||
</Property.Value>
|
||||
</Property.Item>
|
||||
</Property.Table>
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
const STATUS_COLOR: Record<string, string> = {
|
||||
// Run statuses
|
||||
COMPLETED: "#28BF5C",
|
||||
RUNNING: "#3B82F6",
|
||||
FAILED: "#E11D48",
|
||||
CANCELED: "#878C99",
|
||||
// Delivery statuses
|
||||
SUCCEEDED: "#28BF5C",
|
||||
PROCESSING: "#3B82F6",
|
||||
PENDING: "#878C99",
|
||||
};
|
||||
|
||||
function ActivityChart({ activity }: { activity: WebhookActivity }) {
|
||||
const chartConfig: ChartConfig = useMemo(() => {
|
||||
const cfg: ChartConfig = {};
|
||||
for (const status of activity.statuses) {
|
||||
cfg[status] = {
|
||||
label: status.charAt(0) + status.slice(1).toLowerCase(),
|
||||
color: STATUS_COLOR[status] ?? "#9CA3AF",
|
||||
};
|
||||
}
|
||||
return cfg;
|
||||
}, [activity.statuses]);
|
||||
|
||||
const { xAxisFormatter, xAxisTicks, tooltipLabelFormatter } = useMemo(
|
||||
() => buildTimeAxis(activity.data),
|
||||
[activity.data]
|
||||
);
|
||||
|
||||
return (
|
||||
<Chart.Root
|
||||
config={chartConfig}
|
||||
data={activity.data}
|
||||
dataKey="bucket"
|
||||
series={activity.statuses}
|
||||
fillContainer
|
||||
>
|
||||
<Chart.Bar
|
||||
stackId="status"
|
||||
barRadius={0}
|
||||
xAxisProps={{
|
||||
tickFormatter: xAxisFormatter,
|
||||
...(xAxisTicks ? { ticks: xAxisTicks, interval: 0 } : {}),
|
||||
}}
|
||||
tooltipLabelFormatter={tooltipLabelFormatter}
|
||||
/>
|
||||
</Chart.Root>
|
||||
);
|
||||
}
|
||||
|
||||
function ActivityChartSkeleton() {
|
||||
return (
|
||||
<div className="flex min-h-0 flex-1 items-end gap-px rounded-sm">
|
||||
{Array.from({ length: 42 }).map((_, i) => (
|
||||
<div key={i} className="h-full flex-1 bg-charcoal-850" />
|
||||
))}
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
function TableLoading() {
|
||||
return (
|
||||
<div className="flex h-full items-center justify-center">
|
||||
<Spinner className="size-6" />
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
function ChartCard({ title, children }: { title: string; children: ReactNode }) {
|
||||
return (
|
||||
<Card className="h-full overflow-hidden px-0 pb-2 pt-3">
|
||||
<Card.Header>{title}</Card.Header>
|
||||
<div className="min-h-0 flex-1 px-2">{children}</div>
|
||||
</Card>
|
||||
);
|
||||
}
|
||||
|
||||
function buildTimeAxis(data: WebhookActivity["data"]) {
|
||||
const range = data.length >= 2 ? data[data.length - 1].bucket - data[0].bucket : 0;
|
||||
const oneDay = 24 * 60 * 60 * 1000;
|
||||
const showTime = range <= oneDay;
|
||||
|
||||
const xAxisFormatter = (value: number) => {
|
||||
const date = new Date(value);
|
||||
return showTime
|
||||
? date.toLocaleTimeString("en-US", {
|
||||
hour: "2-digit",
|
||||
minute: "2-digit",
|
||||
hour12: false,
|
||||
timeZone: "UTC",
|
||||
})
|
||||
: date.toLocaleDateString("en-US", {
|
||||
month: "short",
|
||||
day: "numeric",
|
||||
timeZone: "UTC",
|
||||
});
|
||||
};
|
||||
|
||||
const xAxisTicks = showTime
|
||||
? undefined
|
||||
: data.filter((d) => new Date(d.bucket).getUTCHours() === 0).map((d) => d.bucket);
|
||||
|
||||
const bucketMs = data.length >= 2 ? data[1].bucket - data[0].bucket : 0;
|
||||
const isSubDayBucket = bucketMs > 0 && bucketMs < oneDay;
|
||||
|
||||
const tooltipLabelFormatter = (_label: string, payload: { payload?: { bucket?: number } }[]) => {
|
||||
const ts = payload?.[0]?.payload?.bucket;
|
||||
if (typeof ts !== "number" || !Number.isFinite(ts)) return _label;
|
||||
const date = new Date(ts);
|
||||
return isSubDayBucket
|
||||
? date.toLocaleString("en-US", {
|
||||
month: "short",
|
||||
day: "numeric",
|
||||
hour: "2-digit",
|
||||
minute: "2-digit",
|
||||
hour12: false,
|
||||
timeZone: "UTC",
|
||||
})
|
||||
: date.toLocaleDateString("en-US", {
|
||||
month: "short",
|
||||
day: "numeric",
|
||||
year: "numeric",
|
||||
timeZone: "UTC",
|
||||
});
|
||||
};
|
||||
|
||||
return { xAxisFormatter, xAxisTicks, tooltipLabelFormatter };
|
||||
}
|
||||
+234
@@ -0,0 +1,234 @@
|
||||
import {
|
||||
type MetaFunction,
|
||||
useNavigation,
|
||||
useRevalidator,
|
||||
useSearchParams,
|
||||
} from "@remix-run/react";
|
||||
import { type LoaderFunctionArgs } from "@remix-run/server-runtime";
|
||||
import type { WebhookDeliveryStatus } from "@trigger.dev/database";
|
||||
import { typedjson, useTypedLoaderData } from "remix-typedjson";
|
||||
import { PageBody } from "~/components/layout/AppLayout";
|
||||
import { DirectionSchema, ListPagination } from "~/components/ListPagination";
|
||||
import { Button } from "~/components/primitives/Buttons";
|
||||
import { NavBar, PageTitle } from "~/components/primitives/PageHeader";
|
||||
import { PulsingDot } from "~/components/primitives/PulsingDot";
|
||||
import { DeliveriesTable } from "~/components/webhookDeliveries/v1/DeliveriesTable";
|
||||
import { useDeliveriesLiveReload } from "~/components/webhookDeliveries/v1/useDeliveriesLiveReload";
|
||||
import {
|
||||
type PossibleWebhook,
|
||||
WebhookDeliveryFilters,
|
||||
} from "~/components/webhookDeliveries/v1/WebhookDeliveryFilters";
|
||||
import { useEnvironment } from "~/hooks/useEnvironment";
|
||||
import { useOrganization } from "~/hooks/useOrganizations";
|
||||
import { useProject } from "~/hooks/useProject";
|
||||
import { type WebhookDeliveryListItem } from "~/presenters/v3/WebhookDetailPresenter.server";
|
||||
import { $replica, webhookReplica } from "~/db.server";
|
||||
import { findProjectBySlug } from "~/models/project.server";
|
||||
import { findEnvironmentBySlug } from "~/models/runtimeEnvironment.server";
|
||||
import { WebhookDeliveriesListPresenter } from "~/presenters/v3/WebhookDeliveriesListPresenter.server";
|
||||
import { clickhouseFactory } from "~/services/clickhouse/clickhouseFactoryInstance.server";
|
||||
import { requireUser } from "~/services/session.server";
|
||||
import { EnvironmentParamSchema } from "~/utils/pathBuilder";
|
||||
import { parseFiniteInt } from "~/utils/searchParams";
|
||||
import { FEATURE_FLAG } from "~/v3/featureFlags";
|
||||
import { flag } from "~/v3/featureFlags.server";
|
||||
|
||||
const VALID_DELIVERY_STATUSES = new Set<string>([
|
||||
"PENDING",
|
||||
"PROCESSING",
|
||||
"SUCCEEDED",
|
||||
"FAILED",
|
||||
"FILTERED",
|
||||
]);
|
||||
|
||||
// Accepts repeated `statuses` params or a single CSV value; drops anything that
|
||||
// isn't one of the four WebhookDeliveryStatus values.
|
||||
function parseStatuses(searchParams: URLSearchParams): WebhookDeliveryStatus[] | undefined {
|
||||
const raw = searchParams
|
||||
.getAll("statuses")
|
||||
.flatMap((value) => value.split(","))
|
||||
.map((value) => value.trim())
|
||||
.filter((value) => VALID_DELIVERY_STATUSES.has(value)) as WebhookDeliveryStatus[];
|
||||
|
||||
return raw.length > 0 ? Array.from(new Set(raw)) : undefined;
|
||||
}
|
||||
|
||||
function parseRepeated(searchParams: URLSearchParams, key: string): string[] | undefined {
|
||||
const values = searchParams.getAll(key).filter((value) => value.length > 0);
|
||||
return values.length > 0 ? values : undefined;
|
||||
}
|
||||
|
||||
export const meta: MetaFunction = () => [{ title: "Deliveries | Webhooks | Trigger.dev" }];
|
||||
|
||||
export const loader = async ({ request, params }: LoaderFunctionArgs) => {
|
||||
const user = await requireUser(request);
|
||||
const userId = user.id;
|
||||
const { organizationSlug, projectParam, envParam } = EnvironmentParamSchema.parse(params);
|
||||
|
||||
const project = await findProjectBySlug(organizationSlug, projectParam, userId);
|
||||
if (!project) throw new Response("Project not found", { status: 404 });
|
||||
const environment = await findEnvironmentBySlug(project.id, envParam, userId);
|
||||
if (!environment) throw new Response("Environment not found", { status: 404 });
|
||||
|
||||
// Feature gate: enabled by a global FeatureFlag OR a per-org override; admins/impersonators
|
||||
// always pass. flag() resolves org override -> global -> default(false).
|
||||
if (!user.admin && !user.isImpersonating) {
|
||||
const org = await $replica.organization.findFirst({
|
||||
where: { id: project.organizationId },
|
||||
select: { featureFlags: true },
|
||||
});
|
||||
const enabled = await flag({
|
||||
key: FEATURE_FLAG.hasWebhooksAccess,
|
||||
defaultValue: false,
|
||||
overrides: (org?.featureFlags as Record<string, unknown>) ?? {},
|
||||
});
|
||||
if (!enabled) throw new Response("Not found", { status: 404 });
|
||||
}
|
||||
|
||||
const url = new URL(request.url);
|
||||
const periodParam = url.searchParams.get("period") ?? undefined;
|
||||
const from = parseFiniteInt(url.searchParams.get("from"));
|
||||
const to = parseFiniteInt(url.searchParams.get("to"));
|
||||
const cursor = url.searchParams.get("cursor") ?? undefined;
|
||||
const directionRaw = url.searchParams.get("direction") ?? undefined;
|
||||
const direction = directionRaw ? DirectionSchema.parse(directionRaw) : undefined;
|
||||
|
||||
const statuses = parseStatuses(url.searchParams);
|
||||
const webhooks = parseRepeated(url.searchParams, "webhooks");
|
||||
const deliveryId = url.searchParams.get("deliveryId") ?? undefined;
|
||||
const runId = url.searchParams.get("runId") ?? undefined;
|
||||
const testParam = url.searchParams.get("test");
|
||||
const isTest = testParam === "only" ? true : testParam === "hide" ? false : undefined;
|
||||
|
||||
// Default to the last 7 days when no explicit window is set, matching the TimeFilter default.
|
||||
const hasExplicitWindow = Boolean(periodParam || from || to);
|
||||
const period = periodParam ?? (hasExplicitWindow ? undefined : "7d");
|
||||
|
||||
const hasFilters = Boolean(
|
||||
statuses || webhooks || deliveryId || runId || testParam || hasExplicitWindow
|
||||
);
|
||||
|
||||
// Distinct handler slugs (one row per handlerWebhookId) for the Webhook picker.
|
||||
const endpoints = await webhookReplica.webhookEndpoint.findMany({
|
||||
where: { runtimeEnvironmentId: environment.id },
|
||||
select: { handlerWebhookId: true, source: true },
|
||||
distinct: ["handlerWebhookId"],
|
||||
});
|
||||
const possibleWebhooks: PossibleWebhook[] = endpoints.map((e) => ({
|
||||
slug: e.handlerWebhookId,
|
||||
source: e.source,
|
||||
}));
|
||||
|
||||
const clickhouse = await clickhouseFactory.getClickhouseForOrganization(
|
||||
project.organizationId,
|
||||
"standard"
|
||||
);
|
||||
|
||||
const presenter = new WebhookDeliveriesListPresenter($replica, clickhouse);
|
||||
const list = await presenter
|
||||
.call({
|
||||
organizationId: project.organizationId,
|
||||
projectId: project.id,
|
||||
environmentId: environment.id,
|
||||
webhooks,
|
||||
statuses,
|
||||
deliveryId,
|
||||
runId,
|
||||
isTest,
|
||||
period,
|
||||
from,
|
||||
to,
|
||||
cursor,
|
||||
direction,
|
||||
})
|
||||
.catch(() => ({ deliveries: [], pagination: {} }));
|
||||
|
||||
return typedjson({
|
||||
deliveries: list.deliveries,
|
||||
pagination: list.pagination,
|
||||
possibleWebhooks,
|
||||
hasFilters,
|
||||
});
|
||||
};
|
||||
|
||||
export default function Page() {
|
||||
const { deliveries, pagination, possibleWebhooks, hasFilters } =
|
||||
useTypedLoaderData<typeof loader>();
|
||||
|
||||
const { visibleDeliveries, newDeliveriesButton } = useLiveDeliveries(deliveries);
|
||||
|
||||
return (
|
||||
<>
|
||||
<NavBar>
|
||||
<PageTitle title="Webhook deliveries" />
|
||||
</NavBar>
|
||||
<PageBody scrollable={false}>
|
||||
<div className="grid h-full max-h-full grid-rows-[auto_1fr] overflow-hidden">
|
||||
<div className="flex items-start justify-between gap-x-2 p-2">
|
||||
<WebhookDeliveryFilters possibleWebhooks={possibleWebhooks} defaultPeriod="7d" />
|
||||
{/* The new-deliveries button sits inline, immediately left of the pager */}
|
||||
<div className="flex items-center gap-x-2">
|
||||
{newDeliveriesButton}
|
||||
<ListPagination list={{ pagination }} />
|
||||
</div>
|
||||
</div>
|
||||
{/* Sits directly in the 1fr row, like the runs, sessions and batches lists. No
|
||||
stickyHeader: that switches Table's container to overflow-visible, which stops it
|
||||
being the scroll container. The header is sticky either way (TableHeader always sets
|
||||
sticky top-0), and the other webhook tables only pass it because an ancestor scrolls. */}
|
||||
<DeliveriesTable deliveries={visibleDeliveries} showWebhook hasFilters={hasFilters} />
|
||||
</div>
|
||||
</PageBody>
|
||||
</>
|
||||
);
|
||||
}
|
||||
|
||||
function useLiveDeliveries(deliveries: WebhookDeliveryListItem[]) {
|
||||
const organization = useOrganization();
|
||||
const project = useProject();
|
||||
const environment = useEnvironment();
|
||||
const navigation = useNavigation();
|
||||
const revalidator = useRevalidator();
|
||||
const [searchParams, setSearchParams] = useSearchParams();
|
||||
|
||||
const { visibleDeliveries, showNewDeliveriesBanner, newDeliveriesCount, dismissNewDeliveries } =
|
||||
useDeliveriesLiveReload({
|
||||
deliveries,
|
||||
isLoading: navigation.state !== "idle",
|
||||
organizationSlug: organization.slug,
|
||||
projectSlug: project.slug,
|
||||
environmentSlug: environment.slug,
|
||||
});
|
||||
|
||||
const onClickShowNewDeliveries = () => {
|
||||
dismissNewDeliveries();
|
||||
if (searchParams.has("cursor") || searchParams.has("direction")) {
|
||||
setSearchParams((prev) => {
|
||||
prev.delete("cursor");
|
||||
prev.delete("direction");
|
||||
return prev;
|
||||
});
|
||||
return;
|
||||
}
|
||||
revalidator.revalidate();
|
||||
};
|
||||
|
||||
const newDeliveriesButton = showNewDeliveriesBanner ? (
|
||||
<span className="flex duration-150 animate-in fade-in-0">
|
||||
<Button
|
||||
variant="secondary/small"
|
||||
className="text-text-bright"
|
||||
onClick={onClickShowNewDeliveries}
|
||||
LeadingIcon={<PulsingDot className="h-2 w-2" />}
|
||||
tooltip="Refresh to see new deliveries"
|
||||
aria-label="New deliveries received. Refresh to see them."
|
||||
>
|
||||
{newDeliveriesCount >= 100
|
||||
? "99+ new deliveries"
|
||||
: `${newDeliveriesCount} new ${newDeliveriesCount === 1 ? "delivery" : "deliveries"}`}
|
||||
</Button>
|
||||
</span>
|
||||
) : null;
|
||||
|
||||
return { visibleDeliveries, newDeliveriesButton };
|
||||
}
|
||||
+451
@@ -0,0 +1,451 @@
|
||||
import { BookOpenIcon } from "@heroicons/react/24/solid";
|
||||
import { type MetaFunction, useRevalidator } from "@remix-run/react";
|
||||
import { type LoaderFunctionArgs } from "@remix-run/server-runtime";
|
||||
import { type ReactNode, useState } from "react";
|
||||
import { typedjson, useTypedLoaderData } from "remix-typedjson";
|
||||
import { z } from "zod";
|
||||
import { AIChatIcon } from "~/assets/icons/AIChatIcon";
|
||||
import { RunsIcon } from "~/assets/icons/RunsIcon";
|
||||
import { WebhookIcon } from "~/assets/icons/WebhookIcon";
|
||||
import { CodeBlock } from "~/components/code/CodeBlock";
|
||||
import { PageBody } from "~/components/layout/AppLayout";
|
||||
import { LinkButton } from "~/components/primitives/Buttons";
|
||||
import { Callout } from "~/components/primitives/Callout";
|
||||
import { CopyableText } from "~/components/primitives/CopyableText";
|
||||
import { TextLink } from "~/components/primitives/TextLink";
|
||||
import { DateTime } from "~/components/primitives/DateTime";
|
||||
import { Header2 } from "~/components/primitives/Headers";
|
||||
import { NavBar, PageAccessories, PageTitle } from "~/components/primitives/PageHeader";
|
||||
import { Paragraph } from "~/components/primitives/Paragraph";
|
||||
import * as Property from "~/components/primitives/PropertyTable";
|
||||
import { TabButton, TabContainer } from "~/components/primitives/Tabs";
|
||||
import {
|
||||
ResizableHandle,
|
||||
ResizablePanel,
|
||||
ResizablePanelGroup,
|
||||
} from "~/components/primitives/Resizable";
|
||||
import { DeliveryStatusBadge } from "~/components/webhookDeliveries/v1/DeliveryStatus";
|
||||
import { DeliveryTimeline } from "~/components/webhookDeliveries/v1/DeliveryTimeline";
|
||||
import { $replica } from "~/db.server";
|
||||
import { env } from "~/env.server";
|
||||
import { useEnvironment } from "~/hooks/useEnvironment";
|
||||
import { useInterval } from "~/hooks/useInterval";
|
||||
import { useOrganization } from "~/hooks/useOrganizations";
|
||||
import { useProject } from "~/hooks/useProject";
|
||||
import { findProjectBySlug } from "~/models/project.server";
|
||||
import { findEnvironmentBySlug } from "~/models/runtimeEnvironment.server";
|
||||
import {
|
||||
WebhookDeliveryDetailPresenter,
|
||||
type WebhookDeliveryDetail,
|
||||
} from "~/presenters/v3/WebhookDeliveryDetailPresenter.server";
|
||||
import { clickhouseFactory } from "~/services/clickhouse/clickhouseFactoryInstance.server";
|
||||
import { requireUser } from "~/services/session.server";
|
||||
import { formatBytes } from "~/utils/columnFormat";
|
||||
import {
|
||||
docsPath,
|
||||
EnvironmentParamSchema,
|
||||
v3RunPath,
|
||||
v3SessionPath,
|
||||
v3WebhooksPath,
|
||||
v3WebhookTaskPath,
|
||||
} from "~/utils/pathBuilder";
|
||||
import { FEATURE_FLAG } from "~/v3/featureFlags";
|
||||
import { flag } from "~/v3/featureFlags.server";
|
||||
|
||||
const DeliveryParamSchema = EnvironmentParamSchema.extend({
|
||||
deliveryParam: z.string(),
|
||||
});
|
||||
|
||||
const EVENT_DISPLAY_CAP = 128 * 1024;
|
||||
|
||||
export const meta: MetaFunction<typeof loader> = ({ data }) => {
|
||||
const friendlyId = (data as { delivery?: WebhookDeliveryDetail } | undefined)?.delivery
|
||||
?.friendlyId;
|
||||
return [
|
||||
{ title: friendlyId ? `${friendlyId} | Deliveries | Trigger.dev` : "Delivery | Trigger.dev" },
|
||||
];
|
||||
};
|
||||
|
||||
export const loader = async ({ request, params }: LoaderFunctionArgs) => {
|
||||
const user = await requireUser(request);
|
||||
const userId = user.id;
|
||||
const { organizationSlug, projectParam, envParam, deliveryParam } =
|
||||
DeliveryParamSchema.parse(params);
|
||||
|
||||
const project = await findProjectBySlug(organizationSlug, projectParam, userId);
|
||||
if (!project) throw new Response("Project not found", { status: 404 });
|
||||
const environment = await findEnvironmentBySlug(project.id, envParam, userId);
|
||||
if (!environment) throw new Response("Environment not found", { status: 404 });
|
||||
|
||||
// Feature gate: same as the deliveries list. Global FeatureFlag OR per-org override;
|
||||
// admins/impersonators always pass.
|
||||
if (!user.admin && !user.isImpersonating) {
|
||||
const org = await $replica.organization.findFirst({
|
||||
where: { id: project.organizationId },
|
||||
select: { featureFlags: true },
|
||||
});
|
||||
const enabled = await flag({
|
||||
key: FEATURE_FLAG.hasWebhooksAccess,
|
||||
defaultValue: false,
|
||||
overrides: (org?.featureFlags as Record<string, unknown>) ?? {},
|
||||
});
|
||||
if (!enabled) throw new Response("Not found", { status: 404 });
|
||||
}
|
||||
|
||||
const clickhouse = await clickhouseFactory.getClickhouseForOrganization(
|
||||
project.organizationId,
|
||||
"standard"
|
||||
);
|
||||
|
||||
const presenter = new WebhookDeliveryDetailPresenter($replica, clickhouse);
|
||||
const delivery = await presenter.call({
|
||||
organizationId: project.organizationId,
|
||||
projectId: project.id,
|
||||
environmentId: environment.id,
|
||||
deliveryFriendlyId: deliveryParam,
|
||||
});
|
||||
|
||||
const rawEventJson =
|
||||
delivery?.parsedEvent != null ? JSON.stringify(delivery.parsedEvent, null, 2) : null;
|
||||
const eventBytes = rawEventJson != null ? Buffer.byteLength(rawEventJson, "utf8") : 0;
|
||||
const eventTruncatedForDisplay = rawEventJson != null && rawEventJson.length > EVENT_DISPLAY_CAP;
|
||||
const eventJson = eventTruncatedForDisplay
|
||||
? rawEventJson!.slice(0, EVENT_DISPLAY_CAP)
|
||||
: rawEventJson;
|
||||
|
||||
// A missing delivery is most often one that aged out of retention (a bookmarked or shared link),
|
||||
// so render a friendly retention-aware state rather than a hard 404.
|
||||
return typedjson({
|
||||
delivery: delivery ? { ...delivery, parsedEvent: null } : delivery,
|
||||
eventJson,
|
||||
eventBytes,
|
||||
eventTruncatedForDisplay,
|
||||
retentionDays: env.WEBHOOK_PARTITION_RETENTION_DAYS,
|
||||
});
|
||||
};
|
||||
|
||||
/** Centred placeholder for a tab whose content was never captured. */
|
||||
function EmptyTabMessage({ children }: { children: ReactNode }) {
|
||||
return (
|
||||
<div className="flex h-full items-center justify-center">
|
||||
<Paragraph variant="base" className="text-center text-text-dimmed">
|
||||
{children}
|
||||
</Paragraph>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
function formatDuration(createdAt: Date, processedAt: Date | null): string | null {
|
||||
if (!processedAt) return null;
|
||||
const ms = processedAt.getTime() - createdAt.getTime();
|
||||
if (ms < 0) return null;
|
||||
if (ms < 1000) return `${ms}ms`;
|
||||
return `${(ms / 1000).toFixed(2)}s`;
|
||||
}
|
||||
|
||||
export default function Page() {
|
||||
const { delivery, eventJson, eventBytes, eventTruncatedForDisplay, retentionDays } =
|
||||
useTypedLoaderData<typeof loader>();
|
||||
const organization = useOrganization();
|
||||
const project = useProject();
|
||||
const environment = useEnvironment();
|
||||
const revalidator = useRevalidator();
|
||||
|
||||
const inFlight = delivery?.status === "PENDING" || delivery?.status === "PROCESSING";
|
||||
useInterval({
|
||||
interval: 3000,
|
||||
pauseWhenHidden: true,
|
||||
disabled: !inFlight,
|
||||
callback: () => revalidator.revalidate(),
|
||||
});
|
||||
|
||||
const deliveriesPath = v3WebhooksPath(organization, project, environment);
|
||||
const [tab, setTab] = useState<"event" | "headers">("event");
|
||||
|
||||
if (!delivery) {
|
||||
return (
|
||||
<>
|
||||
<NavBar>
|
||||
<PageTitle backButton={{ to: deliveriesPath, text: "Deliveries" }} title="Delivery" />
|
||||
</NavBar>
|
||||
<PageBody>
|
||||
<div className="mx-auto flex max-w-md flex-col items-center gap-3 py-16 text-center">
|
||||
<WebhookIcon className="size-8 text-text-dimmed" />
|
||||
<Header2>Delivery not available</Header2>
|
||||
<Paragraph variant="small" className="text-text-dimmed">
|
||||
This delivery couldn't be found. Deliveries are retained for {retentionDays} days, so
|
||||
it may have aged out.
|
||||
</Paragraph>
|
||||
<LinkButton variant="secondary/small" to={deliveriesPath}>
|
||||
Back to deliveries
|
||||
</LinkButton>
|
||||
</div>
|
||||
</PageBody>
|
||||
</>
|
||||
);
|
||||
}
|
||||
|
||||
const runPath = delivery.run
|
||||
? v3RunPath(organization, project, environment, { friendlyId: delivery.run.friendlyId })
|
||||
: undefined;
|
||||
const sessionPath = delivery.session
|
||||
? v3SessionPath(organization, project, environment, { friendlyId: delivery.session.friendlyId })
|
||||
: undefined;
|
||||
const webhookPath = delivery.webhook
|
||||
? v3WebhookTaskPath(organization, project, environment, delivery.webhook.slug)
|
||||
: undefined;
|
||||
|
||||
const headersJson =
|
||||
delivery.headers != null && Object.keys(delivery.headers as object).length > 0
|
||||
? JSON.stringify(delivery.headers, null, 2)
|
||||
: null;
|
||||
const duration = formatDuration(delivery.createdAt, delivery.processedAt);
|
||||
|
||||
return (
|
||||
<>
|
||||
<NavBar>
|
||||
<PageTitle
|
||||
backButton={{ to: deliveriesPath, text: "Deliveries" }}
|
||||
title={
|
||||
<span className="flex items-center gap-2">
|
||||
<WebhookIcon className="size-4.5 text-webhooks" />
|
||||
<span className="font-mono">{delivery.friendlyId}</span>
|
||||
</span>
|
||||
}
|
||||
/>
|
||||
<PageAccessories>
|
||||
<LinkButton
|
||||
variant="docs/small"
|
||||
LeadingIcon={BookOpenIcon}
|
||||
to={docsPath("webhooks/overview")}
|
||||
>
|
||||
Webhooks docs
|
||||
</LinkButton>
|
||||
</PageAccessories>
|
||||
</NavBar>
|
||||
<PageBody scrollable={false}>
|
||||
<ResizablePanelGroup orientation="horizontal" className="max-h-full">
|
||||
<ResizablePanel id="delivery-event" min="300px">
|
||||
<div className="grid h-full grid-rows-[auto_1fr] overflow-hidden">
|
||||
<div className="flex h-10 items-end border-b border-grid-dimmed bg-background-bright pl-3 pr-2">
|
||||
<TabContainer className="-mb-px">
|
||||
<TabButton
|
||||
isActive={tab === "event"}
|
||||
layoutId="delivery-page-tabs"
|
||||
onClick={() => setTab("event")}
|
||||
>
|
||||
Event payload
|
||||
</TabButton>
|
||||
<TabButton
|
||||
isActive={tab === "headers"}
|
||||
layoutId="delivery-page-tabs"
|
||||
onClick={() => setTab("headers")}
|
||||
>
|
||||
Request headers
|
||||
</TabButton>
|
||||
</TabContainer>
|
||||
</div>
|
||||
<div className="overflow-y-auto p-3 scrollbar-thin scrollbar-track-transparent scrollbar-thumb-charcoal-600">
|
||||
{tab === "event" ? (
|
||||
eventJson ? (
|
||||
<div className="flex flex-col gap-2">
|
||||
{eventTruncatedForDisplay ? (
|
||||
<Callout variant="info">
|
||||
This event is {formatBytes(eventBytes)}. Showing the first{" "}
|
||||
{formatBytes(EVENT_DISPLAY_CAP)} for display. The full payload was
|
||||
delivered to your task.
|
||||
</Callout>
|
||||
) : null}
|
||||
<CodeBlock code={eventJson} language="json" showLineNumbers maxLines={1000} />
|
||||
</div>
|
||||
) : (
|
||||
<EmptyTabMessage>
|
||||
No event payload was captured for this delivery.
|
||||
</EmptyTabMessage>
|
||||
)
|
||||
) : headersJson ? (
|
||||
<CodeBlock
|
||||
code={headersJson}
|
||||
language="json"
|
||||
showLineNumbers={false}
|
||||
maxLines={200}
|
||||
/>
|
||||
) : (
|
||||
<EmptyTabMessage>
|
||||
No request headers were captured for this delivery.
|
||||
</EmptyTabMessage>
|
||||
)}
|
||||
</div>
|
||||
</div>
|
||||
</ResizablePanel>
|
||||
|
||||
<ResizableHandle id="delivery-detail-handle" />
|
||||
<ResizablePanel
|
||||
id="delivery-detail"
|
||||
min="280px"
|
||||
default="380px"
|
||||
max="500px"
|
||||
isStaticAtRest
|
||||
>
|
||||
<div className="grid h-full grid-rows-[auto_1fr] overflow-hidden bg-background-bright">
|
||||
<div className="flex h-10 items-center gap-2 border-b border-grid-dimmed pl-3 pr-2">
|
||||
<Header2 className="truncate">Delivery</Header2>
|
||||
</div>
|
||||
<div className="overflow-y-auto px-3 py-3 scrollbar-thin scrollbar-track-transparent scrollbar-thumb-charcoal-600">
|
||||
<DeliveryTimeline delivery={delivery} runPath={runPath} sessionPath={sessionPath} />
|
||||
<Property.Table>
|
||||
<Property.Item>
|
||||
<Property.Label>ID</Property.Label>
|
||||
<Property.Value>
|
||||
<CopyableText
|
||||
value={delivery.friendlyId}
|
||||
className="font-mono text-sm"
|
||||
truncate
|
||||
/>
|
||||
</Property.Value>
|
||||
</Property.Item>
|
||||
<Property.Item>
|
||||
<Property.Label>Status</Property.Label>
|
||||
<Property.Value>
|
||||
<DeliveryStatusBadge status={delivery.status} />
|
||||
</Property.Value>
|
||||
</Property.Item>
|
||||
{delivery.filterReason ? (
|
||||
<Property.Item>
|
||||
<Property.Label>Filter reason</Property.Label>
|
||||
<Property.Value>
|
||||
<span className="text-text-dimmed">{delivery.filterReason}</span>
|
||||
</Property.Value>
|
||||
</Property.Item>
|
||||
) : null}
|
||||
<Property.Item>
|
||||
<Property.Label>Webhook</Property.Label>
|
||||
<Property.Value>
|
||||
{delivery.webhook ? (
|
||||
webhookPath ? (
|
||||
<TextLink to={webhookPath} className="inline-flex items-center gap-1">
|
||||
<WebhookIcon className="size-4 text-webhooks" />
|
||||
{delivery.webhook.slug}
|
||||
</TextLink>
|
||||
) : (
|
||||
<span className="inline-flex items-center gap-1">
|
||||
<WebhookIcon className="size-4 text-webhooks" />
|
||||
{delivery.webhook.slug}
|
||||
</span>
|
||||
)
|
||||
) : (
|
||||
<span className="text-text-dimmed">Unknown</span>
|
||||
)}
|
||||
</Property.Value>
|
||||
</Property.Item>
|
||||
{delivery.session && sessionPath ? (
|
||||
<Property.Item>
|
||||
<Property.Label>Session</Property.Label>
|
||||
<Property.Value>
|
||||
<TextLink
|
||||
to={sessionPath}
|
||||
className="inline-flex items-center gap-1 font-mono text-sm"
|
||||
>
|
||||
<AIChatIcon className="size-4 text-sessions" />
|
||||
{delivery.session.friendlyId}
|
||||
</TextLink>
|
||||
</Property.Value>
|
||||
</Property.Item>
|
||||
) : null}
|
||||
<Property.Item>
|
||||
<Property.Label>Run</Property.Label>
|
||||
<Property.Value>
|
||||
{delivery.run && runPath ? (
|
||||
<TextLink
|
||||
to={runPath}
|
||||
className="inline-flex items-center gap-1 font-mono text-sm"
|
||||
>
|
||||
<RunsIcon className="size-4 text-runs" />
|
||||
{delivery.run.friendlyId}
|
||||
</TextLink>
|
||||
) : (
|
||||
<span className="text-text-dimmed">None</span>
|
||||
)}
|
||||
</Property.Value>
|
||||
</Property.Item>
|
||||
<Property.Item>
|
||||
<Property.Label>External delivery ID</Property.Label>
|
||||
<Property.Value>
|
||||
{delivery.externalDeliveryId ? (
|
||||
<CopyableText
|
||||
value={delivery.externalDeliveryId}
|
||||
className="font-mono text-sm"
|
||||
truncate
|
||||
/>
|
||||
) : (
|
||||
<span className="text-text-dimmed">None</span>
|
||||
)}
|
||||
</Property.Value>
|
||||
</Property.Item>
|
||||
<Property.Item>
|
||||
<Property.Label>Idempotency key</Property.Label>
|
||||
<Property.Value>
|
||||
{delivery.idempotencyKey ? (
|
||||
<CopyableText
|
||||
value={delivery.idempotencyKey}
|
||||
className="font-mono text-sm"
|
||||
truncate
|
||||
/>
|
||||
) : (
|
||||
<span className="text-text-dimmed">None</span>
|
||||
)}
|
||||
</Property.Value>
|
||||
</Property.Item>
|
||||
<Property.Item>
|
||||
<Property.Label>Raw body hash</Property.Label>
|
||||
<Property.Value>
|
||||
{delivery.rawBodyHash ? (
|
||||
<CopyableText
|
||||
value={delivery.rawBodyHash}
|
||||
className="font-mono text-sm"
|
||||
truncate
|
||||
/>
|
||||
) : (
|
||||
<span className="text-text-dimmed">None</span>
|
||||
)}
|
||||
</Property.Value>
|
||||
</Property.Item>
|
||||
<Property.Item>
|
||||
<Property.Label>Created</Property.Label>
|
||||
<Property.Value>
|
||||
<DateTime date={delivery.createdAt} />
|
||||
</Property.Value>
|
||||
</Property.Item>
|
||||
<Property.Item>
|
||||
<Property.Label>Processed</Property.Label>
|
||||
<Property.Value>
|
||||
{delivery.processedAt ? (
|
||||
<DateTime date={delivery.processedAt} />
|
||||
) : (
|
||||
<span className="text-text-dimmed">None</span>
|
||||
)}
|
||||
</Property.Value>
|
||||
</Property.Item>
|
||||
{duration ? (
|
||||
<Property.Item>
|
||||
<Property.Label>Duration</Property.Label>
|
||||
<Property.Value>{duration}</Property.Value>
|
||||
</Property.Item>
|
||||
) : null}
|
||||
{delivery.status === "FAILED" && delivery.errorMessage ? (
|
||||
<Property.Item>
|
||||
<Property.Label>Error</Property.Label>
|
||||
<Property.Value>
|
||||
<span className="text-sm text-error">{delivery.errorMessage}</span>
|
||||
</Property.Value>
|
||||
</Property.Item>
|
||||
) : null}
|
||||
</Property.Table>
|
||||
</div>
|
||||
</div>
|
||||
</ResizablePanel>
|
||||
</ResizablePanelGroup>
|
||||
</PageBody>
|
||||
</>
|
||||
);
|
||||
}
|
||||
+740
@@ -0,0 +1,740 @@
|
||||
import { BookOpenIcon, KeyIcon, SparklesIcon } from "@heroicons/react/24/solid";
|
||||
import { useFetcher, type MetaFunction } from "@remix-run/react";
|
||||
import { type ActionFunctionArgs, type LoaderFunctionArgs } from "@remix-run/server-runtime";
|
||||
import { randomBytes } from "node:crypto";
|
||||
import { WebhookRoutingTarget, WebhookVerifierArtifact } from "@trigger.dev/core/v3";
|
||||
import type { WebhookValueSource } from "@trigger.dev/core/v3";
|
||||
import { Suspense, useEffect, useState } from "react";
|
||||
import { TypedAwait, typeddefer, useTypedLoaderData } from "remix-typedjson";
|
||||
import { z } from "zod";
|
||||
import { WebhookIcon } from "~/assets/icons/WebhookIcon";
|
||||
import { CodeBlock } from "~/components/code/CodeBlock";
|
||||
import { PageBody } from "~/components/layout/AppLayout";
|
||||
import { DirectionSchema, ListPagination } from "~/components/ListPagination";
|
||||
import { Button, LinkButton } from "~/components/primitives/Buttons";
|
||||
import { ClipboardField } from "~/components/primitives/ClipboardField";
|
||||
import { CopyableText } from "~/components/primitives/CopyableText";
|
||||
import { DateTime } from "~/components/primitives/DateTime";
|
||||
import { Dialog, DialogContent, DialogHeader, DialogTrigger } from "~/components/primitives/Dialog";
|
||||
import { Header2, Header3 } from "~/components/primitives/Headers";
|
||||
import { Hint } from "~/components/primitives/Hint";
|
||||
import { Input } from "~/components/primitives/Input";
|
||||
import { Label } from "~/components/primitives/Label";
|
||||
import { NavBar, PageAccessories, PageTitle } from "~/components/primitives/PageHeader";
|
||||
import { Paragraph } from "~/components/primitives/Paragraph";
|
||||
import * as Property from "~/components/primitives/PropertyTable";
|
||||
import {
|
||||
ResizableHandle,
|
||||
ResizablePanel,
|
||||
ResizablePanelGroup,
|
||||
} from "~/components/primitives/Resizable";
|
||||
import { Spinner } from "~/components/primitives/Spinner";
|
||||
import { TextLink } from "~/components/primitives/TextLink";
|
||||
import { TimeFilter } from "~/components/runs/v3/SharedFilters";
|
||||
import { DeliveriesTable } from "~/components/webhookDeliveries/v1/DeliveriesTable";
|
||||
import { EndpointStatusBadge } from "~/components/webhookEndpoints/v1/EndpointStatus";
|
||||
import { $replica, prisma, webhookPrisma } from "~/db.server";
|
||||
import { webhookIngressUrl } from "~/utils/webhookIngressUrl.server";
|
||||
import { useEnvironment } from "~/hooks/useEnvironment";
|
||||
import { useOrganization } from "~/hooks/useOrganizations";
|
||||
import { useProject } from "~/hooks/useProject";
|
||||
import { findProjectBySlug } from "~/models/project.server";
|
||||
import { findEnvironmentBySlug } from "~/models/runtimeEnvironment.server";
|
||||
import {
|
||||
WebhookDetailPresenter,
|
||||
type WebhookEndpointDetail,
|
||||
} from "~/presenters/v3/WebhookDetailPresenter.server";
|
||||
import { clickhouseFactory } from "~/services/clickhouse/clickhouseFactoryInstance.server";
|
||||
import { getSecretStore } from "~/services/secrets/secretStore.server";
|
||||
import { requireUser } from "~/services/session.server";
|
||||
import { docsPath, EnvironmentParamSchema, v3WebhookTaskPath } from "~/utils/pathBuilder";
|
||||
import { parseFiniteInt } from "~/utils/searchParams";
|
||||
import { FEATURE_FLAG } from "~/v3/featureFlags";
|
||||
import { flag } from "~/v3/featureFlags.server";
|
||||
|
||||
const EndpointParamSchema = EnvironmentParamSchema.extend({
|
||||
endpointParam: z.string(),
|
||||
});
|
||||
|
||||
export const meta: MetaFunction<typeof loader> = ({ data }) => {
|
||||
const friendlyId = (data as { endpoint?: WebhookEndpointDetail } | undefined)?.endpoint
|
||||
?.friendlyId;
|
||||
return [
|
||||
{ title: friendlyId ? `${friendlyId} | Endpoints | Trigger.dev` : "Endpoint | Trigger.dev" },
|
||||
];
|
||||
};
|
||||
|
||||
// Shared gate + scope resolution for the loader and action.
|
||||
async function requireWebhookAccess(request: Request, params: LoaderFunctionArgs["params"]) {
|
||||
const user = await requireUser(request);
|
||||
const { organizationSlug, projectParam, envParam, endpointParam } =
|
||||
EndpointParamSchema.parse(params);
|
||||
|
||||
const project = await findProjectBySlug(organizationSlug, projectParam, user.id);
|
||||
if (!project) throw new Response("Project not found", { status: 404 });
|
||||
const environment = await findEnvironmentBySlug(project.id, envParam, user.id);
|
||||
if (!environment) throw new Response("Environment not found", { status: 404 });
|
||||
|
||||
if (!user.admin && !user.isImpersonating) {
|
||||
const org = await $replica.organization.findFirst({
|
||||
where: { id: project.organizationId },
|
||||
select: { featureFlags: true },
|
||||
});
|
||||
const enabled = await flag({
|
||||
key: FEATURE_FLAG.hasWebhooksAccess,
|
||||
defaultValue: false,
|
||||
overrides: (org?.featureFlags as Record<string, unknown>) ?? {},
|
||||
});
|
||||
if (!enabled) throw new Response("Not found", { status: 404 });
|
||||
}
|
||||
|
||||
return { user, project, environment, endpointParam };
|
||||
}
|
||||
|
||||
export const loader = async ({ request, params }: LoaderFunctionArgs) => {
|
||||
const { project, environment, endpointParam } = await requireWebhookAccess(request, params);
|
||||
|
||||
const clickhouse = await clickhouseFactory.getClickhouseForOrganization(
|
||||
project.organizationId,
|
||||
"standard"
|
||||
);
|
||||
const presenter = new WebhookDetailPresenter($replica, clickhouse);
|
||||
|
||||
const endpoint = await presenter.findEndpoint({
|
||||
environmentId: environment.id,
|
||||
endpointFriendlyId: endpointParam,
|
||||
});
|
||||
if (!endpoint) throw new Response("Endpoint not found", { status: 404 });
|
||||
|
||||
const ingestUrl = webhookIngressUrl(endpoint.opaqueId);
|
||||
|
||||
// Parse the tagged-union JSON columns for display (engine validates on write).
|
||||
const routing = WebhookRoutingTarget.safeParse(endpoint.routingTarget);
|
||||
const verifier = WebhookVerifierArtifact.safeParse(endpoint.verifierArtifact);
|
||||
|
||||
const url = new URL(request.url);
|
||||
const periodParam = url.searchParams.get("period") ?? undefined;
|
||||
const from = parseFiniteInt(url.searchParams.get("from"));
|
||||
const to = parseFiniteInt(url.searchParams.get("to"));
|
||||
const hasExplicitWindow = Boolean(periodParam || from || to);
|
||||
const period = periodParam ?? (hasExplicitWindow ? undefined : "7d");
|
||||
const cursor = url.searchParams.get("cursor") ?? undefined;
|
||||
const directionRaw = url.searchParams.get("direction") ?? undefined;
|
||||
const direction = directionRaw ? DirectionSchema.parse(directionRaw) : undefined;
|
||||
|
||||
const deliveriesList = presenter
|
||||
.listDeliveries({
|
||||
organizationId: project.organizationId,
|
||||
projectId: project.id,
|
||||
environmentId: environment.id,
|
||||
webhookEndpointId: endpoint.id,
|
||||
period,
|
||||
from,
|
||||
to,
|
||||
hasExplicitWindow,
|
||||
cursor,
|
||||
direction,
|
||||
})
|
||||
.catch(() => null);
|
||||
|
||||
return typeddefer({
|
||||
endpoint,
|
||||
ingestUrl,
|
||||
routing: routing.success ? routing.data : null,
|
||||
verifier: verifier.success ? verifier.data : null,
|
||||
deliveriesList,
|
||||
});
|
||||
};
|
||||
|
||||
const SetSecretSchema = z.object({
|
||||
intent: z.literal("set-secret"),
|
||||
secret: z.string().trim().min(1, "A signing secret is required"),
|
||||
});
|
||||
|
||||
export const action = async ({ request, params }: ActionFunctionArgs) => {
|
||||
const { project, environment, endpointParam } = await requireWebhookAccess(request, params);
|
||||
|
||||
const clickhouse = await clickhouseFactory.getClickhouseForOrganization(
|
||||
project.organizationId,
|
||||
"standard"
|
||||
);
|
||||
const presenter = new WebhookDetailPresenter($replica, clickhouse);
|
||||
const endpoint = await presenter.findEndpoint({
|
||||
environmentId: environment.id,
|
||||
endpointFriendlyId: endpointParam,
|
||||
});
|
||||
if (!endpoint) throw new Response("Endpoint not found", { status: 404 });
|
||||
|
||||
// Store the plaintext secret encrypted under the DATABASE SecretStore provider, in the exact
|
||||
// shape the engine's resolveSigningSecret reads ({ secret }). Key is namespaced by the endpoint's
|
||||
// internal id; point signingSecretKey at it so verification picks it up.
|
||||
const secretKey = `webhook:signing-secret:${endpoint.id}`;
|
||||
const secretStore = getSecretStore("DATABASE", { prismaClient: prisma });
|
||||
|
||||
const formData = await request.formData();
|
||||
const intent = formData.get("intent");
|
||||
|
||||
// Generate (integrator-supplied secret): mint a strong secret, store it, and return it so the
|
||||
// UI can reveal it ONCE for the integrator to paste into their provider.
|
||||
if (intent === "generate-secret") {
|
||||
const verifier = WebhookVerifierArtifact.safeParse(endpoint.verifierArtifact);
|
||||
if (
|
||||
verifier.success &&
|
||||
"config" in verifier.data &&
|
||||
verifier.data.config.scheme === "asymmetric"
|
||||
) {
|
||||
return {
|
||||
success: false as const,
|
||||
error: "Cannot generate a secret for an asymmetric endpoint; set its public key instead.",
|
||||
};
|
||||
}
|
||||
const secret = `whsec_${randomBytes(32).toString("hex")}`;
|
||||
await secretStore.setSecret(secretKey, { secret });
|
||||
await webhookPrisma.webhookEndpoint.update({
|
||||
where: { id: endpoint.id },
|
||||
data: { signingSecretKey: secretKey },
|
||||
});
|
||||
return { success: true as const, generatedSecret: secret };
|
||||
}
|
||||
|
||||
// Set/Rotate (paste a provider-supplied secret).
|
||||
const submission = SetSecretSchema.safeParse(Object.fromEntries(formData));
|
||||
if (!submission.success) {
|
||||
return { success: false as const, error: submission.error.issues[0]?.message ?? "Invalid" };
|
||||
}
|
||||
await secretStore.setSecret(secretKey, { secret: submission.data.secret });
|
||||
await webhookPrisma.webhookEndpoint.update({
|
||||
where: { id: endpoint.id },
|
||||
data: { signingSecretKey: secretKey },
|
||||
});
|
||||
|
||||
return { success: true as const };
|
||||
};
|
||||
|
||||
export default function Page() {
|
||||
const { endpoint, ingestUrl, routing, verifier, deliveriesList } =
|
||||
useTypedLoaderData<typeof loader>();
|
||||
const organization = useOrganization();
|
||||
const project = useProject();
|
||||
const environment = useEnvironment();
|
||||
|
||||
const handlerPath = v3WebhookTaskPath(
|
||||
organization,
|
||||
project,
|
||||
environment,
|
||||
endpoint.handlerWebhookId
|
||||
);
|
||||
|
||||
return (
|
||||
<>
|
||||
<NavBar>
|
||||
<PageTitle
|
||||
backButton={{ to: handlerPath, text: endpoint.handlerWebhookId }}
|
||||
title={
|
||||
<span className="flex items-center gap-2">
|
||||
<WebhookIcon className="size-4.5 text-webhooks" />
|
||||
<span className="font-mono">{endpoint.friendlyId}</span>
|
||||
<EndpointStatusBadge status={endpoint.status} />
|
||||
</span>
|
||||
}
|
||||
/>
|
||||
<PageAccessories>
|
||||
<LinkButton
|
||||
variant="docs/small"
|
||||
LeadingIcon={BookOpenIcon}
|
||||
to={docsPath("webhooks/overview")}
|
||||
>
|
||||
Webhooks docs
|
||||
</LinkButton>
|
||||
</PageAccessories>
|
||||
</NavBar>
|
||||
<PageBody scrollable={false}>
|
||||
<ResizablePanelGroup orientation="horizontal" className="max-h-full">
|
||||
<ResizablePanel id="endpoint-deliveries" min="300px">
|
||||
<div className="grid h-full grid-rows-[auto_1fr] overflow-hidden">
|
||||
<div className="flex h-10 items-center justify-between gap-2 border-b border-grid-dimmed bg-background-bright pl-3 pr-2">
|
||||
<Header2>Deliveries</Header2>
|
||||
<div className="flex items-center gap-2">
|
||||
<TimeFilter defaultPeriod="7d" labelName="Deliveries" />
|
||||
<Suspense fallback={null}>
|
||||
<TypedAwait resolve={deliveriesList} errorElement={null}>
|
||||
{(list) => (list ? <ListPagination list={list} /> : null)}
|
||||
</TypedAwait>
|
||||
</Suspense>
|
||||
</div>
|
||||
</div>
|
||||
<div className="h-full overflow-y-auto scrollbar-thin scrollbar-track-transparent scrollbar-thumb-charcoal-600">
|
||||
<Suspense fallback={<TableLoading />}>
|
||||
<TypedAwait resolve={deliveriesList} errorElement={<TableLoading />}>
|
||||
{(list) =>
|
||||
list ? (
|
||||
<DeliveriesTable
|
||||
deliveries={list.deliveries}
|
||||
hasFilters={list.hasFilters}
|
||||
showTopBorder={false}
|
||||
stickyHeader
|
||||
/>
|
||||
) : (
|
||||
<TableLoading />
|
||||
)
|
||||
}
|
||||
</TypedAwait>
|
||||
</Suspense>
|
||||
</div>
|
||||
</div>
|
||||
</ResizablePanel>
|
||||
|
||||
<ResizableHandle id="endpoint-detail-handle" />
|
||||
<ResizablePanel
|
||||
id="endpoint-detail"
|
||||
min="320px"
|
||||
default="420px"
|
||||
max="560px"
|
||||
isStaticAtRest
|
||||
>
|
||||
<EndpointSidebar
|
||||
endpoint={endpoint}
|
||||
ingestUrl={ingestUrl}
|
||||
routing={routing}
|
||||
verifier={verifier}
|
||||
handlerPath={handlerPath}
|
||||
/>
|
||||
</ResizablePanel>
|
||||
</ResizablePanelGroup>
|
||||
</PageBody>
|
||||
</>
|
||||
);
|
||||
}
|
||||
|
||||
type LoaderData = ReturnType<typeof useTypedLoaderData<typeof loader>>;
|
||||
|
||||
function EndpointSidebar({
|
||||
endpoint,
|
||||
ingestUrl,
|
||||
routing,
|
||||
verifier,
|
||||
handlerPath,
|
||||
}: {
|
||||
endpoint: WebhookEndpointDetail;
|
||||
ingestUrl: string;
|
||||
routing: LoaderData["routing"];
|
||||
verifier: LoaderData["verifier"];
|
||||
handlerPath: string;
|
||||
}) {
|
||||
const metadataJson =
|
||||
endpoint.metadata != null && Object.keys(endpoint.metadata as object).length > 0
|
||||
? JSON.stringify(endpoint.metadata, null, 2)
|
||||
: null;
|
||||
|
||||
// Asymmetric endpoints store the provider's PUBLIC KEY, not a shared signing secret.
|
||||
const scheme = verifier && verifier.kind !== "bundle" ? verifier.config.scheme : undefined;
|
||||
const credentialNoun = scheme === "asymmetric" ? "public key" : "signing secret";
|
||||
const credentialLabel = scheme === "asymmetric" ? "Public key" : "Signing secret";
|
||||
|
||||
// Generate-and-reveal makes sense when the integrator chooses the secret (and it's an HMAC
|
||||
// shared secret, not a provider public key). "provider" endpoints only paste.
|
||||
const canGenerate =
|
||||
scheme !== "asymmetric" &&
|
||||
(endpoint.secretProvisioning === "integrator" || endpoint.secretProvisioning === "either");
|
||||
|
||||
return (
|
||||
<div className="grid h-full grid-rows-[auto_1fr] overflow-hidden bg-background-bright">
|
||||
<div className="flex items-center gap-2 border-b border-grid-dimmed py-2 pl-3 pr-2">
|
||||
<Header2 className="flex min-w-0 flex-1 items-center gap-1.5">
|
||||
<WebhookIcon className="size-4.5 shrink-0 text-webhooks" />
|
||||
<span className="truncate font-mono">{endpoint.friendlyId}</span>
|
||||
</Header2>
|
||||
</div>
|
||||
<div className="space-y-5 overflow-y-auto px-3 py-3 scrollbar-thin scrollbar-track-transparent scrollbar-thumb-charcoal-600">
|
||||
{/* Connect: the important new bit. Everything an integrator needs to point a provider here. */}
|
||||
<section className="space-y-2">
|
||||
<Header3>Connect</Header3>
|
||||
<Property.Table>
|
||||
<Property.Item>
|
||||
<Property.Label>Webhook URL</Property.Label>
|
||||
<Property.Value>
|
||||
<CopyableText value={ingestUrl} className="font-mono text-xs" />
|
||||
</Property.Value>
|
||||
</Property.Item>
|
||||
<Property.Item>
|
||||
<Property.Label>{credentialLabel}</Property.Label>
|
||||
<Property.Value>
|
||||
<div className="flex flex-col items-start gap-1.5">
|
||||
{endpoint.hasSigningSecret ? (
|
||||
<span className="flex items-center gap-1.5">
|
||||
<span className="size-2 rounded-full bg-success" />
|
||||
<span>Set</span>
|
||||
</span>
|
||||
) : (
|
||||
<span className="text-warning">Not set, all deliveries are rejected</span>
|
||||
)}
|
||||
<div className="flex flex-wrap items-center gap-2">
|
||||
{canGenerate ? (
|
||||
<GenerateSecretDialog hasSigningSecret={endpoint.hasSigningSecret} />
|
||||
) : null}
|
||||
<SetSecretDialog
|
||||
hasSigningSecret={endpoint.hasSigningSecret}
|
||||
credentialNoun={credentialNoun}
|
||||
variant={canGenerate ? "tertiary/small" : "secondary/small"}
|
||||
/>
|
||||
</div>
|
||||
</div>
|
||||
</Property.Value>
|
||||
</Property.Item>
|
||||
</Property.Table>
|
||||
<ProviderSetup verifier={verifier} source={endpoint.source} />
|
||||
</section>
|
||||
|
||||
<section className="space-y-2">
|
||||
<Header3>Routing</Header3>
|
||||
<Property.Table>
|
||||
<Property.Item>
|
||||
<Property.Label>Target</Property.Label>
|
||||
<Property.Value>
|
||||
{routing?.type === "task" ? (
|
||||
<TextLink to={handlerPath} className="font-mono text-xs">
|
||||
{routing.taskId}
|
||||
</TextLink>
|
||||
) : routing?.type === "session" ? (
|
||||
<span className="font-mono text-xs">session: {routing.taskIdentifier}</span>
|
||||
) : (
|
||||
<span className="text-text-dimmed">Unknown</span>
|
||||
)}
|
||||
</Property.Value>
|
||||
</Property.Item>
|
||||
</Property.Table>
|
||||
</section>
|
||||
|
||||
<section className="space-y-2">
|
||||
<Header3>Scope</Header3>
|
||||
<Property.Table>
|
||||
<Property.Item>
|
||||
<Property.Label>Source</Property.Label>
|
||||
<Property.Value>
|
||||
<span className="font-mono text-sm">{endpoint.source}</span>
|
||||
</Property.Value>
|
||||
</Property.Item>
|
||||
<Property.Item>
|
||||
<Property.Label>Tenant</Property.Label>
|
||||
<Property.Value>
|
||||
{endpoint.isDefault ? (
|
||||
<span className="text-text-dimmed">default</span>
|
||||
) : (
|
||||
<span className="font-mono text-sm">{endpoint.tenantId}</span>
|
||||
)}
|
||||
</Property.Value>
|
||||
</Property.Item>
|
||||
<Property.Item>
|
||||
<Property.Label>External ref</Property.Label>
|
||||
<Property.Value>
|
||||
{endpoint.externalRef ? (
|
||||
<span className="font-mono text-sm">{endpoint.externalRef}</span>
|
||||
) : (
|
||||
<span className="text-text-dimmed">None</span>
|
||||
)}
|
||||
</Property.Value>
|
||||
</Property.Item>
|
||||
<Property.Item>
|
||||
<Property.Label>Status</Property.Label>
|
||||
<Property.Value>
|
||||
<EndpointStatusBadge status={endpoint.status} />
|
||||
</Property.Value>
|
||||
</Property.Item>
|
||||
<Property.Item>
|
||||
<Property.Label>Created</Property.Label>
|
||||
<Property.Value>
|
||||
<DateTime date={endpoint.createdAt} />
|
||||
</Property.Value>
|
||||
</Property.Item>
|
||||
<Property.Item>
|
||||
<Property.Label>Updated</Property.Label>
|
||||
<Property.Value>
|
||||
<DateTime date={endpoint.updatedAt} />
|
||||
</Property.Value>
|
||||
</Property.Item>
|
||||
</Property.Table>
|
||||
</section>
|
||||
|
||||
{metadataJson ? (
|
||||
<section className="space-y-2">
|
||||
<Header3>Metadata</Header3>
|
||||
<CodeBlock code={metadataJson} language="json" showLineNumbers={false} maxLines={20} />
|
||||
</section>
|
||||
) : null}
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
function ProviderSetup({ verifier, source }: { verifier: LoaderData["verifier"]; source: string }) {
|
||||
if (!verifier) return null;
|
||||
|
||||
if (verifier.kind === "bundle") {
|
||||
return (
|
||||
<Paragraph variant="small" className="text-text-dimmed">
|
||||
This endpoint uses a custom verifier bundle.
|
||||
</Paragraph>
|
||||
);
|
||||
}
|
||||
|
||||
const presetName = verifier.kind === "preset" ? verifier.preset : null;
|
||||
const config = verifier.config;
|
||||
|
||||
return (
|
||||
<div className="space-y-2">
|
||||
<Paragraph variant="extra-small" className="uppercase text-text-dimmed">
|
||||
Provider setup
|
||||
</Paragraph>
|
||||
<Property.Table>
|
||||
<Property.Item>
|
||||
<Property.Label>Scheme</Property.Label>
|
||||
<Property.Value>
|
||||
<span className="font-mono text-sm">
|
||||
{presetName ? `${presetName} (${config.scheme})` : config.scheme}
|
||||
</span>
|
||||
</Property.Value>
|
||||
</Property.Item>
|
||||
{config.scheme === "hmac" || config.scheme === "asymmetric" ? (
|
||||
<>
|
||||
<Property.Item>
|
||||
<Property.Label>Signature header</Property.Label>
|
||||
<Property.Value>
|
||||
<span className="font-mono text-sm">{config.signatureHeader}</span>
|
||||
</Property.Value>
|
||||
</Property.Item>
|
||||
<Property.Item>
|
||||
<Property.Label>Algorithm</Property.Label>
|
||||
<Property.Value>
|
||||
<span className="font-mono text-sm">
|
||||
{config.algorithm} / {config.encoding}
|
||||
</span>
|
||||
</Property.Value>
|
||||
</Property.Item>
|
||||
{config.timestamp ? (
|
||||
<Property.Item>
|
||||
<Property.Label>Timestamp</Property.Label>
|
||||
<Property.Value>
|
||||
<span className="font-mono text-xs">
|
||||
{describeTimestampSource(config.timestamp.source)}
|
||||
</span>
|
||||
</Property.Value>
|
||||
</Property.Item>
|
||||
) : null}
|
||||
<Property.Item>
|
||||
<Property.Label>Signing string</Property.Label>
|
||||
<Property.Value>
|
||||
<span className="font-mono text-xs">
|
||||
{config.signingString === "raw" ? "raw body" : config.signingString.template}
|
||||
</span>
|
||||
</Property.Value>
|
||||
</Property.Item>
|
||||
{config.scheme === "asymmetric" ? (
|
||||
<Property.Item>
|
||||
<Property.Label>Public key</Property.Label>
|
||||
<Property.Value>
|
||||
<span className="font-mono text-sm">{config.publicKeyEncoding ?? "pem"}</span>
|
||||
</Property.Value>
|
||||
</Property.Item>
|
||||
) : null}
|
||||
</>
|
||||
) : config.scheme === "shared-secret" ? (
|
||||
<>
|
||||
<Property.Item>
|
||||
<Property.Label>Placement</Property.Label>
|
||||
<Property.Value>
|
||||
<span className="font-mono text-sm">{config.placement}</span>
|
||||
</Property.Value>
|
||||
</Property.Item>
|
||||
{config.fieldName ? (
|
||||
<Property.Item>
|
||||
<Property.Label>Field name</Property.Label>
|
||||
<Property.Value>
|
||||
<span className="font-mono text-sm">{config.fieldName}</span>
|
||||
</Property.Value>
|
||||
</Property.Item>
|
||||
) : null}
|
||||
</>
|
||||
) : (
|
||||
<>
|
||||
<Property.Item>
|
||||
<Property.Label>Placement</Property.Label>
|
||||
<Property.Value>
|
||||
<span className="font-mono text-sm">{config.placement}</span>
|
||||
</Property.Value>
|
||||
</Property.Item>
|
||||
<Property.Item>
|
||||
<Property.Label>Param name</Property.Label>
|
||||
<Property.Value>
|
||||
<span className="font-mono text-sm">{config.paramName}</span>
|
||||
</Property.Value>
|
||||
</Property.Item>
|
||||
</>
|
||||
)}
|
||||
</Property.Table>
|
||||
<Hint>
|
||||
{config.scheme === "asymmetric"
|
||||
? `${source} signs with its private key; set its public key above.`
|
||||
: `Sign deliveries with the ${source} scheme above, using the signing secret.`}
|
||||
</Hint>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
// Human-readable description of where the replay timestamp is read from.
|
||||
function describeTimestampSource(source: WebhookValueSource): string {
|
||||
switch (source.from) {
|
||||
case "header":
|
||||
return `header ${source.name}`;
|
||||
case "signatureField":
|
||||
return `field "${source.field}" in signature header`;
|
||||
case "body":
|
||||
return `body ${source.path}`;
|
||||
case "url":
|
||||
return "request URL";
|
||||
case "constant":
|
||||
return "constant";
|
||||
}
|
||||
}
|
||||
|
||||
function SetSecretDialog({
|
||||
hasSigningSecret,
|
||||
credentialNoun,
|
||||
variant = "secondary/small",
|
||||
}: {
|
||||
hasSigningSecret: boolean;
|
||||
credentialNoun: string;
|
||||
variant?: "secondary/small" | "tertiary/small";
|
||||
}) {
|
||||
const fetcher = useFetcher<typeof action>();
|
||||
const [open, setOpen] = useState(false);
|
||||
const isSubmitting = fetcher.state !== "idle";
|
||||
const verb = hasSigningSecret ? "Rotate" : "Set";
|
||||
const isPublicKey = credentialNoun === "public key";
|
||||
|
||||
// Close on a successful save; the loader revalidates and the state flips to "Set".
|
||||
useEffect(() => {
|
||||
if (fetcher.state === "idle" && fetcher.data?.success) {
|
||||
setOpen(false);
|
||||
}
|
||||
}, [fetcher.state, fetcher.data]);
|
||||
|
||||
return (
|
||||
<Dialog open={open} onOpenChange={setOpen}>
|
||||
<DialogTrigger asChild>
|
||||
<Button variant={variant} LeadingIcon={KeyIcon}>
|
||||
{verb} {credentialNoun}
|
||||
</Button>
|
||||
</DialogTrigger>
|
||||
<DialogContent>
|
||||
<DialogHeader>
|
||||
{verb} {credentialNoun}
|
||||
</DialogHeader>
|
||||
<fetcher.Form method="post" className="flex flex-col gap-3 pt-2">
|
||||
<input type="hidden" name="intent" value="set-secret" />
|
||||
<div className="flex flex-col gap-1">
|
||||
<Label htmlFor="secret">{credentialNoun}</Label>
|
||||
<Input
|
||||
id="secret"
|
||||
name="secret"
|
||||
type="text"
|
||||
autoComplete="off"
|
||||
spellCheck={false}
|
||||
placeholder={isPublicKey ? "public key" : "whsec_…"}
|
||||
/>
|
||||
<Hint>
|
||||
{isPublicKey
|
||||
? "The provider's public key. Stored encrypted; deliveries are verified against it."
|
||||
: "Stored encrypted and never shown again. Deliveries are verified against this secret."}
|
||||
</Hint>
|
||||
</div>
|
||||
{fetcher.data && !fetcher.data.success ? (
|
||||
<Paragraph variant="small" className="text-error">
|
||||
{fetcher.data.error}
|
||||
</Paragraph>
|
||||
) : null}
|
||||
<div className="flex justify-end gap-2">
|
||||
<Button
|
||||
type="button"
|
||||
variant="tertiary/small"
|
||||
onClick={() => setOpen(false)}
|
||||
disabled={isSubmitting}
|
||||
>
|
||||
Cancel
|
||||
</Button>
|
||||
<Button type="submit" variant="primary/small" disabled={isSubmitting}>
|
||||
{isSubmitting ? "Saving…" : "Save secret"}
|
||||
</Button>
|
||||
</div>
|
||||
</fetcher.Form>
|
||||
</DialogContent>
|
||||
</Dialog>
|
||||
);
|
||||
}
|
||||
|
||||
// For integrator-supplied secrets (GitHub/GitLab/standard): mint a strong secret server-side,
|
||||
// store it, and reveal it ONCE so the user can paste it into their provider.
|
||||
function GenerateSecretDialog({ hasSigningSecret }: { hasSigningSecret: boolean }) {
|
||||
const fetcher = useFetcher<typeof action>();
|
||||
const [open, setOpen] = useState(false);
|
||||
const isSubmitting = fetcher.state !== "idle";
|
||||
const generated =
|
||||
fetcher.data && "generatedSecret" in fetcher.data ? fetcher.data.generatedSecret : undefined;
|
||||
|
||||
return (
|
||||
<Dialog open={open} onOpenChange={setOpen}>
|
||||
<DialogTrigger asChild>
|
||||
<Button variant="secondary/small" LeadingIcon={SparklesIcon}>
|
||||
{hasSigningSecret ? "Regenerate secret" : "Generate secret"}
|
||||
</Button>
|
||||
</DialogTrigger>
|
||||
<DialogContent>
|
||||
<DialogHeader>
|
||||
{hasSigningSecret ? "Regenerate signing secret" : "Generate signing secret"}
|
||||
</DialogHeader>
|
||||
{generated ? (
|
||||
<div className="flex flex-col gap-3 pt-2">
|
||||
<Paragraph variant="small" className="text-warning">
|
||||
Copy this now. It won't be shown again.
|
||||
</Paragraph>
|
||||
<ClipboardField value={generated} variant="secondary/medium" />
|
||||
<Hint>Paste this into your provider's webhook signing-secret field.</Hint>
|
||||
<div className="flex justify-end">
|
||||
<Button type="button" variant="primary/small" onClick={() => setOpen(false)}>
|
||||
Done
|
||||
</Button>
|
||||
</div>
|
||||
</div>
|
||||
) : (
|
||||
<fetcher.Form method="post" className="flex flex-col gap-3 pt-2">
|
||||
<input type="hidden" name="intent" value="generate-secret" />
|
||||
<Paragraph variant="small" className="text-text-dimmed">
|
||||
Trigger.dev generates a strong signing secret, stores it encrypted, and shows it once
|
||||
so you can paste it into your provider.
|
||||
</Paragraph>
|
||||
<div className="flex justify-end gap-2">
|
||||
<Button
|
||||
type="button"
|
||||
variant="tertiary/small"
|
||||
onClick={() => setOpen(false)}
|
||||
disabled={isSubmitting}
|
||||
>
|
||||
Cancel
|
||||
</Button>
|
||||
<Button type="submit" variant="primary/small" disabled={isSubmitting}>
|
||||
{isSubmitting ? "Generating…" : "Generate secret"}
|
||||
</Button>
|
||||
</div>
|
||||
</fetcher.Form>
|
||||
)}
|
||||
</DialogContent>
|
||||
</Dialog>
|
||||
);
|
||||
}
|
||||
|
||||
function TableLoading() {
|
||||
return (
|
||||
<div className="flex h-full items-center justify-center">
|
||||
<Spinner className="size-6" />
|
||||
</div>
|
||||
);
|
||||
}
|
||||
+10
@@ -0,0 +1,10 @@
|
||||
import { Outlet } from "@remix-run/react";
|
||||
import { PageContainer } from "~/components/layout/AppLayout";
|
||||
|
||||
export default function Page() {
|
||||
return (
|
||||
<PageContainer>
|
||||
<Outlet />
|
||||
</PageContainer>
|
||||
);
|
||||
}
|
||||
@@ -7,8 +7,7 @@ import {
|
||||
type SessionItem,
|
||||
type SessionStatus,
|
||||
} from "@trigger.dev/core/v3";
|
||||
import { SessionId } from "@trigger.dev/core/v3/isomorphic";
|
||||
import type { Prisma, Session } from "@trigger.dev/database";
|
||||
import type { Session } from "@trigger.dev/database";
|
||||
import { $replica, prisma, type PrismaClient } from "~/db.server";
|
||||
import { clickhouseFactory } from "~/services/clickhouse/clickhouseFactoryInstance.server";
|
||||
import { logger } from "~/services/logger.server";
|
||||
@@ -17,8 +16,8 @@ import {
|
||||
ensureRunForSession,
|
||||
type SessionTriggerConfig,
|
||||
} from "~/services/realtime/sessionRunManager.server";
|
||||
import { chatSnapshotStoragePathForSession } from "~/services/realtime/chatSnapshot.server";
|
||||
import {
|
||||
findOrCreateSession,
|
||||
serializeSession,
|
||||
serializeSessionsWithFriendlyRunIds,
|
||||
} from "~/services/realtime/sessions.server";
|
||||
@@ -169,66 +168,18 @@ const { action } = createActionApiRoute(
|
||||
},
|
||||
async ({ authentication, body }) => {
|
||||
try {
|
||||
const { id, friendlyId } = SessionId.generate();
|
||||
|
||||
// Idempotent on (env, externalId): two concurrent POSTs converge
|
||||
// to the same row. We refresh `triggerConfig` on the cached path
|
||||
// so newly-deployed schema changes (e.g. an updated
|
||||
// `clientDataSchema` on the agent) propagate to subsequent runs
|
||||
// — the next `ensureRunForSession` reads back the latest config.
|
||||
let session: Session;
|
||||
let isCached = false;
|
||||
|
||||
const triggerConfigJson = body.triggerConfig as unknown as Prisma.InputJsonValue;
|
||||
|
||||
if (body.externalId) {
|
||||
session = await prisma.session.upsert({
|
||||
where: {
|
||||
runtimeEnvironmentId_externalId: {
|
||||
runtimeEnvironmentId: authentication.environment.id,
|
||||
externalId: body.externalId,
|
||||
},
|
||||
},
|
||||
create: {
|
||||
id,
|
||||
friendlyId,
|
||||
externalId: body.externalId,
|
||||
type: body.type,
|
||||
taskIdentifier: body.taskIdentifier,
|
||||
triggerConfig: triggerConfigJson,
|
||||
tags: body.tags ?? [],
|
||||
metadata: body.metadata as Prisma.InputJsonValue | undefined,
|
||||
expiresAt: body.expiresAt ?? null,
|
||||
projectId: authentication.environment.projectId,
|
||||
runtimeEnvironmentId: authentication.environment.id,
|
||||
environmentType: authentication.environment.type,
|
||||
organizationId: authentication.environment.organizationId,
|
||||
streamBasinName: authentication.environment.organization.streamBasinName,
|
||||
chatSnapshotStoragePath: chatSnapshotStoragePathForSession(friendlyId),
|
||||
},
|
||||
update: { triggerConfig: triggerConfigJson },
|
||||
});
|
||||
isCached = session.id !== id;
|
||||
} else {
|
||||
session = await prisma.session.create({
|
||||
data: {
|
||||
id,
|
||||
friendlyId,
|
||||
type: body.type,
|
||||
taskIdentifier: body.taskIdentifier,
|
||||
triggerConfig: triggerConfigJson,
|
||||
tags: body.tags ?? [],
|
||||
metadata: body.metadata as Prisma.InputJsonValue | undefined,
|
||||
expiresAt: body.expiresAt ?? null,
|
||||
projectId: authentication.environment.projectId,
|
||||
runtimeEnvironmentId: authentication.environment.id,
|
||||
environmentType: authentication.environment.type,
|
||||
organizationId: authentication.environment.organizationId,
|
||||
streamBasinName: authentication.environment.organization.streamBasinName,
|
||||
chatSnapshotStoragePath: chatSnapshotStoragePathForSession(friendlyId),
|
||||
},
|
||||
});
|
||||
}
|
||||
// Idempotent on (env, externalId): two concurrent POSTs converge to the same row, and
|
||||
// `triggerConfig` is refreshed on the cached path so a redeployed config reaches the next run.
|
||||
const { session, isCached } = await findOrCreateSession({
|
||||
environment: authentication.environment,
|
||||
externalId: body.externalId,
|
||||
type: body.type,
|
||||
taskIdentifier: body.taskIdentifier,
|
||||
triggerConfig: body.triggerConfig,
|
||||
tags: body.tags,
|
||||
metadata: body.metadata as Record<string, unknown> | undefined,
|
||||
expiresAt: body.expiresAt,
|
||||
});
|
||||
|
||||
// Reject create on a closed session. The upsert path will return
|
||||
// an already-closed row when the caller reuses an externalId, and
|
||||
|
||||
@@ -0,0 +1,60 @@
|
||||
import { json } from "@remix-run/server-runtime";
|
||||
import { z } from "zod";
|
||||
import { webhookReplica } from "~/db.server";
|
||||
import { clickhouseFactory } from "~/services/clickhouse/clickhouseFactoryInstance.server";
|
||||
import { createActionApiRoute } from "~/services/routeBuilders/apiBuilder.server";
|
||||
import { webhookDeliveriesRepository } from "~/services/webhookDeliveriesRepository/webhookDeliveriesRepository.server";
|
||||
import { webhookEngine } from "~/v3/webhookEngine.server";
|
||||
|
||||
const ParamsSchema = z.object({ deliveryId: z.string() });
|
||||
|
||||
// POST /api/v1/webhooks/deliveries/:deliveryId/replay — re-run the delivery's task from its stored
|
||||
// event as a NEW delivery (we don't keep the raw body, so this re-triggers rather than re-verifies).
|
||||
const { action, loader } = createActionApiRoute(
|
||||
{
|
||||
params: ParamsSchema,
|
||||
method: "POST",
|
||||
allowJWT: true,
|
||||
corsStrategy: "all",
|
||||
authorization: { action: "write", resource: () => ({ type: "webhooks" }) },
|
||||
},
|
||||
async ({ params, authentication }) => {
|
||||
const env = authentication.environment;
|
||||
|
||||
// Resolve the friendly id to the internal (id, createdAt) the engine needs (ClickHouse -> PG).
|
||||
const clickhouse = await clickhouseFactory.getClickhouseForOrganization(
|
||||
env.organizationId,
|
||||
"standard"
|
||||
);
|
||||
const original = await webhookDeliveriesRepository({
|
||||
clickhouse,
|
||||
prisma: webhookReplica,
|
||||
}).getDelivery({
|
||||
organizationId: env.organizationId,
|
||||
projectId: env.project.id,
|
||||
environmentId: env.id,
|
||||
friendlyId: params.deliveryId,
|
||||
});
|
||||
if (!original) return json({ error: "Not found" }, { status: 404 });
|
||||
|
||||
const result = await webhookEngine.replayDelivery({
|
||||
id: original.id,
|
||||
createdAt: original.createdAt,
|
||||
});
|
||||
|
||||
switch (result.outcome) {
|
||||
case "replayed":
|
||||
return json({ deliveryId: result.deliveryFriendlyId, replayedFrom: params.deliveryId });
|
||||
case "delivery_not_found":
|
||||
case "endpoint_not_found":
|
||||
return json({ error: "Not found" }, { status: 404 });
|
||||
case "unsupported_target":
|
||||
return json(
|
||||
{ error: "This delivery's endpoint does not route to a task." },
|
||||
{ status: 400 }
|
||||
);
|
||||
}
|
||||
}
|
||||
);
|
||||
|
||||
export { action, loader };
|
||||
@@ -0,0 +1,25 @@
|
||||
import { json } from "@remix-run/server-runtime";
|
||||
import { z } from "zod";
|
||||
import { findWebhookDeliveryResource } from "~/presenters/v3/ApiWebhookDeliveryPresenter.server";
|
||||
import { createLoaderApiRoute } from "~/services/routeBuilders/apiBuilder.server";
|
||||
|
||||
const ParamsSchema = z.object({
|
||||
deliveryId: z.string(),
|
||||
});
|
||||
|
||||
export const loader = createLoaderApiRoute(
|
||||
{
|
||||
params: ParamsSchema,
|
||||
allowJWT: true,
|
||||
corsStrategy: "all",
|
||||
findResource: async (params, authentication) =>
|
||||
findWebhookDeliveryResource(authentication, params.deliveryId),
|
||||
authorization: {
|
||||
action: "read",
|
||||
resource: () => ({ type: "webhooks" }),
|
||||
},
|
||||
},
|
||||
async ({ resource }) => {
|
||||
return json(resource);
|
||||
}
|
||||
);
|
||||
@@ -0,0 +1,22 @@
|
||||
import { json } from "@remix-run/server-runtime";
|
||||
import {
|
||||
ApiWebhookDeliveryListPresenter,
|
||||
ApiWebhookDeliveryListSearchParams,
|
||||
} from "~/presenters/v3/ApiWebhookDeliveryPresenter.server";
|
||||
import { createLoaderApiRoute } from "~/services/routeBuilders/apiBuilder.server";
|
||||
|
||||
export const loader = createLoaderApiRoute(
|
||||
{
|
||||
searchParams: ApiWebhookDeliveryListSearchParams,
|
||||
findResource: async () => 1, // Collection route — nothing to resolve.
|
||||
allowJWT: true,
|
||||
corsStrategy: "all",
|
||||
authorization: { action: "read", resource: () => ({ type: "webhooks" }) },
|
||||
},
|
||||
async ({ searchParams, authentication }) => {
|
||||
const presenter = new ApiWebhookDeliveryListPresenter();
|
||||
const result = await presenter.call(authentication.environment, searchParams);
|
||||
|
||||
return json(result);
|
||||
}
|
||||
);
|
||||
@@ -0,0 +1,34 @@
|
||||
import { json } from "@remix-run/server-runtime";
|
||||
import { z } from "zod";
|
||||
import { webhookPrisma } from "~/db.server";
|
||||
import { findWebhookEndpointResource } from "~/presenters/v3/ApiWebhookEndpointPresenter.server";
|
||||
import { createActionApiRoute } from "~/services/routeBuilders/apiBuilder.server";
|
||||
|
||||
const ParamsSchema = z.object({ endpointId: z.string() });
|
||||
|
||||
// POST /api/v1/webhooks/endpoints/:endpointId/disable — pause an endpoint (ingress returns 404).
|
||||
const { action, loader } = createActionApiRoute(
|
||||
{
|
||||
params: ParamsSchema,
|
||||
method: "POST",
|
||||
allowJWT: true,
|
||||
corsStrategy: "all",
|
||||
authorization: { action: "write", resource: () => ({ type: "webhooks" }) },
|
||||
},
|
||||
async ({ params, authentication }) => {
|
||||
const env = authentication.environment;
|
||||
const endpoint = await webhookPrisma.webhookEndpoint.findFirst({
|
||||
where: { friendlyId: params.endpointId, runtimeEnvironmentId: env.id },
|
||||
});
|
||||
if (!endpoint) return json({ error: "Not found" }, { status: 404 });
|
||||
|
||||
await webhookPrisma.webhookEndpoint.update({
|
||||
where: { id: endpoint.id },
|
||||
data: { status: "INACTIVE", manuallyDeactivatedAt: new Date() },
|
||||
});
|
||||
|
||||
return json(await findWebhookEndpointResource(authentication, params.endpointId));
|
||||
}
|
||||
);
|
||||
|
||||
export { action, loader };
|
||||
@@ -0,0 +1,34 @@
|
||||
import { json } from "@remix-run/server-runtime";
|
||||
import { z } from "zod";
|
||||
import { webhookPrisma } from "~/db.server";
|
||||
import { findWebhookEndpointResource } from "~/presenters/v3/ApiWebhookEndpointPresenter.server";
|
||||
import { createActionApiRoute } from "~/services/routeBuilders/apiBuilder.server";
|
||||
|
||||
const ParamsSchema = z.object({ endpointId: z.string() });
|
||||
|
||||
// POST /api/v1/webhooks/endpoints/:endpointId/enable — resume a paused endpoint.
|
||||
const { action, loader } = createActionApiRoute(
|
||||
{
|
||||
params: ParamsSchema,
|
||||
method: "POST",
|
||||
allowJWT: true,
|
||||
corsStrategy: "all",
|
||||
authorization: { action: "write", resource: () => ({ type: "webhooks" }) },
|
||||
},
|
||||
async ({ params, authentication }) => {
|
||||
const env = authentication.environment;
|
||||
const endpoint = await webhookPrisma.webhookEndpoint.findFirst({
|
||||
where: { friendlyId: params.endpointId, runtimeEnvironmentId: env.id },
|
||||
});
|
||||
if (!endpoint) return json({ error: "Not found" }, { status: 404 });
|
||||
|
||||
await webhookPrisma.webhookEndpoint.update({
|
||||
where: { id: endpoint.id },
|
||||
data: { status: "ACTIVE", manuallyDeactivatedAt: null },
|
||||
});
|
||||
|
||||
return json(await findWebhookEndpointResource(authentication, params.endpointId));
|
||||
}
|
||||
);
|
||||
|
||||
export { action, loader };
|
||||
@@ -0,0 +1,50 @@
|
||||
import { json } from "@remix-run/server-runtime";
|
||||
import { randomBytes } from "node:crypto";
|
||||
import { WebhookVerifierArtifact } from "@trigger.dev/core/v3";
|
||||
import { z } from "zod";
|
||||
import { prisma, webhookPrisma } from "~/db.server";
|
||||
import { createActionApiRoute } from "~/services/routeBuilders/apiBuilder.server";
|
||||
import { getSecretStore } from "~/services/secrets/secretStore.server";
|
||||
|
||||
const ParamsSchema = z.object({ endpointId: z.string() });
|
||||
|
||||
// POST /api/v1/webhooks/endpoints/:endpointId/rotate-secret — mint a new signing secret and return
|
||||
// it ONCE. Only for schemes we generate (hmac / shared-secret); asymmetric endpoints set a public key.
|
||||
const { action, loader } = createActionApiRoute(
|
||||
{
|
||||
params: ParamsSchema,
|
||||
method: "POST",
|
||||
allowJWT: true,
|
||||
corsStrategy: "all",
|
||||
authorization: { action: "write", resource: () => ({ type: "webhooks" }) },
|
||||
},
|
||||
async ({ params, authentication }) => {
|
||||
const env = authentication.environment;
|
||||
const endpoint = await webhookPrisma.webhookEndpoint.findFirst({
|
||||
where: { friendlyId: params.endpointId, runtimeEnvironmentId: env.id },
|
||||
});
|
||||
if (!endpoint) return json({ error: "Not found" }, { status: 404 });
|
||||
|
||||
const parsed = WebhookVerifierArtifact.safeParse(endpoint.verifierArtifact);
|
||||
if (parsed.success && "config" in parsed.data && parsed.data.config.scheme === "asymmetric") {
|
||||
return json(
|
||||
{
|
||||
error: "Cannot generate a secret for an asymmetric endpoint; set its public key instead.",
|
||||
},
|
||||
{ status: 400 }
|
||||
);
|
||||
}
|
||||
|
||||
const secret = `whsec_${randomBytes(32).toString("hex")}`;
|
||||
const secretKey = `webhook:signing-secret:${endpoint.id}`;
|
||||
await getSecretStore("DATABASE", { prismaClient: prisma }).setSecret(secretKey, { secret });
|
||||
await webhookPrisma.webhookEndpoint.update({
|
||||
where: { id: endpoint.id },
|
||||
data: { signingSecretKey: secretKey },
|
||||
});
|
||||
|
||||
return json({ id: endpoint.friendlyId, secretSet: true as const, secret });
|
||||
}
|
||||
);
|
||||
|
||||
export { action, loader };
|
||||
@@ -0,0 +1,25 @@
|
||||
import { json } from "@remix-run/server-runtime";
|
||||
import { z } from "zod";
|
||||
import { findWebhookEndpointResource } from "~/presenters/v3/ApiWebhookEndpointPresenter.server";
|
||||
import { createLoaderApiRoute } from "~/services/routeBuilders/apiBuilder.server";
|
||||
|
||||
const ParamsSchema = z.object({
|
||||
endpointId: z.string(),
|
||||
});
|
||||
|
||||
export const loader = createLoaderApiRoute(
|
||||
{
|
||||
params: ParamsSchema,
|
||||
allowJWT: true,
|
||||
corsStrategy: "all",
|
||||
findResource: async (params, authentication) =>
|
||||
findWebhookEndpointResource(authentication, params.endpointId),
|
||||
authorization: {
|
||||
action: "read",
|
||||
resource: () => ({ type: "webhooks" }),
|
||||
},
|
||||
},
|
||||
async ({ resource }) => {
|
||||
return json(resource);
|
||||
}
|
||||
);
|
||||
@@ -0,0 +1,22 @@
|
||||
import { json } from "@remix-run/server-runtime";
|
||||
import {
|
||||
ApiWebhookEndpointListPresenter,
|
||||
ApiWebhookEndpointListSearchParams,
|
||||
} from "~/presenters/v3/ApiWebhookEndpointPresenter.server";
|
||||
import { createLoaderApiRoute } from "~/services/routeBuilders/apiBuilder.server";
|
||||
|
||||
export const loader = createLoaderApiRoute(
|
||||
{
|
||||
searchParams: ApiWebhookEndpointListSearchParams,
|
||||
findResource: async () => 1, // Collection route — nothing to resolve.
|
||||
allowJWT: true,
|
||||
corsStrategy: "all",
|
||||
authorization: { action: "read", resource: () => ({ type: "webhooks" }) },
|
||||
},
|
||||
async ({ searchParams, authentication }) => {
|
||||
const presenter = new ApiWebhookEndpointListPresenter();
|
||||
const result = await presenter.call(authentication.environment, searchParams);
|
||||
|
||||
return json(result);
|
||||
}
|
||||
);
|
||||
@@ -10,6 +10,7 @@ import {
|
||||
resolveSessionWithWriterFallback,
|
||||
} from "~/services/realtime/sessions.server";
|
||||
import { getRealtimeStreamInstance } from "~/services/realtime/v1StreamsGlobal.server";
|
||||
import { stripClientWebhookActionSource } from "~/services/realtime/sanitizeSessionInput.server";
|
||||
import {
|
||||
claimSessionStreamPart,
|
||||
drainSessionStreamWaitpoints,
|
||||
@@ -137,7 +138,11 @@ const { action, loader } = createActionApiRoute(
|
||||
|
||||
const addressingKey = canonicalSessionAddressingKey(session, params.session);
|
||||
|
||||
const part = await request.text();
|
||||
let part = await request.text();
|
||||
if (params.io === "in") {
|
||||
part = stripClientWebhookActionSource(part);
|
||||
}
|
||||
|
||||
const clientPartId = request.headers.get("X-Part-Id");
|
||||
const partId = clientPartId ?? nanoid(7);
|
||||
|
||||
|
||||
+69
-16
@@ -22,6 +22,8 @@ const RequestSchema = z.object({
|
||||
payloadSchema: z.string().max(50_000).optional(),
|
||||
currentPayload: z.string().max(50_000).optional(),
|
||||
isAgent: z.enum(["true", "false"]).optional(),
|
||||
payloadKind: z.enum(["standard", "agent", "webhook"]).optional(),
|
||||
providerSource: z.string().max(100).optional(),
|
||||
});
|
||||
|
||||
export async function action({ request, params }: ActionFunctionArgs) {
|
||||
@@ -65,20 +67,31 @@ export async function action({ request, params }: ActionFunctionArgs) {
|
||||
);
|
||||
}
|
||||
|
||||
const { prompt, taskIdentifier, payloadSchema, currentPayload, isAgent } = submission.data;
|
||||
const agentMode = isAgent === "true";
|
||||
const {
|
||||
prompt,
|
||||
taskIdentifier,
|
||||
payloadSchema,
|
||||
currentPayload,
|
||||
isAgent,
|
||||
payloadKind,
|
||||
providerSource,
|
||||
} = submission.data;
|
||||
const kind = payloadKind ?? (isAgent === "true" ? "agent" : "standard");
|
||||
|
||||
logger.info("[AI payload] Generating payload", {
|
||||
taskIdentifier,
|
||||
hasPayloadSchema: !!payloadSchema,
|
||||
hasCurrentPayload: !!currentPayload,
|
||||
promptLength: prompt.length,
|
||||
agentMode,
|
||||
kind,
|
||||
});
|
||||
|
||||
const systemPrompt = agentMode
|
||||
? buildAgentClientDataPrompt(taskIdentifier, payloadSchema, currentPayload)
|
||||
: buildSystemPrompt(taskIdentifier, payloadSchema, currentPayload);
|
||||
const systemPrompt =
|
||||
kind === "webhook"
|
||||
? buildWebhookEventPrompt(providerSource, currentPayload)
|
||||
: kind === "agent"
|
||||
? buildAgentClientDataPrompt(taskIdentifier, payloadSchema, currentPayload)
|
||||
: buildSystemPrompt(taskIdentifier, payloadSchema, currentPayload);
|
||||
|
||||
const stream = new ReadableStream({
|
||||
async start(controller) {
|
||||
@@ -101,16 +114,19 @@ export async function action({ request, params }: ActionFunctionArgs) {
|
||||
abortSignal: getRequestAbortSignal(),
|
||||
system: systemPrompt,
|
||||
prompt,
|
||||
tools: {
|
||||
getTaskSourceCode: tool({
|
||||
description:
|
||||
"Look up the source code of the task to understand what payload shape it expects. Use this when there is no JSON Schema available and you need to infer the payload structure from the task implementation.",
|
||||
inputSchema: z.object({}),
|
||||
execute: async () => {
|
||||
return getTaskSourceCode(environment.id, environment.type, taskIdentifier);
|
||||
},
|
||||
}),
|
||||
},
|
||||
tools:
|
||||
kind === "webhook"
|
||||
? {}
|
||||
: {
|
||||
getTaskSourceCode: tool({
|
||||
description:
|
||||
"Look up the source code of the task to understand what payload shape it expects. Use this when there is no JSON Schema available and you need to infer the payload structure from the task implementation.",
|
||||
inputSchema: z.object({}),
|
||||
execute: async () => {
|
||||
return getTaskSourceCode(environment.id, environment.type, taskIdentifier);
|
||||
},
|
||||
}),
|
||||
},
|
||||
stopWhen: stepCountIs(3),
|
||||
});
|
||||
|
||||
@@ -293,6 +309,43 @@ Use this as context but generate new client data based on the user's prompt.`;
|
||||
return prompt;
|
||||
}
|
||||
|
||||
function buildWebhookEventPrompt(providerSource?: string, currentPayload?: string): string {
|
||||
const provider = providerSource && providerSource !== "custom" ? providerSource : undefined;
|
||||
|
||||
let prompt = `You are generating a realistic example webhook event body${
|
||||
provider ? ` for the "${provider}" provider` : ""
|
||||
}.
|
||||
|
||||
Return ONLY the raw JSON event body that the provider would POST to a webhook endpoint, wrapped in a \`\`\`json code block. Do NOT include HTTP headers, a signature, or any {event, headers} envelope — just the event JSON itself.
|
||||
|
||||
Requirements:
|
||||
- Generate a realistic, well-formed event payload matching the shape ${
|
||||
provider ? `the "${provider}" provider actually sends` : "a typical webhook provider sends"
|
||||
}.
|
||||
- Use plausible values (real-looking ids with the provider's prefixes, unix/ISO timestamps, emails, amounts).
|
||||
- Include the fields a consumer branches on (e.g. an event "type"/"event" discriminator and a "data"/"object" payload) when the provider uses them.
|
||||
- The JSON must be valid and parseable.`;
|
||||
|
||||
if (provider) {
|
||||
prompt += `
|
||||
|
||||
If you know the "${provider}" webhook event schema, follow it closely (field names, nesting, id prefixes) and pick a common, representative event type for this provider.`;
|
||||
}
|
||||
|
||||
if (currentPayload) {
|
||||
prompt += `
|
||||
|
||||
The current event body in the editor is:
|
||||
\`\`\`json
|
||||
${currentPayload}
|
||||
\`\`\`
|
||||
|
||||
Use it as context but generate a new event based on the user's prompt.`;
|
||||
}
|
||||
|
||||
return prompt;
|
||||
}
|
||||
|
||||
function buildSystemPrompt(
|
||||
taskIdentifier: string,
|
||||
payloadSchema?: string,
|
||||
|
||||
+174
@@ -0,0 +1,174 @@
|
||||
import { type LoaderFunctionArgs } from "@remix-run/server-runtime";
|
||||
import { typedjson } from "remix-typedjson";
|
||||
import { z } from "zod";
|
||||
import { type WebhookDeliveryStatus } from "@trigger.dev/database";
|
||||
import { $replica, webhookReplica } from "~/db.server";
|
||||
import { WebhookDeliveriesListPresenter } from "~/presenters/v3/WebhookDeliveriesListPresenter.server";
|
||||
import { resolveDeliveryRunTargets } from "~/presenters/v3/WebhookDetailPresenter.server";
|
||||
import { clickhouseFactory } from "~/services/clickhouse/clickhouseFactoryInstance.server";
|
||||
import { loadProjectEnvironmentFromRequest } from "~/services/loadProjectEnvironmentFromRequest.server";
|
||||
import { requireUser } from "~/services/session.server";
|
||||
import { FEATURE_FLAG } from "~/v3/featureFlags";
|
||||
import { flag } from "~/v3/featureFlags.server";
|
||||
import {
|
||||
type ListedWebhookDelivery,
|
||||
webhookDeliveriesRepository,
|
||||
} from "~/services/webhookDeliveriesRepository/webhookDeliveriesRepository.server";
|
||||
|
||||
const deliveryIdsQueryParam = z
|
||||
.string()
|
||||
.optional()
|
||||
.transform((value) => {
|
||||
const ids =
|
||||
value
|
||||
?.split(",")
|
||||
.map((id) => id.trim())
|
||||
.filter(Boolean) ?? [];
|
||||
return [...new Set(ids)].slice(0, 100);
|
||||
});
|
||||
|
||||
const SearchParamsSchema = z.object({
|
||||
webhookEndpointId: z.string().optional(),
|
||||
deliveryIds: deliveryIdsQueryParam,
|
||||
includeNewDeliveries: z
|
||||
.string()
|
||||
.optional()
|
||||
.transform((value) => value === "true"),
|
||||
since: z.coerce.number().optional(),
|
||||
to: z.coerce.number().optional(),
|
||||
});
|
||||
|
||||
const KNOWN_DELIVERY_STATUSES: readonly WebhookDeliveryStatus[] = [
|
||||
"PENDING",
|
||||
"PROCESSING",
|
||||
"SUCCEEDED",
|
||||
"FAILED",
|
||||
"FILTERED",
|
||||
];
|
||||
|
||||
/**
|
||||
* Parse the deliveries-list filters so the new-deliveries count applies the same ones the list is
|
||||
* showing (its badge must never count events the filtered list would exclude). Mirrors the top-level
|
||||
* list's param parsing: repeated or CSV `statuses`, repeated `webhooks`, `deliveryId`, `runId`, `test`.
|
||||
*/
|
||||
function parseListFilters(searchParams: URLSearchParams) {
|
||||
const statusValues = searchParams
|
||||
.getAll("statuses")
|
||||
.flatMap((value) => value.split(","))
|
||||
.map((value) => value.trim())
|
||||
.filter((value): value is WebhookDeliveryStatus =>
|
||||
KNOWN_DELIVERY_STATUSES.includes(value as WebhookDeliveryStatus)
|
||||
);
|
||||
const webhooks = searchParams.getAll("webhooks").filter((value) => value.length > 0);
|
||||
const testParam = searchParams.get("test");
|
||||
return {
|
||||
statuses: statusValues.length > 0 ? statusValues : undefined,
|
||||
webhooks: webhooks.length > 0 ? webhooks : undefined,
|
||||
deliveryId: searchParams.get("deliveryId") ?? undefined,
|
||||
runId: searchParams.get("runId") ?? undefined,
|
||||
isTest: testParam === "only" ? true : testParam === "hide" ? false : undefined,
|
||||
};
|
||||
}
|
||||
|
||||
export type LiveDeliveryFields = {
|
||||
friendlyId: string;
|
||||
status: ListedWebhookDelivery["status"];
|
||||
runId: string | null;
|
||||
run: { friendlyId: string } | null;
|
||||
session: { friendlyId: string; externalId: string | null } | null;
|
||||
errorMessage: string | null;
|
||||
processedAt: Date | null;
|
||||
};
|
||||
|
||||
export function mapDeliveryToLiveFields(
|
||||
delivery: ListedWebhookDelivery,
|
||||
targets: {
|
||||
runFriendlyIdById: Map<string, string>;
|
||||
sessionByRunId: Map<string, { friendlyId: string; externalId: string | null }>;
|
||||
}
|
||||
): LiveDeliveryFields {
|
||||
const runFriendlyId = delivery.runId ? targets.runFriendlyIdById.get(delivery.runId) : undefined;
|
||||
return {
|
||||
friendlyId: delivery.friendlyId,
|
||||
status: delivery.status,
|
||||
runId: delivery.runId,
|
||||
run: runFriendlyId ? { friendlyId: runFriendlyId } : null,
|
||||
session: delivery.runId ? (targets.sessionByRunId.get(delivery.runId) ?? null) : null,
|
||||
errorMessage: delivery.errorMessage,
|
||||
processedAt: delivery.processedAt,
|
||||
};
|
||||
}
|
||||
|
||||
export async function loader({ request, params }: LoaderFunctionArgs) {
|
||||
const url = new URL(request.url);
|
||||
const { webhookEndpointId, deliveryIds, includeNewDeliveries, since, to } =
|
||||
SearchParamsSchema.parse(Object.fromEntries(url.searchParams));
|
||||
|
||||
const newDeliveriesSince = includeNewDeliveries && since !== undefined ? since : undefined;
|
||||
|
||||
if (deliveryIds.length === 0 && newDeliveriesSince === undefined) {
|
||||
return typedjson({ deliveries: [] });
|
||||
}
|
||||
|
||||
const { project, environment } = await loadProjectEnvironmentFromRequest(request, params);
|
||||
|
||||
const user = await requireUser(request);
|
||||
if (!user.admin && !user.isImpersonating) {
|
||||
const org = await $replica.organization.findFirst({
|
||||
where: { id: project.organizationId },
|
||||
select: { featureFlags: true },
|
||||
});
|
||||
const enabled = await flag({
|
||||
key: FEATURE_FLAG.hasWebhooksAccess,
|
||||
defaultValue: false,
|
||||
overrides: (org?.featureFlags as Record<string, unknown>) ?? {},
|
||||
});
|
||||
if (!enabled) throw new Response("Not found", { status: 404 });
|
||||
}
|
||||
|
||||
const clickhouse = await clickhouseFactory.getClickhouseForOrganization(
|
||||
project.organizationId,
|
||||
"standard"
|
||||
);
|
||||
const repository = webhookDeliveriesRepository({ clickhouse, prisma: webhookReplica });
|
||||
|
||||
const [deliveries, newDeliveriesResult] = await Promise.all([
|
||||
deliveryIds.length > 0
|
||||
? repository
|
||||
.getDeliveriesByFriendlyIds({
|
||||
organizationId: project.organizationId,
|
||||
projectId: project.id,
|
||||
environmentId: environment.id,
|
||||
friendlyIds: deliveryIds,
|
||||
})
|
||||
.then(async (rows) => {
|
||||
const targets = await resolveDeliveryRunTargets($replica, rows);
|
||||
return rows.map((row) => mapDeliveryToLiveFields(row, targets));
|
||||
})
|
||||
: Promise.resolve([]),
|
||||
newDeliveriesSince !== undefined
|
||||
? (async () => {
|
||||
const filters = parseListFilters(url.searchParams);
|
||||
const count = await new WebhookDeliveriesListPresenter(
|
||||
$replica,
|
||||
clickhouse
|
||||
).countNewDeliveries({
|
||||
organizationId: project.organizationId,
|
||||
projectId: project.id,
|
||||
environmentId: environment.id,
|
||||
webhookEndpointId,
|
||||
...filters,
|
||||
since: newDeliveriesSince,
|
||||
to,
|
||||
});
|
||||
return { count, since: newDeliveriesSince };
|
||||
})()
|
||||
: Promise.resolve(undefined),
|
||||
]);
|
||||
|
||||
if (newDeliveriesResult) {
|
||||
return typedjson({ deliveries, ...newDeliveriesResult });
|
||||
}
|
||||
|
||||
return typedjson({ deliveries });
|
||||
}
|
||||
+148
@@ -0,0 +1,148 @@
|
||||
import { type LoaderFunctionArgs } from "@remix-run/server-runtime";
|
||||
import { WebhookVerifierArtifact } from "@trigger.dev/core/v3";
|
||||
import { typedjson } from "remix-typedjson";
|
||||
import { z } from "zod";
|
||||
import { $replica, webhookReplica } from "~/db.server";
|
||||
import { findProjectBySlug } from "~/models/project.server";
|
||||
import { findEnvironmentBySlug } from "~/models/runtimeEnvironment.server";
|
||||
import { WebhookDetailPresenter } from "~/presenters/v3/WebhookDetailPresenter.server";
|
||||
import { clickhouseFactory } from "~/services/clickhouse/clickhouseFactoryInstance.server";
|
||||
import { requireUser } from "~/services/session.server";
|
||||
import { webhookDeliveriesRepository } from "~/services/webhookDeliveriesRepository/webhookDeliveriesRepository.server";
|
||||
import { EnvironmentParamSchema } from "~/utils/pathBuilder";
|
||||
import { FEATURE_FLAG } from "~/v3/featureFlags";
|
||||
import { flag } from "~/v3/featureFlags.server";
|
||||
|
||||
const ParamsSchema = EnvironmentParamSchema.extend({ endpointParam: z.string() });
|
||||
|
||||
const RECENT_DELIVERIES_LIMIT = 20;
|
||||
|
||||
export type ReplaySourceDelivery = {
|
||||
friendlyId: string;
|
||||
status: string;
|
||||
isTest: boolean;
|
||||
externalDeliveryId: string;
|
||||
createdAt: string;
|
||||
};
|
||||
|
||||
export type ReplaySourceData =
|
||||
| { kind: "list"; deliveries: ReplaySourceDelivery[] }
|
||||
| { kind: "payload"; body: string; headers: Record<string, string> }
|
||||
| { kind: "error"; error: string };
|
||||
|
||||
const HEADER_DENYLIST = new Set([
|
||||
"content-type",
|
||||
"content-length",
|
||||
"host",
|
||||
"accept",
|
||||
"accept-encoding",
|
||||
"accept-language",
|
||||
"user-agent",
|
||||
"connection",
|
||||
"cache-control",
|
||||
"pragma",
|
||||
"origin",
|
||||
"referer",
|
||||
"x-trigger-test",
|
||||
]);
|
||||
|
||||
function isNoiseHeader(lower: string): boolean {
|
||||
return lower.startsWith("sec-") || HEADER_DENYLIST.has(lower);
|
||||
}
|
||||
|
||||
function signatureHeaderNames(artifact: unknown): Set<string> {
|
||||
const names = new Set<string>();
|
||||
const parsed = WebhookVerifierArtifact.safeParse(artifact);
|
||||
if (!parsed.success || parsed.data.kind === "bundle") return names;
|
||||
const config = parsed.data.config;
|
||||
if (config.scheme === "hmac" || config.scheme === "asymmetric") {
|
||||
names.add(config.signatureHeader.toLowerCase());
|
||||
if (config.timestamp?.source.from === "header") {
|
||||
names.add(config.timestamp.source.name.toLowerCase());
|
||||
}
|
||||
}
|
||||
return names;
|
||||
}
|
||||
|
||||
export async function loader({ request, params }: LoaderFunctionArgs) {
|
||||
const user = await requireUser(request);
|
||||
const { organizationSlug, projectParam, envParam, endpointParam } = ParamsSchema.parse(params);
|
||||
|
||||
const project = await findProjectBySlug(organizationSlug, projectParam, user.id);
|
||||
if (!project) return typedjson({ kind: "error", error: "Project not found" } as ReplaySourceData);
|
||||
const environment = await findEnvironmentBySlug(project.id, envParam, user.id);
|
||||
if (!environment)
|
||||
return typedjson({ kind: "error", error: "Environment not found" } as ReplaySourceData);
|
||||
|
||||
if (!user.admin && !user.isImpersonating) {
|
||||
const org = await $replica.organization.findFirst({
|
||||
where: { id: project.organizationId },
|
||||
select: { featureFlags: true },
|
||||
});
|
||||
const enabled = await flag({
|
||||
key: FEATURE_FLAG.hasWebhooksAccess,
|
||||
defaultValue: false,
|
||||
overrides: (org?.featureFlags as Record<string, unknown>) ?? {},
|
||||
});
|
||||
if (!enabled) return typedjson({ kind: "error", error: "Not found" } as ReplaySourceData);
|
||||
}
|
||||
|
||||
const clickhouse = await clickhouseFactory.getClickhouseForOrganization(
|
||||
project.organizationId,
|
||||
"standard"
|
||||
);
|
||||
const presenter = new WebhookDetailPresenter($replica, clickhouse);
|
||||
const endpoint = await presenter.findEndpoint({
|
||||
environmentId: environment.id,
|
||||
endpointFriendlyId: endpointParam,
|
||||
});
|
||||
if (!endpoint)
|
||||
return typedjson({ kind: "error", error: "Endpoint not found" } as ReplaySourceData);
|
||||
|
||||
const repository = webhookDeliveriesRepository({ clickhouse, prisma: webhookReplica });
|
||||
|
||||
const url = new URL(request.url);
|
||||
const deliveryId = url.searchParams.get("deliveryId") ?? undefined;
|
||||
|
||||
if (deliveryId) {
|
||||
const delivery = await repository.getDelivery({
|
||||
organizationId: project.organizationId,
|
||||
projectId: project.id,
|
||||
environmentId: environment.id,
|
||||
friendlyId: deliveryId,
|
||||
});
|
||||
if (!delivery || delivery.webhookEndpointId !== endpoint.id) {
|
||||
return typedjson({ kind: "error", error: "Delivery not found" } as ReplaySourceData);
|
||||
}
|
||||
const body =
|
||||
delivery.parsedEvent != null ? JSON.stringify(delivery.parsedEvent, null, 2) : "{}";
|
||||
const strip = signatureHeaderNames(endpoint.verifierArtifact);
|
||||
const rawHeaders = (delivery.headers ?? {}) as Record<string, unknown>;
|
||||
const headers: Record<string, string> = {};
|
||||
for (const [key, value] of Object.entries(rawHeaders)) {
|
||||
const lower = key.toLowerCase();
|
||||
if (isNoiseHeader(lower) || strip.has(lower)) continue;
|
||||
if (typeof value === "string") headers[key] = value;
|
||||
}
|
||||
return typedjson({ kind: "payload", body, headers } as ReplaySourceData);
|
||||
}
|
||||
|
||||
const { deliveries } = await repository.listDeliveries({
|
||||
organizationId: project.organizationId,
|
||||
projectId: project.id,
|
||||
environmentId: environment.id,
|
||||
webhookEndpointId: endpoint.id,
|
||||
page: { size: RECENT_DELIVERIES_LIMIT },
|
||||
});
|
||||
|
||||
return typedjson({
|
||||
kind: "list",
|
||||
deliveries: deliveries.map((delivery) => ({
|
||||
friendlyId: delivery.friendlyId,
|
||||
status: delivery.status,
|
||||
isTest: delivery.isTest,
|
||||
externalDeliveryId: delivery.externalDeliveryId,
|
||||
createdAt: delivery.createdAt.toISOString(),
|
||||
})),
|
||||
} as ReplaySourceData);
|
||||
}
|
||||
+221
@@ -0,0 +1,221 @@
|
||||
import { signWithVerifierConfig } from "@internal/webhook-engine";
|
||||
import { type ActionFunctionArgs, redirect } from "@remix-run/server-runtime";
|
||||
import { WebhookVerifierArtifact } from "@trigger.dev/core/v3";
|
||||
import { z } from "zod";
|
||||
import { $replica, prisma } from "~/db.server";
|
||||
import { env } from "~/env.server";
|
||||
import { findProjectBySlug } from "~/models/project.server";
|
||||
import { findEnvironmentBySlug } from "~/models/runtimeEnvironment.server";
|
||||
import { WebhookDetailPresenter } from "~/presenters/v3/WebhookDetailPresenter.server";
|
||||
import { clickhouseFactory } from "~/services/clickhouse/clickhouseFactoryInstance.server";
|
||||
import { getSecretStore } from "~/services/secrets/secretStore.server";
|
||||
import { requireUser } from "~/services/session.server";
|
||||
import { webhookConsoleSendRateLimiter } from "~/services/webhookConsoleSendRateLimit.server";
|
||||
import { EnvironmentParamSchema } from "~/utils/pathBuilder";
|
||||
import { webhookIngressUrl } from "~/utils/webhookIngressUrl.server";
|
||||
import { webhookEngine } from "~/v3/webhookEngine.server";
|
||||
import { FEATURE_FLAG } from "~/v3/featureFlags";
|
||||
import { flag } from "~/v3/featureFlags.server";
|
||||
|
||||
const ParamsSchema = EnvironmentParamSchema.extend({ endpointParam: z.string() });
|
||||
|
||||
const SigningSecretSchema = z.object({ secret: z.string() });
|
||||
|
||||
const SendSchema = z.object({
|
||||
body: z.string(),
|
||||
headers: z.record(z.string()).optional(),
|
||||
signatureMode: z.enum(["signed", "unsigned", "tampered", "simulate"]).default("signed"),
|
||||
redirect: z.boolean().default(true),
|
||||
});
|
||||
|
||||
export type WebhookSendResult =
|
||||
| {
|
||||
success: true;
|
||||
httpStatus: number;
|
||||
deliveryId?: string;
|
||||
deduplicated?: boolean;
|
||||
handshake?: boolean;
|
||||
responseBody: string;
|
||||
}
|
||||
| { success: false; error: string; notSignable?: boolean };
|
||||
|
||||
/**
|
||||
* Authenticated dashboard test-send. Session/cookie auth via requireUser; authorization via
|
||||
* findProjectBySlug (org membership) + findEnvironmentBySlug (env access) + the webhooks feature flag.
|
||||
* The delivery is injected in-process through the engine (verify -> filter -> route -> run), never by
|
||||
* looping back through the public ingress, so it does not consume the provider's ingress rate budget
|
||||
* and is not gated by WEBHOOK_INGRESS_ENABLED.
|
||||
*/
|
||||
export async function action({ request, params }: ActionFunctionArgs): Promise<WebhookSendResult> {
|
||||
const user = await requireUser(request);
|
||||
const { organizationSlug, projectParam, envParam, endpointParam } = ParamsSchema.parse(params);
|
||||
|
||||
if (env.WEBHOOK_ENABLED !== "1") {
|
||||
return { success: false, error: "Webhooks are not enabled on this instance." };
|
||||
}
|
||||
|
||||
const project = await findProjectBySlug(organizationSlug, projectParam, user.id);
|
||||
if (!project) return { success: false, error: "Project not found" };
|
||||
const environment = await findEnvironmentBySlug(project.id, envParam, user.id);
|
||||
if (!environment) return { success: false, error: "Environment not found" };
|
||||
|
||||
if (!user.admin && !user.isImpersonating) {
|
||||
const org = await $replica.organization.findFirst({
|
||||
where: { id: project.organizationId },
|
||||
select: { featureFlags: true },
|
||||
});
|
||||
const enabled = await flag({
|
||||
key: FEATURE_FLAG.hasWebhooksAccess,
|
||||
defaultValue: false,
|
||||
overrides: (org?.featureFlags as Record<string, unknown>) ?? {},
|
||||
});
|
||||
if (!enabled) return { success: false, error: "Not found" };
|
||||
}
|
||||
|
||||
const rateLimit = await webhookConsoleSendRateLimiter.limit(user.id);
|
||||
if (!rateLimit.success) {
|
||||
return { success: false, error: "Too many test sends. Wait a moment and try again." };
|
||||
}
|
||||
|
||||
const parsed = SendSchema.safeParse(await request.json());
|
||||
if (!parsed.success) {
|
||||
return { success: false, error: parsed.error.issues[0]?.message ?? "Invalid request" };
|
||||
}
|
||||
const { body, headers: providedHeaders, signatureMode, redirect: shouldRedirect } = parsed.data;
|
||||
|
||||
const rawBody = new TextEncoder().encode(body);
|
||||
const limitBytes = env.WEBHOOK_INGRESS_BODY_SIZE_LIMIT_MB * 1024 * 1024;
|
||||
if (rawBody.length > limitBytes) {
|
||||
return {
|
||||
success: false,
|
||||
error: `Body exceeds the ${env.WEBHOOK_INGRESS_BODY_SIZE_LIMIT_MB} MB limit.`,
|
||||
};
|
||||
}
|
||||
|
||||
const clickhouse = await clickhouseFactory.getClickhouseForOrganization(
|
||||
project.organizationId,
|
||||
"standard"
|
||||
);
|
||||
const presenter = new WebhookDetailPresenter($replica, clickhouse);
|
||||
const endpoint = await presenter.findEndpoint({
|
||||
environmentId: environment.id,
|
||||
endpointFriendlyId: endpointParam,
|
||||
});
|
||||
if (!endpoint) return { success: false, error: "Endpoint not found" };
|
||||
|
||||
const verifier = WebhookVerifierArtifact.safeParse(endpoint.verifierArtifact);
|
||||
if (!verifier.success) return { success: false, error: "Endpoint has no verifier config" };
|
||||
if (verifier.data.kind === "bundle") {
|
||||
return { success: false, error: "Bundle verifiers cannot be sent from the console" };
|
||||
}
|
||||
const config = verifier.data.config;
|
||||
|
||||
const ingressUrl = webhookIngressUrl(endpoint.opaqueId);
|
||||
const baseHeaders = { ...(providedHeaders ?? {}) };
|
||||
|
||||
let ingestHeaders: Record<string, string> = baseHeaders;
|
||||
let ingestBody: Uint8Array = rawBody;
|
||||
let ingestUrl = ingressUrl;
|
||||
|
||||
if (signatureMode === "signed") {
|
||||
if (!endpoint.hasSigningSecret) {
|
||||
return { success: false, error: "This endpoint has no signing secret. Set one first." };
|
||||
}
|
||||
const secretStore = getSecretStore("DATABASE", { prismaClient: prisma });
|
||||
const stored = await secretStore.getSecret(
|
||||
SigningSecretSchema,
|
||||
`webhook:signing-secret:${endpoint.id}`
|
||||
);
|
||||
if (!stored?.secret) {
|
||||
return { success: false, error: "The signing secret could not be read." };
|
||||
}
|
||||
const signed = signWithVerifierConfig({
|
||||
config,
|
||||
secret: stored.secret,
|
||||
rawBody,
|
||||
url: ingressUrl,
|
||||
headers: baseHeaders,
|
||||
});
|
||||
if (!signed.ok) {
|
||||
return { success: false, error: signed.error, notSignable: signed.notSignable };
|
||||
}
|
||||
ingestHeaders = signed.headers;
|
||||
ingestBody = signed.body;
|
||||
ingestUrl = signed.url;
|
||||
} else if (signatureMode === "tampered") {
|
||||
const bogus = signWithVerifierConfig({
|
||||
config,
|
||||
secret: `tampered-${Date.now()}`,
|
||||
rawBody,
|
||||
url: ingressUrl,
|
||||
headers: baseHeaders,
|
||||
});
|
||||
if (bogus.ok) {
|
||||
ingestHeaders = bogus.headers;
|
||||
ingestBody = bogus.body;
|
||||
ingestUrl = bogus.url;
|
||||
} else if (config.scheme === "hmac" || config.scheme === "asymmetric") {
|
||||
ingestHeaders = { ...baseHeaders, [config.signatureHeader]: "deadbeef" };
|
||||
}
|
||||
}
|
||||
|
||||
const finalHeaders: Record<string, string> = {
|
||||
"content-type": "application/json",
|
||||
...ingestHeaders,
|
||||
"x-trigger-test": "1",
|
||||
};
|
||||
|
||||
const ingestInput = {
|
||||
opaqueId: endpoint.opaqueId,
|
||||
rawBytes: ingestBody,
|
||||
headers: finalHeaders,
|
||||
url: ingestUrl,
|
||||
};
|
||||
const result =
|
||||
signatureMode === "simulate"
|
||||
? await webhookEngine.simulateInject(ingestInput)
|
||||
: await webhookEngine.ingest(ingestInput);
|
||||
|
||||
const deliveryPathFor = (friendlyId: string) =>
|
||||
`/orgs/${organizationSlug}/projects/${projectParam}/env/${envParam}/webhooks/deliveries/${friendlyId}`;
|
||||
|
||||
switch (result.outcome) {
|
||||
case "accepted": {
|
||||
const friendlyId = result.deliveryFriendlyId;
|
||||
if (shouldRedirect) throw redirect(deliveryPathFor(friendlyId));
|
||||
return {
|
||||
success: true,
|
||||
httpStatus: 200,
|
||||
deliveryId: friendlyId,
|
||||
responseBody: JSON.stringify({ received: true, deliveryId: friendlyId }),
|
||||
};
|
||||
}
|
||||
case "handshake":
|
||||
return { success: true, httpStatus: 200, handshake: true, responseBody: result.body };
|
||||
case "duplicate": {
|
||||
const friendlyId = result.deliveryId;
|
||||
if (shouldRedirect && friendlyId) throw redirect(deliveryPathFor(friendlyId));
|
||||
return {
|
||||
success: true,
|
||||
httpStatus: 200,
|
||||
deliveryId: friendlyId,
|
||||
deduplicated: true,
|
||||
responseBody: JSON.stringify({ received: true, deliveryId: friendlyId }),
|
||||
};
|
||||
}
|
||||
case "verification_failed":
|
||||
return {
|
||||
success: false,
|
||||
error: result.error ?? "Signature verification failed. No delivery was recorded.",
|
||||
};
|
||||
case "secret_missing":
|
||||
return { success: false, error: "This endpoint has no signing secret. Set one first." };
|
||||
case "endpoint_not_found":
|
||||
case "endpoint_inactive":
|
||||
return { success: false, error: "This endpoint is not active." };
|
||||
case "enqueue_failed":
|
||||
return { success: false, error: result.error ?? "Failed to record the delivery." };
|
||||
default:
|
||||
return { success: false, error: "The delivery could not be sent." };
|
||||
}
|
||||
}
|
||||
+122
@@ -0,0 +1,122 @@
|
||||
import {
|
||||
categoryLabel,
|
||||
categoryOrder,
|
||||
getProvider,
|
||||
getSample,
|
||||
sampleManifest,
|
||||
} from "@internal/webhook-sources";
|
||||
import { type LoaderFunctionArgs } from "@remix-run/server-runtime";
|
||||
import { typedjson } from "remix-typedjson";
|
||||
import { $replica } from "~/db.server";
|
||||
import { findProjectBySlug } from "~/models/project.server";
|
||||
import { requireUser } from "~/services/session.server";
|
||||
import { EnvironmentParamSchema } from "~/utils/pathBuilder";
|
||||
import { FEATURE_FLAG } from "~/v3/featureFlags";
|
||||
import { flag } from "~/v3/featureFlags.server";
|
||||
|
||||
export type WebhookSampleMeta = {
|
||||
provider: string;
|
||||
providerLabel?: string;
|
||||
presetId?: string;
|
||||
eventType: string;
|
||||
name: string;
|
||||
description?: string;
|
||||
};
|
||||
|
||||
/** Provider metadata joined from the registry, for the producer pane. */
|
||||
export type WebhookProviderMeta = {
|
||||
id: string;
|
||||
label: string;
|
||||
category?: string;
|
||||
categoryLabel?: string;
|
||||
docsUrl?: string;
|
||||
/** Set when the provider maps to a verifier preset (i.e. round-trippable / first-class). */
|
||||
preset?: string;
|
||||
eventCount: number;
|
||||
};
|
||||
|
||||
export type WebhookSamplesData =
|
||||
| { kind: "manifest"; providers: WebhookProviderMeta[]; samples: WebhookSampleMeta[] }
|
||||
| { kind: "body"; body: string; extraHeaders?: Record<string, string> }
|
||||
| { kind: "error"; error: string };
|
||||
|
||||
function titleCase(id: string): string {
|
||||
return id.charAt(0).toUpperCase() + id.slice(1);
|
||||
}
|
||||
|
||||
export async function loader({ request, params }: LoaderFunctionArgs) {
|
||||
const user = await requireUser(request);
|
||||
const { organizationSlug, projectParam } = EnvironmentParamSchema.parse(params);
|
||||
|
||||
const project = await findProjectBySlug(organizationSlug, projectParam, user.id);
|
||||
if (!project)
|
||||
return typedjson({ kind: "error", error: "Project not found" } as WebhookSamplesData);
|
||||
|
||||
if (!user.admin && !user.isImpersonating) {
|
||||
const org = await $replica.organization.findFirst({
|
||||
where: { id: project.organizationId },
|
||||
select: { featureFlags: true },
|
||||
});
|
||||
const enabled = await flag({
|
||||
key: FEATURE_FLAG.hasWebhooksAccess,
|
||||
defaultValue: false,
|
||||
overrides: (org?.featureFlags as Record<string, unknown>) ?? {},
|
||||
});
|
||||
if (!enabled) throw new Response("Not found", { status: 404 });
|
||||
}
|
||||
|
||||
const url = new URL(request.url);
|
||||
const provider = url.searchParams.get("provider") ?? undefined;
|
||||
const eventType = url.searchParams.get("eventType") ?? undefined;
|
||||
|
||||
if (provider && eventType) {
|
||||
const sample = getSample(provider, eventType);
|
||||
if (!sample)
|
||||
return typedjson({ kind: "error", error: "Sample not found" } as WebhookSamplesData);
|
||||
return typedjson({
|
||||
kind: "body",
|
||||
body: JSON.stringify(sample.body, null, 2),
|
||||
extraHeaders: sample.extraHeaders,
|
||||
} as WebhookSamplesData);
|
||||
}
|
||||
|
||||
const manifest = sampleManifest().filter((sample) => getProvider(sample.provider));
|
||||
|
||||
const items: WebhookSampleMeta[] = manifest.map((sample) => {
|
||||
const entry = getProvider(sample.provider);
|
||||
return {
|
||||
provider: sample.provider,
|
||||
providerLabel: entry?.label ?? sample.providerLabel,
|
||||
presetId: sample.presetId,
|
||||
eventType: sample.eventType,
|
||||
name: sample.name,
|
||||
description: sample.description,
|
||||
};
|
||||
});
|
||||
|
||||
const providerMap = new Map<string, WebhookProviderMeta>();
|
||||
for (const sample of manifest) {
|
||||
const existing = providerMap.get(sample.provider);
|
||||
if (existing) {
|
||||
existing.eventCount += 1;
|
||||
continue;
|
||||
}
|
||||
const entry = getProvider(sample.provider);
|
||||
providerMap.set(sample.provider, {
|
||||
id: sample.provider,
|
||||
label: entry?.label ?? sample.providerLabel ?? titleCase(sample.provider),
|
||||
category: entry?.category,
|
||||
categoryLabel: entry ? categoryLabel(entry.category) : undefined,
|
||||
docsUrl: entry?.docsUrl,
|
||||
preset: entry?.preset,
|
||||
eventCount: 1,
|
||||
});
|
||||
}
|
||||
|
||||
const providers = [...providerMap.values()].sort((a, b) => {
|
||||
const cat = categoryOrder(a.category ?? "") - categoryOrder(b.category ?? "");
|
||||
return cat !== 0 ? cat : a.label.localeCompare(b.label);
|
||||
});
|
||||
|
||||
return typedjson({ kind: "manifest", providers, samples: items } as WebhookSamplesData);
|
||||
}
|
||||
@@ -0,0 +1,75 @@
|
||||
import { type ActionFunctionArgs, json } from "@remix-run/server-runtime";
|
||||
import { env } from "~/env.server";
|
||||
import { logger } from "~/services/logger.server";
|
||||
import { readBodyWithCap } from "~/utils/readBodyWithCap.server";
|
||||
import { webhookIngressRateLimiter } from "~/services/webhookIngressRateLimit.server";
|
||||
import { webhookEngine } from "~/v3/webhookEngine.server";
|
||||
|
||||
// Public, unauthenticated webhook ingress. A Remix `action` (NOT
|
||||
// createActionApiRoute, which parses JSON) so we can capture the raw bytes the
|
||||
// signature scheme verifies. The engine resolves the endpoint (and its env id +
|
||||
// type) from the globally-unique opaqueId, so this route runs no env query.
|
||||
export async function action({ request, params }: ActionFunctionArgs) {
|
||||
if (request.method !== "POST") {
|
||||
return json({ error: "Method not allowed" }, { status: 405 });
|
||||
}
|
||||
if (env.WEBHOOK_ENABLED !== "1" || env.WEBHOOK_INGRESS_ENABLED !== "1") {
|
||||
return json({ error: "Not found" }, { status: 404 });
|
||||
}
|
||||
|
||||
const opaqueId = params.opaqueId;
|
||||
if (!opaqueId) return json({ error: "Not found" }, { status: 404 });
|
||||
|
||||
// Per-opaqueId rate limit FIRST, before any DB or secret work.
|
||||
const rl = await webhookIngressRateLimiter.limit(opaqueId);
|
||||
if (!rl.success) {
|
||||
logger.info("webhook ingress rate limited", { opaqueId });
|
||||
return json({ error: "Too many requests" }, { status: 429 });
|
||||
}
|
||||
|
||||
// Content-Length is a cheap fast-path reject; the capped streaming read is the real enforcement
|
||||
// (a chunked request can omit/understate Content-Length and would otherwise buffer unbounded).
|
||||
const limitBytes = env.WEBHOOK_INGRESS_BODY_SIZE_LIMIT_MB * 1024 * 1024;
|
||||
const contentLength = Number(request.headers.get("content-length") ?? "0");
|
||||
if (Number.isFinite(contentLength) && contentLength > limitBytes) {
|
||||
return json({ error: "Payload too large" }, { status: 413 });
|
||||
}
|
||||
|
||||
const rawBytes = await readBodyWithCap(request, limitBytes);
|
||||
if (rawBytes === null) {
|
||||
return json({ error: "Payload too large" }, { status: 413 });
|
||||
}
|
||||
|
||||
const headers: Record<string, string> = {};
|
||||
request.headers.forEach((v, k) => (headers[k] = v));
|
||||
|
||||
const result = await webhookEngine.ingest({
|
||||
opaqueId,
|
||||
rawBytes,
|
||||
headers,
|
||||
url: request.url, // url-secret reads this; never logged with its query string
|
||||
});
|
||||
|
||||
switch (result.outcome) {
|
||||
case "accepted":
|
||||
logger.info("webhook ingress accepted", { opaqueId, deliveryId: result.deliveryId });
|
||||
return json({ received: true, deliveryId: result.deliveryFriendlyId }, { status: 200 });
|
||||
case "handshake":
|
||||
// Provider handshake echo (e.g. Slack url_verification): the challenge value, plain text, 200.
|
||||
return new Response(result.body, { status: 200, headers: { "content-type": "text/plain" } });
|
||||
case "duplicate":
|
||||
return json({ received: true, deliveryId: result.deliveryId }, { status: 200 });
|
||||
case "endpoint_not_found":
|
||||
case "endpoint_inactive":
|
||||
return json({ error: "Not found" }, { status: 404 });
|
||||
case "secret_missing":
|
||||
logger.warn("webhook ingress rejected: signing secret unset", { opaqueId });
|
||||
return json({ error: "Bad request" }, { status: 400 });
|
||||
case "verification_failed":
|
||||
logger.info("webhook ingress verification failed", { opaqueId });
|
||||
return json({ error: "Bad request" }, { status: 400 });
|
||||
case "enqueue_failed":
|
||||
logger.error("webhook ingress enqueue failed", { opaqueId, error: result.error });
|
||||
return json({ error: "Internal error" }, { status: 500 });
|
||||
}
|
||||
}
|
||||
@@ -191,6 +191,33 @@ function initializeSessionsReplicationClickhouseClient(): ClickHouse {
|
||||
});
|
||||
}
|
||||
|
||||
const defaultWebhookDeliveriesReplicationClickhouseClient = singleton(
|
||||
"webhookDeliveriesReplicationClickhouseClient",
|
||||
initializeWebhookDeliveriesReplicationClickhouseClient
|
||||
);
|
||||
|
||||
function initializeWebhookDeliveriesReplicationClickhouseClient(): ClickHouse {
|
||||
if (!env.WEBHOOK_DELIVERIES_REPLICATION_CLICKHOUSE_URL) {
|
||||
// Webhook deliveries replication worker gates on this URL; factory may still resolve "webhook_deliveries_replication" for tests.
|
||||
return defaultClickhouseClient;
|
||||
}
|
||||
|
||||
const url = new URL(env.WEBHOOK_DELIVERIES_REPLICATION_CLICKHOUSE_URL);
|
||||
url.searchParams.delete("secure");
|
||||
|
||||
return new ClickHouse({
|
||||
url: url.toString(),
|
||||
name: "webhook-deliveries-replication",
|
||||
keepAlive: {
|
||||
enabled: env.SESSION_REPLICATION_KEEP_ALIVE_ENABLED === "1",
|
||||
idleSocketTtl: env.SESSION_REPLICATION_KEEP_ALIVE_IDLE_SOCKET_TTL_MS,
|
||||
},
|
||||
logLevel: env.SESSION_REPLICATION_CLICKHOUSE_LOG_LEVEL,
|
||||
compression: { request: true },
|
||||
maxOpenConnections: env.SESSION_REPLICATION_MAX_OPEN_CONNECTIONS,
|
||||
});
|
||||
}
|
||||
|
||||
/** Run-engine PendingVersionSystem lookup (`RUN_ENGINE_CLICKHOUSE_URL`);
|
||||
* falls back to the default client if unset. */
|
||||
const defaultRunEngineClickhouseClient = singleton(
|
||||
@@ -398,6 +425,7 @@ export type ClientType =
|
||||
| "events"
|
||||
| "replication"
|
||||
| "sessions_replication"
|
||||
| "webhook_deliveries_replication"
|
||||
| "logs"
|
||||
| "query"
|
||||
| "admin"
|
||||
@@ -439,6 +467,9 @@ function buildOrgClickhouseClient(url: string, clientType: ClientType): ClickHou
|
||||
maxOpenConnections: env.RUN_REPLICATION_MAX_OPEN_CONNECTIONS,
|
||||
});
|
||||
case "sessions_replication":
|
||||
// Webhook deliveries replication shares the sessions replication ClickHouse
|
||||
// client config (same infra, both replication writers).
|
||||
case "webhook_deliveries_replication":
|
||||
return new ClickHouse({
|
||||
url: parsed.toString(),
|
||||
name,
|
||||
@@ -566,6 +597,8 @@ export class ClickhouseFactory {
|
||||
return defaultRunsReplicationClickhouseClient;
|
||||
case "sessions_replication":
|
||||
return defaultSessionsReplicationClickhouseClient;
|
||||
case "webhook_deliveries_replication":
|
||||
return defaultWebhookDeliveriesReplicationClickhouseClient;
|
||||
case "logs":
|
||||
return defaultLogsClickhouseClient;
|
||||
case "query":
|
||||
|
||||
@@ -0,0 +1,25 @@
|
||||
/**
|
||||
* Strip a client-forged `actionSource: "webhook"` from a session `.in` append part.
|
||||
*
|
||||
* Only the hosted webhook ingress may claim webhook trust, and it appends server-side rather than
|
||||
* through the client append route. A client with session write access could otherwise send a record
|
||||
* carrying `actionSource: "webhook"`, which the run loop uses to skip action-schema validation. We
|
||||
* downgrade it here (delete the field) so the record is validated as a normal client action.
|
||||
*/
|
||||
export function stripClientWebhookActionSource(part: string): string {
|
||||
if (!part.includes('"actionSource"')) return part;
|
||||
|
||||
let record: { payload?: { actionSource?: string } } | undefined;
|
||||
try {
|
||||
record = JSON.parse(part) as { payload?: { actionSource?: string } };
|
||||
} catch {
|
||||
return part;
|
||||
}
|
||||
|
||||
if (record?.payload?.actionSource === "webhook") {
|
||||
delete record.payload.actionSource;
|
||||
return JSON.stringify(record);
|
||||
}
|
||||
|
||||
return part;
|
||||
}
|
||||
@@ -1,7 +1,10 @@
|
||||
import type { PrismaClient, Session } from "@trigger.dev/database";
|
||||
import type { SessionItem } from "@trigger.dev/core/v3";
|
||||
import type { Prisma, PrismaClient, Session } from "@trigger.dev/database";
|
||||
import type { SessionItem, SessionTriggerConfig } from "@trigger.dev/core/v3";
|
||||
import { SessionId } from "@trigger.dev/core/v3/isomorphic";
|
||||
import type { RunStore } from "@internal/run-store";
|
||||
import { $replica, prisma } from "~/db.server";
|
||||
import type { AuthenticatedEnvironment } from "~/services/apiAuth.server";
|
||||
import { chatSnapshotStoragePathForSession } from "~/services/realtime/chatSnapshot.server";
|
||||
import { runStore as defaultRunStore } from "~/v3/runStore.server";
|
||||
|
||||
import { boundedIn } from "@trigger.dev/database";
|
||||
@@ -63,6 +66,71 @@ export function isSessionFriendlyIdForm(value: string): boolean {
|
||||
return value.startsWith(SESSION_FRIENDLY_ID_PREFIX);
|
||||
}
|
||||
|
||||
/**
|
||||
* Find-or-create a Session row. Idempotent on `(environment, externalId)`: two concurrent callers
|
||||
* converge to the same row (and `triggerConfig` is refreshed on the cached path so a redeployed
|
||||
* config propagates to the next run). Shared by `POST /api/v1/sessions` and the webhook session
|
||||
* delivery port so the create-field set (org/env scoping, streamBasin, snapshot path) stays in one place.
|
||||
*/
|
||||
export async function findOrCreateSession(params: {
|
||||
environment: AuthenticatedEnvironment;
|
||||
externalId?: string;
|
||||
type: string;
|
||||
taskIdentifier: string;
|
||||
triggerConfig: SessionTriggerConfig;
|
||||
tags?: string[];
|
||||
metadata?: Record<string, unknown>;
|
||||
expiresAt?: Date | null;
|
||||
}): Promise<{ session: Session; isCached: boolean }> {
|
||||
const { id, friendlyId } = SessionId.generate();
|
||||
const env = params.environment;
|
||||
const triggerConfigJson = params.triggerConfig as unknown as Prisma.InputJsonValue;
|
||||
|
||||
const common = {
|
||||
type: params.type,
|
||||
taskIdentifier: params.taskIdentifier,
|
||||
triggerConfig: triggerConfigJson,
|
||||
tags: params.tags ?? [],
|
||||
metadata: params.metadata as Prisma.InputJsonValue | undefined,
|
||||
expiresAt: params.expiresAt ?? null,
|
||||
projectId: env.projectId,
|
||||
runtimeEnvironmentId: env.id,
|
||||
environmentType: env.type,
|
||||
organizationId: env.organizationId,
|
||||
streamBasinName: env.organization.streamBasinName,
|
||||
chatSnapshotStoragePath: chatSnapshotStoragePathForSession(friendlyId),
|
||||
};
|
||||
|
||||
if (params.externalId) {
|
||||
const session = await prisma.session.upsert({
|
||||
where: {
|
||||
runtimeEnvironmentId_externalId: {
|
||||
runtimeEnvironmentId: env.id,
|
||||
externalId: params.externalId,
|
||||
},
|
||||
},
|
||||
create: { id, friendlyId, externalId: params.externalId, ...common },
|
||||
update: { triggerConfig: triggerConfigJson },
|
||||
});
|
||||
return { session, isCached: session.id !== id };
|
||||
}
|
||||
|
||||
const session = await prisma.session.create({ data: { id, friendlyId, ...common } });
|
||||
return { session, isCached: false };
|
||||
}
|
||||
|
||||
/** Find a session by externalId without creating one (resume-only channel delivery, e.g. startOn). */
|
||||
export async function findSessionByExternalId(
|
||||
environment: AuthenticatedEnvironment,
|
||||
externalId: string
|
||||
): Promise<Session | null> {
|
||||
return prisma.session.findUnique({
|
||||
where: {
|
||||
runtimeEnvironmentId_externalId: { runtimeEnvironmentId: environment.id, externalId },
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Canonicalise the addressing key used for everything stream-level: the
|
||||
* S2 stream path and the run-engine waitpoint cache key. `chat.agent`
|
||||
|
||||
@@ -16,6 +16,7 @@ import {
|
||||
function toTriggerSource(source: string | undefined): TaskTriggerSource {
|
||||
const normalized = source?.toUpperCase();
|
||||
if (normalized === "AGENT") return "AGENT";
|
||||
if (normalized === "WEBHOOK") return "WEBHOOK";
|
||||
if (normalized === "SCHEDULED" || normalized === "SCHEDULE") return "SCHEDULED";
|
||||
return "STANDARD";
|
||||
}
|
||||
|
||||
@@ -0,0 +1,14 @@
|
||||
import { Ratelimit } from "@upstash/ratelimit";
|
||||
import { RateLimiter, type Duration } from "./rateLimiter.server";
|
||||
|
||||
/**
|
||||
* Per-user limiter for the authenticated webhook console test-send. The console injects deliveries
|
||||
* in-process (not through the public ingress), so it does NOT ride the per-opaqueId ingress limiter;
|
||||
* this keeps a single user's testing from flooding the engine + task queue, keyed by user id.
|
||||
*/
|
||||
export const webhookConsoleSendRateLimiter = new RateLimiter({
|
||||
keyPrefix: "webhook-console-send",
|
||||
limiter: Ratelimit.fixedWindow(30, "60 s" as Duration),
|
||||
logSuccess: false,
|
||||
logFailure: true,
|
||||
});
|
||||
@@ -0,0 +1,92 @@
|
||||
import invariant from "tiny-invariant";
|
||||
import { env } from "~/env.server";
|
||||
import { clickhouseFactory } from "~/services/clickhouse/clickhouseFactoryInstance.server";
|
||||
import { singleton } from "~/utils/singleton";
|
||||
import { meter, provider } from "~/v3/tracer.server";
|
||||
import { WebhookDeliveriesReplicationService } from "./webhookDeliveriesReplicationService.server";
|
||||
import { signalsEmitter } from "./signals.server";
|
||||
|
||||
export const webhookDeliveriesReplicationInstance = singleton(
|
||||
"webhookDeliveriesReplicationInstance",
|
||||
initializeWebhookDeliveriesReplicationInstance
|
||||
);
|
||||
|
||||
function initializeWebhookDeliveriesReplicationInstance() {
|
||||
const { DATABASE_URL } = process.env;
|
||||
invariant(typeof DATABASE_URL === "string", "DATABASE_URL env var not set");
|
||||
|
||||
if (!env.WEBHOOK_DELIVERIES_REPLICATION_CLICKHOUSE_URL) {
|
||||
console.log("🗃️ Webhook deliveries replication service not enabled");
|
||||
return;
|
||||
}
|
||||
|
||||
console.log("🗃️ Webhook deliveries replication service enabled");
|
||||
|
||||
const service = new WebhookDeliveriesReplicationService({
|
||||
clickhouseFactory,
|
||||
// Follows the webhook writer DB (where WebhookDelivery physically lives once split).
|
||||
pgConnectionUrl: env.WEBHOOK_DATABASE_URL ?? DATABASE_URL,
|
||||
serviceName: "webhook-deliveries-replication",
|
||||
slotName: env.WEBHOOK_DELIVERIES_REPLICATION_SLOT_NAME,
|
||||
publicationName: env.WEBHOOK_DELIVERIES_REPLICATION_PUBLICATION_NAME,
|
||||
// The source WebhookDelivery is a partitioned parent; without this the slot stays silent.
|
||||
publishViaPartitionRoot: true,
|
||||
redisOptions: {
|
||||
keyPrefix: "webhook-deliveries-replication:",
|
||||
port: env.RUN_REPLICATION_REDIS_PORT ?? undefined,
|
||||
host: env.RUN_REPLICATION_REDIS_HOST ?? undefined,
|
||||
username: env.RUN_REPLICATION_REDIS_USERNAME ?? undefined,
|
||||
password: env.RUN_REPLICATION_REDIS_PASSWORD ?? undefined,
|
||||
enableAutoPipelining: true,
|
||||
...(env.RUN_REPLICATION_REDIS_TLS_DISABLED === "true" ? {} : { tls: {} }),
|
||||
},
|
||||
maxFlushConcurrency: env.WEBHOOK_DELIVERIES_REPLICATION_MAX_FLUSH_CONCURRENCY,
|
||||
flushIntervalMs: env.WEBHOOK_DELIVERIES_REPLICATION_FLUSH_INTERVAL_MS,
|
||||
flushBatchSize: env.WEBHOOK_DELIVERIES_REPLICATION_FLUSH_BATCH_SIZE,
|
||||
leaderLockTimeoutMs: env.WEBHOOK_DELIVERIES_REPLICATION_LEADER_LOCK_TIMEOUT_MS,
|
||||
leaderLockExtendIntervalMs: env.WEBHOOK_DELIVERIES_REPLICATION_LEADER_LOCK_EXTEND_INTERVAL_MS,
|
||||
leaderLockAcquireAdditionalTimeMs:
|
||||
env.WEBHOOK_DELIVERIES_REPLICATION_LEADER_LOCK_ADDITIONAL_TIME_MS,
|
||||
leaderLockRetryIntervalMs: env.WEBHOOK_DELIVERIES_REPLICATION_LEADER_LOCK_RETRY_INTERVAL_MS,
|
||||
ackIntervalSeconds: env.WEBHOOK_DELIVERIES_REPLICATION_ACK_INTERVAL_SECONDS,
|
||||
logLevel: env.WEBHOOK_DELIVERIES_REPLICATION_LOG_LEVEL,
|
||||
waitForAsyncInsert: env.WEBHOOK_DELIVERIES_REPLICATION_WAIT_FOR_ASYNC_INSERT === "1",
|
||||
tracer: provider.getTracer("webhook-deliveries-replication-service"),
|
||||
meter,
|
||||
insertMaxRetries: env.WEBHOOK_DELIVERIES_REPLICATION_INSERT_MAX_RETRIES,
|
||||
insertBaseDelayMs: env.WEBHOOK_DELIVERIES_REPLICATION_INSERT_BASE_DELAY_MS,
|
||||
insertMaxDelayMs: env.WEBHOOK_DELIVERIES_REPLICATION_INSERT_MAX_DELAY_MS,
|
||||
insertStrategy: env.WEBHOOK_DELIVERIES_REPLICATION_INSERT_STRATEGY,
|
||||
});
|
||||
|
||||
if (env.WEBHOOK_DELIVERIES_REPLICATION_ENABLED === "1") {
|
||||
// Gate start() on the org data-stores registry being loaded. Starting earlier would
|
||||
// race the registry load — sync factory lookups would return `null` and route org-scoped
|
||||
// deliveries to the default ClickHouse, writing them to the wrong cluster.
|
||||
clickhouseFactory
|
||||
.isReady()
|
||||
.then(() => service.start())
|
||||
.then(() => {
|
||||
console.log("🗃️ Webhook deliveries replication service started");
|
||||
})
|
||||
.catch((error) => {
|
||||
console.error("🗃️ Webhook deliveries replication service failed to start", {
|
||||
error,
|
||||
});
|
||||
});
|
||||
|
||||
// SIGTERM/SIGINT fire during process teardown; wrap the async shutdown so an
|
||||
// unhandled rejection doesn't bubble past process exit.
|
||||
const shutdownWebhookDeliveriesReplication = () => {
|
||||
service.shutdown().catch((error) => {
|
||||
console.error("🗃️ Webhook deliveries replication service shutdown error", {
|
||||
error,
|
||||
});
|
||||
});
|
||||
};
|
||||
signalsEmitter.on("SIGTERM", shutdownWebhookDeliveriesReplication);
|
||||
signalsEmitter.on("SIGINT", shutdownWebhookDeliveriesReplication);
|
||||
}
|
||||
|
||||
return service;
|
||||
}
|
||||
@@ -0,0 +1,821 @@
|
||||
import type { ClickHouse, WebhookDeliveryInsertArray } from "@internal/clickhouse";
|
||||
import { getWebhookDeliveryField } from "@internal/clickhouse";
|
||||
import { type RedisOptions } from "@internal/redis";
|
||||
import {
|
||||
LogicalReplicationClient,
|
||||
type MessageDelete,
|
||||
type MessageInsert,
|
||||
type MessageUpdate,
|
||||
type PgoutputMessage,
|
||||
} from "@internal/replication";
|
||||
import {
|
||||
getMeter,
|
||||
recordSpanError,
|
||||
startSpan,
|
||||
trace,
|
||||
type Counter,
|
||||
type Histogram,
|
||||
type Meter,
|
||||
type Tracer,
|
||||
} from "@internal/tracing";
|
||||
import { Logger, type LogLevel } from "@trigger.dev/core/logger";
|
||||
import { tryCatch } from "@trigger.dev/core/utils";
|
||||
import { type WebhookDelivery } from "@trigger.dev/database";
|
||||
import EventEmitter from "node:events";
|
||||
import type { ClickhouseFactory } from "~/services/clickhouse/clickhouseFactory.server";
|
||||
import { ConcurrentFlushScheduler } from "./runsReplicationService.server";
|
||||
|
||||
interface TransactionEvent<T = any> {
|
||||
tag: "insert" | "update" | "delete";
|
||||
data: T;
|
||||
raw: MessageInsert | MessageUpdate | MessageDelete;
|
||||
}
|
||||
|
||||
interface Transaction<T = any> {
|
||||
beginStartTimestamp: number;
|
||||
commitLsn: string | null;
|
||||
commitEndLsn: string | null;
|
||||
xid: number;
|
||||
events: TransactionEvent<T>[];
|
||||
replicationLagMs: number;
|
||||
}
|
||||
|
||||
export type WebhookDeliveriesReplicationServiceOptions = {
|
||||
clickhouseFactory: ClickhouseFactory;
|
||||
pgConnectionUrl: string;
|
||||
serviceName: string;
|
||||
slotName: string;
|
||||
publicationName: string;
|
||||
publishViaPartitionRoot?: boolean;
|
||||
redisOptions: RedisOptions;
|
||||
maxFlushConcurrency?: number;
|
||||
flushIntervalMs?: number;
|
||||
flushBatchSize?: number;
|
||||
leaderLockTimeoutMs?: number;
|
||||
leaderLockExtendIntervalMs?: number;
|
||||
leaderLockAcquireAdditionalTimeMs?: number;
|
||||
leaderLockRetryIntervalMs?: number;
|
||||
ackIntervalSeconds?: number;
|
||||
acknowledgeTimeoutMs?: number;
|
||||
logger?: Logger;
|
||||
logLevel?: LogLevel;
|
||||
tracer?: Tracer;
|
||||
meter?: Meter;
|
||||
waitForAsyncInsert?: boolean;
|
||||
insertStrategy?: "insert" | "insert_async";
|
||||
// Retry configuration for insert operations
|
||||
insertMaxRetries?: number;
|
||||
insertBaseDelayMs?: number;
|
||||
insertMaxDelayMs?: number;
|
||||
};
|
||||
|
||||
type WebhookDeliveryInsert = {
|
||||
_version: bigint;
|
||||
delivery: WebhookDelivery;
|
||||
event: "insert" | "update" | "delete";
|
||||
};
|
||||
|
||||
export type WebhookDeliveriesReplicationServiceEvents = {
|
||||
message: [
|
||||
{ lsn: string; message: PgoutputMessage; service: WebhookDeliveriesReplicationService },
|
||||
];
|
||||
batchFlushed: [{ flushId: string; deliveryInserts: WebhookDeliveryInsertArray[] }];
|
||||
};
|
||||
|
||||
export class WebhookDeliveriesReplicationService {
|
||||
private _isSubscribed = false;
|
||||
private _currentTransaction:
|
||||
| (Omit<Transaction<WebhookDelivery>, "commitEndLsn" | "replicationLagMs"> & {
|
||||
commitEndLsn?: string | null;
|
||||
replicationLagMs?: number;
|
||||
})
|
||||
| null = null;
|
||||
|
||||
private _replicationClient: LogicalReplicationClient;
|
||||
private _concurrentFlushScheduler: ConcurrentFlushScheduler<WebhookDeliveryInsert>;
|
||||
private logger: Logger;
|
||||
private _isShuttingDown = false;
|
||||
private _isShutDownComplete = false;
|
||||
private _tracer: Tracer;
|
||||
private _meter: Meter;
|
||||
private _currentParseDurationMs: number | null = null;
|
||||
private _lastAcknowledgedAt: number | null = null;
|
||||
private _acknowledgeTimeoutMs: number;
|
||||
private _latestCommitEndLsn: string | null = null;
|
||||
private _lastAcknowledgedLsn: string | null = null;
|
||||
private _acknowledgeInterval: NodeJS.Timeout | null = null;
|
||||
// Retry configuration
|
||||
private _insertMaxRetries: number;
|
||||
private _insertBaseDelayMs: number;
|
||||
private _insertMaxDelayMs: number;
|
||||
private _insertStrategy: "insert" | "insert_async";
|
||||
|
||||
// Metrics
|
||||
private _replicationLagHistogram: Histogram;
|
||||
private _batchesFlushedCounter: Counter;
|
||||
private _batchSizeHistogram: Histogram;
|
||||
private _deliveriesInsertedCounter: Counter;
|
||||
private _insertRetriesCounter: Counter;
|
||||
private _eventsProcessedCounter: Counter;
|
||||
private _flushDurationHistogram: Histogram;
|
||||
|
||||
public readonly events: EventEmitter<WebhookDeliveriesReplicationServiceEvents>;
|
||||
|
||||
constructor(private readonly options: WebhookDeliveriesReplicationServiceOptions) {
|
||||
this.logger =
|
||||
options.logger ??
|
||||
new Logger("WebhookDeliveriesReplicationService", options.logLevel ?? "info");
|
||||
this.events = new EventEmitter();
|
||||
this._tracer = options.tracer ?? trace.getTracer("webhook-deliveries-replication-service");
|
||||
this._meter = options.meter ?? getMeter("webhook-deliveries-replication");
|
||||
|
||||
// Initialize metrics
|
||||
this._replicationLagHistogram = this._meter.createHistogram(
|
||||
"webhook_deliveries_replication.replication_lag_ms",
|
||||
{
|
||||
description: "Replication lag from Postgres commit to processing",
|
||||
unit: "ms",
|
||||
}
|
||||
);
|
||||
|
||||
this._batchesFlushedCounter = this._meter.createCounter(
|
||||
"webhook_deliveries_replication.batches_flushed",
|
||||
{
|
||||
description: "Total batches flushed to ClickHouse",
|
||||
}
|
||||
);
|
||||
|
||||
this._batchSizeHistogram = this._meter.createHistogram(
|
||||
"webhook_deliveries_replication.batch_size",
|
||||
{
|
||||
description: "Number of items per batch flush",
|
||||
unit: "items",
|
||||
}
|
||||
);
|
||||
|
||||
this._deliveriesInsertedCounter = this._meter.createCounter(
|
||||
"webhook_deliveries_replication.deliveries_inserted",
|
||||
{
|
||||
description: "Webhook delivery inserts to ClickHouse",
|
||||
unit: "inserts",
|
||||
}
|
||||
);
|
||||
|
||||
this._insertRetriesCounter = this._meter.createCounter(
|
||||
"webhook_deliveries_replication.insert_retries",
|
||||
{
|
||||
description: "Insert retry attempts",
|
||||
}
|
||||
);
|
||||
|
||||
this._eventsProcessedCounter = this._meter.createCounter(
|
||||
"webhook_deliveries_replication.events_processed",
|
||||
{
|
||||
description: "Replication events processed (inserts, updates, deletes)",
|
||||
}
|
||||
);
|
||||
|
||||
this._flushDurationHistogram = this._meter.createHistogram(
|
||||
"webhook_deliveries_replication.flush_duration_ms",
|
||||
{
|
||||
description: "Duration of batch flush operations",
|
||||
unit: "ms",
|
||||
}
|
||||
);
|
||||
|
||||
this._acknowledgeTimeoutMs = options.acknowledgeTimeoutMs ?? 1_000;
|
||||
|
||||
this._insertStrategy = options.insertStrategy ?? "insert";
|
||||
|
||||
this._replicationClient = new LogicalReplicationClient({
|
||||
pgConfig: {
|
||||
connectionString: options.pgConnectionUrl,
|
||||
},
|
||||
name: options.serviceName,
|
||||
slotName: options.slotName,
|
||||
publicationName: options.publicationName,
|
||||
table: "WebhookDelivery",
|
||||
publishViaPartitionRoot: options.publishViaPartitionRoot,
|
||||
redisOptions: options.redisOptions,
|
||||
autoAcknowledge: false,
|
||||
publicationActions: ["insert", "update", "delete"],
|
||||
logger: options.logger ?? new Logger("LogicalReplicationClient", options.logLevel ?? "info"),
|
||||
leaderLockTimeoutMs: options.leaderLockTimeoutMs ?? 30_000,
|
||||
leaderLockExtendIntervalMs: options.leaderLockExtendIntervalMs ?? 10_000,
|
||||
ackIntervalSeconds: options.ackIntervalSeconds ?? 10,
|
||||
leaderLockAcquireAdditionalTimeMs: options.leaderLockAcquireAdditionalTimeMs ?? 10_000,
|
||||
leaderLockRetryIntervalMs: options.leaderLockRetryIntervalMs ?? 500,
|
||||
tracer: options.tracer,
|
||||
});
|
||||
|
||||
this._concurrentFlushScheduler = new ConcurrentFlushScheduler<WebhookDeliveryInsert>({
|
||||
batchSize: options.flushBatchSize ?? 50,
|
||||
flushInterval: options.flushIntervalMs ?? 100,
|
||||
maxConcurrency: options.maxFlushConcurrency ?? 100,
|
||||
callback: this.#flushBatch.bind(this),
|
||||
// Key-based deduplication to reduce duplicates sent to ClickHouse
|
||||
getKey: (item) => {
|
||||
if (!item?.delivery?.id) {
|
||||
this.logger.warn("Skipping replication event with null delivery", { event: item });
|
||||
return null;
|
||||
}
|
||||
return `${item.event}_${item.delivery.id}`;
|
||||
},
|
||||
// Keep the delivery with the higher version (latest)
|
||||
// and take the last occurrence for that version.
|
||||
// Items originating from the same DB transaction have the same version.
|
||||
shouldReplace: (existing, incoming) => incoming._version >= existing._version,
|
||||
logger: new Logger("ConcurrentFlushScheduler", options.logLevel ?? "info"),
|
||||
tracer: options.tracer,
|
||||
});
|
||||
|
||||
this._replicationClient.events.on("data", async ({ lsn, log, parseDuration }) => {
|
||||
this.#handleData(lsn, log, parseDuration);
|
||||
});
|
||||
|
||||
this._replicationClient.events.on("heartbeat", async ({ lsn, shouldRespond }) => {
|
||||
if (this._isShuttingDown) return;
|
||||
if (this._isShutDownComplete) return;
|
||||
|
||||
if (shouldRespond) {
|
||||
this._lastAcknowledgedLsn = lsn;
|
||||
await this._replicationClient.acknowledge(lsn);
|
||||
}
|
||||
});
|
||||
|
||||
this._replicationClient.events.on("error", (error) => {
|
||||
this.logger.error("Replication client error", {
|
||||
error,
|
||||
});
|
||||
});
|
||||
|
||||
this._replicationClient.events.on("start", () => {
|
||||
this.logger.info("Replication client started");
|
||||
});
|
||||
|
||||
this._replicationClient.events.on("acknowledge", ({ lsn }) => {
|
||||
this.logger.debug("Acknowledged", { lsn });
|
||||
});
|
||||
|
||||
this._replicationClient.events.on("leaderElection", (isLeader) => {
|
||||
this.logger.info("Leader election", { isLeader });
|
||||
});
|
||||
|
||||
// Initialize retry configuration
|
||||
this._insertMaxRetries = options.insertMaxRetries ?? 3;
|
||||
this._insertBaseDelayMs = options.insertBaseDelayMs ?? 100;
|
||||
this._insertMaxDelayMs = options.insertMaxDelayMs ?? 2000;
|
||||
}
|
||||
|
||||
public async shutdown() {
|
||||
if (this._isShuttingDown) return;
|
||||
|
||||
this._isShuttingDown = true;
|
||||
|
||||
this.logger.info("Initiating shutdown of webhook deliveries replication service");
|
||||
|
||||
if (!this._currentTransaction) {
|
||||
this.logger.info("No transaction to commit, shutting down immediately");
|
||||
await this._replicationClient.stop();
|
||||
this._isSubscribed = false;
|
||||
this._isShutDownComplete = true;
|
||||
return;
|
||||
}
|
||||
|
||||
this._concurrentFlushScheduler.shutdown();
|
||||
}
|
||||
|
||||
async start() {
|
||||
if (this._isSubscribed) {
|
||||
this.logger.debug("Replication client already started, skipping start");
|
||||
return;
|
||||
}
|
||||
|
||||
this.logger.info("Starting replication client", {
|
||||
lastLsn: this._latestCommitEndLsn,
|
||||
});
|
||||
|
||||
await this._replicationClient.subscribe(this._latestCommitEndLsn ?? undefined);
|
||||
|
||||
this._acknowledgeInterval = setInterval(this.#acknowledgeLatestTransaction.bind(this), 1000);
|
||||
this._concurrentFlushScheduler.start();
|
||||
this._isSubscribed = true;
|
||||
}
|
||||
|
||||
async stop() {
|
||||
this.logger.info("Stopping replication client");
|
||||
|
||||
await this._replicationClient.stop();
|
||||
|
||||
if (this._acknowledgeInterval) {
|
||||
clearInterval(this._acknowledgeInterval);
|
||||
this._acknowledgeInterval = null;
|
||||
}
|
||||
|
||||
this._isSubscribed = false;
|
||||
}
|
||||
|
||||
async teardown() {
|
||||
this.logger.info("Teardown replication client");
|
||||
|
||||
await this._replicationClient.teardown();
|
||||
|
||||
if (this._acknowledgeInterval) {
|
||||
clearInterval(this._acknowledgeInterval);
|
||||
this._acknowledgeInterval = null;
|
||||
}
|
||||
|
||||
this._isSubscribed = false;
|
||||
}
|
||||
|
||||
#handleData(lsn: string, message: PgoutputMessage, parseDuration: bigint) {
|
||||
this.logger.debug("Handling data", {
|
||||
lsn,
|
||||
tag: message.tag,
|
||||
parseDuration,
|
||||
});
|
||||
|
||||
this.events.emit("message", { lsn, message, service: this });
|
||||
|
||||
switch (message.tag) {
|
||||
case "begin": {
|
||||
if (this._isShuttingDown || this._isShutDownComplete) {
|
||||
return;
|
||||
}
|
||||
|
||||
this._currentTransaction = {
|
||||
beginStartTimestamp: Date.now(),
|
||||
commitLsn: message.commitLsn,
|
||||
xid: message.xid,
|
||||
events: [],
|
||||
};
|
||||
|
||||
this._currentParseDurationMs = Number(parseDuration) / 1_000_000;
|
||||
|
||||
break;
|
||||
}
|
||||
case "insert": {
|
||||
if (!this._currentTransaction) {
|
||||
return;
|
||||
}
|
||||
|
||||
if (this._currentParseDurationMs) {
|
||||
this._currentParseDurationMs =
|
||||
this._currentParseDurationMs + Number(parseDuration) / 1_000_000;
|
||||
}
|
||||
|
||||
this._currentTransaction.events.push({
|
||||
tag: message.tag,
|
||||
data: message.new as WebhookDelivery,
|
||||
raw: message,
|
||||
});
|
||||
break;
|
||||
}
|
||||
case "update": {
|
||||
if (!this._currentTransaction) {
|
||||
return;
|
||||
}
|
||||
|
||||
if (this._currentParseDurationMs) {
|
||||
this._currentParseDurationMs =
|
||||
this._currentParseDurationMs + Number(parseDuration) / 1_000_000;
|
||||
}
|
||||
|
||||
this._currentTransaction.events.push({
|
||||
tag: message.tag,
|
||||
data: message.new as WebhookDelivery,
|
||||
raw: message,
|
||||
});
|
||||
break;
|
||||
}
|
||||
case "delete": {
|
||||
if (!this._currentTransaction) {
|
||||
return;
|
||||
}
|
||||
|
||||
if (this._currentParseDurationMs) {
|
||||
this._currentParseDurationMs =
|
||||
this._currentParseDurationMs + Number(parseDuration) / 1_000_000;
|
||||
}
|
||||
|
||||
this._currentTransaction.events.push({
|
||||
tag: message.tag,
|
||||
data: message.old as WebhookDelivery,
|
||||
raw: message,
|
||||
});
|
||||
|
||||
break;
|
||||
}
|
||||
case "commit": {
|
||||
if (!this._currentTransaction) {
|
||||
return;
|
||||
}
|
||||
|
||||
if (this._currentParseDurationMs) {
|
||||
this._currentParseDurationMs =
|
||||
this._currentParseDurationMs + Number(parseDuration) / 1_000_000;
|
||||
}
|
||||
|
||||
const replicationLagMs = Date.now() - Number(message.commitTime / 1000n);
|
||||
this._currentTransaction.commitEndLsn = message.commitEndLsn;
|
||||
this._currentTransaction.replicationLagMs = replicationLagMs;
|
||||
const transaction = this._currentTransaction as Transaction<WebhookDelivery>;
|
||||
this._currentTransaction = null;
|
||||
|
||||
if (transaction.commitEndLsn) {
|
||||
this._latestCommitEndLsn = transaction.commitEndLsn;
|
||||
}
|
||||
|
||||
this.#handleTransaction(transaction);
|
||||
break;
|
||||
}
|
||||
default: {
|
||||
this.logger.debug("Unknown message tag", {
|
||||
pgMessage: message,
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#handleTransaction(transaction: Transaction<WebhookDelivery>) {
|
||||
if (this._isShutDownComplete) return;
|
||||
|
||||
if (this._isShuttingDown) {
|
||||
this._replicationClient.stop().finally(() => {
|
||||
this._isSubscribed = false;
|
||||
this._isShutDownComplete = true;
|
||||
});
|
||||
}
|
||||
|
||||
// If there are no events, do nothing
|
||||
if (transaction.events.length === 0) {
|
||||
return;
|
||||
}
|
||||
|
||||
if (!transaction.commitEndLsn) {
|
||||
this.logger.error("Transaction has no commit end lsn", {
|
||||
transaction,
|
||||
});
|
||||
|
||||
return;
|
||||
}
|
||||
|
||||
const lsnToUInt64Start = process.hrtime.bigint();
|
||||
|
||||
// If there are events, we need to handle them
|
||||
const _version = lsnToUInt64(transaction.commitEndLsn);
|
||||
|
||||
const lsnToUInt64DurationMs = Number(process.hrtime.bigint() - lsnToUInt64Start) / 1_000_000;
|
||||
|
||||
this._concurrentFlushScheduler.addToBatch(
|
||||
transaction.events.map((event) => ({
|
||||
_version,
|
||||
delivery: event.data,
|
||||
event: event.tag,
|
||||
}))
|
||||
);
|
||||
|
||||
// Record metrics
|
||||
this._replicationLagHistogram.record(transaction.replicationLagMs);
|
||||
|
||||
// Count events by type
|
||||
for (const event of transaction.events) {
|
||||
this._eventsProcessedCounter.add(1, { event_type: event.tag });
|
||||
}
|
||||
|
||||
this.logger.debug("handle_transaction", {
|
||||
transaction: {
|
||||
xid: transaction.xid,
|
||||
commitLsn: transaction.commitLsn,
|
||||
commitEndLsn: transaction.commitEndLsn,
|
||||
events: transaction.events.length,
|
||||
parseDurationMs: this._currentParseDurationMs,
|
||||
lsnToUInt64DurationMs,
|
||||
version: _version.toString(),
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
async #acknowledgeLatestTransaction() {
|
||||
if (!this._latestCommitEndLsn) {
|
||||
return;
|
||||
}
|
||||
|
||||
if (this._lastAcknowledgedLsn === this._latestCommitEndLsn) {
|
||||
return;
|
||||
}
|
||||
|
||||
const now = Date.now();
|
||||
|
||||
if (this._lastAcknowledgedAt) {
|
||||
const timeSinceLastAcknowledged = now - this._lastAcknowledgedAt;
|
||||
// If we've already acknowledged within the last second, don't acknowledge again
|
||||
if (timeSinceLastAcknowledged < this._acknowledgeTimeoutMs) {
|
||||
return;
|
||||
}
|
||||
}
|
||||
|
||||
this._lastAcknowledgedAt = now;
|
||||
this._lastAcknowledgedLsn = this._latestCommitEndLsn;
|
||||
|
||||
this.logger.debug("acknowledge_latest_transaction", {
|
||||
commitEndLsn: this._latestCommitEndLsn,
|
||||
lastAcknowledgedAt: this._lastAcknowledgedAt,
|
||||
});
|
||||
|
||||
const [ackError] = await tryCatch(
|
||||
this._replicationClient.acknowledge(this._latestCommitEndLsn)
|
||||
);
|
||||
|
||||
if (ackError) {
|
||||
this.logger.error("Error acknowledging transaction", { ackError });
|
||||
}
|
||||
|
||||
if (this._isShutDownComplete && this._acknowledgeInterval) {
|
||||
clearInterval(this._acknowledgeInterval);
|
||||
}
|
||||
}
|
||||
|
||||
async #flushBatch(flushId: string, batch: Array<WebhookDeliveryInsert>) {
|
||||
if (batch.length === 0) {
|
||||
return;
|
||||
}
|
||||
|
||||
this.logger.debug("Flushing batch", {
|
||||
flushId,
|
||||
batchSize: batch.length,
|
||||
});
|
||||
|
||||
const flushStartTime = performance.now();
|
||||
|
||||
await startSpan(this._tracer, "flushBatch", async (span) => {
|
||||
const routeCache = new Map<string, ClickHouse>();
|
||||
const groups = new Map<ClickHouse, { deliveryInserts: WebhookDeliveryInsertArray[] }>();
|
||||
|
||||
for (const item of batch) {
|
||||
if (!item.delivery.organizationId) {
|
||||
continue;
|
||||
}
|
||||
|
||||
let client = routeCache.get(item.delivery.organizationId);
|
||||
if (!client) {
|
||||
client = this.options.clickhouseFactory.getClickhouseForOrganizationSync(
|
||||
item.delivery.organizationId,
|
||||
"webhook_deliveries_replication"
|
||||
);
|
||||
routeCache.set(item.delivery.organizationId, client);
|
||||
}
|
||||
|
||||
let group = groups.get(client);
|
||||
if (!group) {
|
||||
group = { deliveryInserts: [] };
|
||||
groups.set(client, group);
|
||||
}
|
||||
|
||||
group.deliveryInserts.push(
|
||||
toWebhookDeliveryInsertArray(item.delivery, item._version, item.event === "delete")
|
||||
);
|
||||
}
|
||||
|
||||
// batch inserts in clickhouse are more performant if the items
|
||||
// are pre-sorted by the primary key
|
||||
const sortDeliveryInserts = (rows: WebhookDeliveryInsertArray[]) =>
|
||||
rows.sort((a, b) => {
|
||||
const aOrgId = getWebhookDeliveryField(a, "organization_id");
|
||||
const bOrgId = getWebhookDeliveryField(b, "organization_id");
|
||||
if (aOrgId !== bOrgId) {
|
||||
return aOrgId < bOrgId ? -1 : 1;
|
||||
}
|
||||
const aProjId = getWebhookDeliveryField(a, "project_id");
|
||||
const bProjId = getWebhookDeliveryField(b, "project_id");
|
||||
if (aProjId !== bProjId) {
|
||||
return aProjId < bProjId ? -1 : 1;
|
||||
}
|
||||
const aEnvId = getWebhookDeliveryField(a, "environment_id");
|
||||
const bEnvId = getWebhookDeliveryField(b, "environment_id");
|
||||
if (aEnvId !== bEnvId) {
|
||||
return aEnvId < bEnvId ? -1 : 1;
|
||||
}
|
||||
const aCreatedAt = getWebhookDeliveryField(a, "created_at");
|
||||
const bCreatedAt = getWebhookDeliveryField(b, "created_at");
|
||||
if (aCreatedAt !== bCreatedAt) {
|
||||
return aCreatedAt - bCreatedAt;
|
||||
}
|
||||
const aDeliveryId = getWebhookDeliveryField(a, "delivery_id");
|
||||
const bDeliveryId = getWebhookDeliveryField(b, "delivery_id");
|
||||
if (aDeliveryId === bDeliveryId) return 0;
|
||||
return aDeliveryId < bDeliveryId ? -1 : 1;
|
||||
});
|
||||
|
||||
const combinedDeliveryInserts: WebhookDeliveryInsertArray[] = [];
|
||||
let deliveryError: Error | null = null;
|
||||
|
||||
// Sequential per-group flush — matches runsReplicationService for the same reason
|
||||
// (parallel writes have hit Linux net.ipv4.tcp_wmem buffer pressure at high throughput).
|
||||
for (const [clickhouse, group] of groups) {
|
||||
sortDeliveryInserts(group.deliveryInserts);
|
||||
combinedDeliveryInserts.push(...group.deliveryInserts);
|
||||
|
||||
const [insErr] = await this.#insertWithRetry(
|
||||
(attempt) => this.#insertDeliveryInserts(clickhouse, group.deliveryInserts, attempt),
|
||||
"delivery inserts",
|
||||
flushId
|
||||
);
|
||||
if (insErr && !deliveryError) {
|
||||
deliveryError = insErr;
|
||||
}
|
||||
|
||||
if (!insErr) {
|
||||
this._deliveriesInsertedCounter.add(group.deliveryInserts.length);
|
||||
}
|
||||
}
|
||||
|
||||
span.setAttribute("delivery_inserts", combinedDeliveryInserts.length);
|
||||
|
||||
this.logger.debug("Flushing inserts", {
|
||||
flushId,
|
||||
deliveryInserts: combinedDeliveryInserts.length,
|
||||
clickhouseGroups: groups.size,
|
||||
});
|
||||
|
||||
if (deliveryError) {
|
||||
this.logger.error("Error inserting delivery inserts", {
|
||||
error: deliveryError,
|
||||
flushId,
|
||||
});
|
||||
recordSpanError(span, deliveryError);
|
||||
}
|
||||
|
||||
this.logger.debug("Flushed inserts", {
|
||||
flushId,
|
||||
deliveryInserts: combinedDeliveryInserts.length,
|
||||
});
|
||||
|
||||
this.events.emit("batchFlushed", { flushId, deliveryInserts: combinedDeliveryInserts });
|
||||
|
||||
const flushDurationMs = performance.now() - flushStartTime;
|
||||
const hasErrors = deliveryError !== null;
|
||||
|
||||
this._batchSizeHistogram.record(batch.length);
|
||||
this._flushDurationHistogram.record(flushDurationMs);
|
||||
this._batchesFlushedCounter.add(1, { success: !hasErrors });
|
||||
});
|
||||
}
|
||||
|
||||
// New method to handle inserts with retry logic for connection errors
|
||||
async #insertWithRetry<T>(
|
||||
insertFn: (attempt: number) => Promise<T>,
|
||||
operationName: string,
|
||||
flushId: string
|
||||
): Promise<[Error | null, T | null]> {
|
||||
let lastError: Error | null = null;
|
||||
|
||||
for (let attempt = 1; attempt <= this._insertMaxRetries; attempt++) {
|
||||
try {
|
||||
const result = await insertFn(attempt);
|
||||
return [null, result];
|
||||
} catch (error) {
|
||||
lastError = error instanceof Error ? error : new Error(String(error));
|
||||
|
||||
// Check if this is a retryable error
|
||||
if (this.#isRetryableError(lastError)) {
|
||||
const delay = this.#calculateRetryDelay(attempt);
|
||||
|
||||
this.logger.warn(`Retrying WebhookDeliveriesReplication insert due to error`, {
|
||||
operationName,
|
||||
flushId,
|
||||
attempt,
|
||||
maxRetries: this._insertMaxRetries,
|
||||
error: lastError.message,
|
||||
delay,
|
||||
});
|
||||
|
||||
// Record retry metric
|
||||
this._insertRetriesCounter.add(1, { operation: "webhook_deliveries" });
|
||||
|
||||
await new Promise((resolve) => setTimeout(resolve, delay));
|
||||
continue;
|
||||
}
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
return [lastError, null];
|
||||
}
|
||||
|
||||
// Retry all errors except known permanent ones
|
||||
#isRetryableError(error: Error): boolean {
|
||||
const errorMessage = error.message.toLowerCase();
|
||||
|
||||
// Permanent errors that should NOT be retried
|
||||
const permanentErrorPatterns = [
|
||||
"authentication failed",
|
||||
"permission denied",
|
||||
"invalid credentials",
|
||||
"table not found",
|
||||
"database not found",
|
||||
"column not found",
|
||||
"schema mismatch",
|
||||
"invalid query",
|
||||
"syntax error",
|
||||
"type error",
|
||||
"constraint violation",
|
||||
"duplicate key",
|
||||
"foreign key violation",
|
||||
];
|
||||
|
||||
// If it's a known permanent error, don't retry
|
||||
if (permanentErrorPatterns.some((pattern) => errorMessage.includes(pattern))) {
|
||||
return false;
|
||||
}
|
||||
|
||||
// Retry everything else
|
||||
return true;
|
||||
}
|
||||
|
||||
#calculateRetryDelay(attempt: number): number {
|
||||
// Exponential backoff: baseDelay, baseDelay*2, baseDelay*4, etc.
|
||||
const delay = Math.min(
|
||||
this._insertBaseDelayMs * Math.pow(2, attempt - 1),
|
||||
this._insertMaxDelayMs
|
||||
);
|
||||
|
||||
// Add some jitter to prevent thundering herd
|
||||
const jitter = Math.random() * 100;
|
||||
return delay + jitter;
|
||||
}
|
||||
|
||||
#getClickhouseInsertSettings() {
|
||||
if (this._insertStrategy === "insert") {
|
||||
return {};
|
||||
}
|
||||
|
||||
return {
|
||||
async_insert: 1 as const,
|
||||
async_insert_max_data_size: "1000000",
|
||||
async_insert_busy_timeout_ms: 1000,
|
||||
wait_for_async_insert: this.options.waitForAsyncInsert ? (1 as const) : (0 as const),
|
||||
};
|
||||
}
|
||||
|
||||
async #insertDeliveryInserts(
|
||||
clickhouse: ClickHouse,
|
||||
deliveryInserts: WebhookDeliveryInsertArray[],
|
||||
attempt: number
|
||||
) {
|
||||
if (deliveryInserts.length === 0) {
|
||||
return;
|
||||
}
|
||||
return await startSpan(this._tracer, "insertDeliveryInserts", async (span) => {
|
||||
const [insertError, insertResult] = await clickhouse.webhookDeliveries.insertCompactArrays(
|
||||
deliveryInserts,
|
||||
{
|
||||
params: {
|
||||
clickhouse_settings: this.#getClickhouseInsertSettings(),
|
||||
},
|
||||
}
|
||||
);
|
||||
|
||||
if (insertError) {
|
||||
this.logger.error("Error inserting delivery inserts attempt", {
|
||||
error: insertError,
|
||||
attempt,
|
||||
});
|
||||
|
||||
recordSpanError(span, insertError);
|
||||
throw insertError;
|
||||
}
|
||||
|
||||
return insertResult;
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
function toWebhookDeliveryInsertArray(
|
||||
delivery: WebhookDelivery,
|
||||
version: bigint,
|
||||
isDeleted: boolean
|
||||
): WebhookDeliveryInsertArray {
|
||||
return [
|
||||
delivery.runtimeEnvironmentId,
|
||||
delivery.organizationId,
|
||||
delivery.projectId,
|
||||
delivery.id,
|
||||
delivery.webhookEndpointId,
|
||||
delivery.environmentType,
|
||||
delivery.friendlyId,
|
||||
delivery.externalDeliveryId ?? "",
|
||||
delivery.runId ?? "",
|
||||
delivery.status,
|
||||
delivery.isTest ? 1 : 0,
|
||||
delivery.createdAt.getTime(),
|
||||
delivery.updatedAt.getTime(),
|
||||
version.toString(),
|
||||
isDeleted ? 1 : 0,
|
||||
];
|
||||
}
|
||||
|
||||
function lsnToUInt64(lsn: string): bigint {
|
||||
const [seg, off] = lsn.split("/");
|
||||
return (BigInt("0x" + seg) << 32n) | BigInt("0x" + off);
|
||||
}
|
||||
+382
@@ -0,0 +1,382 @@
|
||||
import { type ClickhouseQueryBuilder } from "@internal/clickhouse";
|
||||
import { WebhookDeliveryId } from "@trigger.dev/core/v3/isomorphic";
|
||||
import { boundedIn } from "@trigger.dev/database";
|
||||
import { createdAtMsBounds, deliveryIdsCreatedAtBounds } from "./deliveryIdBounds";
|
||||
import { decodeRunsCursor, encodeRunsCursor } from "../runsRepository/runsCursor.server";
|
||||
import {
|
||||
type CountDeliveriesByEndpointOptions,
|
||||
type DetailedWebhookDelivery,
|
||||
type FilterWebhookDeliveriesOptions,
|
||||
type GetDeliveriesByFriendlyIdsOptions,
|
||||
type GetWebhookDeliveryOptions,
|
||||
type IWebhookDeliveriesRepository,
|
||||
type ListedWebhookDelivery,
|
||||
type ListWebhookDeliveriesOptions,
|
||||
type WebhookDeliveriesRepositoryOptions,
|
||||
type WebhookDeliveryIdsPage,
|
||||
} from "./webhookDeliveriesRepository.server";
|
||||
|
||||
type DeliveryCursorRow = { deliveryId: string; createdAt: number };
|
||||
|
||||
const DELIVERY_DETAIL_SELECT = {
|
||||
id: true,
|
||||
friendlyId: true,
|
||||
webhookEndpointId: true,
|
||||
runtimeEnvironmentId: true,
|
||||
environmentType: true,
|
||||
status: true,
|
||||
externalDeliveryId: true,
|
||||
idempotencyKey: true,
|
||||
runId: true,
|
||||
rawBodyHash: true,
|
||||
parsedEvent: true,
|
||||
headers: true,
|
||||
errorMessage: true,
|
||||
filterReason: true,
|
||||
createdAt: true,
|
||||
updatedAt: true,
|
||||
processedAt: true,
|
||||
} as const;
|
||||
|
||||
const DELIVERY_LIST_SELECT = {
|
||||
id: true,
|
||||
friendlyId: true,
|
||||
webhookEndpointId: true,
|
||||
runtimeEnvironmentId: true,
|
||||
status: true,
|
||||
isTest: true,
|
||||
externalDeliveryId: true,
|
||||
runId: true,
|
||||
createdAt: true,
|
||||
processedAt: true,
|
||||
errorMessage: true,
|
||||
} as const;
|
||||
|
||||
export class ClickHouseWebhookDeliveriesRepository implements IWebhookDeliveriesRepository {
|
||||
constructor(private readonly options: WebhookDeliveriesRepositoryOptions) {}
|
||||
|
||||
get name() {
|
||||
return "clickhouse";
|
||||
}
|
||||
|
||||
/**
|
||||
* Runs the keyset-paginated query and returns `{ deliveryId, createdAt }` rows
|
||||
* (one extra beyond `page.size` to signal "has more"). The ordering is always
|
||||
* the composite `(created_at, delivery_id)`; the cursor predicate must match
|
||||
* it. The cursor is the shared opaque `runsCursor` token — its `runId` field
|
||||
* carries the delivery id here.
|
||||
*/
|
||||
private async listDeliveryRows(
|
||||
options: ListWebhookDeliveriesOptions
|
||||
): Promise<DeliveryCursorRow[]> {
|
||||
const queryBuilder = this.options.clickhouse.webhookDeliveries.queryBuilder();
|
||||
applyDeliveryFiltersToQueryBuilder(queryBuilder, options);
|
||||
|
||||
const forward = options.page.direction === "forward" || !options.page.direction;
|
||||
|
||||
if (options.page.cursor) {
|
||||
const decoded = decodeRunsCursor(options.page.cursor);
|
||||
|
||||
if (forward) {
|
||||
if (decoded.kind === "composite") {
|
||||
queryBuilder.where(
|
||||
"(created_at, delivery_id) < (fromUnixTimestamp64Milli({cursorCreatedAt: Int64}), {deliveryId: String})",
|
||||
{ cursorCreatedAt: decoded.createdAt, deliveryId: decoded.runId }
|
||||
);
|
||||
} else {
|
||||
queryBuilder.where("delivery_id < {deliveryId: String}", { deliveryId: decoded.runId });
|
||||
}
|
||||
queryBuilder.orderBy("created_at DESC, delivery_id DESC");
|
||||
} else {
|
||||
if (decoded.kind === "composite") {
|
||||
queryBuilder.where(
|
||||
"(created_at, delivery_id) > (fromUnixTimestamp64Milli({cursorCreatedAt: Int64}), {deliveryId: String})",
|
||||
{ cursorCreatedAt: decoded.createdAt, deliveryId: decoded.runId }
|
||||
);
|
||||
} else {
|
||||
queryBuilder.where("delivery_id > {deliveryId: String}", { deliveryId: decoded.runId });
|
||||
}
|
||||
queryBuilder.orderBy("created_at ASC, delivery_id ASC");
|
||||
}
|
||||
|
||||
queryBuilder.limit(options.page.size + 1);
|
||||
} else {
|
||||
// Initial page - no cursor provided
|
||||
queryBuilder.orderBy("created_at DESC, delivery_id DESC").limit(options.page.size + 1);
|
||||
}
|
||||
|
||||
const [queryError, result] = await queryBuilder.execute();
|
||||
|
||||
if (queryError) {
|
||||
throw queryError;
|
||||
}
|
||||
|
||||
return result.map((row) => ({ deliveryId: row.delivery_id, createdAt: row.created_at_ms }));
|
||||
}
|
||||
|
||||
/**
|
||||
* Turns the (size + 1) result rows into the actual page of rows plus the
|
||||
* forward/backward cursors. Mirrors ClickHouseRunsRepository.listRunIds; the
|
||||
* cursor tokens are composite so pagination can't duplicate or skip
|
||||
* deliveries. Shared by listDeliveryIds and listDeliveries so the hydration
|
||||
* path can reuse the page rows (and their created_at values) directly.
|
||||
*/
|
||||
private buildPage(
|
||||
rows: DeliveryCursorRow[],
|
||||
options: ListWebhookDeliveriesOptions
|
||||
): {
|
||||
pageRows: DeliveryCursorRow[];
|
||||
pagination: { nextCursor: string | null; previousCursor: string | null };
|
||||
} {
|
||||
// listDeliveryRows fetches one extra row beyond page.size to detect "has more".
|
||||
const hasMore = rows.length > options.page.size;
|
||||
|
||||
const cursorFor = (row: DeliveryCursorRow | undefined): string | null =>
|
||||
row ? encodeRunsCursor(row.createdAt, row.deliveryId) : null;
|
||||
|
||||
let nextCursor: string | null = null;
|
||||
let previousCursor: string | null = null;
|
||||
|
||||
const direction = options.page.direction ?? "forward";
|
||||
switch (direction) {
|
||||
case "forward": {
|
||||
previousCursor = options.page.cursor ? cursorFor(rows.at(0)) : null;
|
||||
if (hasMore) {
|
||||
// The next cursor is the last delivery on this page.
|
||||
nextCursor = cursorFor(rows[options.page.size - 1]);
|
||||
}
|
||||
break;
|
||||
}
|
||||
case "backward": {
|
||||
const reversedRows = [...rows].reverse();
|
||||
if (hasMore) {
|
||||
previousCursor = cursorFor(reversedRows.at(1));
|
||||
nextCursor = cursorFor(reversedRows.at(options.page.size));
|
||||
} else {
|
||||
// No newer rows, so there's no previous (newer) page. The next
|
||||
// (older) cursor is the oldest row on this page = rows[0] (rows are
|
||||
// ASC here). Index by the actual row count, not page.size — on a
|
||||
// partial page (fewer than page.size rows) page.size-1 overshoots
|
||||
// and would null the cursor, stranding forward navigation.
|
||||
nextCursor = cursorFor(rows.at(0));
|
||||
}
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
// The page is always the first `page.size` rows of the result. listDeliveryRows
|
||||
// fetches one extra row only to detect `hasMore`; that extra row is the
|
||||
// farthest from the cursor in BOTH directions (forward orders DESC, backward
|
||||
// orders ASC), so it's always the trailing element to drop — never the
|
||||
// leading one.
|
||||
const pageRows = rows.slice(0, options.page.size);
|
||||
|
||||
return { pageRows, pagination: { nextCursor, previousCursor } };
|
||||
}
|
||||
|
||||
async listDeliveryIds(options: ListWebhookDeliveriesOptions): Promise<WebhookDeliveryIdsPage> {
|
||||
const rows = await this.listDeliveryRows(options);
|
||||
const { pageRows, pagination } = this.buildPage(rows, options);
|
||||
|
||||
return { deliveryIds: pageRows.map((row) => row.deliveryId), pagination };
|
||||
}
|
||||
|
||||
async listDeliveries(options: ListWebhookDeliveriesOptions) {
|
||||
const rows = await this.listDeliveryRows(options);
|
||||
const { pageRows, pagination } = this.buildPage(rows, options);
|
||||
|
||||
if (pageRows.length === 0) {
|
||||
return { deliveries: [], pagination };
|
||||
}
|
||||
|
||||
const deliveryIds = pageRows.map((row) => row.deliveryId);
|
||||
|
||||
// PARTITION PRUNING: webhookDelivery is RANGE-partitioned on createdAt. An
|
||||
// `id IN (...)` query without a createdAt predicate scans every child
|
||||
// partition, so derive a [min, max] range from the CH page and pass it
|
||||
// through. This is the one place webhook hydration diverges from runs.
|
||||
const bounds = createdAtMsBounds(pageRows.map((row) => row.createdAt));
|
||||
|
||||
// CH gives the ordered id list; Postgres hydrates the full lean rows by PK id.
|
||||
const deliveries = await this.options.prisma.webhookDelivery.findMany({
|
||||
where: {
|
||||
id: { in: boundedIn(deliveryIds) },
|
||||
...(bounds ? { createdAt: bounds } : {}),
|
||||
},
|
||||
select: DELIVERY_LIST_SELECT,
|
||||
});
|
||||
|
||||
// Re-order to CH order (findMany does not preserve `in` order).
|
||||
const byId = new Map(deliveries.map((d) => [d.id, d]));
|
||||
let result = deliveryIds
|
||||
.map((id) => byId.get(id))
|
||||
.filter((d): d is (typeof deliveries)[number] => Boolean(d));
|
||||
|
||||
// ClickHouse is slightly delayed, so re-filter status in memory too.
|
||||
if (options.statuses && options.statuses.length > 0) {
|
||||
result = result.filter((d) => options.statuses!.includes(d.status));
|
||||
}
|
||||
|
||||
return { deliveries: result, pagination };
|
||||
}
|
||||
|
||||
async countDeliveries(options: FilterWebhookDeliveriesOptions) {
|
||||
const queryBuilder = this.options.clickhouse.webhookDeliveries.countQueryBuilder();
|
||||
applyDeliveryFiltersToQueryBuilder(queryBuilder, options);
|
||||
|
||||
const [queryError, result] = await queryBuilder.execute();
|
||||
|
||||
if (queryError) {
|
||||
throw queryError;
|
||||
}
|
||||
|
||||
if (result.length === 0) {
|
||||
throw new Error("No count rows returned");
|
||||
}
|
||||
|
||||
return result[0].count;
|
||||
}
|
||||
|
||||
async countDeliveriesByEndpoint(
|
||||
options: CountDeliveriesByEndpointOptions
|
||||
): Promise<Map<string, number>> {
|
||||
if (options.webhookEndpointIds.length === 0) {
|
||||
return new Map();
|
||||
}
|
||||
|
||||
const queryBuilder = this.options.clickhouse.webhookDeliveries.groupedCountQueryBuilder();
|
||||
queryBuilder
|
||||
.where("organization_id = {organizationId: String}", {
|
||||
organizationId: options.organizationId,
|
||||
})
|
||||
.where("project_id = {projectId: String}", { projectId: options.projectId })
|
||||
.where("environment_id = {environmentId: String}", { environmentId: options.environmentId })
|
||||
.where("webhook_endpoint_id IN {webhookEndpointIds: Array(String)}", {
|
||||
webhookEndpointIds: options.webhookEndpointIds,
|
||||
})
|
||||
.where("created_at >= fromUnixTimestamp64Milli({period: Int64})", {
|
||||
period: Date.now() - options.period,
|
||||
})
|
||||
.groupBy("webhook_endpoint_id");
|
||||
|
||||
const [queryError, rows] = await queryBuilder.execute();
|
||||
|
||||
if (queryError) {
|
||||
throw queryError;
|
||||
}
|
||||
|
||||
const counts = new Map<string, number>();
|
||||
for (const row of rows) {
|
||||
counts.set(row.webhook_endpoint_id, row.count);
|
||||
}
|
||||
return counts;
|
||||
}
|
||||
|
||||
/**
|
||||
* A point lookup: pure Postgres, never ClickHouse. `friendlyId` is `whd_` + the row id, so we
|
||||
* strip the prefix and hit the composite-PK index (scoped by environment). ClickHouse is for
|
||||
* aggregations and for filtering/ordering a list into ids, never for selecting a row's columns.
|
||||
*
|
||||
* `WebhookDelivery` is RANGE-partitioned on `createdAt`, so a bare `id` predicate can't prune and
|
||||
* probes every partition. The delivery id is time-encoded (see `WebhookDeliveryId`) with the same
|
||||
* timestamp the engine stores as `createdAt`, so we recover it from the id and add it as an exact
|
||||
* predicate to prune to the row's partition.
|
||||
*/
|
||||
async getDelivery(options: GetWebhookDeliveryOptions): Promise<DetailedWebhookDelivery | null> {
|
||||
const id = WebhookDeliveryId.toId(options.friendlyId);
|
||||
const createdAt = WebhookDeliveryId.parseTimestamp(options.friendlyId);
|
||||
|
||||
return this.options.prisma.webhookDelivery.findFirst({
|
||||
where: {
|
||||
id,
|
||||
runtimeEnvironmentId: options.environmentId,
|
||||
...(createdAt ? { createdAt } : {}),
|
||||
},
|
||||
select: DELIVERY_DETAIL_SELECT,
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Hydrate a known set of deliveries by friendlyId. Pure Postgres: the caller (the live poll)
|
||||
* already has the ids, so there is nothing for ClickHouse to filter or order.
|
||||
*
|
||||
* The ids are time-encoded (see `WebhookDeliveryId`), so we bound the query to the span of their
|
||||
* mint timestamps, which equal the rows' `createdAt`. That prunes the RANGE-partitioned table to
|
||||
* the visible page's few days instead of probing all of retention on every poll.
|
||||
*/
|
||||
async getDeliveriesByFriendlyIds(
|
||||
options: GetDeliveriesByFriendlyIdsOptions
|
||||
): Promise<ListedWebhookDelivery[]> {
|
||||
const ids = options.friendlyIds.map((friendlyId) => WebhookDeliveryId.toId(friendlyId));
|
||||
if (ids.length === 0) return [];
|
||||
|
||||
const bounds = deliveryIdsCreatedAtBounds(options.friendlyIds);
|
||||
|
||||
return this.options.prisma.webhookDelivery.findMany({
|
||||
where: {
|
||||
id: { in: boundedIn(ids) },
|
||||
runtimeEnvironmentId: options.environmentId,
|
||||
...(bounds ? { createdAt: bounds } : {}),
|
||||
},
|
||||
select: DELIVERY_LIST_SELECT,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
function applyDeliveryFiltersToQueryBuilder<T>(
|
||||
queryBuilder: ClickhouseQueryBuilder<T>,
|
||||
options: FilterWebhookDeliveriesOptions
|
||||
) {
|
||||
queryBuilder
|
||||
.where("organization_id = {organizationId: String}", {
|
||||
organizationId: options.organizationId,
|
||||
})
|
||||
.where("project_id = {projectId: String}", { projectId: options.projectId })
|
||||
.where("environment_id = {environmentId: String}", { environmentId: options.environmentId });
|
||||
|
||||
if (options.webhookEndpointId) {
|
||||
queryBuilder.where("webhook_endpoint_id = {webhookEndpointId: String}", {
|
||||
webhookEndpointId: options.webhookEndpointId,
|
||||
});
|
||||
}
|
||||
|
||||
if (options.webhookEndpointIds && options.webhookEndpointIds.length > 0) {
|
||||
queryBuilder.where("webhook_endpoint_id IN {webhookEndpointIds: Array(String)}", {
|
||||
webhookEndpointIds: options.webhookEndpointIds,
|
||||
});
|
||||
}
|
||||
|
||||
if (options.deliveryId) {
|
||||
queryBuilder.where(
|
||||
"(friendly_id = {deliveryId: String} OR external_delivery_id = {deliveryId: String})",
|
||||
{ deliveryId: options.deliveryId }
|
||||
);
|
||||
}
|
||||
|
||||
if (options.runId) {
|
||||
queryBuilder.where("run_id = {runId: String}", { runId: options.runId });
|
||||
}
|
||||
|
||||
if (options.statuses && options.statuses.length > 0) {
|
||||
queryBuilder.where("status IN {statuses: Array(String)}", { statuses: options.statuses });
|
||||
}
|
||||
|
||||
if (options.isTest !== undefined) {
|
||||
queryBuilder.where("is_test = {isTest: UInt8}", { isTest: options.isTest ? 1 : 0 });
|
||||
}
|
||||
|
||||
// PARTITION PRUNING: the list MUST carry a created_at range.
|
||||
if (options.period) {
|
||||
queryBuilder.where("created_at >= fromUnixTimestamp64Milli({period: Int64})", {
|
||||
period: new Date(Date.now() - options.period).getTime(),
|
||||
});
|
||||
}
|
||||
if (options.from) {
|
||||
queryBuilder.where("created_at >= fromUnixTimestamp64Milli({from: Int64})", {
|
||||
from: options.from,
|
||||
});
|
||||
}
|
||||
if (options.to) {
|
||||
queryBuilder.where("created_at <= fromUnixTimestamp64Milli({to: Int64})", { to: options.to });
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,55 @@
|
||||
import { WebhookDeliveryId } from "@trigger.dev/core/v3/isomorphic";
|
||||
|
||||
/**
|
||||
* Single-pass min/max over a set of `createdAt` timestamps (unix ms), returned as a Prisma
|
||||
* `{ gte, lte }` range for partition-pruning the RANGE-partitioned `WebhookDelivery` table.
|
||||
*
|
||||
* Avoids `Math.min(...spread)` / `Math.max(...spread)`: the spread builds an O(n) argument list and
|
||||
* throws "Maximum call stack size exceeded" once the array is large (~1e5+ elements). Returns
|
||||
* `undefined` for an empty set, so the caller adds no `createdAt` predicate.
|
||||
*/
|
||||
export function createdAtMsBounds(msValues: number[]): { gte: Date; lte: Date } | undefined {
|
||||
let min = Number.POSITIVE_INFINITY;
|
||||
let max = Number.NEGATIVE_INFINITY;
|
||||
|
||||
for (const ms of msValues) {
|
||||
if (ms < min) min = ms;
|
||||
if (ms > max) max = ms;
|
||||
}
|
||||
|
||||
if (min === Number.POSITIVE_INFINITY) return undefined;
|
||||
return { gte: new Date(min), lte: new Date(max) };
|
||||
}
|
||||
|
||||
/**
|
||||
* Compute the `createdAt` span covering a set of webhook delivery friendlyIds, for partition-pruning
|
||||
* a lookup by id on the RANGE-partitioned `WebhookDelivery` table.
|
||||
*
|
||||
* A delivery id body is `base32hex(big-endian ms timestamp then random bytes)`, and base32hex is
|
||||
* order-preserving, so lexical id order equals chronological order. The earliest and latest
|
||||
* timestamps therefore sit at the lexical extremes, and we recover the span by decoding only those
|
||||
* two ids instead of all N. The embedded timestamp equals the row's `createdAt`, so `[gte, lte]`
|
||||
* covers every row in the set exactly.
|
||||
*
|
||||
* Returns `undefined` for an empty set, or if an extreme id fails to decode, so the caller adds no
|
||||
* `createdAt` predicate and the lookup stays correct (just unpruned).
|
||||
*/
|
||||
export function deliveryIdsCreatedAtBounds(
|
||||
friendlyIds: string[]
|
||||
): { gte: Date; lte: Date } | undefined {
|
||||
if (friendlyIds.length === 0) return undefined;
|
||||
|
||||
let minBody = WebhookDeliveryId.toId(friendlyIds[0]!);
|
||||
let maxBody = minBody;
|
||||
for (let i = 1; i < friendlyIds.length; i++) {
|
||||
const body = WebhookDeliveryId.toId(friendlyIds[i]!);
|
||||
if (body < minBody) minBody = body;
|
||||
if (body > maxBody) maxBody = body;
|
||||
}
|
||||
|
||||
const gte = WebhookDeliveryId.parseTimestamp(minBody);
|
||||
const lte = WebhookDeliveryId.parseTimestamp(maxBody);
|
||||
if (!gte || !lte) return undefined;
|
||||
|
||||
return { gte, lte };
|
||||
}
|
||||
+226
@@ -0,0 +1,226 @@
|
||||
import { type ClickHouse } from "@internal/clickhouse";
|
||||
import { type Tracer } from "@internal/tracing";
|
||||
import { type Logger, type LogLevel } from "@trigger.dev/core/logger";
|
||||
import { type Prisma, type WebhookDeliveryStatus } from "@trigger.dev/database";
|
||||
import { type WebhookReplicaDatabase } from "~/db.server";
|
||||
import { startActiveSpan } from "~/v3/tracer.server";
|
||||
import { ClickHouseWebhookDeliveriesRepository } from "./clickhouseWebhookDeliveriesRepository.server";
|
||||
|
||||
export type WebhookDeliveriesRepositoryOptions = {
|
||||
clickhouse: ClickHouse;
|
||||
prisma: WebhookReplicaDatabase;
|
||||
logger?: Logger;
|
||||
logLevel?: LogLevel;
|
||||
tracer?: Tracer;
|
||||
};
|
||||
|
||||
export type FilterWebhookDeliveriesOptions = {
|
||||
organizationId: string;
|
||||
projectId: string;
|
||||
environmentId: string;
|
||||
webhookEndpointId?: string;
|
||||
webhookEndpointIds?: string[]; // webhook (handler) filter, resolved to endpoint ids by the presenter
|
||||
deliveryId?: string; // matches the delivery friendlyId OR external delivery id
|
||||
runId?: string; // INTERNAL run id (presenter resolves friendly -> internal)
|
||||
statuses?: WebhookDeliveryStatus[];
|
||||
isTest?: boolean;
|
||||
period?: number;
|
||||
from?: number;
|
||||
to?: number;
|
||||
};
|
||||
|
||||
export type ListWebhookDeliveriesOptions = FilterWebhookDeliveriesOptions & {
|
||||
page: { size: number; cursor?: string; direction?: "forward" | "backward" };
|
||||
};
|
||||
|
||||
export type WebhookDeliveryIdsPage = {
|
||||
deliveryIds: string[];
|
||||
pagination: { nextCursor: string | null; previousCursor: string | null };
|
||||
};
|
||||
|
||||
export type ListedWebhookDelivery = Prisma.WebhookDeliveryGetPayload<{
|
||||
select: {
|
||||
id: true;
|
||||
friendlyId: true;
|
||||
webhookEndpointId: true;
|
||||
runtimeEnvironmentId: true;
|
||||
status: true;
|
||||
isTest: true;
|
||||
externalDeliveryId: true;
|
||||
runId: true;
|
||||
createdAt: true;
|
||||
processedAt: true;
|
||||
errorMessage: true;
|
||||
};
|
||||
}>;
|
||||
|
||||
// The detail view selects more than the list, including the size-capped
|
||||
// `parsedEvent` snapshot (which the list omits).
|
||||
export type DetailedWebhookDelivery = Prisma.WebhookDeliveryGetPayload<{
|
||||
select: {
|
||||
id: true;
|
||||
friendlyId: true;
|
||||
webhookEndpointId: true;
|
||||
runtimeEnvironmentId: true;
|
||||
environmentType: true;
|
||||
status: true;
|
||||
externalDeliveryId: true;
|
||||
idempotencyKey: true;
|
||||
runId: true;
|
||||
rawBodyHash: true;
|
||||
parsedEvent: true;
|
||||
headers: true;
|
||||
errorMessage: true;
|
||||
filterReason: true;
|
||||
createdAt: true;
|
||||
updatedAt: true;
|
||||
processedAt: true;
|
||||
};
|
||||
}>;
|
||||
|
||||
export type GetWebhookDeliveryOptions = {
|
||||
organizationId: string;
|
||||
projectId: string;
|
||||
environmentId: string;
|
||||
friendlyId: string;
|
||||
};
|
||||
|
||||
export type GetDeliveriesByFriendlyIdsOptions = {
|
||||
organizationId: string;
|
||||
projectId: string;
|
||||
environmentId: string;
|
||||
friendlyIds: string[];
|
||||
};
|
||||
|
||||
export type CountDeliveriesByEndpointOptions = {
|
||||
organizationId: string;
|
||||
projectId: string;
|
||||
environmentId: string;
|
||||
webhookEndpointIds: string[];
|
||||
period: number; // lookback window in ms
|
||||
};
|
||||
|
||||
export interface IWebhookDeliveriesRepository {
|
||||
name: string;
|
||||
listDeliveryIds(options: ListWebhookDeliveriesOptions): Promise<WebhookDeliveryIdsPage>;
|
||||
listDeliveries(options: ListWebhookDeliveriesOptions): Promise<{
|
||||
deliveries: ListedWebhookDelivery[];
|
||||
pagination: { nextCursor: string | null; previousCursor: string | null };
|
||||
}>;
|
||||
getDeliveriesByFriendlyIds(
|
||||
options: GetDeliveriesByFriendlyIdsOptions
|
||||
): Promise<ListedWebhookDelivery[]>;
|
||||
countDeliveries(options: FilterWebhookDeliveriesOptions): Promise<number>;
|
||||
// One grouped query for many endpoints -> Map<endpointId, count> (avoids an N+1 of count queries).
|
||||
countDeliveriesByEndpoint(
|
||||
options: CountDeliveriesByEndpointOptions
|
||||
): Promise<Map<string, number>>;
|
||||
getDelivery(options: GetWebhookDeliveryOptions): Promise<DetailedWebhookDelivery | null>;
|
||||
}
|
||||
|
||||
export class WebhookDeliveriesRepository implements IWebhookDeliveriesRepository {
|
||||
private readonly clickHouseRepository: ClickHouseWebhookDeliveriesRepository;
|
||||
|
||||
constructor(private readonly options: WebhookDeliveriesRepositoryOptions) {
|
||||
this.clickHouseRepository = new ClickHouseWebhookDeliveriesRepository(options);
|
||||
}
|
||||
|
||||
get name() {
|
||||
return this.clickHouseRepository.name;
|
||||
}
|
||||
|
||||
async listDeliveryIds(options: ListWebhookDeliveriesOptions) {
|
||||
return startActiveSpan(
|
||||
"webhookDeliveriesRepository.listDeliveryIds",
|
||||
async () => this.clickHouseRepository.listDeliveryIds(options),
|
||||
{
|
||||
attributes: {
|
||||
"repository.name": "clickhouse",
|
||||
organizationId: options.organizationId,
|
||||
projectId: options.projectId,
|
||||
environmentId: options.environmentId,
|
||||
},
|
||||
}
|
||||
);
|
||||
}
|
||||
|
||||
async listDeliveries(options: ListWebhookDeliveriesOptions) {
|
||||
return startActiveSpan(
|
||||
"webhookDeliveriesRepository.listDeliveries",
|
||||
async () => this.clickHouseRepository.listDeliveries(options),
|
||||
{
|
||||
attributes: {
|
||||
"repository.name": "clickhouse",
|
||||
organizationId: options.organizationId,
|
||||
projectId: options.projectId,
|
||||
environmentId: options.environmentId,
|
||||
},
|
||||
}
|
||||
);
|
||||
}
|
||||
|
||||
async getDeliveriesByFriendlyIds(options: GetDeliveriesByFriendlyIdsOptions) {
|
||||
return startActiveSpan(
|
||||
"webhookDeliveriesRepository.getDeliveriesByFriendlyIds",
|
||||
async () => this.clickHouseRepository.getDeliveriesByFriendlyIds(options),
|
||||
{
|
||||
attributes: {
|
||||
"repository.name": "clickhouse",
|
||||
organizationId: options.organizationId,
|
||||
projectId: options.projectId,
|
||||
environmentId: options.environmentId,
|
||||
},
|
||||
}
|
||||
);
|
||||
}
|
||||
|
||||
async countDeliveries(options: FilterWebhookDeliveriesOptions) {
|
||||
return startActiveSpan(
|
||||
"webhookDeliveriesRepository.countDeliveries",
|
||||
async () => this.clickHouseRepository.countDeliveries(options),
|
||||
{
|
||||
attributes: {
|
||||
"repository.name": "clickhouse",
|
||||
organizationId: options.organizationId,
|
||||
projectId: options.projectId,
|
||||
environmentId: options.environmentId,
|
||||
},
|
||||
}
|
||||
);
|
||||
}
|
||||
|
||||
async countDeliveriesByEndpoint(options: CountDeliveriesByEndpointOptions) {
|
||||
return startActiveSpan(
|
||||
"webhookDeliveriesRepository.countDeliveriesByEndpoint",
|
||||
async () => this.clickHouseRepository.countDeliveriesByEndpoint(options),
|
||||
{
|
||||
attributes: {
|
||||
"repository.name": "clickhouse",
|
||||
organizationId: options.organizationId,
|
||||
projectId: options.projectId,
|
||||
environmentId: options.environmentId,
|
||||
},
|
||||
}
|
||||
);
|
||||
}
|
||||
|
||||
async getDelivery(options: GetWebhookDeliveryOptions) {
|
||||
return startActiveSpan(
|
||||
"webhookDeliveriesRepository.getDelivery",
|
||||
async () => this.clickHouseRepository.getDelivery(options),
|
||||
{
|
||||
attributes: {
|
||||
"repository.name": "clickhouse",
|
||||
organizationId: options.organizationId,
|
||||
projectId: options.projectId,
|
||||
environmentId: options.environmentId,
|
||||
},
|
||||
}
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
// Inline-constructed per request (NOT a module-level singleton), like RunsRepository.
|
||||
export function webhookDeliveriesRepository(opts: WebhookDeliveriesRepositoryOptions) {
|
||||
return new WebhookDeliveriesRepository(opts);
|
||||
}
|
||||
@@ -0,0 +1,31 @@
|
||||
import type { RequestHandler } from "express";
|
||||
import { Ratelimit } from "@upstash/ratelimit";
|
||||
import { env } from "~/env.server";
|
||||
import { logger } from "./logger.server";
|
||||
import { RateLimiter, type Duration } from "./rateLimiter.server";
|
||||
|
||||
const ipLimiter = new RateLimiter({
|
||||
keyPrefix: "webhook-ingress-ip",
|
||||
limiter: Ratelimit.fixedWindow(
|
||||
env.WEBHOOK_INGRESS_IP_RATE_LIMIT_TOKENS,
|
||||
env.WEBHOOK_INGRESS_IP_RATE_LIMIT_WINDOW as Duration
|
||||
),
|
||||
});
|
||||
|
||||
// Coarse per-IP gate mounted in server.ts ahead of the Remix handler. The
|
||||
// per-opaqueId limiter (webhookIngressRateLimit.server) is the real protection.
|
||||
export const webhookIngressIpRateLimiter: RequestHandler = async (req, res, next) => {
|
||||
if (!req.path.startsWith("/webhooks/v1/ingest/")) return next();
|
||||
const ip =
|
||||
(req.headers["x-forwarded-for"] as string)?.split(",")[0]?.trim() || req.ip || "unknown";
|
||||
try {
|
||||
const { success } = await ipLimiter.limit(ip);
|
||||
if (!success) {
|
||||
res.status(429).json({ error: "Too many requests" });
|
||||
return;
|
||||
}
|
||||
} catch (error) {
|
||||
logger.warn("webhookIngressIpRateLimiter: limiter error, allowing request", { error });
|
||||
}
|
||||
next();
|
||||
};
|
||||
@@ -0,0 +1,16 @@
|
||||
import { Ratelimit } from "@upstash/ratelimit";
|
||||
import { env } from "~/env.server";
|
||||
import { RateLimiter, type Duration } from "./rateLimiter.server";
|
||||
|
||||
// Per-opaqueId fixed-window limiter for the unauthenticated webhook ingress
|
||||
// route. apiRateLimiter/engineRateLimiter only match /api and /engine and key
|
||||
// off the auth header, so they never see this request; this is the real
|
||||
// protection (the per-IP limiter in server.ts is coarse).
|
||||
export const webhookIngressRateLimiter = new RateLimiter({
|
||||
keyPrefix: "webhook-ingress",
|
||||
limiter: Ratelimit.fixedWindow(
|
||||
env.WEBHOOK_INGRESS_RATE_LIMIT_TOKENS,
|
||||
env.WEBHOOK_INGRESS_RATE_LIMIT_WINDOW as Duration
|
||||
),
|
||||
logFailure: true,
|
||||
});
|
||||
@@ -209,6 +209,7 @@
|
||||
--color-aiMetrics: var(--color-green-500);
|
||||
--color-errors: var(--color-amber-500);
|
||||
--color-agents: var(--color-purple-500);
|
||||
--color-webhooks: var(--color-teal-500);
|
||||
--color-sessions: var(--color-pink-500);
|
||||
--color-playgrounds: var(--color-fuchsia-500);
|
||||
--color-models: var(--color-violet-500);
|
||||
|
||||
@@ -35,6 +35,7 @@ export const ENV_PAGE_TARGETS: ReadonlyMap<string, DeeplinkTarget> = new Map([
|
||||
["tasks", { landing: "", prefix: "tasks" }],
|
||||
["test", page("test")],
|
||||
["waitpoints", { landing: "waitpoints/tokens", prefix: "waitpoints/tokens" }],
|
||||
["webhooks", page("webhooks")],
|
||||
]);
|
||||
|
||||
export const DEEPLINK_PATH_PREFIX = "/_";
|
||||
|
||||
@@ -402,6 +402,52 @@ export function v3AgentTaskPath(
|
||||
)}`;
|
||||
}
|
||||
|
||||
export function v3WebhooksPath(
|
||||
organization: OrgForPath,
|
||||
project: ProjectForPath,
|
||||
environment: EnvironmentForPath
|
||||
) {
|
||||
// Top-level Webhooks section landing = the cross-endpoint deliveries list.
|
||||
return `${v3EnvironmentPath(organization, project, environment)}/webhooks`;
|
||||
}
|
||||
|
||||
export function v3WebhookTaskPath(
|
||||
organization: OrgForPath,
|
||||
project: ProjectForPath,
|
||||
environment: EnvironmentForPath,
|
||||
webhookSlug: string
|
||||
) {
|
||||
return `${v3EnvironmentPath(organization, project, environment)}/webhooks/${encodeURIComponent(
|
||||
webhookSlug
|
||||
)}`;
|
||||
}
|
||||
|
||||
export function v3WebhookDeliveryPath(
|
||||
organization: OrgForPath,
|
||||
project: ProjectForPath,
|
||||
environment: EnvironmentForPath,
|
||||
deliveryFriendlyId: string
|
||||
) {
|
||||
return `${v3EnvironmentPath(
|
||||
organization,
|
||||
project,
|
||||
environment
|
||||
)}/webhooks/deliveries/${encodeURIComponent(deliveryFriendlyId)}`;
|
||||
}
|
||||
|
||||
export function v3WebhookEndpointPath(
|
||||
organization: OrgForPath,
|
||||
project: ProjectForPath,
|
||||
environment: EnvironmentForPath,
|
||||
endpointFriendlyId: string
|
||||
) {
|
||||
return `${v3EnvironmentPath(
|
||||
organization,
|
||||
project,
|
||||
environment
|
||||
)}/webhooks/endpoints/${encodeURIComponent(endpointFriendlyId)}`;
|
||||
}
|
||||
|
||||
export function v3StandardTaskPath(
|
||||
organization: OrgForPath,
|
||||
project: ProjectForPath,
|
||||
|
||||
@@ -0,0 +1,40 @@
|
||||
// Read a request body, aborting as soon as the accumulated bytes exceed `limitBytes`. A chunked
|
||||
// upload can omit or understate Content-Length, so reading the stream incrementally (instead of
|
||||
// request.arrayBuffer(), which buffers the whole stream first) caps the memory a single request can
|
||||
// force us to hold before rejection. Returns null when the body is over the limit.
|
||||
export async function readBodyWithCap(
|
||||
request: Request,
|
||||
limitBytes: number
|
||||
): Promise<Uint8Array | null> {
|
||||
if (!request.body) {
|
||||
return new Uint8Array(0);
|
||||
}
|
||||
|
||||
const reader = request.body.getReader();
|
||||
const chunks: Uint8Array[] = [];
|
||||
let total = 0;
|
||||
|
||||
try {
|
||||
while (true) {
|
||||
const { done, value } = await reader.read();
|
||||
if (done) break;
|
||||
if (!value) continue;
|
||||
total += value.byteLength;
|
||||
if (total > limitBytes) {
|
||||
await reader.cancel();
|
||||
return null;
|
||||
}
|
||||
chunks.push(value);
|
||||
}
|
||||
} finally {
|
||||
reader.releaseLock();
|
||||
}
|
||||
|
||||
const out = new Uint8Array(total);
|
||||
let offset = 0;
|
||||
for (const chunk of chunks) {
|
||||
out.set(chunk, offset);
|
||||
offset += chunk.byteLength;
|
||||
}
|
||||
return out;
|
||||
}
|
||||
@@ -0,0 +1,11 @@
|
||||
import { env } from "~/env.server";
|
||||
|
||||
// Public origin webhook providers POST to. A dedicated WEBHOOK_INGRESS_ORIGIN (e.g.
|
||||
// https://webhook.trigger.dev) takes precedence; otherwise it rides the API/app origin.
|
||||
export function webhookIngressOrigin(): string {
|
||||
return env.WEBHOOK_INGRESS_ORIGIN ?? env.API_ORIGIN ?? env.APP_ORIGIN;
|
||||
}
|
||||
|
||||
export function webhookIngressUrl(opaqueId: string): string {
|
||||
return `${webhookIngressOrigin()}/webhooks/v1/ingest/${opaqueId}`;
|
||||
}
|
||||
@@ -5,6 +5,7 @@ export const FEATURE_FLAG = {
|
||||
taskEventRepository: "taskEventRepository",
|
||||
hasQueryAccess: "hasQueryAccess",
|
||||
hasLogsPageAccess: "hasLogsPageAccess",
|
||||
hasWebhooksAccess: "hasWebhooksAccess",
|
||||
hasAiAccess: "hasAiAccess",
|
||||
hasDashboardAgentAccess: "hasDashboardAgentAccess",
|
||||
dashboardAgentTurnEvalsEnabled: "dashboardAgentTurnEvalsEnabled",
|
||||
@@ -41,6 +42,7 @@ export const FeatureFlagCatalog = {
|
||||
[FEATURE_FLAG.taskEventRepository]: z.enum(["clickhouse", "clickhouse_v2", "postgres"]),
|
||||
[FEATURE_FLAG.hasQueryAccess]: z.coerce.boolean(),
|
||||
[FEATURE_FLAG.hasLogsPageAccess]: z.coerce.boolean(),
|
||||
[FEATURE_FLAG.hasWebhooksAccess]: z.coerce.boolean(),
|
||||
[FEATURE_FLAG.hasAiAccess]: z.coerce.boolean(),
|
||||
// Gates the in-dashboard AI agent panel. Controllable globally and per-org
|
||||
// (org wins). Defaults off via DASHBOARD_AGENT_ENABLED.
|
||||
|
||||
@@ -16,6 +16,11 @@ import { runsReplicationInstance } from "~/services/runsReplicationInstance.serv
|
||||
// effect the bundler must preserve. See TRI-9864.
|
||||
import { sessionsReplicationInstance } from "~/services/sessionsReplicationInstance.server";
|
||||
(globalThis as Record<string, unknown>).__sessionsReplicationInstance = sessionsReplicationInstance;
|
||||
// Same reference-hold as the sessions replicator above (and the same
|
||||
// `void`-tree-shaking caveat) for the webhook deliveries replication singleton.
|
||||
import { webhookDeliveriesReplicationInstance } from "~/services/webhookDeliveriesReplicationInstance.server";
|
||||
(globalThis as Record<string, unknown>).__webhookDeliveriesReplicationInstance =
|
||||
webhookDeliveriesReplicationInstance;
|
||||
import { singleton } from "~/utils/singleton";
|
||||
import { tracer } from "../tracer.server";
|
||||
import { $replica } from "~/db.server";
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
import { BackgroundWorkerMetadata, tryCatch } from "@trigger.dev/core/v3";
|
||||
import { CURRENT_DEPLOYMENT_LABEL } from "@trigger.dev/core/v3/isomorphic";
|
||||
import type { PrismaClientOrTransaction, WorkerDeployment } from "@trigger.dev/database";
|
||||
import { webhookPrisma } from "~/db.server";
|
||||
import { logger } from "~/services/logger.server";
|
||||
import { syncTaskIdentifiers } from "~/services/taskIdentifierRegistry.server";
|
||||
import {
|
||||
@@ -9,7 +10,7 @@ import {
|
||||
} from "~/services/taskMetadataCache.server";
|
||||
import { taskMetadataCacheInstance } from "~/services/taskMetadataCacheInstance.server";
|
||||
import { BaseService, ServiceValidationError } from "./baseService.server";
|
||||
import { syncDeclarativeSchedules } from "./createBackgroundWorker.server";
|
||||
import { syncDeclarativeSchedules, syncDeclarativeWebhooks } from "./createBackgroundWorker.server";
|
||||
import { compareDeploymentVersions } from "../utils/deploymentVersions";
|
||||
|
||||
export type ChangeCurrentDeploymentDirection = "promote" | "rollback";
|
||||
@@ -217,5 +218,12 @@ export class ChangeCurrentDeploymentService extends BaseService {
|
||||
}
|
||||
|
||||
await syncDeclarativeSchedules(parsed.data.tasks, worker, environment, this._prisma);
|
||||
await syncDeclarativeWebhooks(
|
||||
parsed.data.webhooks,
|
||||
worker,
|
||||
environment,
|
||||
this._prisma,
|
||||
webhookPrisma
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -2,16 +2,24 @@ import type {
|
||||
BackgroundWorkerMetadata,
|
||||
BackgroundWorkerSourceFileMetadata,
|
||||
CreateBackgroundWorkerRequestBody,
|
||||
FilterAst,
|
||||
PromptResource,
|
||||
QueueManifest,
|
||||
TaskResource,
|
||||
WebhookResource,
|
||||
} from "@trigger.dev/core/v3";
|
||||
import { tryCatch } from "@trigger.dev/core/v3";
|
||||
import { BackgroundWorkerId, stringifyDuration } from "@trigger.dev/core/v3/isomorphic";
|
||||
import { FILTER_AST_VERSION, tryCatch } from "@trigger.dev/core/v3";
|
||||
import { FilterParseError, parseFilter } from "@internal/webhook-engine";
|
||||
import {
|
||||
BackgroundWorkerId,
|
||||
WebhookEndpointId,
|
||||
stringifyDuration,
|
||||
} from "@trigger.dev/core/v3/isomorphic";
|
||||
import { randomBytes } from "node:crypto";
|
||||
import type { BackgroundWorker, TaskQueue, TaskQueueType } from "@trigger.dev/database";
|
||||
import cronstrue from "cronstrue";
|
||||
import type { PrismaClientOrTransaction } from "~/db.server";
|
||||
import { $transaction, Prisma, boundedIn } from "~/db.server";
|
||||
import type { PrismaClientOrTransaction, WebhookDatabase } from "~/db.server";
|
||||
import { $transaction, Prisma, boundedIn, webhookPrisma } from "~/db.server";
|
||||
import { sanitizeQueueName } from "~/models/taskQueue.server";
|
||||
import type { AuthenticatedEnvironment } from "~/services/apiAuth.server";
|
||||
import { logger } from "~/services/logger.server";
|
||||
@@ -200,6 +208,34 @@ export class CreateBackgroundWorkerService extends BaseService {
|
||||
throw new ServiceValidationError("Error syncing declarative schedules");
|
||||
}
|
||||
|
||||
const [webhooksError] = await tryCatch(
|
||||
syncDeclarativeWebhooks(
|
||||
body.metadata.webhooks,
|
||||
backgroundWorker,
|
||||
environment,
|
||||
this._prisma,
|
||||
webhookPrisma
|
||||
)
|
||||
);
|
||||
if (webhooksError) {
|
||||
if (webhooksError instanceof ServiceValidationError) {
|
||||
logger.warn("Error syncing declarative webhooks", {
|
||||
error: webhooksError.message,
|
||||
backgroundWorker,
|
||||
environment,
|
||||
});
|
||||
throw webhooksError;
|
||||
}
|
||||
|
||||
logger.error("Error syncing declarative webhooks", {
|
||||
error: webhooksError,
|
||||
backgroundWorker,
|
||||
environment,
|
||||
});
|
||||
|
||||
throw new ServiceValidationError("Error syncing declarative webhooks");
|
||||
}
|
||||
|
||||
const [syncIdentifiersError] = await tryCatch(
|
||||
syncTaskIdentifiers(
|
||||
environment.id,
|
||||
@@ -354,7 +390,7 @@ async function createWorkerTask(
|
||||
): Promise<TaskMetadataEntry | null> {
|
||||
// Hoisted so the P2002 catch branch can return the same entry shape.
|
||||
let queue: TaskQueue | undefined;
|
||||
let resolvedTriggerSource: "SCHEDULED" | "AGENT" | "STANDARD" | undefined;
|
||||
let resolvedTriggerSource: "SCHEDULED" | "AGENT" | "WEBHOOK" | "STANDARD" | undefined;
|
||||
let resolvedTtl: string | null | undefined;
|
||||
|
||||
try {
|
||||
@@ -380,7 +416,9 @@ async function createWorkerTask(
|
||||
? ("SCHEDULED" as const)
|
||||
: task.triggerSource === "agent"
|
||||
? ("AGENT" as const)
|
||||
: ("STANDARD" as const);
|
||||
: task.triggerSource === "webhook"
|
||||
? ("WEBHOOK" as const)
|
||||
: ("STANDARD" as const);
|
||||
|
||||
resolvedTtl =
|
||||
typeof task.ttl === "number" ? (stringifyDuration(task.ttl) ?? null) : (task.ttl ?? null);
|
||||
@@ -640,6 +678,145 @@ export class CreateDeclarativeScheduleError extends Error {
|
||||
}
|
||||
}
|
||||
|
||||
// TODO: centralize (the P2 dynamic webhooks.create() API will also mint opaqueIds).
|
||||
function generateOpaqueId(): string {
|
||||
return randomBytes(16).toString("base64url");
|
||||
}
|
||||
|
||||
export async function syncDeclarativeWebhooks(
|
||||
webhooks: WebhookResource[] | undefined,
|
||||
worker: BackgroundWorker,
|
||||
environment: AuthenticatedEnvironment,
|
||||
prisma: PrismaClientOrTransaction,
|
||||
// Endpoint rows live on the webhook DB; the task-existence check below stays on the main client.
|
||||
webhookPrisma: WebhookDatabase
|
||||
) {
|
||||
if (webhooks === undefined) return;
|
||||
|
||||
const existing = await webhookPrisma.webhookEndpoint.findMany({
|
||||
where: {
|
||||
runtimeEnvironmentId: environment.id,
|
||||
endpointTenantId: "",
|
||||
endpointExternalRef: "",
|
||||
},
|
||||
});
|
||||
const missing = new Set(existing.map((e) => e.handlerWebhookId));
|
||||
|
||||
for (const wh of webhooks) {
|
||||
// Both routing targets resolve to a task: a fan-out webhook to its own task, a session webhook to
|
||||
// the claiming agent. Validate the target exists in this worker so a bad route fails at sync.
|
||||
const targetTaskSlug =
|
||||
wh.routingTarget.type === "task" ? wh.routingTarget.taskId : wh.routingTarget.taskIdentifier;
|
||||
const taskExists = await prisma.backgroundWorkerTask.findFirst({
|
||||
where: { workerId: worker.id, slug: targetTaskSlug },
|
||||
select: { id: true },
|
||||
});
|
||||
if (!taskExists) {
|
||||
throw new ServiceValidationError(
|
||||
`Webhook "${wh.id}" routes to unknown task "${targetTaskSlug}"`
|
||||
);
|
||||
}
|
||||
|
||||
missing.delete(wh.id);
|
||||
|
||||
if (
|
||||
"config" in wh.verifierArtifact &&
|
||||
wh.verifierArtifact.config.scheme === "url-secret" &&
|
||||
wh.verifierArtifact.config.placement === "path"
|
||||
) {
|
||||
throw new ServiceValidationError(
|
||||
`Webhook "${wh.id}" uses url-secret verification with path placement, which cannot be verified on the hosted ingress URL. Use query placement or a header-based scheme.`
|
||||
);
|
||||
}
|
||||
|
||||
// Compile `filter` into a FilterAst, once here at sync. A bad filter fails the deploy with a clear
|
||||
// message rather than surfacing at ingest. Re-deploying without a filter nulls the columns.
|
||||
let filterNode: FilterAst | undefined;
|
||||
if (wh.filter) {
|
||||
try {
|
||||
filterNode = parseFilter(wh.filter);
|
||||
} catch (error) {
|
||||
if (error instanceof FilterParseError) {
|
||||
throw new ServiceValidationError(
|
||||
`Webhook "${wh.id}" has an invalid filter: ${error.message}`
|
||||
);
|
||||
}
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
const filterData = {
|
||||
filter: wh.filter ?? null,
|
||||
filterAst: filterNode ? (filterNode as unknown as Prisma.InputJsonValue) : Prisma.DbNull,
|
||||
filterAstVersion: filterNode ? FILTER_AST_VERSION : null,
|
||||
};
|
||||
|
||||
// Validate a session target's startOn like the route filter: a bad predicate fails the deploy, not ingest.
|
||||
if (wh.routingTarget.type === "session" && wh.routingTarget.startOn) {
|
||||
try {
|
||||
parseFilter(wh.routingTarget.startOn);
|
||||
} catch (error) {
|
||||
if (error instanceof FilterParseError) {
|
||||
throw new ServiceValidationError(
|
||||
`Webhook "${wh.id}" has an invalid startOn: ${error.message}`
|
||||
);
|
||||
}
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
const found = existing.find((e) => e.handlerWebhookId === wh.id);
|
||||
if (found) {
|
||||
await webhookPrisma.webhookEndpoint.update({
|
||||
where: { id: found.id },
|
||||
data: {
|
||||
source: wh.source,
|
||||
routingTarget: wh.routingTarget as unknown as Prisma.InputJsonValue,
|
||||
verifierArtifact: wh.verifierArtifact as unknown as Prisma.InputJsonValue,
|
||||
secretProvisioning: wh.secretProvisioning ?? "either",
|
||||
metadata: (wh.metadata ?? {}) as unknown as Prisma.InputJsonValue,
|
||||
...(found.manuallyDeactivatedAt === null ? { status: "ACTIVE" as const } : {}),
|
||||
...filterData,
|
||||
},
|
||||
});
|
||||
} else {
|
||||
const { id, friendlyId } = WebhookEndpointId.generate();
|
||||
await webhookPrisma.webhookEndpoint.create({
|
||||
data: {
|
||||
id,
|
||||
friendlyId,
|
||||
opaqueId: generateOpaqueId(), // CSPRNG, NOT a friendlyId
|
||||
organizationId: environment.organizationId,
|
||||
projectId: environment.projectId,
|
||||
runtimeEnvironmentId: environment.id,
|
||||
environmentType: environment.type,
|
||||
endpointTenantId: "",
|
||||
endpointExternalRef: "",
|
||||
source: wh.source,
|
||||
handlerWebhookId: wh.id,
|
||||
routingTarget: wh.routingTarget as unknown as Prisma.InputJsonValue,
|
||||
verifierArtifact: wh.verifierArtifact as unknown as Prisma.InputJsonValue,
|
||||
secretProvisioning: wh.secretProvisioning ?? "either",
|
||||
metadata: (wh.metadata ?? {}) as unknown as Prisma.InputJsonValue,
|
||||
status: "ACTIVE",
|
||||
...filterData,
|
||||
},
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
if (missing.size > 0) {
|
||||
await webhookPrisma.webhookEndpoint.updateMany({
|
||||
where: {
|
||||
runtimeEnvironmentId: environment.id,
|
||||
endpointTenantId: "",
|
||||
endpointExternalRef: "",
|
||||
handlerWebhookId: { in: boundedIn(Array.from(missing)) },
|
||||
},
|
||||
data: { status: "INACTIVE" },
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
export async function syncDeclarativeSchedules(
|
||||
tasks: TaskResource[],
|
||||
worker: BackgroundWorker,
|
||||
|
||||
@@ -13,11 +13,13 @@ import {
|
||||
createBackgroundFiles,
|
||||
createWorkerResources,
|
||||
syncDeclarativeSchedules,
|
||||
syncDeclarativeWebhooks,
|
||||
} from "./createBackgroundWorker.server";
|
||||
import { findOrCreateBackgroundWorker } from "./createDeploymentBackgroundWorkerV4/findOrCreateBackgroundWorker.server";
|
||||
import { TimeoutDeploymentService } from "./timeoutDeployment.server";
|
||||
import { recordDeploymentOutcome } from "./recordDeploymentOutcome.server";
|
||||
import { env } from "~/env.server";
|
||||
import { webhookPrisma } from "~/db.server";
|
||||
|
||||
export class CreateDeploymentBackgroundWorkerServiceV4 extends BaseService {
|
||||
private readonly _taskMetaCache: TaskMetadataCache;
|
||||
@@ -228,6 +230,29 @@ export class CreateDeploymentBackgroundWorkerServiceV4 extends BaseService {
|
||||
throw serviceError;
|
||||
}
|
||||
|
||||
const [webhooksError] = await tryCatch(
|
||||
syncDeclarativeWebhooks(
|
||||
body.metadata.webhooks,
|
||||
backgroundWorker,
|
||||
environment,
|
||||
this._prisma,
|
||||
webhookPrisma
|
||||
)
|
||||
);
|
||||
|
||||
if (webhooksError) {
|
||||
logger.error("Error syncing declarative webhooks", { error: webhooksError });
|
||||
|
||||
const serviceError =
|
||||
webhooksError instanceof ServiceValidationError
|
||||
? webhooksError
|
||||
: new ServiceValidationError("Error syncing declarative webhooks");
|
||||
|
||||
await this.#failBackgroundWorkerDeployment(deployment, serviceError, environment);
|
||||
|
||||
throw serviceError;
|
||||
}
|
||||
|
||||
// Guarded BUILDING → DEPLOYING transition. `updateMany` for optimistic concurrency control
|
||||
const { count: updatedCount } = await this._prisma.workerDeployment.updateMany({
|
||||
where: {
|
||||
|
||||
@@ -0,0 +1,285 @@
|
||||
import { WebhookEngine } from "@internal/webhook-engine";
|
||||
import type { WebhookDeliverTaskErrorType } from "@internal/webhook-engine";
|
||||
import { tryCatch } from "@trigger.dev/core/utils";
|
||||
import { z } from "zod";
|
||||
import { prisma, webhookPrisma } from "~/db.server";
|
||||
import { env } from "~/env.server";
|
||||
import { findEnvironmentById } from "~/models/runtimeEnvironment.server";
|
||||
import { logger } from "~/services/logger.server";
|
||||
import { S2RealtimeStreams } from "~/services/realtime/s2realtimeStreams.server";
|
||||
import {
|
||||
ensureRunForSession,
|
||||
type SessionTriggerConfig,
|
||||
} from "~/services/realtime/sessionRunManager.server";
|
||||
import { findOrCreateSession, findSessionByExternalId } from "~/services/realtime/sessions.server";
|
||||
import { getRealtimeStreamInstance } from "~/services/realtime/v1StreamsGlobal.server";
|
||||
import {
|
||||
claimSessionStreamPart,
|
||||
drainSessionStreamWaitpoints,
|
||||
releaseSessionStreamPart,
|
||||
} from "~/services/sessionStreamWaitpointCache.server";
|
||||
import { getSecretStore } from "~/services/secrets/secretStore.server";
|
||||
import { singleton } from "~/utils/singleton";
|
||||
import { engine as runEngine } from "./runEngine.server";
|
||||
import { ServiceValidationError } from "./services/common.server";
|
||||
import { TriggerTaskService } from "./services/triggerTask.server";
|
||||
import { meter, tracer } from "./tracer.server";
|
||||
|
||||
export const webhookEngine = singleton("WebhookEngine", createWebhookEngine);
|
||||
|
||||
export type { WebhookEngine };
|
||||
|
||||
// The plaintext signing secret is stored under the "DATABASE" SecretStore
|
||||
// provider as { secret: string } (same shape as environment variables).
|
||||
const SigningSecretSchema = z.object({ secret: z.string() });
|
||||
|
||||
function createWebhookEngine() {
|
||||
// The engine owns the webhook tables, so it runs on the webhook DB client. The signing-secret
|
||||
// store stays on the main client below (SecretStore is control-plane, not part of the split).
|
||||
const secretStore = getSecretStore("DATABASE", { prismaClient: prisma });
|
||||
|
||||
const engine = new WebhookEngine({
|
||||
prisma: webhookPrisma,
|
||||
logLevel: env.WEBHOOK_ENGINE_LOG_LEVEL,
|
||||
disabled: env.WEBHOOK_ENABLED !== "1",
|
||||
redis: {
|
||||
host: env.WEBHOOK_WORKER_REDIS_HOST ?? "localhost",
|
||||
port: env.WEBHOOK_WORKER_REDIS_PORT ?? 6379,
|
||||
username: env.WEBHOOK_WORKER_REDIS_USERNAME,
|
||||
password: env.WEBHOOK_WORKER_REDIS_PASSWORD,
|
||||
keyPrefix: "webhook:",
|
||||
enableAutoPipelining: true,
|
||||
...(env.WEBHOOK_WORKER_REDIS_TLS_DISABLED === "true" ? {} : { tls: {} }),
|
||||
},
|
||||
worker: {
|
||||
concurrency: env.WEBHOOK_WORKER_CONCURRENCY_LIMIT,
|
||||
workers: env.WEBHOOK_WORKER_CONCURRENCY_WORKERS,
|
||||
tasksPerWorker: env.WEBHOOK_WORKER_CONCURRENCY_TASKS_PER_WORKER,
|
||||
pollIntervalMs: env.WEBHOOK_WORKER_POLL_INTERVAL,
|
||||
shutdownTimeoutMs: env.WEBHOOK_WORKER_SHUTDOWN_TIMEOUT_MS,
|
||||
disabled: env.WEBHOOK_ENABLED !== "1" || env.WEBHOOK_WORKER_ENABLED !== "true",
|
||||
},
|
||||
partitions: {
|
||||
ensureSchedule: env.WEBHOOK_PARTITION_ENSURE_SCHEDULE,
|
||||
ensureJitterInMs: env.WEBHOOK_PARTITION_ENSURE_JITTER_MS,
|
||||
lookaheadDays: env.WEBHOOK_PARTITION_LOOKAHEAD_DAYS,
|
||||
retentionDays: env.WEBHOOK_PARTITION_RETENTION_DAYS,
|
||||
},
|
||||
frontGate: {
|
||||
defaultTtlSeconds: env.WEBHOOK_FRONT_GATE_DEFAULT_TTL_SECONDS,
|
||||
maxTtlSeconds: env.WEBHOOK_FRONT_GATE_MAX_TTL_SECONDS,
|
||||
},
|
||||
endpointCache: {
|
||||
ttlMs: env.WEBHOOK_ENDPOINT_CACHE_TTL_MS,
|
||||
maxSize: env.WEBHOOK_ENDPOINT_CACHE_MAX_SIZE,
|
||||
},
|
||||
tracer,
|
||||
meter,
|
||||
resolveSigningSecret: async (key) => {
|
||||
const value = await secretStore.getSecret(SigningSecretSchema, key);
|
||||
// Fail closed: an unset/empty secret returns undefined so ingest rejects.
|
||||
return value?.secret || undefined;
|
||||
},
|
||||
triggerTask: async ({
|
||||
environmentId,
|
||||
taskId,
|
||||
idempotencyKey,
|
||||
idempotencyKeyExpiresAt,
|
||||
payload,
|
||||
headers,
|
||||
identityTags,
|
||||
endpointMetadata,
|
||||
}) => {
|
||||
try {
|
||||
const environment = await findEnvironmentById(environmentId);
|
||||
if (!environment) {
|
||||
return { success: false, errorType: "NOT_FOUND", error: "Environment not found" };
|
||||
}
|
||||
|
||||
const triggerService = new TriggerTaskService();
|
||||
|
||||
const result = await triggerService.call(
|
||||
taskId,
|
||||
environment,
|
||||
{
|
||||
// The webhook task run receives a { event, headers } envelope; the SDK's webhook()
|
||||
// run unwraps it into onEvent({ event, headers }).
|
||||
payload: { event: payload, headers },
|
||||
options: {
|
||||
tags: identityTags,
|
||||
metadata: (endpointMetadata as Record<string, unknown>) ?? undefined,
|
||||
},
|
||||
},
|
||||
{
|
||||
idempotencyKey,
|
||||
idempotencyKeyExpiresAt,
|
||||
triggerSource: "webhook",
|
||||
triggerAction: "trigger",
|
||||
customIcon: "webhook",
|
||||
}
|
||||
);
|
||||
|
||||
return { success: !!result, runId: result?.run.id };
|
||||
} catch (error) {
|
||||
const errorMessage = error instanceof Error ? error.message : String(error);
|
||||
let errorType: WebhookDeliverTaskErrorType = "SYSTEM_ERROR";
|
||||
|
||||
if (
|
||||
error instanceof ServiceValidationError &&
|
||||
errorMessage.includes("queue size limit for this environment has been reached")
|
||||
) {
|
||||
errorType = "QUEUE_LIMIT";
|
||||
}
|
||||
|
||||
return { success: false, error: errorMessage, errorType };
|
||||
}
|
||||
},
|
||||
// Route a verified delivery to a session: find-or-create it, then append a webhook action to `.in`.
|
||||
// The run boots on a preload payload (so onChatStart fires), then reads the action from `.in`.
|
||||
deliverToSession: async ({
|
||||
environmentId,
|
||||
taskIdentifier,
|
||||
externalId,
|
||||
deliverAs,
|
||||
actionType,
|
||||
connectorId,
|
||||
event,
|
||||
source,
|
||||
headers,
|
||||
deliveryId,
|
||||
triggerConfigTemplate,
|
||||
isSessionStart,
|
||||
}) => {
|
||||
try {
|
||||
const environment = await findEnvironmentById(environmentId);
|
||||
if (!environment) {
|
||||
return { success: false, errorType: "NOT_FOUND", error: "Environment not found" };
|
||||
}
|
||||
|
||||
const template = (triggerConfigTemplate ?? {}) as Partial<SessionTriggerConfig>;
|
||||
const triggerConfig: SessionTriggerConfig = {
|
||||
...template,
|
||||
basePayload: {
|
||||
messages: [],
|
||||
trigger: "preload",
|
||||
chatId: externalId,
|
||||
...(template.basePayload ?? {}),
|
||||
},
|
||||
};
|
||||
|
||||
// Resume an existing session; otherwise only START one when the event is a session-start
|
||||
// (startOn). Resume-only with no session yet -> ignore (no session, no run, no egress).
|
||||
const existing = await findSessionByExternalId(environment, externalId);
|
||||
if (!existing && !isSessionStart) {
|
||||
return {
|
||||
success: true,
|
||||
skipped: true,
|
||||
skippedReason: "startOn: not a session-start event",
|
||||
};
|
||||
}
|
||||
const { session, isCached } = existing
|
||||
? { session: existing, isCached: true }
|
||||
: await findOrCreateSession({
|
||||
environment,
|
||||
externalId,
|
||||
type: "chat.agent",
|
||||
taskIdentifier,
|
||||
triggerConfig,
|
||||
});
|
||||
|
||||
if (session.closedAt || (session.expiresAt && session.expiresAt.getTime() < Date.now())) {
|
||||
return { success: false, error: "Session is closed or expired" };
|
||||
}
|
||||
|
||||
// Boot / revive the run, then append the action. The run reads it from `.in`.
|
||||
const ensureResult = await ensureRunForSession({
|
||||
session,
|
||||
environment,
|
||||
reason: isCached ? "continuation" : "initial",
|
||||
});
|
||||
|
||||
const realtimeStream = getRealtimeStreamInstance(environment, "v2", { session });
|
||||
if (!(realtimeStream instanceof S2RealtimeStreams)) {
|
||||
return { success: false, error: "Session channels require the S2 realtime backend" };
|
||||
}
|
||||
|
||||
const addressingKey = session.externalId ?? session.friendlyId;
|
||||
// "action" (chat.event) -> onAction envelope; "message" (channels) -> a turn whose message the
|
||||
// run derives by applying the connector's inbound() to the raw event.
|
||||
const payload =
|
||||
deliverAs === "message"
|
||||
? {
|
||||
chatId: externalId,
|
||||
trigger: "submit-message",
|
||||
channelEvent: { connectorId, event, source, headers, deliveryId },
|
||||
}
|
||||
: {
|
||||
chatId: externalId,
|
||||
trigger: "action",
|
||||
actionSource: "webhook",
|
||||
action: { type: actionType, event, source, headers, deliveryId },
|
||||
};
|
||||
const part = JSON.stringify({ kind: "message", payload });
|
||||
|
||||
// deliveryId as the part id → a deliver-job retry re-claims the same id and skips a duplicate
|
||||
// append. The S2 record is durable, so a run that boots later still reads it.
|
||||
const wonClaim = await claimSessionStreamPart(
|
||||
environment.id,
|
||||
addressingKey,
|
||||
"in",
|
||||
deliveryId
|
||||
);
|
||||
if (wonClaim) {
|
||||
const [appendError] = await tryCatch(
|
||||
realtimeStream.appendPartToSessionStream(part, deliveryId, addressingKey, "in")
|
||||
);
|
||||
if (appendError) {
|
||||
// Nothing landed — release the claim so a retry re-appends the same id.
|
||||
await releaseSessionStreamPart(environment.id, addressingKey, "in", deliveryId);
|
||||
// A ServiceValidationError (e.g. record too large) is terminal; anything else is transient.
|
||||
if (appendError instanceof ServiceValidationError) {
|
||||
return { success: false, error: appendError.message };
|
||||
}
|
||||
throw appendError;
|
||||
}
|
||||
}
|
||||
|
||||
// Wake any `.in` waitpoints the run registered (best-effort; the record is durable in S2).
|
||||
const [drainError, waitpointIds] = await tryCatch(
|
||||
drainSessionStreamWaitpoints(environment.id, addressingKey, "in")
|
||||
);
|
||||
if (drainError) {
|
||||
logger.error("deliverToSession: failed to drain session waitpoints", {
|
||||
externalId,
|
||||
error: drainError,
|
||||
});
|
||||
} else if (waitpointIds && waitpointIds.length > 0) {
|
||||
await Promise.all(
|
||||
waitpointIds.map((waitpointId) =>
|
||||
tryCatch(
|
||||
runEngine.completeWaitpoint({
|
||||
id: waitpointId,
|
||||
output: { value: part, type: "application/json", isError: false },
|
||||
})
|
||||
)
|
||||
)
|
||||
);
|
||||
}
|
||||
|
||||
return { success: true, runId: ensureResult.runId };
|
||||
} catch (error) {
|
||||
const errorMessage = error instanceof Error ? error.message : String(error);
|
||||
let errorType: WebhookDeliverTaskErrorType = "SYSTEM_ERROR";
|
||||
if (
|
||||
error instanceof ServiceValidationError &&
|
||||
errorMessage.includes("queue size limit for this environment has been reached")
|
||||
) {
|
||||
errorType = "QUEUE_LIMIT";
|
||||
}
|
||||
return { success: false, error: errorMessage, errorType };
|
||||
}
|
||||
},
|
||||
});
|
||||
|
||||
return engine;
|
||||
}
|
||||
@@ -21,6 +21,7 @@
|
||||
"db:seed": "tsx seed.ts",
|
||||
"db:seed:ai-spans": "tsx seed-ai-spans.mts",
|
||||
"db:seed:queue-metrics": "tsx seed-queue-metrics.mts",
|
||||
"db:seed:webhooks": "tsx seed-webhook-deliveries.ts",
|
||||
"upload:sourcemaps": "bash ./upload-sourcemaps.sh",
|
||||
"test": "vitest --no-file-parallelism",
|
||||
"eval:dev": "evalite watch"
|
||||
@@ -66,6 +67,8 @@
|
||||
"@internal/schedule-engine": "workspace:*",
|
||||
"@internal/tracing": "workspace:*",
|
||||
"@internal/tsql": "workspace:*",
|
||||
"@internal/webhook-engine": "workspace:*",
|
||||
"@internal/webhook-sources": "workspace:*",
|
||||
"@internationalized/date": "^3.5.1",
|
||||
"@jsonhero/schema-infer": "^0.1.5",
|
||||
"@kapaai/react-sdk": "^0.1.3",
|
||||
|
||||
@@ -0,0 +1,599 @@
|
||||
import "dotenv/config";
|
||||
import { randomBytes, createHash } from "node:crypto";
|
||||
import { nanoid } from "nanoid";
|
||||
import { prisma, webhookPrisma } from "./app/db.server";
|
||||
|
||||
/**
|
||||
* Seed a rich, stable spread of webhook endpoints and deliveries for design work.
|
||||
*
|
||||
* Attaches several endpoints (varied providers, verifier schemes, and status) and a
|
||||
* spread of deliveries across the last two weeks (every delivery status, realistic
|
||||
* payloads + headers, a mix of test and live) to an existing DEVELOPMENT environment.
|
||||
* Writes the delivery rows to both Postgres (the source of every displayed field) and
|
||||
* ClickHouse (which orders and paginates the Deliveries list).
|
||||
*
|
||||
* By default it targets the first DEVELOPMENT environment the local user can see; set
|
||||
* WEBHOOK_SEED_PROJECT to a project name to pick a specific one.
|
||||
*
|
||||
* Re-runnable: it clears that environment's webhook deliveries first, so each run yields
|
||||
* the same clean dataset. Stale ClickHouse rows self-hide because the list drops any
|
||||
* ordered id whose Postgres row is gone.
|
||||
*
|
||||
* pnpm --filter webapp run db:seed:webhooks
|
||||
* pnpm --filter webapp run db:seed:webhooks -- 60 # deliveries per endpoint
|
||||
*/
|
||||
|
||||
const PER_ENDPOINT = Number.parseInt(process.argv[2] ?? "", 10) || 45;
|
||||
const SPREAD_DAYS = 14;
|
||||
const DAY_MS = 86_400_000;
|
||||
|
||||
type Scheme = "hmac" | "shared-secret" | "url-secret" | "asymmetric";
|
||||
|
||||
type EndpointSpec = {
|
||||
source: string;
|
||||
handlerWebhookId: string;
|
||||
scheme: Scheme;
|
||||
verifierArtifact: unknown;
|
||||
secretProvisioning: "provider" | "integrator" | "either";
|
||||
hasSecret: boolean;
|
||||
status: "ACTIVE" | "INACTIVE";
|
||||
tenantId: string;
|
||||
externalRef: string;
|
||||
events: Array<{ type: string; body: () => Record<string, unknown> }>;
|
||||
signatureHeader: string;
|
||||
userAgent: string;
|
||||
};
|
||||
|
||||
function hmacArtifact(preset: string, header: string, extra: Record<string, unknown> = {}) {
|
||||
return {
|
||||
kind: "preset" as const,
|
||||
preset,
|
||||
config: {
|
||||
scheme: "hmac" as const,
|
||||
algorithm: "sha256" as const,
|
||||
encoding: "hex" as const,
|
||||
signatureHeader: header,
|
||||
signingString: "raw" as const,
|
||||
...extra,
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
const ENDPOINTS: EndpointSpec[] = [
|
||||
{
|
||||
source: "stripe",
|
||||
handlerWebhookId: "stripe-webhook",
|
||||
scheme: "hmac",
|
||||
signatureHeader: "Stripe-Signature",
|
||||
userAgent: "Stripe/1.0 (+https://stripe.com/docs/webhooks)",
|
||||
verifierArtifact: hmacArtifact("stripe", "Stripe-Signature", {
|
||||
signature: { itemSeparator: ",", fieldSeparator: "=", field: "v1", trim: true },
|
||||
timestamp: {
|
||||
source: { from: "signatureField", field: "t" },
|
||||
unit: "seconds",
|
||||
toleranceSeconds: 300,
|
||||
},
|
||||
signingString: {
|
||||
template: "{t}.{body}",
|
||||
vars: { t: { from: "signatureField", field: "t" } },
|
||||
},
|
||||
}),
|
||||
secretProvisioning: "provider",
|
||||
hasSecret: true,
|
||||
status: "ACTIVE",
|
||||
tenantId: "",
|
||||
externalRef: "",
|
||||
events: [
|
||||
{
|
||||
type: "payment_intent.succeeded",
|
||||
body: () =>
|
||||
stripeEvent("payment_intent.succeeded", {
|
||||
amount: 4200,
|
||||
currency: "usd",
|
||||
status: "succeeded",
|
||||
}),
|
||||
},
|
||||
{
|
||||
type: "charge.refunded",
|
||||
body: () =>
|
||||
stripeEvent("charge.refunded", { amount: 4200, currency: "usd", refunded: true }),
|
||||
},
|
||||
{
|
||||
type: "customer.subscription.updated",
|
||||
body: () =>
|
||||
stripeEvent("customer.subscription.updated", { status: "active", plan: "pro_monthly" }),
|
||||
},
|
||||
{
|
||||
type: "invoice.paid",
|
||||
body: () => stripeEvent("invoice.paid", { amount_paid: 9900, currency: "usd" }),
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
source: "github",
|
||||
handlerWebhookId: "github-webhook",
|
||||
scheme: "hmac",
|
||||
signatureHeader: "X-Hub-Signature-256",
|
||||
userAgent: "GitHub-Hookshot/a1b2c3",
|
||||
verifierArtifact: hmacArtifact("github", "X-Hub-Signature-256"),
|
||||
secretProvisioning: "integrator",
|
||||
hasSecret: true,
|
||||
status: "ACTIVE",
|
||||
tenantId: "",
|
||||
externalRef: "",
|
||||
events: [
|
||||
{
|
||||
type: "push",
|
||||
body: () => ({
|
||||
ref: "refs/heads/main",
|
||||
commits: [{ id: nanoid(), message: "fix: tidy up" }],
|
||||
pusher: { name: "octocat" },
|
||||
}),
|
||||
},
|
||||
{
|
||||
type: "pull_request",
|
||||
body: () => ({
|
||||
action: "opened",
|
||||
number: 42,
|
||||
pull_request: { title: "Add webhooks", user: { login: "octocat" } },
|
||||
}),
|
||||
},
|
||||
{
|
||||
type: "issues",
|
||||
body: () => ({
|
||||
action: "opened",
|
||||
issue: { number: 7, title: "Docs typo", user: { login: "hubot" } },
|
||||
}),
|
||||
},
|
||||
{
|
||||
type: "star",
|
||||
body: () => ({
|
||||
action: "created",
|
||||
starred_at: new Date().toISOString(),
|
||||
sender: { login: "fan" },
|
||||
}),
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
source: "slack",
|
||||
handlerWebhookId: "slack-channel",
|
||||
scheme: "hmac",
|
||||
signatureHeader: "X-Slack-Signature",
|
||||
userAgent: "Slackbot 1.0 (+https://api.slack.com/robots)",
|
||||
verifierArtifact: hmacArtifact("slack", "X-Slack-Signature", {
|
||||
timestamp: {
|
||||
source: { from: "header", name: "X-Slack-Request-Timestamp" },
|
||||
unit: "seconds",
|
||||
toleranceSeconds: 300,
|
||||
},
|
||||
signingString: {
|
||||
template: "v0:{t}:{body}",
|
||||
vars: { t: { from: "header", name: "X-Slack-Request-Timestamp" } },
|
||||
},
|
||||
}),
|
||||
secretProvisioning: "provider",
|
||||
hasSecret: true,
|
||||
status: "ACTIVE",
|
||||
tenantId: "T0288ANLG",
|
||||
externalRef: "app_mention",
|
||||
events: [
|
||||
{
|
||||
type: "app_mention",
|
||||
body: () => ({
|
||||
type: "event_callback",
|
||||
event: {
|
||||
type: "app_mention",
|
||||
text: "<@U123> ship it",
|
||||
user: "U0G9QF9C6",
|
||||
channel: "C0288ANLG",
|
||||
},
|
||||
}),
|
||||
},
|
||||
{
|
||||
type: "message",
|
||||
body: () => ({
|
||||
type: "event_callback",
|
||||
event: { type: "message", text: "any update?", user: "U0G9QF9C6", channel: "C0288ANLG" },
|
||||
}),
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
source: "svix",
|
||||
handlerWebhookId: "svix-webhook",
|
||||
scheme: "hmac",
|
||||
signatureHeader: "webhook-signature",
|
||||
userAgent: "Svix-Webhooks/1.4",
|
||||
verifierArtifact: hmacArtifact("svix", "webhook-signature", {
|
||||
encoding: "base64",
|
||||
signature: { itemSeparator: " ", fieldSeparator: ",", field: "v1" },
|
||||
secret: { encoding: "base64", stripPrefix: "whsec_" },
|
||||
}),
|
||||
secretProvisioning: "provider",
|
||||
hasSecret: false,
|
||||
status: "ACTIVE",
|
||||
tenantId: "",
|
||||
externalRef: "",
|
||||
events: [
|
||||
{
|
||||
type: "invoice.created",
|
||||
body: () => ({ type: "invoice.created", data: { id: `in_${nanoid()}`, total: 1200 } }),
|
||||
},
|
||||
{
|
||||
type: "message.sent",
|
||||
body: () => ({
|
||||
type: "message.sent",
|
||||
data: { id: `msg_${nanoid()}`, to: "user@example.com" },
|
||||
}),
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
source: "discord",
|
||||
handlerWebhookId: "discord-interactions",
|
||||
scheme: "asymmetric",
|
||||
signatureHeader: "X-Signature-Ed25519",
|
||||
userAgent: "Discord-Interactions/1.0 (+https://discord.com)",
|
||||
verifierArtifact: {
|
||||
kind: "preset" as const,
|
||||
preset: "discord",
|
||||
config: {
|
||||
scheme: "asymmetric" as const,
|
||||
algorithm: "ed25519" as const,
|
||||
encoding: "hex" as const,
|
||||
signatureHeader: "X-Signature-Ed25519",
|
||||
timestamp: {
|
||||
source: { from: "header" as const, name: "X-Signature-Timestamp" },
|
||||
unit: "seconds" as const,
|
||||
},
|
||||
signingString: {
|
||||
template: "{t}{body}",
|
||||
vars: { t: { from: "header" as const, name: "X-Signature-Timestamp" } },
|
||||
},
|
||||
publicKeyEncoding: "raw-hex" as const,
|
||||
},
|
||||
},
|
||||
secretProvisioning: "provider",
|
||||
hasSecret: true,
|
||||
status: "ACTIVE",
|
||||
tenantId: "",
|
||||
externalRef: "",
|
||||
events: [
|
||||
{
|
||||
type: "INTERACTION",
|
||||
body: () => ({ type: 2, data: { name: "deploy" }, member: { user: { username: "dev" } } }),
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
source: "custom",
|
||||
handlerWebhookId: "orders-webhook",
|
||||
scheme: "shared-secret",
|
||||
signatureHeader: "X-Webhook-Token",
|
||||
userAgent: "acme-orders/2.3",
|
||||
verifierArtifact: {
|
||||
kind: "config" as const,
|
||||
config: {
|
||||
scheme: "shared-secret" as const,
|
||||
placement: "header" as const,
|
||||
fieldName: "X-Webhook-Token",
|
||||
},
|
||||
},
|
||||
secretProvisioning: "either",
|
||||
hasSecret: true,
|
||||
status: "INACTIVE",
|
||||
tenantId: "acct_9f2",
|
||||
externalRef: "orders",
|
||||
events: [
|
||||
{
|
||||
type: "order.created",
|
||||
body: () => ({
|
||||
event: "order.created",
|
||||
orderId: `ord_${nanoid()}`,
|
||||
total: 129.99,
|
||||
currency: "USD",
|
||||
}),
|
||||
},
|
||||
{
|
||||
type: "order.fulfilled",
|
||||
body: () => ({ event: "order.fulfilled", orderId: `ord_${nanoid()}`, carrier: "ups" }),
|
||||
},
|
||||
],
|
||||
},
|
||||
];
|
||||
|
||||
function stripeEvent(type: string, data: Record<string, unknown>): Record<string, unknown> {
|
||||
return {
|
||||
id: `evt_${nanoid()}`,
|
||||
object: "event",
|
||||
type,
|
||||
created: Math.floor(Date.now() / 1000),
|
||||
data: { object: { id: `obj_${nanoid()}`, ...data } },
|
||||
};
|
||||
}
|
||||
|
||||
const STATUS_WEIGHTS: Array<[string, number]> = [
|
||||
["SUCCEEDED", 68],
|
||||
["FAILED", 12],
|
||||
["FILTERED", 11],
|
||||
["PROCESSING", 4],
|
||||
["PENDING", 5],
|
||||
];
|
||||
|
||||
const FAIL_REASONS = [
|
||||
"Signature verification failed",
|
||||
"Signing secret not set for endpoint",
|
||||
"Timestamp outside tolerance window",
|
||||
];
|
||||
|
||||
function pickStatus(): string {
|
||||
const total = STATUS_WEIGHTS.reduce((s, [, w]) => s + w, 0);
|
||||
let r = Math.random() * total;
|
||||
for (const [status, w] of STATUS_WEIGHTS) {
|
||||
if ((r -= w) <= 0) return status;
|
||||
}
|
||||
return "SUCCEEDED";
|
||||
}
|
||||
|
||||
function biasedCreatedAt(now: number): Date {
|
||||
const r = Math.random() ** 1.7;
|
||||
return new Date(now - r * SPREAD_DAYS * DAY_MS);
|
||||
}
|
||||
|
||||
function dayPartitionName(d: Date): { name: string; lo: string; hi: string } {
|
||||
const floor = new Date(Date.UTC(d.getUTCFullYear(), d.getUTCMonth(), d.getUTCDate()));
|
||||
const hi = new Date(floor.getTime() + DAY_MS);
|
||||
const y = floor.getUTCFullYear();
|
||||
const m = String(floor.getUTCMonth() + 1).padStart(2, "0");
|
||||
const day = String(floor.getUTCDate()).padStart(2, "0");
|
||||
return {
|
||||
name: `WebhookDelivery_${y}_${m}_${day}`,
|
||||
lo: floor.toISOString(),
|
||||
hi: hi.toISOString(),
|
||||
};
|
||||
}
|
||||
|
||||
async function main() {
|
||||
const user = await prisma.user.findUnique({ where: { email: "local@trigger.dev" } });
|
||||
if (!user) {
|
||||
console.error("User local@trigger.dev not found. Run `pnpm run db:seed` first.");
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
const projectName = process.env.WEBHOOK_SEED_PROJECT;
|
||||
const runtimeEnv = await prisma.runtimeEnvironment.findFirst({
|
||||
where: {
|
||||
type: "DEVELOPMENT",
|
||||
organization: { members: { some: { userId: user.id } } },
|
||||
...(projectName ? { project: { name: projectName } } : {}),
|
||||
},
|
||||
include: { project: true, organization: true },
|
||||
orderBy: { createdAt: "asc" },
|
||||
});
|
||||
if (!runtimeEnv) {
|
||||
console.error(
|
||||
projectName
|
||||
? `No DEVELOPMENT environment found for project "${projectName}". Run \`pnpm run db:seed\` first.`
|
||||
: "No DEVELOPMENT environment found. Run `pnpm run db:seed` first."
|
||||
);
|
||||
process.exit(1);
|
||||
}
|
||||
const org = runtimeEnv.organization;
|
||||
const project = runtimeEnv.project;
|
||||
console.log(`Seeding into ${org.title} / ${project.name} (env ${runtimeEnv.slug})`);
|
||||
|
||||
const scope = {
|
||||
organizationId: org.id,
|
||||
projectId: project.id,
|
||||
runtimeEnvironmentId: runtimeEnv.id,
|
||||
environmentType: runtimeEnv.type,
|
||||
};
|
||||
|
||||
console.log("Clearing previous demo deliveries...");
|
||||
await webhookPrisma.webhookDelivery.deleteMany({
|
||||
where: { runtimeEnvironmentId: runtimeEnv.id },
|
||||
});
|
||||
|
||||
const endpoints: Array<{ spec: EndpointSpec; id: string; friendlyId: string }> = [];
|
||||
for (const spec of ENDPOINTS) {
|
||||
const friendlyId = `wh_${nanoid()}`;
|
||||
const created = await webhookPrisma.webhookEndpoint.upsert({
|
||||
where: {
|
||||
runtimeEnvironmentId_handlerWebhookId_endpointTenantId_endpointExternalRef: {
|
||||
runtimeEnvironmentId: runtimeEnv.id,
|
||||
handlerWebhookId: spec.handlerWebhookId,
|
||||
endpointTenantId: spec.tenantId,
|
||||
endpointExternalRef: spec.externalRef,
|
||||
},
|
||||
},
|
||||
update: {
|
||||
status: spec.status,
|
||||
verifierArtifact: spec.verifierArtifact as object,
|
||||
secretProvisioning: spec.secretProvisioning,
|
||||
},
|
||||
create: {
|
||||
friendlyId,
|
||||
opaqueId: randomBytes(16).toString("base64url"),
|
||||
...scope,
|
||||
endpointTenantId: spec.tenantId,
|
||||
endpointExternalRef: spec.externalRef,
|
||||
source: spec.source,
|
||||
handlerWebhookId: spec.handlerWebhookId,
|
||||
routingTarget: { type: "task", taskId: spec.handlerWebhookId } as object,
|
||||
verifierArtifact: spec.verifierArtifact as object,
|
||||
secretProvisioning: spec.secretProvisioning,
|
||||
signingSecretKey: spec.hasSecret ? `webhook:signing-secret:seed-${spec.source}` : null,
|
||||
status: spec.status,
|
||||
metadata: { seeded: true, provider: spec.source } as object,
|
||||
},
|
||||
});
|
||||
endpoints.push({ spec, id: created.id, friendlyId: created.friendlyId });
|
||||
}
|
||||
console.log(`Endpoints ready: ${endpoints.length}`);
|
||||
|
||||
const now = Date.now();
|
||||
type Row = {
|
||||
id: string;
|
||||
friendlyId: string;
|
||||
endpointId: string;
|
||||
source: string;
|
||||
status: string;
|
||||
isTest: boolean;
|
||||
externalDeliveryId: string;
|
||||
parsedEvent: Record<string, unknown>;
|
||||
headers: Record<string, string>;
|
||||
errorMessage: string | null;
|
||||
filterReason: string | null;
|
||||
createdAt: Date;
|
||||
processedAt: Date | null;
|
||||
};
|
||||
|
||||
const rows: Row[] = [];
|
||||
for (const ep of endpoints) {
|
||||
for (let i = 0; i < PER_ENDPOINT; i++) {
|
||||
const event = ep.spec.events[Math.floor(Math.random() * ep.spec.events.length)];
|
||||
const body = event.body();
|
||||
const status =
|
||||
ep.spec.status === "INACTIVE" && Math.random() < 0.5 ? "FILTERED" : pickStatus();
|
||||
const createdAt = biasedCreatedAt(now);
|
||||
const terminal = status !== "PENDING" && status !== "PROCESSING";
|
||||
const processedAt = terminal
|
||||
? new Date(createdAt.getTime() + 80 + Math.floor(Math.random() * 1100))
|
||||
: null;
|
||||
const externalDeliveryId = `${ep.spec.source}_${nanoid()}`;
|
||||
const headers: Record<string, string> = {
|
||||
"content-type": "application/json",
|
||||
"user-agent": ep.spec.userAgent,
|
||||
accept: "*/*",
|
||||
[ep.spec.signatureHeader.toLowerCase()]:
|
||||
status === "FAILED" ? "tampered" : `sig_${nanoid()}`,
|
||||
};
|
||||
// friendlyId must be the id plus the prefix, matching WebhookDeliveryId. The detail
|
||||
// lookup strips "whd_" and queries Postgres by `id`, so minting the two independently
|
||||
// makes every seeded delivery's detail page 404.
|
||||
const deliveryId = nanoid();
|
||||
rows.push({
|
||||
id: deliveryId,
|
||||
friendlyId: `whd_${deliveryId}`,
|
||||
endpointId: ep.id,
|
||||
source: ep.spec.source,
|
||||
status,
|
||||
isTest: Math.random() < 0.15,
|
||||
externalDeliveryId,
|
||||
parsedEvent: body,
|
||||
headers,
|
||||
errorMessage:
|
||||
status === "FAILED"
|
||||
? FAIL_REASONS[Math.floor(Math.random() * FAIL_REASONS.length)]
|
||||
: null,
|
||||
filterReason:
|
||||
status === "FILTERED"
|
||||
? `event.type "${String(body.type ?? (body as { event?: string }).event ?? "unknown")}" did not match the endpoint filter`
|
||||
: null,
|
||||
createdAt,
|
||||
processedAt,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
const days = new Map<string, { lo: string; hi: string }>();
|
||||
for (const r of rows) {
|
||||
const p = dayPartitionName(r.createdAt);
|
||||
days.set(p.name, { lo: p.lo, hi: p.hi });
|
||||
}
|
||||
for (const [name, { lo, hi }] of days) {
|
||||
await webhookPrisma.$executeRawUnsafe(
|
||||
`CREATE TABLE IF NOT EXISTS "${name}" PARTITION OF "WebhookDelivery" FOR VALUES FROM ('${lo}') TO ('${hi}')`
|
||||
);
|
||||
}
|
||||
console.log(`Ensured ${days.size} daily partitions.`);
|
||||
|
||||
for (let i = 0; i < rows.length; i += 500) {
|
||||
const chunk = rows.slice(i, i + 500);
|
||||
await webhookPrisma.webhookDelivery.createMany({
|
||||
data: chunk.map((r) => ({
|
||||
id: r.id,
|
||||
friendlyId: r.friendlyId,
|
||||
webhookEndpointId: r.endpointId,
|
||||
...scope,
|
||||
externalDeliveryId: r.externalDeliveryId,
|
||||
idempotencyKey: r.externalDeliveryId,
|
||||
runId: null,
|
||||
status: r.status as never,
|
||||
isTest: r.isTest,
|
||||
parsedEvent: r.parsedEvent as object,
|
||||
headers: r.headers as object,
|
||||
rawBodyHash: createHash("sha256").update(JSON.stringify(r.parsedEvent)).digest("hex"),
|
||||
errorMessage: r.errorMessage,
|
||||
filterReason: r.filterReason,
|
||||
createdAt: r.createdAt,
|
||||
updatedAt: r.processedAt ?? r.createdAt,
|
||||
processedAt: r.processedAt,
|
||||
})),
|
||||
});
|
||||
}
|
||||
console.log(`Inserted ${rows.length} deliveries into Postgres.`);
|
||||
|
||||
const clickhouseUrl =
|
||||
process.env.WEBHOOK_DELIVERIES_REPLICATION_CLICKHOUSE_URL ?? process.env.CLICKHOUSE_URL;
|
||||
if (!clickhouseUrl) {
|
||||
console.error(
|
||||
"No ClickHouse URL (set WEBHOOK_DELIVERIES_REPLICATION_CLICKHOUSE_URL or CLICKHOUSE_URL)."
|
||||
);
|
||||
process.exit(1);
|
||||
}
|
||||
const chUrl = new URL(clickhouseUrl);
|
||||
const auth = chUrl.username
|
||||
? "Basic " + Buffer.from(`${chUrl.username}:${chUrl.password}`).toString("base64")
|
||||
: undefined;
|
||||
const chEndpoint = `${chUrl.protocol}//${chUrl.host}/`;
|
||||
|
||||
const fmt = (d: Date) => d.toISOString().replace("T", " ").replace("Z", "");
|
||||
const ndjson = rows
|
||||
.map((r) => {
|
||||
const updated = r.processedAt ?? r.createdAt;
|
||||
return JSON.stringify({
|
||||
environment_id: scope.runtimeEnvironmentId,
|
||||
organization_id: scope.organizationId,
|
||||
project_id: scope.projectId,
|
||||
delivery_id: r.id,
|
||||
webhook_endpoint_id: r.endpointId,
|
||||
environment_type: scope.environmentType,
|
||||
friendly_id: r.friendlyId,
|
||||
external_delivery_id: r.externalDeliveryId,
|
||||
run_id: "",
|
||||
status: r.status,
|
||||
is_test: r.isTest ? 1 : 0,
|
||||
created_at: fmt(r.createdAt),
|
||||
updated_at: fmt(updated),
|
||||
_version: String(updated.getTime()),
|
||||
_is_deleted: 0,
|
||||
});
|
||||
})
|
||||
.join("\n");
|
||||
|
||||
const insertQuery = "INSERT INTO trigger_dev.webhook_deliveries_v1 FORMAT JSONEachRow";
|
||||
const chResponse = await fetch(`${chEndpoint}?query=${encodeURIComponent(insertQuery)}`, {
|
||||
method: "POST",
|
||||
headers: { "content-type": "application/x-ndjson", ...(auth ? { authorization: auth } : {}) },
|
||||
body: ndjson,
|
||||
});
|
||||
if (!chResponse.ok) {
|
||||
console.error(`ClickHouse insert failed (${chResponse.status}): ${await chResponse.text()}`);
|
||||
process.exit(1);
|
||||
}
|
||||
console.log(`Inserted ${rows.length} delivery rows into ClickHouse.`);
|
||||
|
||||
const port = process.env.REMIX_APP_PORT ?? process.env.PORT ?? "3030";
|
||||
console.log("\nDone.");
|
||||
console.log(
|
||||
`Deliveries: http://localhost:${port}/orgs/${org.slug}/projects/${project.slug}/env/${runtimeEnv.slug}/webhooks`
|
||||
);
|
||||
process.exit(0);
|
||||
}
|
||||
|
||||
main().catch((err) => {
|
||||
console.error(err);
|
||||
process.exit(1);
|
||||
});
|
||||
+14
-1
@@ -145,8 +145,18 @@ async function startServer() {
|
||||
// log dominates log volume. HTTP_ACCESS_LOG_DISABLED suppresses successful
|
||||
// (2xx) access logs; non-2xx responses are always logged so errors stay visible.
|
||||
const suppressSuccessfulAccessLogs = process.env.HTTP_ACCESS_LOG_DISABLED === "1";
|
||||
// Strip the query string from webhook ingress URLs (they may carry a
|
||||
// url-secret) before they reach the access log. Other paths pass through.
|
||||
morgan.token("url-redacted", (req: any) => {
|
||||
const url: string = req.originalUrl ?? req.url ?? "";
|
||||
if (url.startsWith("/webhooks/v1/ingest/")) {
|
||||
const q = url.indexOf("?");
|
||||
return q === -1 ? url : url.slice(0, q);
|
||||
}
|
||||
return url;
|
||||
});
|
||||
app.use(
|
||||
morgan("tiny", {
|
||||
morgan(":method :url-redacted :status :res[content-length] - :response-time ms", {
|
||||
skip: (_req, res) =>
|
||||
suppressSuccessfulAccessLogs && res.statusCode >= 200 && res.statusCode < 300,
|
||||
})
|
||||
@@ -189,6 +199,8 @@ async function startServer() {
|
||||
const runWithHttpContext: RunWithHttpContextFunction = build.entry.module.runWithHttpContext;
|
||||
const tenantContextMiddleware: RequestHandler = build.entry.module.tenantContextMiddleware;
|
||||
const dashboardAgentBodyCap: RequestHandler = build.entry.module.dashboardAgentBodyCap;
|
||||
const webhookIngressIpRateLimiter: RequestHandler =
|
||||
build.entry.module.webhookIngressIpRateLimiter;
|
||||
|
||||
app.use((req, res, next) => {
|
||||
// helpful headers:
|
||||
@@ -240,6 +252,7 @@ async function startServer() {
|
||||
app.use(deploymentRateLimiter);
|
||||
app.use(engineRateLimiter);
|
||||
app.use(otlpRateLimiter);
|
||||
app.use(webhookIngressIpRateLimiter);
|
||||
|
||||
app.use(tenantContextMiddleware);
|
||||
|
||||
|
||||
@@ -0,0 +1,106 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
import {
|
||||
buildDeliveryTimelineItems,
|
||||
type BuildDeliveryTimelineInput,
|
||||
} from "~/components/webhookDeliveries/v1/buildDeliveryTimelineItems";
|
||||
|
||||
const createdAt = new Date("2026-07-13T10:32:04.120Z");
|
||||
const processedAt = new Date("2026-07-13T10:32:05.320Z");
|
||||
|
||||
function baseDelivery(overrides: Partial<BuildDeliveryTimelineInput>): BuildDeliveryTimelineInput {
|
||||
return {
|
||||
status: "PENDING",
|
||||
createdAt,
|
||||
processedAt: null,
|
||||
errorMessage: null,
|
||||
filterReason: null,
|
||||
run: null,
|
||||
session: null,
|
||||
...overrides,
|
||||
};
|
||||
}
|
||||
|
||||
describe("buildDeliveryTimelineItems", () => {
|
||||
it("PENDING renders Received + an in-flight routing line, no terminal node", () => {
|
||||
const items = buildDeliveryTimelineItems(baseDelivery({ status: "PENDING" }));
|
||||
|
||||
expect(items.map((i) => i.id)).toEqual(["received", "routing"]);
|
||||
const line = items[1];
|
||||
expect(line.type).toBe("line");
|
||||
expect(line).toMatchObject({ from: createdAt, to: null, state: "inprogress" });
|
||||
});
|
||||
|
||||
it("PROCESSING is also in-flight (no terminal node)", () => {
|
||||
const items = buildDeliveryTimelineItems(baseDelivery({ status: "PROCESSING" }));
|
||||
|
||||
expect(items.map((i) => i.id)).toEqual(["received", "routing"]);
|
||||
expect(items[1]).toMatchObject({ to: null, state: "inprogress" });
|
||||
});
|
||||
|
||||
it("SUCCEEDED renders a Delivered terminal node with the run/session target", () => {
|
||||
const items = buildDeliveryTimelineItems(
|
||||
baseDelivery({
|
||||
status: "SUCCEEDED",
|
||||
processedAt,
|
||||
run: { friendlyId: "run_2f4b" },
|
||||
session: { friendlyId: "sess_xyz", externalId: "cust_1" },
|
||||
})
|
||||
);
|
||||
|
||||
expect(items.map((i) => i.id)).toEqual(["received", "routing", "delivered"]);
|
||||
expect(items[1]).toMatchObject({ from: createdAt, to: processedAt, state: "complete" });
|
||||
const terminal = items[2];
|
||||
expect(terminal).toMatchObject({
|
||||
type: "event",
|
||||
title: "Delivered",
|
||||
state: "complete",
|
||||
date: processedAt,
|
||||
target: {
|
||||
run: { friendlyId: "run_2f4b" },
|
||||
session: { friendlyId: "sess_xyz", externalId: "cust_1" },
|
||||
},
|
||||
});
|
||||
});
|
||||
|
||||
it("FAILED renders a Failed terminal node carrying the error message", () => {
|
||||
const items = buildDeliveryTimelineItems(
|
||||
baseDelivery({ status: "FAILED", processedAt, errorMessage: "queue limit exceeded" })
|
||||
);
|
||||
|
||||
expect(items.map((i) => i.id)).toEqual(["received", "routing", "failed"]);
|
||||
expect(items[1]).toMatchObject({ state: "error" });
|
||||
expect(items[2]).toMatchObject({
|
||||
title: "Failed",
|
||||
state: "error",
|
||||
note: "queue limit exceeded",
|
||||
});
|
||||
});
|
||||
|
||||
it("FILTERED renders a dimmed Filtered node with the reason and no run target", () => {
|
||||
const items = buildDeliveryTimelineItems(
|
||||
baseDelivery({
|
||||
status: "FILTERED",
|
||||
processedAt,
|
||||
filterReason: "action != opened",
|
||||
})
|
||||
);
|
||||
|
||||
expect(items.map((i) => i.id)).toEqual(["received", "routing", "filtered"]);
|
||||
expect(items[1]).toMatchObject({ state: "delayed", variant: "light" });
|
||||
const terminal = items[2];
|
||||
expect(terminal).toMatchObject({
|
||||
type: "event",
|
||||
title: "Filtered",
|
||||
state: "delayed",
|
||||
note: "action != opened",
|
||||
});
|
||||
expect((terminal as { target?: unknown }).target).toBeUndefined();
|
||||
});
|
||||
|
||||
it("Received is always the first node and marked complete", () => {
|
||||
for (const status of ["PENDING", "PROCESSING", "SUCCEEDED", "FAILED", "FILTERED"] as const) {
|
||||
const items = buildDeliveryTimelineItems(baseDelivery({ status, processedAt }));
|
||||
expect(items[0]).toMatchObject({ id: "received", title: "Received", state: "complete" });
|
||||
}
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,73 @@
|
||||
import { describe, expect, it, vi } from "vitest";
|
||||
import { WebhookDeliveryId } from "@trigger.dev/core/v3/isomorphic";
|
||||
import {
|
||||
createdAtMsBounds,
|
||||
deliveryIdsCreatedAtBounds,
|
||||
} from "../app/services/webhookDeliveriesRepository/deliveryIdBounds";
|
||||
|
||||
function idAt(iso: string): string {
|
||||
vi.setSystemTime(new Date(iso));
|
||||
return WebhookDeliveryId.generate().friendlyId;
|
||||
}
|
||||
|
||||
describe("deliveryIdsCreatedAtBounds", () => {
|
||||
it("returns undefined for an empty set", () => {
|
||||
expect(deliveryIdsCreatedAtBounds([])).toBeUndefined();
|
||||
});
|
||||
|
||||
it("returns a zero-width span for a single id (gte == lte == its mint time)", () => {
|
||||
vi.useFakeTimers();
|
||||
try {
|
||||
const at = "2026-08-11T10:00:00.000Z";
|
||||
const friendlyId = idAt(at);
|
||||
const bounds = deliveryIdsCreatedAtBounds([friendlyId]);
|
||||
expect(bounds?.gte.toISOString()).toBe(at);
|
||||
expect(bounds?.lte.toISOString()).toBe(at);
|
||||
} finally {
|
||||
vi.useRealTimers();
|
||||
}
|
||||
});
|
||||
|
||||
it("spans the earliest and latest mint times across ids, regardless of input order", () => {
|
||||
vi.useFakeTimers();
|
||||
try {
|
||||
const early = idAt("2026-08-09T00:00:00.000Z");
|
||||
const mid = idAt("2026-08-10T12:00:00.000Z");
|
||||
const late = idAt("2026-08-11T23:59:59.000Z");
|
||||
const bounds = deliveryIdsCreatedAtBounds([mid, late, early]);
|
||||
expect(bounds?.gte.toISOString()).toBe("2026-08-09T00:00:00.000Z");
|
||||
expect(bounds?.lte.toISOString()).toBe("2026-08-11T23:59:59.000Z");
|
||||
} finally {
|
||||
vi.useRealTimers();
|
||||
}
|
||||
});
|
||||
|
||||
it("returns undefined when an id fails to decode, so the caller skips pruning", () => {
|
||||
expect(deliveryIdsCreatedAtBounds(["whd_notavaliddeliveryid"])).toBeUndefined();
|
||||
});
|
||||
});
|
||||
|
||||
describe("createdAtMsBounds", () => {
|
||||
it("returns undefined for an empty set", () => {
|
||||
expect(createdAtMsBounds([])).toBeUndefined();
|
||||
});
|
||||
|
||||
it("returns a zero-width span for a single value", () => {
|
||||
const bounds = createdAtMsBounds([1_000]);
|
||||
expect(bounds?.gte.getTime()).toBe(1_000);
|
||||
expect(bounds?.lte.getTime()).toBe(1_000);
|
||||
});
|
||||
|
||||
it("spans the smallest and largest value regardless of input order", () => {
|
||||
const bounds = createdAtMsBounds([50, 10, 30, 90, 40]);
|
||||
expect(bounds?.gte.getTime()).toBe(10);
|
||||
expect(bounds?.lte.getTime()).toBe(90);
|
||||
});
|
||||
|
||||
it("handles a large input without a stack overflow (unlike Math.min(...spread))", () => {
|
||||
const values = Array.from({ length: 300_000 }, (_, i) => i);
|
||||
const bounds = createdAtMsBounds(values);
|
||||
expect(bounds?.gte.getTime()).toBe(0);
|
||||
expect(bounds?.lte.getTime()).toBe(299_999);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,45 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { readBodyWithCap } from "~/utils/readBodyWithCap.server";
|
||||
|
||||
function streamingRequest(chunks: Uint8Array[]): Request {
|
||||
const body = new ReadableStream<Uint8Array>({
|
||||
pull(controller) {
|
||||
const next = chunks.shift();
|
||||
if (next) controller.enqueue(next);
|
||||
else controller.close();
|
||||
},
|
||||
});
|
||||
return new Request("https://example.com/ingest", {
|
||||
method: "POST",
|
||||
body,
|
||||
// @ts-expect-error duplex isn't in the lib types yet but Node requires it for a stream body
|
||||
duplex: "half",
|
||||
});
|
||||
}
|
||||
|
||||
const chunk = (n: number) => new Uint8Array(n).fill(122);
|
||||
|
||||
describe("readBodyWithCap", () => {
|
||||
it("returns the full body when under the cap", async () => {
|
||||
const bytes = await readBodyWithCap(streamingRequest([chunk(100), chunk(100)]), 1024);
|
||||
expect(bytes).not.toBeNull();
|
||||
expect(bytes!.byteLength).toBe(200);
|
||||
});
|
||||
|
||||
it("returns null when the streamed body exceeds the cap (no full buffering)", async () => {
|
||||
const bytes = await readBodyWithCap(
|
||||
streamingRequest([chunk(1024), chunk(1024), chunk(1024)]),
|
||||
2048
|
||||
);
|
||||
expect(bytes).toBeNull();
|
||||
});
|
||||
|
||||
it("treats an empty body as zero bytes", async () => {
|
||||
const bytes = await readBodyWithCap(
|
||||
new Request("https://example.com/ingest", { method: "POST" }),
|
||||
1024
|
||||
);
|
||||
expect(bytes).not.toBeNull();
|
||||
expect(bytes!.byteLength).toBe(0);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,35 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { stripClientWebhookActionSource } from "~/services/realtime/sanitizeSessionInput.server";
|
||||
|
||||
const record = (payload: Record<string, unknown>) => JSON.stringify({ kind: "message", payload });
|
||||
|
||||
describe("stripClientWebhookActionSource", () => {
|
||||
it("removes a client-forged webhook actionSource so the action is validated normally", () => {
|
||||
const forged = record({
|
||||
chatId: "c1",
|
||||
trigger: "action",
|
||||
actionSource: "webhook",
|
||||
action: { type: "refund", amount: 9999 },
|
||||
});
|
||||
|
||||
const cleaned = JSON.parse(stripClientWebhookActionSource(forged));
|
||||
expect(cleaned.payload.actionSource).toBeUndefined();
|
||||
expect(cleaned.payload.action).toEqual({ type: "refund", amount: 9999 });
|
||||
expect(cleaned.payload.trigger).toBe("action");
|
||||
});
|
||||
|
||||
it("leaves a non-webhook actionSource untouched", () => {
|
||||
const part = record({ trigger: "action", actionSource: "client", action: { type: "ping" } });
|
||||
expect(stripClientWebhookActionSource(part)).toBe(part);
|
||||
});
|
||||
|
||||
it("leaves a normal message part untouched (fast path, no parse)", () => {
|
||||
const part = record({ trigger: "submit-message", message: { role: "user", parts: [] } });
|
||||
expect(stripClientWebhookActionSource(part)).toBe(part);
|
||||
});
|
||||
|
||||
it("leaves a malformed part untouched", () => {
|
||||
const part = '{"kind":"message","payload":{"actionSource":"webhook"';
|
||||
expect(stripClientWebhookActionSource(part)).toBe(part);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,224 @@
|
||||
import { containerTest } from "@internal/testcontainers";
|
||||
import type { WebhookResource } from "@trigger.dev/core/v3";
|
||||
import type { BackgroundWorker, PrismaClient } from "@trigger.dev/database";
|
||||
import { describe, expect, vi } from "vitest";
|
||||
import type { AuthenticatedEnvironment } from "~/services/apiAuth.server";
|
||||
import { syncDeclarativeWebhooks } from "~/v3/services/createBackgroundWorker.server";
|
||||
|
||||
vi.setConfig({ testTimeout: 60_000 });
|
||||
|
||||
type WorkerArg = Parameters<typeof syncDeclarativeWebhooks>[1];
|
||||
const noWorker = {} as unknown as WorkerArg;
|
||||
|
||||
async function seedProjectWithEnv(prisma: PrismaClient) {
|
||||
const slug = `sdw_${Math.random().toString(36).slice(2, 10)}`;
|
||||
const organization = await prisma.organization.create({ data: { title: slug, slug } });
|
||||
const project = await prisma.project.create({
|
||||
data: { name: slug, slug, organizationId: organization.id, externalRef: slug },
|
||||
});
|
||||
const environment = await prisma.runtimeEnvironment.create({
|
||||
data: {
|
||||
slug: "prod",
|
||||
type: "PRODUCTION",
|
||||
projectId: project.id,
|
||||
organizationId: organization.id,
|
||||
apiKey: `tr_prod_${slug}`,
|
||||
pkApiKey: `pk_prod_${slug}`,
|
||||
shortcode: `p${slug.slice(0, 5)}`,
|
||||
},
|
||||
});
|
||||
return { organization, project, environment };
|
||||
}
|
||||
|
||||
async function seedWorkerWithTask(
|
||||
prisma: PrismaClient,
|
||||
project: { id: string },
|
||||
environment: { id: string },
|
||||
taskSlug: string
|
||||
): Promise<BackgroundWorker> {
|
||||
const suffix = Math.random().toString(36).slice(2, 10);
|
||||
const worker = await prisma.backgroundWorker.create({
|
||||
data: {
|
||||
friendlyId: `worker_${suffix}`,
|
||||
contentHash: `hash_${suffix}`,
|
||||
version: "20260101.1",
|
||||
metadata: {},
|
||||
projectId: project.id,
|
||||
runtimeEnvironmentId: environment.id,
|
||||
},
|
||||
});
|
||||
await prisma.backgroundWorkerTask.create({
|
||||
data: {
|
||||
friendlyId: `task_${suffix}`,
|
||||
slug: taskSlug,
|
||||
filePath: `src/trigger/${taskSlug}.ts`,
|
||||
workerId: worker.id,
|
||||
projectId: project.id,
|
||||
runtimeEnvironmentId: environment.id,
|
||||
},
|
||||
});
|
||||
return worker;
|
||||
}
|
||||
|
||||
async function seedEndpoint(
|
||||
prisma: PrismaClient,
|
||||
base: { organizationId: string; projectId: string; runtimeEnvironmentId: string },
|
||||
handlerWebhookId: string,
|
||||
status: "ACTIVE" | "INACTIVE",
|
||||
manuallyDeactivatedAt: Date | null = null
|
||||
) {
|
||||
const suffix = Math.random().toString(36).slice(2, 10);
|
||||
return prisma.webhookEndpoint.create({
|
||||
data: {
|
||||
friendlyId: `wh_${suffix}`,
|
||||
opaqueId: `op_${suffix}${Math.random().toString(36).slice(2, 10)}`,
|
||||
organizationId: base.organizationId,
|
||||
projectId: base.projectId,
|
||||
runtimeEnvironmentId: base.runtimeEnvironmentId,
|
||||
environmentType: "PRODUCTION",
|
||||
source: "stripe",
|
||||
handlerWebhookId,
|
||||
routingTarget: { type: "task", taskId: "handle-stripe" },
|
||||
verifierArtifact: { kind: "bundle", bundleUrl: "https://example.test/v.js", hash: "h" },
|
||||
status,
|
||||
manuallyDeactivatedAt,
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
function makeWebhookResource(id: string, taskId: string): WebhookResource {
|
||||
return {
|
||||
id,
|
||||
filePath: `src/trigger/${id}.ts`,
|
||||
source: "stripe",
|
||||
verifierArtifact: { kind: "bundle", bundleUrl: "https://example.test/v.js", hash: "h" },
|
||||
routingTarget: { type: "task", taskId },
|
||||
};
|
||||
}
|
||||
|
||||
const asEnv = (env: unknown) => env as AuthenticatedEnvironment;
|
||||
|
||||
describe("syncDeclarativeWebhooks status reconciliation", () => {
|
||||
containerTest(
|
||||
"an absent webhooks list (older client) does not deactivate existing endpoints",
|
||||
async ({ prisma }) => {
|
||||
const { organization, project, environment } = await seedProjectWithEnv(prisma);
|
||||
const endpoint = await seedEndpoint(
|
||||
prisma,
|
||||
{
|
||||
organizationId: organization.id,
|
||||
projectId: project.id,
|
||||
runtimeEnvironmentId: environment.id,
|
||||
},
|
||||
"declared-webhook",
|
||||
"ACTIVE"
|
||||
);
|
||||
|
||||
await syncDeclarativeWebhooks(undefined, noWorker, asEnv(environment), prisma, prisma);
|
||||
|
||||
const after = await prisma.webhookEndpoint.findUniqueOrThrow({ where: { id: endpoint.id } });
|
||||
expect(after.status).toBe("ACTIVE");
|
||||
}
|
||||
);
|
||||
|
||||
containerTest(
|
||||
"an explicit empty list deactivates endpoints that are no longer declared",
|
||||
async ({ prisma }) => {
|
||||
const { organization, project, environment } = await seedProjectWithEnv(prisma);
|
||||
const endpoint = await seedEndpoint(
|
||||
prisma,
|
||||
{
|
||||
organizationId: organization.id,
|
||||
projectId: project.id,
|
||||
runtimeEnvironmentId: environment.id,
|
||||
},
|
||||
"declared-webhook",
|
||||
"ACTIVE"
|
||||
);
|
||||
|
||||
await syncDeclarativeWebhooks([], noWorker, asEnv(environment), prisma, prisma);
|
||||
|
||||
const after = await prisma.webhookEndpoint.findUniqueOrThrow({ where: { id: endpoint.id } });
|
||||
expect(after.status).toBe("INACTIVE");
|
||||
}
|
||||
);
|
||||
|
||||
containerTest(
|
||||
"a redeploy does not re-activate an endpoint disabled via the API",
|
||||
async ({ prisma }) => {
|
||||
const { organization, project, environment } = await seedProjectWithEnv(prisma);
|
||||
const worker = await seedWorkerWithTask(prisma, project, environment, "handle-stripe");
|
||||
const endpoint = await seedEndpoint(
|
||||
prisma,
|
||||
{
|
||||
organizationId: organization.id,
|
||||
projectId: project.id,
|
||||
runtimeEnvironmentId: environment.id,
|
||||
},
|
||||
"declared-webhook",
|
||||
"INACTIVE",
|
||||
new Date()
|
||||
);
|
||||
|
||||
await syncDeclarativeWebhooks(
|
||||
[makeWebhookResource("declared-webhook", "handle-stripe")],
|
||||
worker,
|
||||
asEnv(environment),
|
||||
prisma,
|
||||
prisma
|
||||
);
|
||||
|
||||
const after = await prisma.webhookEndpoint.findUniqueOrThrow({ where: { id: endpoint.id } });
|
||||
expect(after.status).toBe("INACTIVE");
|
||||
expect(after.manuallyDeactivatedAt).not.toBeNull();
|
||||
}
|
||||
);
|
||||
|
||||
containerTest(
|
||||
"a redeploy re-activates an endpoint auto-deactivated when it was removed then re-declared",
|
||||
async ({ prisma }) => {
|
||||
const { organization, project, environment } = await seedProjectWithEnv(prisma);
|
||||
const worker = await seedWorkerWithTask(prisma, project, environment, "handle-stripe");
|
||||
const endpoint = await seedEndpoint(
|
||||
prisma,
|
||||
{
|
||||
organizationId: organization.id,
|
||||
projectId: project.id,
|
||||
runtimeEnvironmentId: environment.id,
|
||||
},
|
||||
"declared-webhook",
|
||||
"INACTIVE",
|
||||
null
|
||||
);
|
||||
|
||||
await syncDeclarativeWebhooks(
|
||||
[makeWebhookResource("declared-webhook", "handle-stripe")],
|
||||
worker,
|
||||
asEnv(environment),
|
||||
prisma,
|
||||
prisma
|
||||
);
|
||||
|
||||
const after = await prisma.webhookEndpoint.findUniqueOrThrow({ where: { id: endpoint.id } });
|
||||
expect(after.status).toBe("ACTIVE");
|
||||
}
|
||||
);
|
||||
|
||||
containerTest("a newly declared webhook creates an active endpoint", async ({ prisma }) => {
|
||||
const { project, environment } = await seedProjectWithEnv(prisma);
|
||||
const worker = await seedWorkerWithTask(prisma, project, environment, "handle-stripe");
|
||||
|
||||
await syncDeclarativeWebhooks(
|
||||
[makeWebhookResource("brand-new-webhook", "handle-stripe")],
|
||||
worker,
|
||||
asEnv(environment),
|
||||
prisma,
|
||||
prisma
|
||||
);
|
||||
|
||||
const created = await prisma.webhookEndpoint.findFirst({
|
||||
where: { runtimeEnvironmentId: environment.id, handlerWebhookId: "brand-new-webhook" },
|
||||
});
|
||||
expect(created?.status).toBe("ACTIVE");
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,31 @@
|
||||
-- +goose Up
|
||||
|
||||
CREATE TABLE IF NOT EXISTS trigger_dev.webhook_deliveries_v1
|
||||
(
|
||||
environment_id String,
|
||||
organization_id String,
|
||||
project_id String,
|
||||
delivery_id String,
|
||||
webhook_endpoint_id String,
|
||||
|
||||
environment_type LowCardinality(String),
|
||||
friendly_id String,
|
||||
|
||||
external_delivery_id String DEFAULT '',
|
||||
run_id String DEFAULT '',
|
||||
|
||||
status LowCardinality(String),
|
||||
|
||||
is_test UInt8 DEFAULT 0,
|
||||
|
||||
created_at DateTime64(3),
|
||||
updated_at DateTime64(3),
|
||||
|
||||
_version UInt64,
|
||||
_is_deleted UInt8 DEFAULT 0
|
||||
)
|
||||
ENGINE = ReplacingMergeTree(_version, _is_deleted)
|
||||
PARTITION BY toYYYYMM(created_at)
|
||||
ORDER BY (organization_id, project_id, environment_id, created_at, delivery_id)
|
||||
TTL toDateTime(created_at) + INTERVAL 60 DAY
|
||||
SETTINGS ttl_only_drop_parts = 1, materialize_ttl_recalculate_only = 1;
|
||||
@@ -48,6 +48,12 @@ import {
|
||||
getSessionsQueryBuilder,
|
||||
insertSessionsCompactArrays,
|
||||
} from "./sessions.js";
|
||||
import {
|
||||
getWebhookDeliveriesQueryBuilder,
|
||||
getWebhookDeliveriesCountQueryBuilder,
|
||||
getWebhookDeliveriesGroupedCountQueryBuilder,
|
||||
insertWebhookDeliveriesCompactArrays,
|
||||
} from "./webhookDeliveries.js";
|
||||
import {
|
||||
getGlobalModelMetrics,
|
||||
getGlobalModelComparison,
|
||||
@@ -79,6 +85,7 @@ export type * from "./queueMetrics.js";
|
||||
export type * from "./llmModelAggregates.js";
|
||||
export type * from "./errors.js";
|
||||
export type * from "./sessions.js";
|
||||
export type * from "./webhookDeliveries.js";
|
||||
export type * from "./client/queryBuilder.js";
|
||||
|
||||
// Re-export column constants, indices, and type-safe accessors
|
||||
@@ -93,6 +100,11 @@ export {
|
||||
} from "./taskRuns.js";
|
||||
|
||||
export { SESSION_COLUMNS, SESSION_INDEX, getSessionField } from "./sessions.js";
|
||||
export {
|
||||
WEBHOOK_DELIVERY_COLUMNS,
|
||||
WEBHOOK_DELIVERY_INDEX,
|
||||
getWebhookDeliveryField,
|
||||
} from "./webhookDeliveries.js";
|
||||
|
||||
// TSQL query execution
|
||||
export {
|
||||
@@ -301,6 +313,15 @@ export class ClickHouse {
|
||||
};
|
||||
}
|
||||
|
||||
get webhookDeliveries() {
|
||||
return {
|
||||
insertCompactArrays: insertWebhookDeliveriesCompactArrays(this.writer),
|
||||
queryBuilder: getWebhookDeliveriesQueryBuilder(this.reader),
|
||||
countQueryBuilder: getWebhookDeliveriesCountQueryBuilder(this.reader),
|
||||
groupedCountQueryBuilder: getWebhookDeliveriesGroupedCountQueryBuilder(this.reader),
|
||||
};
|
||||
}
|
||||
|
||||
get taskEventsV2() {
|
||||
return {
|
||||
insert: insertTaskEventsV2(this.writer),
|
||||
|
||||
@@ -0,0 +1,155 @@
|
||||
import type { ClickHouseSettings } from "@clickhouse/client";
|
||||
import { z } from "zod";
|
||||
import type { ClickhouseReader, ClickhouseWriter } from "./client/types.js";
|
||||
|
||||
export const WebhookDeliveryV1 = z.object({
|
||||
environment_id: z.string(),
|
||||
organization_id: z.string(),
|
||||
project_id: z.string(),
|
||||
delivery_id: z.string(),
|
||||
webhook_endpoint_id: z.string(),
|
||||
environment_type: z.string(),
|
||||
friendly_id: z.string(),
|
||||
external_delivery_id: z.string().default(""),
|
||||
run_id: z.string().default(""),
|
||||
status: z.string(),
|
||||
is_test: z.number().int().default(0),
|
||||
created_at: z.number().int(),
|
||||
updated_at: z.number().int(),
|
||||
_version: z.string(),
|
||||
_is_deleted: z.number().int().default(0),
|
||||
});
|
||||
export type WebhookDeliveryV1 = z.input<typeof WebhookDeliveryV1>;
|
||||
|
||||
export const WEBHOOK_DELIVERY_COLUMNS = [
|
||||
"environment_id",
|
||||
"organization_id",
|
||||
"project_id",
|
||||
"delivery_id",
|
||||
"webhook_endpoint_id",
|
||||
"environment_type",
|
||||
"friendly_id",
|
||||
"external_delivery_id",
|
||||
"run_id",
|
||||
"status",
|
||||
"is_test",
|
||||
"created_at",
|
||||
"updated_at",
|
||||
"_version",
|
||||
"_is_deleted",
|
||||
] as const;
|
||||
|
||||
export type WebhookDeliveryColumnName = (typeof WEBHOOK_DELIVERY_COLUMNS)[number];
|
||||
|
||||
export const WEBHOOK_DELIVERY_INDEX = Object.fromEntries(
|
||||
WEBHOOK_DELIVERY_COLUMNS.map((col, idx) => [col, idx])
|
||||
) as { readonly [K in WebhookDeliveryColumnName]: number };
|
||||
|
||||
export type WebhookDeliveryFieldTypes = {
|
||||
environment_id: string;
|
||||
organization_id: string;
|
||||
project_id: string;
|
||||
delivery_id: string;
|
||||
webhook_endpoint_id: string;
|
||||
environment_type: string;
|
||||
friendly_id: string;
|
||||
external_delivery_id: string;
|
||||
run_id: string;
|
||||
status: string;
|
||||
is_test: number;
|
||||
created_at: number;
|
||||
updated_at: number;
|
||||
_version: string;
|
||||
_is_deleted: number;
|
||||
};
|
||||
|
||||
export type WebhookDeliveryInsertArray = [
|
||||
environment_id: string,
|
||||
organization_id: string,
|
||||
project_id: string,
|
||||
delivery_id: string,
|
||||
webhook_endpoint_id: string,
|
||||
environment_type: string,
|
||||
friendly_id: string,
|
||||
external_delivery_id: string,
|
||||
run_id: string,
|
||||
status: string,
|
||||
is_test: number,
|
||||
created_at: number,
|
||||
updated_at: number,
|
||||
_version: string,
|
||||
_is_deleted: number,
|
||||
];
|
||||
|
||||
export function getWebhookDeliveryField<K extends WebhookDeliveryColumnName>(
|
||||
row: WebhookDeliveryInsertArray,
|
||||
field: K
|
||||
): WebhookDeliveryFieldTypes[K] {
|
||||
return row[WEBHOOK_DELIVERY_INDEX[field]] as WebhookDeliveryFieldTypes[K];
|
||||
}
|
||||
|
||||
export function insertWebhookDeliveriesCompactArrays(
|
||||
ch: ClickhouseWriter,
|
||||
settings?: ClickHouseSettings
|
||||
) {
|
||||
return ch.insertCompactRaw({
|
||||
name: "insertWebhookDeliveriesCompactArrays",
|
||||
table: "trigger_dev.webhook_deliveries_v1",
|
||||
columns: WEBHOOK_DELIVERY_COLUMNS,
|
||||
settings, // no enable_json_type
|
||||
});
|
||||
}
|
||||
|
||||
export const WebhookDeliveryV1QueryResult = z.object({
|
||||
delivery_id: z.string(),
|
||||
created_at_ms: z.number().int(),
|
||||
});
|
||||
export type WebhookDeliveryV1QueryResult = z.infer<typeof WebhookDeliveryV1QueryResult>;
|
||||
|
||||
export function getWebhookDeliveriesQueryBuilder(
|
||||
ch: ClickhouseReader,
|
||||
settings?: ClickHouseSettings
|
||||
) {
|
||||
return ch.queryBuilder({
|
||||
name: "getWebhookDeliveries",
|
||||
baseQuery:
|
||||
"SELECT delivery_id, toUnixTimestamp64Milli(created_at) AS created_at_ms FROM trigger_dev.webhook_deliveries_v1 FINAL",
|
||||
schema: WebhookDeliveryV1QueryResult,
|
||||
settings,
|
||||
});
|
||||
}
|
||||
|
||||
export function getWebhookDeliveriesCountQueryBuilder(
|
||||
ch: ClickhouseReader,
|
||||
settings?: ClickHouseSettings
|
||||
) {
|
||||
return ch.queryBuilder({
|
||||
name: "getWebhookDeliveriesCount",
|
||||
baseQuery: "SELECT count() as count FROM trigger_dev.webhook_deliveries_v1 FINAL",
|
||||
schema: z.object({ count: z.number().int() }),
|
||||
settings,
|
||||
});
|
||||
}
|
||||
|
||||
export const WebhookDeliveryGroupedCountResult = z.object({
|
||||
webhook_endpoint_id: z.string(),
|
||||
count: z.number().int(),
|
||||
});
|
||||
export type WebhookDeliveryGroupedCountResult = z.infer<typeof WebhookDeliveryGroupedCountResult>;
|
||||
|
||||
// Per-endpoint delivery counts in one query (caller adds the scope/period WHERE + GROUP BY). Uses
|
||||
// count(DISTINCT delivery_id) instead of FINAL: it dedupes the ReplacingMergeTree version rows (one
|
||||
// per status transition) without the full-merge cost of FINAL, which matters when counting per
|
||||
// endpoint for a list.
|
||||
export function getWebhookDeliveriesGroupedCountQueryBuilder(
|
||||
ch: ClickhouseReader,
|
||||
settings?: ClickHouseSettings
|
||||
) {
|
||||
return ch.queryBuilder({
|
||||
name: "getWebhookDeliveriesGroupedCount",
|
||||
baseQuery:
|
||||
"SELECT webhook_endpoint_id, count(DISTINCT delivery_id) AS count FROM trigger_dev.webhook_deliveries_v1",
|
||||
schema: WebhookDeliveryGroupedCountResult,
|
||||
settings,
|
||||
});
|
||||
}
|
||||
+5
@@ -0,0 +1,5 @@
|
||||
-- AlterEnum
|
||||
-- Standalone enum-only migration (Q5): the new value is committed before any
|
||||
-- table migration or code references it, so the wrapped-transaction restriction
|
||||
-- on using a freshly added enum value never bites.
|
||||
ALTER TYPE "public"."TaskTriggerSource" ADD VALUE 'WEBHOOK';
|
||||
+76
@@ -0,0 +1,76 @@
|
||||
-- CreateEnum
|
||||
CREATE TYPE "public"."WebhookEndpointStatus" AS ENUM ('ACTIVE', 'INACTIVE', 'DELETING');
|
||||
|
||||
-- CreateEnum
|
||||
CREATE TYPE "public"."WebhookDeliveryStatus" AS ENUM ('PENDING', 'PROCESSING', 'SUCCEEDED', 'FAILED');
|
||||
|
||||
-- NOTE (Q5): ALTER TYPE "TaskTriggerSource" ADD VALUE 'WEBHOOK' lives in the earlier
|
||||
-- 20260622120730_add_webhook_trigger_source migration, not here.
|
||||
|
||||
-- CreateTable
|
||||
CREATE TABLE "public"."WebhookEndpoint" (
|
||||
"id" TEXT NOT NULL,
|
||||
"friendlyId" TEXT NOT NULL,
|
||||
"opaqueId" TEXT NOT NULL,
|
||||
"organizationId" TEXT NOT NULL,
|
||||
"projectId" TEXT NOT NULL,
|
||||
"runtimeEnvironmentId" TEXT NOT NULL,
|
||||
"environmentType" "public"."RuntimeEnvironmentType" NOT NULL,
|
||||
"endpointTenantId" TEXT NOT NULL DEFAULT '',
|
||||
"endpointExternalRef" TEXT NOT NULL DEFAULT '',
|
||||
"source" TEXT NOT NULL,
|
||||
"handlerWebhookId" TEXT NOT NULL,
|
||||
"routingTarget" JSONB NOT NULL,
|
||||
"verifierArtifact" JSONB NOT NULL,
|
||||
"metadata" JSONB NOT NULL DEFAULT '{}',
|
||||
"signingSecretKey" TEXT,
|
||||
"status" "public"."WebhookEndpointStatus" NOT NULL DEFAULT 'ACTIVE',
|
||||
"createdAt" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||||
"updatedAt" TIMESTAMP(3) NOT NULL,
|
||||
|
||||
CONSTRAINT "WebhookEndpoint_pkey" PRIMARY KEY ("id")
|
||||
);
|
||||
|
||||
-- CreateTable (HAND-EDITED: PARTITION BY RANGE clause; clone of TaskEventPartitioned migration.sql:54)
|
||||
CREATE TABLE "public"."WebhookDelivery" (
|
||||
"id" TEXT NOT NULL,
|
||||
"friendlyId" TEXT NOT NULL,
|
||||
"webhookEndpointId" TEXT NOT NULL,
|
||||
"organizationId" TEXT NOT NULL,
|
||||
"projectId" TEXT NOT NULL,
|
||||
"runtimeEnvironmentId" TEXT NOT NULL,
|
||||
"environmentType" "public"."RuntimeEnvironmentType" NOT NULL,
|
||||
"externalDeliveryId" TEXT NOT NULL,
|
||||
"idempotencyKey" TEXT NOT NULL,
|
||||
"runId" TEXT,
|
||||
"status" "public"."WebhookDeliveryStatus" NOT NULL DEFAULT 'PENDING',
|
||||
"isTest" BOOLEAN NOT NULL DEFAULT false,
|
||||
"parsedEvent" JSONB,
|
||||
"rawBodyHash" TEXT,
|
||||
"errorMessage" TEXT,
|
||||
"createdAt" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||||
"updatedAt" TIMESTAMP(3) NOT NULL,
|
||||
"processedAt" TIMESTAMP(3),
|
||||
|
||||
CONSTRAINT "WebhookDelivery_pkey" PRIMARY KEY ("id","createdAt")
|
||||
) PARTITION BY RANGE ("createdAt");
|
||||
|
||||
-- CreateIndex
|
||||
CREATE UNIQUE INDEX "WebhookEndpoint_friendlyId_key" ON "public"."WebhookEndpoint"("friendlyId");
|
||||
|
||||
-- CreateIndex
|
||||
CREATE UNIQUE INDEX "WebhookEndpoint_opaqueId_key" ON "public"."WebhookEndpoint"("opaqueId");
|
||||
|
||||
-- CreateIndex
|
||||
CREATE INDEX "WebhookEndpoint_runtimeEnvironmentId_source_idx" ON "public"."WebhookEndpoint"("runtimeEnvironmentId", "source");
|
||||
|
||||
-- CreateIndex
|
||||
CREATE INDEX "WebhookEndpoint_projectId_createdAt_idx" ON "public"."WebhookEndpoint"("projectId", "createdAt" DESC);
|
||||
|
||||
-- CreateIndex
|
||||
CREATE UNIQUE INDEX "WebhookEndpoint_runtimeEnvironmentId_handlerWebhookId_endpo_key" ON "public"."WebhookEndpoint"("runtimeEnvironmentId", "handlerWebhookId", "endpointTenantId", "endpointExternalRef");
|
||||
|
||||
-- CreateIndex (on the PARENT; auto-propagates to every child at PARTITION OF time)
|
||||
CREATE INDEX "WebhookDelivery_webhookEndpointId_createdAt_idx" ON "public"."WebhookDelivery"("webhookEndpointId", "createdAt" DESC);
|
||||
|
||||
-- No DEFAULT partition on purpose: retention drops use DETACH ... CONCURRENTLY, which a default forbids.
|
||||
+4
@@ -0,0 +1,4 @@
|
||||
-- Add the inbound request headers to webhook deliveries. Surfaced to the webhook task via
|
||||
-- onEvent({ headers }). Nullable + additive; the ADD COLUMN on the RANGE-partitioned parent
|
||||
-- cascades to all child partitions automatically.
|
||||
ALTER TABLE "WebhookDelivery" ADD COLUMN "headers" JSONB;
|
||||
+3
@@ -0,0 +1,3 @@
|
||||
-- Who supplies the signing secret/key for a webhook endpoint ("provider" | "integrator" | "either").
|
||||
-- Drives the dashboard Connect UI (paste vs generate). Synced from the declared source.
|
||||
ALTER TABLE "WebhookEndpoint" ADD COLUMN "secretProvisioning" TEXT NOT NULL DEFAULT 'either';
|
||||
+11
@@ -0,0 +1,11 @@
|
||||
-- Webhook delivery filters: a server-side predicate gates routing (not receipt). A verified delivery
|
||||
-- that doesn't match becomes a FILTERED row (no run/session) with the reason recorded; the compiled
|
||||
-- filter AST is stored on the endpoint and evaluated at ingest.
|
||||
|
||||
ALTER TYPE "WebhookDeliveryStatus" ADD VALUE 'FILTERED';
|
||||
|
||||
ALTER TABLE "WebhookDelivery" ADD COLUMN "filterReason" TEXT;
|
||||
|
||||
ALTER TABLE "WebhookEndpoint" ADD COLUMN "filter" TEXT;
|
||||
ALTER TABLE "WebhookEndpoint" ADD COLUMN "filterAst" JSONB;
|
||||
ALTER TABLE "WebhookEndpoint" ADD COLUMN "filterAstVersion" INTEGER;
|
||||
+2
@@ -0,0 +1,2 @@
|
||||
-- AlterTable
|
||||
ALTER TABLE "public"."WebhookEndpoint" ADD COLUMN "manuallyDeactivatedAt" TIMESTAMP(3);
|
||||
@@ -766,6 +766,102 @@ enum TaskTriggerSource {
|
||||
STANDARD
|
||||
SCHEDULED
|
||||
AGENT
|
||||
WEBHOOK
|
||||
}
|
||||
|
||||
model WebhookEndpoint {
|
||||
id String @id @default(cuid())
|
||||
friendlyId String @unique // wh_... (WebhookEndpointId IdUtil "wh")
|
||||
|
||||
/// URL key. CSPRNG, NOT a friendlyId: crypto.randomBytes(16) base64url.
|
||||
/// Generated by the create path (deploy-sync / P2 API), NOT by @default.
|
||||
opaqueId String @unique // global unique (Q2): ingress resolves the endpoint by opaqueId alone
|
||||
|
||||
// scope (plain columns, NO @relation -> no FK, matching TaskEventPartitioned)
|
||||
organizationId String
|
||||
projectId String
|
||||
runtimeEnvironmentId String
|
||||
environmentType RuntimeEnvironmentType // denormalized from the env at deploy-sync (immutable per endpoint), Q3
|
||||
|
||||
// tenant scope. "" sentinel on declared (P1); P2 dynamic create fills these.
|
||||
// Sentinel not null so the 4-column @@unique actually bites.
|
||||
endpointTenantId String @default("")
|
||||
endpointExternalRef String @default("")
|
||||
|
||||
source String // provider tag e.g. "stripe","slack","github"
|
||||
handlerWebhookId String // declared webhook() id (string ref, GOLDEN LAW, no relation)
|
||||
routingTarget Json // RoutingTarget tagged union ({ type: "task" } | { type: "session" })
|
||||
verifierArtifact Json // VerifierArtifact tagged union (config|preset in P1)
|
||||
filter String? // source filter DSL string (display/round-trip)
|
||||
filterAst Json? // compiled FilterAst, evaluated at ingest; null = route all
|
||||
filterAstVersion Int? // re-parse `filter` on a format bump
|
||||
metadata Json @default("{}") // arbitrary user metadata; flows into the webhook task
|
||||
|
||||
// who supplies the secret/key; drives the Connect UI (paste vs generate). From the source.
|
||||
secretProvisioning String @default("either") // "provider" | "integrator" | "either"
|
||||
|
||||
/// SecretReference.key string. Plain String, NO @relation -> no FK to SecretReference.
|
||||
signingSecretKey String?
|
||||
|
||||
status WebhookEndpointStatus @default(ACTIVE)
|
||||
/// When an operator disabled the endpoint via the dashboard/API. Null means the declarative sync
|
||||
/// owns the status: a redeploy that re-declares a previously-removed (auto-deactivated) webhook
|
||||
/// reactivates it. Non-null means the operator disabled it, so the sync leaves the status alone.
|
||||
manuallyDeactivatedAt DateTime?
|
||||
createdAt DateTime @default(now())
|
||||
updatedAt DateTime @updatedAt
|
||||
|
||||
@@unique([runtimeEnvironmentId, handlerWebhookId, endpointTenantId, endpointExternalRef]) // deploy-sync key
|
||||
@@index([runtimeEnvironmentId, source])
|
||||
@@index([projectId, createdAt(sort: Desc)])
|
||||
}
|
||||
|
||||
enum WebhookEndpointStatus {
|
||||
ACTIVE
|
||||
INACTIVE
|
||||
DELETING
|
||||
}
|
||||
|
||||
model WebhookDelivery {
|
||||
id String @default(cuid())
|
||||
friendlyId String // whd_... (WebhookDeliveryId IdUtil "whd"). Uniqueness scoped by the composite PK.
|
||||
|
||||
// plain columns, NO @relation -> no FK
|
||||
webhookEndpointId String
|
||||
organizationId String
|
||||
projectId String
|
||||
runtimeEnvironmentId String
|
||||
environmentType RuntimeEnvironmentType
|
||||
|
||||
externalDeliveryId String // provider id (Stripe event id, GitHub X-GitHub-Delivery)
|
||||
idempotencyKey String // = externalDeliveryId; passed to Run Engine when routing to a task
|
||||
runId String? // the triggered run, if task target
|
||||
|
||||
status WebhookDeliveryStatus @default(PENDING)
|
||||
|
||||
/// Set from the x-trigger-test ingress header; marks console/test-send deliveries so the list can filter them.
|
||||
isTest Boolean @default(false)
|
||||
|
||||
parsedEvent Json? // size-capped snapshot of the verified event (full event lives in ClickHouse)
|
||||
headers Json? // inbound request headers, surfaced to the webhook task via onEvent({ headers })
|
||||
rawBodyHash String? // sha256 of raw bytes; cheap P2 replay anchor
|
||||
errorMessage String?
|
||||
filterReason String? // why a FILTERED delivery was not routed (failing clause + actual value)
|
||||
|
||||
createdAt DateTime @default(now())
|
||||
updatedAt DateTime @updatedAt // Q14: real column; the status transitions update it, maps straight to CH updated_at
|
||||
processedAt DateTime?
|
||||
|
||||
@@id([id, createdAt]) // composite PK: the partition key MUST be in every PK/unique on a partitioned table
|
||||
@@index([webhookEndpointId, createdAt(sort: Desc)]) // dashboard recent-deliveries; also partition-pruned
|
||||
}
|
||||
|
||||
enum WebhookDeliveryStatus {
|
||||
PENDING
|
||||
PROCESSING
|
||||
SUCCEEDED
|
||||
FAILED
|
||||
FILTERED
|
||||
}
|
||||
|
||||
model PlaygroundConversation {
|
||||
|
||||
@@ -18,6 +18,25 @@ export type PrismaClientOrTransaction = PrismaClient | PrismaTransactionClient;
|
||||
|
||||
export type PrismaReplicaClient = Omit<PrismaClient, "$transaction">;
|
||||
|
||||
// Narrow client views for the webhook feature's tables, prepping it to run on a
|
||||
// dedicated Postgres: control-plane models are absent, so touching one on the
|
||||
// webhook client (or spanning both DBs in a $transaction) is a compile error.
|
||||
export type WebhookDatabase = Pick<
|
||||
PrismaClient,
|
||||
| "webhookEndpoint"
|
||||
| "webhookDelivery"
|
||||
| "$transaction"
|
||||
| "$queryRaw"
|
||||
| "$queryRawUnsafe"
|
||||
| "$executeRaw"
|
||||
| "$executeRawUnsafe"
|
||||
>;
|
||||
|
||||
export type WebhookReplicaDatabase = Pick<
|
||||
PrismaReplicaClient,
|
||||
"webhookEndpoint" | "webhookDelivery" | "$queryRaw" | "$queryRawUnsafe"
|
||||
>;
|
||||
|
||||
export { Decimal };
|
||||
|
||||
function isTransactionClient(prisma: PrismaClientOrTransaction): prisma is PrismaTransactionClient {
|
||||
|
||||
@@ -35,6 +35,12 @@ export interface LogicalReplicationClientOptions {
|
||||
* The name of the publication to use.
|
||||
*/
|
||||
publicationName: string;
|
||||
/**
|
||||
* Whether to create the publication with `publish_via_partition_root = true`.
|
||||
* Required when the replicated table is a partitioned parent: without it, child
|
||||
* partitions publish nothing and the slot stays silent (default: false).
|
||||
*/
|
||||
publishViaPartitionRoot?: boolean;
|
||||
/**
|
||||
* A connected Redis client instance for Redlock.
|
||||
*/
|
||||
@@ -616,12 +622,18 @@ export class LogicalReplicationClient {
|
||||
return true;
|
||||
}
|
||||
|
||||
const publicationWithOptions: string[] = [];
|
||||
if (this.options.publicationActions) {
|
||||
publicationWithOptions.push(`publish = '${this.options.publicationActions.join(", ")}'`);
|
||||
}
|
||||
if (this.options.publishViaPartitionRoot) {
|
||||
publicationWithOptions.push("publish_via_partition_root = true");
|
||||
}
|
||||
|
||||
const [createError] = await tryCatch(
|
||||
this.client.query(
|
||||
`CREATE PUBLICATION "${this.options.publicationName}" FOR TABLE "${this.options.table}" ${
|
||||
this.options.publicationActions
|
||||
? `WITH (publish = '${this.options.publicationActions.join(", ")}')`
|
||||
: ""
|
||||
publicationWithOptions.length > 0 ? `WITH (${publicationWithOptions.join(", ")})` : ""
|
||||
};`
|
||||
)
|
||||
);
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user