chore(claude): Improve claude code instructions (#3161)
Also includes a claude.md audit workflow for PRs
This commit is contained in:
Eric Allam
@@ -0,0 +1,13 @@
|
||||
---
|
||||
paths:
|
||||
- "internal-packages/database/**"
|
||||
---
|
||||
|
||||
# Database Migration Safety
|
||||
|
||||
- When adding indexes to **existing tables**, use `CREATE INDEX CONCURRENTLY IF NOT EXISTS` to avoid table locks. These must be in their own separate migration file (one index per file).
|
||||
- Indexes on **newly created tables** (same migration as `CREATE TABLE`) do not need CONCURRENTLY.
|
||||
- When indexing a **new column on an existing table**, split into two migrations: first `ADD COLUMN IF NOT EXISTS`, then `CREATE INDEX CONCURRENTLY IF NOT EXISTS` in a separate file.
|
||||
- After generating a migration with Prisma, remove extraneous lines for: `_BackgroundWorkerToBackgroundWorkerFile`, `_BackgroundWorkerToTaskQueue`, `_TaskRunToTaskRunTag`, `_WaitpointRunConnections`, `_completedWaitpoints`, `SecretStore_key_idx`, and unrelated TaskRun indexes.
|
||||
- Never drop columns or tables without explicit approval.
|
||||
- New code should target `RunEngineVersion.V2` only.
|
||||
@@ -0,0 +1,14 @@
|
||||
---
|
||||
paths:
|
||||
- "docs/**"
|
||||
---
|
||||
|
||||
# Documentation Writing Rules
|
||||
|
||||
- Use Mintlify MDX format. Frontmatter: `title`, `description`, `sidebarTitle` (optional).
|
||||
- After creating a new page, add it to `docs.json` navigation under the correct group.
|
||||
- Use Mintlify components: `<Note>`, `<Warning>`, `<Info>`, `<Tip>`, `<CodeGroup>`, `<Expandable>`, `<Steps>`/`<Step>`.
|
||||
- Code examples should be complete and runnable where possible.
|
||||
- Always import from `@trigger.dev/sdk`, never `@trigger.dev/sdk/v3`.
|
||||
- Keep paragraphs short. Use headers to break up content.
|
||||
- Link to related pages using relative paths (e.g., `[Tasks](/tasks/overview)`).
|
||||
@@ -0,0 +1,33 @@
|
||||
---
|
||||
paths:
|
||||
- "apps/webapp/app/v3/**"
|
||||
---
|
||||
|
||||
# Legacy V1 Engine Code in `app/v3/`
|
||||
|
||||
The `v3/` directory name is misleading - most code here is actively used by the current V2 engine. Only the specific files below are legacy V1-only code.
|
||||
|
||||
## V1-Only Files - Never Modify
|
||||
|
||||
- `marqs/` directory (entire MarQS queue system: sharedQueueConsumer, devQueueConsumer, fairDequeuingStrategy, devPubSub)
|
||||
- `legacyRunEngineWorker.server.ts` (V1 background job worker)
|
||||
- `services/triggerTaskV1.server.ts` (deprecated V1 task triggering)
|
||||
- `services/cancelTaskRunV1.server.ts` (deprecated V1 cancellation)
|
||||
- `authenticatedSocketConnection.server.ts` (V1 dev WebSocket using DevQueueConsumer)
|
||||
- `sharedSocketConnection.ts` (V1 shared queue socket using SharedQueueConsumer)
|
||||
|
||||
## V1/V2 Branching Pattern
|
||||
|
||||
Some services act as routers that branch on `RunEngineVersion`:
|
||||
- `services/cancelTaskRun.server.ts` - calls V1 service or `engine.cancelRun()` for V2
|
||||
- `services/batchTriggerV3.server.ts` - uses marqs for V1 path, run-engine for V2
|
||||
|
||||
When editing these shared services, only modify V2 code paths.
|
||||
|
||||
## V2 Modern Stack
|
||||
|
||||
- **Run lifecycle**: `@internal/run-engine` (internal-packages/run-engine)
|
||||
- **Background jobs**: `@trigger.dev/redis-worker` (not graphile-worker/zodworker)
|
||||
- **Queue operations**: RunQueue inside run-engine (not MarQS)
|
||||
- **V2 engine singleton**: `runEngine.server.ts`, `runEngineHandlers.server.ts`
|
||||
- **V2 workers**: `commonWorker.server.ts`, `alertsWorker.server.ts`, `batchTriggerWorker.server.ts`
|
||||
@@ -0,0 +1,12 @@
|
||||
---
|
||||
paths:
|
||||
- "packages/**"
|
||||
---
|
||||
|
||||
# Public Package Rules
|
||||
|
||||
- Changes to `packages/` are **customer-facing**. Always add a changeset: `pnpm run changeset:add`
|
||||
- Default to **patch**. Get maintainer approval for minor. Never select major without explicit approval.
|
||||
- `@trigger.dev/core`: **Never import the root**. Always use subpath imports (e.g., `@trigger.dev/core/v3`).
|
||||
- Do NOT update `rules/` or `.claude/skills/trigger-dev-tasks/` unless explicitly asked. These are maintained in separate dedicated passes.
|
||||
- Test changes using `references/hello-world` reference project.
|
||||
@@ -0,0 +1,23 @@
|
||||
---
|
||||
paths:
|
||||
- "apps/**"
|
||||
---
|
||||
|
||||
# Server App Changes
|
||||
|
||||
When modifying server apps (webapp, supervisor, coordinator, etc.) with **no package changes**, add a `.server-changes/` file instead of a changeset:
|
||||
|
||||
```bash
|
||||
cat > .server-changes/descriptive-name.md << 'EOF'
|
||||
---
|
||||
area: webapp
|
||||
type: fix
|
||||
---
|
||||
|
||||
Brief description of what changed and why.
|
||||
EOF
|
||||
```
|
||||
|
||||
- **area**: `webapp` | `supervisor` | `coordinator` | `kubernetes-provider` | `docker-provider`
|
||||
- **type**: `feature` | `fix` | `improvement` | `breaking`
|
||||
- If the PR also touches `packages/`, just the changeset is sufficient (no `.server-changes/` needed).
|
||||
@@ -0,0 +1,69 @@
|
||||
name: 📝 CLAUDE.md Audit
|
||||
|
||||
on:
|
||||
pull_request:
|
||||
types: [opened, ready_for_review, synchronize]
|
||||
paths-ignore:
|
||||
- "docs/**"
|
||||
- ".changeset/**"
|
||||
- ".server-changes/**"
|
||||
- "**/*.md"
|
||||
- "references/**"
|
||||
|
||||
concurrency:
|
||||
group: claude-md-audit-${{ github.event.pull_request.number }}
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
audit:
|
||||
if: github.event.pull_request.draft == false
|
||||
runs-on: ubuntu-latest
|
||||
permissions:
|
||||
contents: read
|
||||
pull-requests: write
|
||||
issues: write
|
||||
id-token: write
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
fetch-depth: 0
|
||||
|
||||
- name: Run Claude Code
|
||||
id: claude
|
||||
uses: anthropics/claude-code-action@v1
|
||||
with:
|
||||
claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}
|
||||
use_sticky_comment: true
|
||||
|
||||
claude_args: |
|
||||
--max-turns 15
|
||||
--allowedTools "Read,Glob,Grep,Bash(git diff:*)"
|
||||
|
||||
prompt: |
|
||||
You are reviewing a PR to check whether any CLAUDE.md files or .claude/rules/ files need updating.
|
||||
|
||||
## Your task
|
||||
|
||||
1. Run `git diff origin/main...HEAD --name-only` to see which files changed in this PR.
|
||||
2. For each changed directory, check if there's a CLAUDE.md in that directory or a parent directory.
|
||||
3. Determine if any CLAUDE.md or .claude/rules/ file should be updated based on the changes. Consider:
|
||||
- New files/directories that aren't covered by existing documentation
|
||||
- Changed architecture or patterns that contradict current CLAUDE.md guidance
|
||||
- New dependencies, services, or infrastructure that Claude should know about
|
||||
- Renamed or moved files that are referenced in CLAUDE.md
|
||||
- Changes to build commands, test patterns, or development workflows
|
||||
|
||||
## Response format
|
||||
|
||||
If NO updates are needed, respond with exactly:
|
||||
✅ CLAUDE.md files look current for this PR.
|
||||
|
||||
If updates ARE needed, respond with a short list:
|
||||
📝 **CLAUDE.md updates suggested:**
|
||||
- `path/to/CLAUDE.md`: [what should be added/changed]
|
||||
- `.claude/rules/file.md`: [what should be added/changed]
|
||||
|
||||
Keep suggestions specific and brief. Only flag things that would actually mislead Claude in future sessions.
|
||||
Do NOT suggest updates for trivial changes (bug fixes, small refactors within existing patterns).
|
||||
Do NOT suggest creating new CLAUDE.md files - only updates to existing ones.
|
||||
@@ -1,73 +1,47 @@
|
||||
# CLAUDE.md
|
||||
|
||||
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
||||
This file provides guidance to Claude Code when working with this repository. Subdirectory CLAUDE.md files provide deeper context when you navigate into specific areas.
|
||||
|
||||
## Build and Development Commands
|
||||
|
||||
This is a pnpm 10.23.0 monorepo using Turborepo. Run commands from root with `pnpm run`.
|
||||
|
||||
### Essential Commands
|
||||
|
||||
```bash
|
||||
# Start Docker services (PostgreSQL, Redis, Electric)
|
||||
pnpm run docker
|
||||
|
||||
# Run database migrations
|
||||
pnpm run db:migrate
|
||||
|
||||
# Seed the database (required for reference projects)
|
||||
pnpm run db:seed
|
||||
pnpm run docker # Start Docker services (PostgreSQL, Redis, Electric)
|
||||
pnpm run db:migrate # Run database migrations
|
||||
pnpm run db:seed # Seed the database (required for reference projects)
|
||||
|
||||
# Build packages (required before running)
|
||||
pnpm run build --filter webapp && pnpm run build --filter trigger.dev && pnpm run build --filter @trigger.dev/sdk
|
||||
|
||||
# Run webapp in development mode (http://localhost:3030)
|
||||
pnpm run dev --filter webapp
|
||||
|
||||
# Build and watch for changes (CLI and packages)
|
||||
pnpm run dev --filter trigger.dev --filter "@trigger.dev/*"
|
||||
pnpm run dev --filter webapp # Run webapp (http://localhost:3030)
|
||||
pnpm run dev --filter trigger.dev --filter "@trigger.dev/*" # Watch CLI and packages
|
||||
```
|
||||
|
||||
### Testing
|
||||
## Testing
|
||||
|
||||
We use vitest exclusively. **Never mock anything** - use testcontainers instead.
|
||||
|
||||
```bash
|
||||
# Run all tests for a package
|
||||
pnpm run test --filter webapp
|
||||
|
||||
# Run a single test file (preferred - cd into directory first)
|
||||
pnpm run test --filter webapp # All tests for a package
|
||||
cd internal-packages/run-engine
|
||||
pnpm run test ./src/engine/tests/ttl.test.ts --run
|
||||
|
||||
# May need to build dependencies first
|
||||
pnpm run build --filter @internal/run-engine
|
||||
pnpm run test ./src/engine/tests/ttl.test.ts --run # Single test file
|
||||
pnpm run build --filter @internal/run-engine # May need to build deps first
|
||||
```
|
||||
|
||||
Test files go next to source files (e.g., `MyService.ts` → `MyService.test.ts`).
|
||||
Test files go next to source files (e.g., `MyService.ts` -> `MyService.test.ts`).
|
||||
|
||||
#### Testcontainers for Redis/PostgreSQL
|
||||
### Testcontainers for Redis/PostgreSQL
|
||||
|
||||
```typescript
|
||||
import { redisTest, postgresTest, containerTest } from "@internal/testcontainers";
|
||||
|
||||
// Redis only
|
||||
redisTest("should use redis", async ({ redisOptions }) => {
|
||||
/* ... */
|
||||
});
|
||||
|
||||
// PostgreSQL only
|
||||
postgresTest("should use postgres", async ({ prisma }) => {
|
||||
/* ... */
|
||||
});
|
||||
|
||||
// Both Redis and PostgreSQL
|
||||
containerTest("should use both", async ({ prisma, redisOptions }) => {
|
||||
/* ... */
|
||||
});
|
||||
redisTest("should use redis", async ({ redisOptions }) => { /* ... */ });
|
||||
postgresTest("should use postgres", async ({ prisma }) => { /* ... */ });
|
||||
containerTest("should use both", async ({ prisma, redisOptions }) => { /* ... */ });
|
||||
```
|
||||
|
||||
### Changesets and Server Changes
|
||||
## Changesets and Server Changes
|
||||
|
||||
When modifying any public package (`packages/*` or `integrations/*`), add a changeset:
|
||||
|
||||
@@ -77,246 +51,112 @@ pnpm run changeset:add
|
||||
|
||||
- Default to **patch** for bug fixes and minor changes
|
||||
- Confirm with maintainers before selecting **minor** (new features)
|
||||
- **Never** select major (breaking changes) without explicit approval
|
||||
- **Never** select major without explicit approval
|
||||
|
||||
When modifying only server components (`apps/webapp/`, `apps/supervisor/`, etc.) with no package changes, add a `.server-changes/` file instead:
|
||||
|
||||
```bash
|
||||
# Create a file with a descriptive name
|
||||
cat > .server-changes/fix-batch-queue-stalls.md << 'EOF'
|
||||
---
|
||||
area: webapp
|
||||
type: fix
|
||||
---
|
||||
|
||||
Speed up batch queue processing by removing stalls and fixing retry race
|
||||
EOF
|
||||
```
|
||||
|
||||
- **area**: `webapp` | `supervisor` | `coordinator` | `kubernetes-provider` | `docker-provider`
|
||||
- **type**: `feature` | `fix` | `improvement` | `breaking`
|
||||
- **Mixed PRs** (both packages and server): just the changeset is enough, no `.server-changes/` file needed
|
||||
- See `.server-changes/README.md` for full documentation
|
||||
When modifying only server components (`apps/webapp/`, `apps/supervisor/`, etc.) with no package changes, add a `.server-changes/` file instead. See `.server-changes/README.md` for format and documentation.
|
||||
|
||||
## Architecture Overview
|
||||
|
||||
### Request Flow
|
||||
|
||||
User API call -> Webapp routes -> Services -> RunEngine -> Redis Queue -> Supervisor -> Container execution -> Results back through RunEngine -> ClickHouse (analytics) + PostgreSQL (state)
|
||||
|
||||
### Apps
|
||||
|
||||
- **apps/webapp**: Remix 2.1.0 app - main API, dashboard, and Docker image. Uses Express server.
|
||||
- **apps/supervisor**: Node.js app handling task execution, interfacing with Docker/Kubernetes.
|
||||
- **apps/webapp**: Remix 2.1.0 app - main API, dashboard, orchestration. Uses Express server.
|
||||
- **apps/supervisor**: Manages task execution containers (Docker/Kubernetes).
|
||||
|
||||
### Public Packages
|
||||
|
||||
- **packages/trigger-sdk** (`@trigger.dev/sdk`): Main SDK
|
||||
- **packages/cli-v3** (`trigger.dev`): CLI package
|
||||
- **packages/core** (`@trigger.dev/core`): Shared code between SDK and webapp. Import subpaths only (never root).
|
||||
- **packages/build**: Build extensions and types
|
||||
- **packages/trigger-sdk** (`@trigger.dev/sdk`): Main SDK for writing tasks
|
||||
- **packages/cli-v3** (`trigger.dev`): CLI - also bundles code that goes into customer task images
|
||||
- **packages/core** (`@trigger.dev/core`): Shared types. **Import subpaths only** (never root).
|
||||
- **packages/build** (`@trigger.dev/build`): Build extensions and types
|
||||
- **packages/react-hooks**: React hooks for realtime and triggering
|
||||
- **packages/redis-worker** (`@trigger.dev/redis-worker`): Custom Redis-based background job system
|
||||
- **packages/redis-worker** (`@trigger.dev/redis-worker`): Redis-based background job system
|
||||
|
||||
### Internal Packages
|
||||
|
||||
- **internal-packages/database** (`@trigger.dev/database`): Prisma 6.14.0 client and schema
|
||||
- **internal-packages/clickhouse** (`@internal/clickhouse`): ClickHouse client and schema migrations
|
||||
- **internal-packages/run-engine** (`@internal/run-engine`): "Run Engine 2.0" - run lifecycle management
|
||||
- **internal-packages/redis** (`@internal/redis`): Redis client creation utilities
|
||||
- **internal-packages/testcontainers** (`@internal/testcontainers`): Test helpers for Redis/PostgreSQL containers
|
||||
- **internal-packages/zodworker** (`@internal/zodworker`): Graphile-worker wrapper (being replaced by redis-worker)
|
||||
- **internal-packages/database**: Prisma 6.14.0 client and schema (PostgreSQL)
|
||||
- **internal-packages/clickhouse**: ClickHouse client, schema migrations, analytics queries
|
||||
- **internal-packages/run-engine**: "Run Engine 2.0" - core run lifecycle management
|
||||
- **internal-packages/redis**: Redis client creation utilities (ioredis)
|
||||
- **internal-packages/testcontainers**: Test helpers for Redis/PostgreSQL containers
|
||||
- **internal-packages/schedule-engine**: Durable cron scheduling
|
||||
- **internal-packages/zodworker**: Graphile-worker wrapper (DEPRECATED - use redis-worker)
|
||||
|
||||
### Legacy V1 Engine Code
|
||||
|
||||
The `apps/webapp/app/v3/` directory name is misleading - most code there is actively used by V2. Only specific files are V1-only legacy (MarQS queue, triggerTaskV1, cancelTaskRunV1, etc.). See `apps/webapp/CLAUDE.md` for the exact list. When you encounter V1/V2 branching in services, only modify V2 code paths. All new work uses Run Engine 2.0 (`@internal/run-engine`) and redis-worker.
|
||||
|
||||
### Documentation
|
||||
|
||||
Docs live in `docs/` as a Mintlify site (MDX format). See `docs/CLAUDE.md` for conventions.
|
||||
|
||||
### Reference Projects
|
||||
|
||||
The `references/` directory contains test workspaces for developing and testing new SDK and platform features. Use these projects (e.g., `references/hello-world`) to manually test changes to the CLI, SDK, core packages, and webapp before submitting PRs.
|
||||
|
||||
## Webapp Development
|
||||
|
||||
### Key Locations
|
||||
|
||||
- Trigger API: `apps/webapp/app/routes/api.v1.tasks.$taskId.trigger.ts`
|
||||
- Batch trigger: `apps/webapp/app/routes/api.v1.tasks.batch.ts`
|
||||
- Prisma setup: `apps/webapp/app/db.server.ts`
|
||||
- Run engine config: `apps/webapp/app/v3/runEngine.server.ts`
|
||||
- Services: `apps/webapp/app/v3/services/**/*.server.ts`
|
||||
- Presenters: `apps/webapp/app/v3/presenters/**/*.server.ts`
|
||||
- OTEL endpoints: `apps/webapp/app/routes/otel.v1.logs.ts`, `otel.v1.traces.ts`
|
||||
|
||||
### Environment Variables
|
||||
|
||||
Access via `env` export from `apps/webapp/app/env.server.ts`, never `process.env` directly.
|
||||
|
||||
For testable code, **never import env.server.ts** in test files. Pass configuration as options instead. Example pattern:
|
||||
|
||||
- `realtimeClient.server.ts` (testable service)
|
||||
- `realtimeClientGlobal.server.ts` (configuration)
|
||||
|
||||
### Legacy vs Run Engine 2.0
|
||||
|
||||
The codebase is transitioning from the "legacy run engine" (spread across codebase) to "Run Engine 2.0" (`@internal/run-engine`). Focus on Run Engine 2.0 for new work.
|
||||
The `references/` directory contains test workspaces for testing SDK and platform features. Use `references/hello-world` to manually test changes before submitting PRs.
|
||||
|
||||
## Docker Image Guidelines
|
||||
|
||||
When updating Docker image references in `docker/Dockerfile` or other container files:
|
||||
When updating Docker image references:
|
||||
|
||||
- **Always use multiplatform/index digests**, not architecture-specific digests
|
||||
- Architecture-specific digests (e.g., for `linux/amd64` only) will cause CI failures on different build environments
|
||||
- On Docker Hub, the multiplatform digest is shown on the main image page, while architecture-specific digests are listed under "OS/ARCH"
|
||||
- Example: Use `node:20.20-bullseye-slim@sha256:abc123...` where the digest is from the multiplatform index, not from a specific OS/ARCH variant
|
||||
|
||||
## Database Migrations (PostgreSQL)
|
||||
|
||||
1. Edit `internal-packages/database/prisma/schema.prisma`
|
||||
2. Create migration:
|
||||
```bash
|
||||
cd internal-packages/database
|
||||
pnpm run db:migrate:dev:create --name "add_new_column"
|
||||
```
|
||||
3. **Important**: Generated migration includes extraneous changes. Remove lines related to:
|
||||
- `_BackgroundWorkerToBackgroundWorkerFile`
|
||||
- `_BackgroundWorkerToTaskQueue`
|
||||
- `_TaskRunToTaskRunTag`
|
||||
- `_WaitpointRunConnections`
|
||||
- `_completedWaitpoints`
|
||||
- `SecretStore_key_idx`
|
||||
- Various `TaskRun` indexes unless you added them
|
||||
4. Apply migration:
|
||||
```bash
|
||||
pnpm run db:migrate:deploy && pnpm run generate
|
||||
```
|
||||
|
||||
### Index Migration Rules
|
||||
|
||||
- Indexes **must use CONCURRENTLY** to avoid table locks
|
||||
- **CONCURRENTLY indexes must be in their own separate migration file** - they cannot be combined with other schema changes
|
||||
|
||||
## ClickHouse Migrations
|
||||
|
||||
ClickHouse migrations use Goose format and live in `internal-packages/clickhouse/schema/`.
|
||||
|
||||
1. Create a new numbered SQL file (e.g., `010_add_new_column.sql`)
|
||||
2. Use Goose markers:
|
||||
|
||||
```sql
|
||||
-- +goose Up
|
||||
ALTER TABLE trigger_dev.your_table
|
||||
ADD COLUMN new_column String DEFAULT '';
|
||||
|
||||
-- +goose Down
|
||||
ALTER TABLE trigger_dev.your_table
|
||||
DROP COLUMN new_column;
|
||||
```
|
||||
|
||||
Follow naming conventions in `internal-packages/clickhouse/README.md`:
|
||||
|
||||
- `raw_` prefix for input tables
|
||||
- `_v1`, `_v2` suffixes for versioning
|
||||
- `_mv_v1` suffix for materialized views
|
||||
- Architecture-specific digests cause CI failures on different build environments
|
||||
- Use the digest from the main Docker Hub page, not from a specific OS/ARCH variant
|
||||
|
||||
## Writing Trigger.dev Tasks
|
||||
|
||||
Always import from `@trigger.dev/sdk`. Never use `@trigger.dev/sdk/v3` or deprecated `client.defineJob` pattern.
|
||||
Always import from `@trigger.dev/sdk`. Never use `@trigger.dev/sdk/v3` or deprecated `client.defineJob`.
|
||||
|
||||
```typescript
|
||||
import { task } from "@trigger.dev/sdk";
|
||||
|
||||
// Every task must be exported
|
||||
export const myTask = task({
|
||||
id: "my-task", // Unique ID
|
||||
id: "my-task",
|
||||
run: async (payload: { message: string }) => {
|
||||
// Task logic - no timeouts
|
||||
// Task logic
|
||||
},
|
||||
});
|
||||
```
|
||||
|
||||
### SDK Documentation Rules
|
||||
|
||||
The `rules/` directory contains versioned documentation for writing Trigger.dev tasks, distributed to users via the SDK installer. Current version is defined in `rules/manifest.json`.
|
||||
|
||||
- `rules/4.3.0/` - Latest: batch trigger v2 (1,000 items, 3MB payloads), debouncing
|
||||
- `rules/4.1.0/` - Realtime streams v2, updated config
|
||||
- `rules/4.0.0/` - Base v4 SDK documentation
|
||||
|
||||
When adding new SDK features, create a new version directory with only the files that changed from the previous version. Update `manifest.json` to point unchanged files to previous versions.
|
||||
|
||||
### Claude Code Skill
|
||||
|
||||
The `.claude/skills/trigger-dev-tasks/` skill provides Claude Code with Trigger.dev task expertise. It includes:
|
||||
|
||||
- `SKILL.md` - Core instructions and patterns
|
||||
- Reference files for basic tasks, advanced tasks, scheduled tasks, realtime, and config
|
||||
|
||||
Keep the skill in sync with the latest rules version when SDK features change.
|
||||
The `rules/` directory contains versioned SDK documentation distributed via the SDK installer. Current version: `rules/manifest.json`. Do NOT update `rules/` or `.claude/skills/trigger-dev-tasks/` unless explicitly asked - these are maintained in separate dedicated passes.
|
||||
|
||||
## Testing with hello-world Reference Project
|
||||
|
||||
First-time setup:
|
||||
|
||||
1. Run `pnpm run db:seed` to seed the database (creates the hello-world project)
|
||||
1. `pnpm run db:seed` to seed the database
|
||||
2. Build CLI: `pnpm run build --filter trigger.dev && pnpm i`
|
||||
3. Authorize CLI: `cd references/hello-world && pnpm exec trigger login -a http://localhost:3030`
|
||||
3. Authorize: `cd references/hello-world && pnpm exec trigger login -a http://localhost:3030`
|
||||
|
||||
Running:
|
||||
|
||||
```bash
|
||||
cd references/hello-world
|
||||
pnpm exec trigger dev # or with --log-level debug
|
||||
```
|
||||
Running: `cd references/hello-world && pnpm exec trigger dev`
|
||||
|
||||
## Local Task Testing Workflow
|
||||
|
||||
This workflow enables Claude Code to run the webapp and trigger dev simultaneously, trigger tasks, and inspect results for testing code changes.
|
||||
|
||||
### Step 1: Start Webapp in Background
|
||||
|
||||
```bash
|
||||
# Run from repo root with run_in_background: true
|
||||
pnpm run dev --filter webapp
|
||||
```
|
||||
|
||||
Verify webapp is running:
|
||||
|
||||
```bash
|
||||
curl -s http://localhost:3030/healthcheck # Should return 200
|
||||
curl -s http://localhost:3030/healthcheck # Verify running
|
||||
```
|
||||
|
||||
### Step 2: Start Trigger Dev in Background
|
||||
|
||||
```bash
|
||||
# Run from hello-world directory with run_in_background: true
|
||||
cd references/hello-world && pnpm exec trigger dev
|
||||
# Wait for "Local worker ready [node]"
|
||||
```
|
||||
|
||||
The worker will build and register tasks. Check output for "Local worker ready [node]" message.
|
||||
|
||||
### Step 3: Trigger and Monitor Tasks via MCP
|
||||
|
||||
Use the Trigger.dev MCP tools to interact with tasks:
|
||||
|
||||
```
|
||||
# Get current worker and registered tasks
|
||||
mcp__trigger__get_current_worker(projectRef: "proj_rrkpdguyagvsoktglnod", environment: "dev")
|
||||
|
||||
# Trigger a task
|
||||
mcp__trigger__trigger_task(
|
||||
projectRef: "proj_rrkpdguyagvsoktglnod",
|
||||
environment: "dev",
|
||||
taskId: "hello-world",
|
||||
payload: {"message": "Hello from Claude"}
|
||||
)
|
||||
|
||||
# List runs to see status
|
||||
mcp__trigger__list_runs(
|
||||
projectRef: "proj_rrkpdguyagvsoktglnod",
|
||||
environment: "dev",
|
||||
taskIdentifier: "hello-world",
|
||||
limit: 5
|
||||
)
|
||||
mcp__trigger__trigger_task(projectRef: "proj_rrkpdguyagvsoktglnod", environment: "dev", taskId: "hello-world", payload: {"message": "Hello"})
|
||||
mcp__trigger__list_runs(projectRef: "proj_rrkpdguyagvsoktglnod", environment: "dev", taskIdentifier: "hello-world", limit: 5)
|
||||
```
|
||||
|
||||
### Step 4: Monitor Execution
|
||||
|
||||
- Check trigger dev output file for real-time execution logs
|
||||
- Successful runs show: `Task | Run ID | Success (Xms)`
|
||||
- Dashboard available at: http://localhost:3030/orgs/references-9dfd/projects/hello-world-97DT/env/dev/runs
|
||||
|
||||
### Key Project Refs
|
||||
|
||||
- hello-world: `proj_rrkpdguyagvsoktglnod`
|
||||
Dashboard: http://localhost:3030/orgs/references-9dfd/projects/hello-world-97DT/env/dev/runs
|
||||
|
||||
@@ -0,0 +1,20 @@
|
||||
# Supervisor
|
||||
|
||||
Node.js app that manages task execution containers. Receives work from the platform, starts Docker/Kubernetes containers, monitors execution, and reports results.
|
||||
|
||||
## Key Directories
|
||||
|
||||
- `src/services/` - Core service logic
|
||||
- `src/workloadManager/` - Container orchestration abstraction (Docker or Kubernetes)
|
||||
- `src/workloadServer/` - HTTP server for workload communication (heartbeats, snapshots)
|
||||
- `src/clients/` - Platform communication (webapp/coordinator)
|
||||
- `src/env.ts` - Environment configuration
|
||||
|
||||
## Architecture
|
||||
|
||||
- **WorkloadManager**: Abstracts Docker vs Kubernetes execution
|
||||
- **SupervisorSession**: Manages the dequeue loop with EWMA-based dynamic scaling
|
||||
- **ResourceMonitor**: Tracks CPU/memory during execution
|
||||
- **PodCleaner/FailedPodHandler**: Kubernetes-specific cleanup
|
||||
|
||||
Communicates with the platform via Socket.io and HTTP. Receives task assignments through the dequeue protocol from the webapp.
|
||||
@@ -0,0 +1,58 @@
|
||||
# Webapp
|
||||
|
||||
Remix 2.1.0 app serving as the main API, dashboard, and orchestration engine. Uses an Express server (`server.ts`).
|
||||
|
||||
## Key File Locations
|
||||
|
||||
- **Trigger API**: `app/routes/api.v1.tasks.$taskId.trigger.ts`
|
||||
- **Batch trigger**: `app/routes/api.v1.tasks.batch.ts`
|
||||
- **OTEL endpoints**: `app/routes/otel.v1.logs.ts`, `app/routes/otel.v1.traces.ts`
|
||||
- **Prisma setup**: `app/db.server.ts`
|
||||
- **Run engine config**: `app/v3/runEngine.server.ts`
|
||||
- **Services**: `app/v3/services/**/*.server.ts`
|
||||
- **Presenters**: `app/v3/presenters/**/*.server.ts`
|
||||
|
||||
## Route Convention
|
||||
|
||||
Routes use Remix flat-file convention with dot-separated segments:
|
||||
`api.v1.tasks.$taskId.trigger.ts` -> `/api/v1/tasks/:taskId/trigger`
|
||||
|
||||
## Environment Variables
|
||||
|
||||
Access via `env` export from `app/env.server.ts`. **Never use `process.env` directly.**
|
||||
|
||||
For testable code, **never import env.server.ts** in test files. Pass configuration as options instead:
|
||||
- `realtimeClient.server.ts` (testable service, takes config as constructor arg)
|
||||
- `realtimeClientGlobal.server.ts` (creates singleton with env config)
|
||||
|
||||
## Run Engine 2.0
|
||||
|
||||
The webapp integrates `@internal/run-engine` via `app/v3/runEngine.server.ts`. This is the singleton engine instance. Services in `app/v3/services/` call engine methods for all run lifecycle operations (triggering, completing, cancelling, etc.).
|
||||
|
||||
The `engineVersion.server.ts` file determines V1 vs V2 for a given environment. New code should always target V2.
|
||||
|
||||
## Background Workers
|
||||
|
||||
Background job workers use `@trigger.dev/redis-worker`:
|
||||
- `app/v3/commonWorker.server.ts`
|
||||
- `app/v3/alertsWorker.server.ts`
|
||||
- `app/v3/batchTriggerWorker.server.ts`
|
||||
|
||||
Do NOT add new jobs using zodworker/graphile-worker (legacy).
|
||||
|
||||
## Real-time
|
||||
|
||||
- Socket.io: `app/v3/handleSocketIo.server.ts`, `app/v3/handleWebsockets.server.ts`
|
||||
- Electric SQL: Powers real-time data sync for the dashboard
|
||||
|
||||
## Legacy V1 Code
|
||||
|
||||
The `app/v3/` directory name is misleading - most code is actively used by V2. Only these specific files are V1-only legacy:
|
||||
- `app/v3/marqs/` (old MarQS queue system)
|
||||
- `app/v3/legacyRunEngineWorker.server.ts`
|
||||
- `app/v3/services/triggerTaskV1.server.ts`
|
||||
- `app/v3/services/cancelTaskRunV1.server.ts`
|
||||
- `app/v3/authenticatedSocketConnection.server.ts`
|
||||
- `app/v3/sharedSocketConnection.ts`
|
||||
|
||||
Some services (e.g., `cancelTaskRun.server.ts`, `batchTriggerV3.server.ts`) branch on `RunEngineVersion` to support both V1 and V2. When editing these, only modify V2 code paths.
|
||||
@@ -0,0 +1,55 @@
|
||||
# Documentation
|
||||
|
||||
Mintlify-based documentation site for Trigger.dev.
|
||||
|
||||
## Configuration
|
||||
|
||||
- Main config: `docs.json` - defines navigation, theme, metadata
|
||||
- Navigation structure: `docs.json` -> `navigation.dropdowns` -> groups -> pages
|
||||
|
||||
## Writing Docs
|
||||
|
||||
Pages are MDX files. Frontmatter format:
|
||||
|
||||
```yaml
|
||||
---
|
||||
title: "Page Title"
|
||||
description: "Brief description for SEO and previews"
|
||||
sidebarTitle: "Short Title" # Optional, shown in sidebar if different from title
|
||||
---
|
||||
```
|
||||
|
||||
## Adding a New Page
|
||||
|
||||
1. Create the MDX file in the appropriate directory
|
||||
2. Add the page path to `docs.json` navigation (under the correct group)
|
||||
|
||||
## Mintlify Components
|
||||
|
||||
Use these components for structured content:
|
||||
|
||||
- `<Note>` - General notes
|
||||
- `<Warning>` - Important warnings
|
||||
- `<Info>` - Informational callouts
|
||||
- `<Tip>` - Helpful tips
|
||||
- `<CodeGroup>` - Multi-language/multi-file code examples
|
||||
- `<Expandable>` - Collapsible content sections
|
||||
- `<Steps>` / `<Step>` - Step-by-step instructions
|
||||
- `<Card>` / `<CardGroup>` - Card layouts for navigation
|
||||
|
||||
## Code Examples
|
||||
|
||||
- Always import from `@trigger.dev/sdk` (never `@trigger.dev/sdk/v3`)
|
||||
- Make code examples complete and runnable where possible
|
||||
- Use language tags in code fences: `typescript`, `bash`, `json`
|
||||
|
||||
## Directory Structure
|
||||
|
||||
- `documentation/` - Core conceptual docs
|
||||
- `guides/` - How-to guides
|
||||
- `config/` - Configuration reference
|
||||
- `deployment/` - Deployment guides
|
||||
- `tasks/` - Task documentation
|
||||
- `realtime/` - Real-time features
|
||||
- `runs/` - Run management
|
||||
- `images/` - Image assets
|
||||
@@ -0,0 +1,30 @@
|
||||
# ClickHouse Package
|
||||
|
||||
`@internal/clickhouse` - ClickHouse client for analytics and observability data.
|
||||
|
||||
## Migrations
|
||||
|
||||
Goose-format SQL migrations live in `schema/`. Create new numbered files:
|
||||
|
||||
```sql
|
||||
-- +goose Up
|
||||
ALTER TABLE trigger_dev.your_table
|
||||
ADD COLUMN new_column String DEFAULT '';
|
||||
|
||||
-- +goose Down
|
||||
ALTER TABLE trigger_dev.your_table
|
||||
DROP COLUMN new_column;
|
||||
```
|
||||
|
||||
## Naming Conventions
|
||||
|
||||
- `raw_` prefix for input tables (where data lands first)
|
||||
- `_v1`, `_v2` suffixes for table versioning
|
||||
- `_mv_v1` suffix for materialized views
|
||||
- `_per_day`, `_per_month` for aggregation tables
|
||||
|
||||
See `README.md` in this directory for full naming convention documentation.
|
||||
|
||||
## Purpose
|
||||
|
||||
Stores time-series data for task run analytics, event streams, and performance metrics. Separate from PostgreSQL to handle high-volume writes from task execution.
|
||||
@@ -0,0 +1,60 @@
|
||||
# Database Package
|
||||
|
||||
Prisma 6.14.0 client and schema for PostgreSQL (`@trigger.dev/database`).
|
||||
|
||||
## Schema
|
||||
|
||||
Located at `prisma/schema.prisma`. Key models include TaskRun, BackgroundWorker, BackgroundWorkerTask, WorkerDeployment, RuntimeEnvironment, and Project.
|
||||
|
||||
### Engine Versions
|
||||
|
||||
```prisma
|
||||
enum RunEngineVersion {
|
||||
V1 // Legacy (MarQS + Graphile) - DEPRECATED
|
||||
V2 // Current (run-engine + redis-worker)
|
||||
}
|
||||
```
|
||||
|
||||
New code should always target V2.
|
||||
|
||||
## Creating Migrations
|
||||
|
||||
1. Edit `prisma/schema.prisma`
|
||||
2. Generate migration:
|
||||
```bash
|
||||
cd internal-packages/database
|
||||
pnpm run db:migrate:dev:create --name "descriptive_name"
|
||||
```
|
||||
3. **Clean up generated migration** - remove extraneous lines for:
|
||||
- `_BackgroundWorkerToBackgroundWorkerFile`
|
||||
- `_BackgroundWorkerToTaskQueue`
|
||||
- `_TaskRunToTaskRunTag`
|
||||
- `_WaitpointRunConnections`
|
||||
- `_completedWaitpoints`
|
||||
- `SecretStore_key_idx`
|
||||
- Various `TaskRun` indexes (unless you added them)
|
||||
4. Apply migration:
|
||||
```bash
|
||||
pnpm run db:migrate:deploy && pnpm run generate
|
||||
```
|
||||
|
||||
## Index Migration Rules
|
||||
|
||||
When adding indexes to **existing tables**:
|
||||
|
||||
- Use `CREATE INDEX CONCURRENTLY IF NOT EXISTS` to avoid table locks in production
|
||||
- CONCURRENTLY indexes **must be in their own separate migration file** - they cannot be combined with other schema changes (PostgreSQL requirement)
|
||||
- Only add one index per migration file
|
||||
- Pre-apply the index manually in production before deploying the migration (Prisma will skip creation if the index already exists)
|
||||
|
||||
Indexes on **newly created tables** (in the same migration as `CREATE TABLE`) do not need CONCURRENTLY and can be in the same migration file.
|
||||
|
||||
When adding an index on a **new column on an existing table**, use two migrations:
|
||||
1. First migration: `ALTER TABLE ... ADD COLUMN IF NOT EXISTS ...` (the column)
|
||||
2. Second migration: `CREATE INDEX CONCURRENTLY IF NOT EXISTS ...` (the index, in its own file)
|
||||
|
||||
See `README.md` in this directory and `ai/references/migrations.md` for the full index workflow.
|
||||
|
||||
## Read Replicas
|
||||
|
||||
Use `$replica` from `~/db.server` for read-heavy queries in the webapp.
|
||||
@@ -0,0 +1,46 @@
|
||||
# Run Engine 2.0
|
||||
|
||||
Core run lifecycle management system (`@internal/run-engine`). This is where ALL new run lifecycle logic should go - not in `apps/webapp/app/v3/services/` directly.
|
||||
|
||||
## Architecture
|
||||
|
||||
The `RunEngine` class (`src/engine/index.ts`) orchestrates modular systems:
|
||||
|
||||
### Systems (`src/engine/systems/`)
|
||||
|
||||
Each system handles one concern:
|
||||
- **BatchSystem**: Batch trigger processing with DRR (Deficit Round Robin)
|
||||
- **CheckpointSystem**: Execution checkpoints for recovery
|
||||
- **DebounceSystem**: Configurable debouncing with delay
|
||||
- **DelayedRunSystem**: TTL-based delayed execution
|
||||
- **DequeueSystem**: Pulls runs from queue, assigns to workers
|
||||
- **EnqueueSystem**: Adds runs to queue with ordering
|
||||
- **ExecutionSnapshotSystem**: Stores/restores run state for warm restarts
|
||||
- **PendingVersionSystem**: Version management for deployments
|
||||
- **RunAttemptSystem**: Individual execution attempts (retries, heartbeats)
|
||||
- **TTLSystem**: Automatic run expiration
|
||||
- **WaitpointSystem**: Synchronization primitive for waiting between tasks
|
||||
|
||||
### Queue and Locking
|
||||
|
||||
- **RunQueue** (`src/run-queue/`): Redis-backed fair queue with concurrency management
|
||||
- **BatchQueue** (`src/batch-queue/`): Batch processing queue
|
||||
- **RunLocker** (`src/locking.ts`): Redis locks preventing concurrent run execution
|
||||
|
||||
## Key Design Patterns
|
||||
|
||||
- Event-driven via EventBus
|
||||
- OpenTelemetry tracer/meter integration
|
||||
- Redis for distributed locks and queues
|
||||
- Prisma for persistence with read-only replica support (`readOnlyPrisma`)
|
||||
|
||||
## Testing
|
||||
|
||||
Tests live in `src/engine/tests/` and use testcontainers (Redis + PostgreSQL):
|
||||
|
||||
```bash
|
||||
cd internal-packages/run-engine
|
||||
pnpm run test ./src/engine/tests/ttl.test.ts --run
|
||||
```
|
||||
|
||||
May need to build dependencies first: `pnpm run build --filter @internal/run-engine`
|
||||
@@ -0,0 +1,38 @@
|
||||
# CLI Package
|
||||
|
||||
The `trigger.dev` CLI package, published as `trigger.dev` on npm. Executable: `trigger`.
|
||||
|
||||
## Dev vs Deploy
|
||||
|
||||
### Dev Mode (`src/dev/`)
|
||||
Runs tasks locally in the user's Node.js process. No containers involved. Uses `src/dev/` for the dev command, connects to the local webapp for coordination.
|
||||
|
||||
### Deploy Mode (`src/deploy/`)
|
||||
Bundles task code and builds Docker images for production:
|
||||
1. **Bundle**: `src/build/` bundles worker code using the build system
|
||||
2. **Archive**: `src/deploy/archiveContext.ts` packages files for deployment
|
||||
3. **Build image**: `src/deploy/buildImage.ts` creates Docker images (local Docker/Depot or remote builds)
|
||||
4. **Push**: Pushes image to registry, registers with webapp API
|
||||
|
||||
## Customer Task Images
|
||||
|
||||
Code in `src/entryPoints/` runs **inside customer containers** - this is a different runtime environment from the CLI itself. Changes here affect deployed task execution directly.
|
||||
|
||||
The build system (`src/build/`) uses the config from `trigger.config.ts` in user projects to determine what to bundle, which build extensions to apply, and how to structure the output.
|
||||
|
||||
## Commands
|
||||
|
||||
CLI command definitions live in `src/commands/`. Key commands:
|
||||
- `dev.ts` - Local development mode
|
||||
- `deploy.ts` - Production deployment
|
||||
- `init.ts` - Project initialization
|
||||
- `login.ts` - Authentication
|
||||
- `promote.ts` - Deployment promotion
|
||||
|
||||
## MCP Server
|
||||
|
||||
`src/mcp/` provides an MCP server for AI-assisted task development.
|
||||
|
||||
## SDK Documentation Rules
|
||||
|
||||
The `rules/` directory at the repo root contains versioned SDK documentation that gets installed alongside customer projects. Update both `rules/` and `.claude/skills/trigger-dev-tasks/` when SDK features change.
|
||||
@@ -0,0 +1,25 @@
|
||||
# Core Package
|
||||
|
||||
`@trigger.dev/core` - shared types, schemas, and utilities used across the SDK, CLI, and webapp.
|
||||
|
||||
## Critical Import Rule
|
||||
|
||||
**NEVER import the root** (`@trigger.dev/core`). Always use subpath imports:
|
||||
|
||||
```typescript
|
||||
import { ... } from "@trigger.dev/core/v3";
|
||||
import { ... } from "@trigger.dev/core/v3/utils";
|
||||
import { ... } from "@trigger.dev/core/logger";
|
||||
import { ... } from "@trigger.dev/core/schemas";
|
||||
```
|
||||
|
||||
## Cross-Cutting Impact
|
||||
|
||||
Changes here affect both the customer-facing SDK and the server-side webapp. Exercise caution - breaking changes can affect deployed user tasks and the platform simultaneously.
|
||||
|
||||
## Contents
|
||||
|
||||
- Protocol definitions and message types
|
||||
- API schemas (Zod validation)
|
||||
- Shared constants and enums
|
||||
- Utility functions used across packages
|
||||
@@ -0,0 +1,19 @@
|
||||
# Redis Worker
|
||||
|
||||
`@trigger.dev/redis-worker` - custom Redis-based background job system. **This replaces graphile-worker/zodworker** for all new background job needs.
|
||||
|
||||
## Key Files
|
||||
|
||||
- `src/worker.ts` - Worker loop and job processing with concurrency control
|
||||
- `src/queue.ts` - Redis-backed job queue abstraction
|
||||
- `src/fair-queue/` - Fair dequeueing algorithm for queue selection
|
||||
|
||||
## Usage
|
||||
|
||||
Used by the webapp for background jobs (alerting, batch processing, common tasks) and by the run engine for TTL expiration and batch operations.
|
||||
|
||||
All new background jobs in the webapp should use redis-worker. Do NOT add new jobs to zodworker (`@internal/zodworker`) or graphile-worker.
|
||||
|
||||
## Testing
|
||||
|
||||
Uses ioredis. Tests use testcontainers for Redis.
|
||||
@@ -0,0 +1,27 @@
|
||||
# Trigger.dev SDK
|
||||
|
||||
`@trigger.dev/sdk` - the main customer-facing SDK for writing background tasks.
|
||||
|
||||
## Import Rules
|
||||
|
||||
Always import from `@trigger.dev/sdk`. Never use `@trigger.dev/sdk/v3` (deprecated path alias).
|
||||
|
||||
## Key Exports
|
||||
|
||||
- `task` - Define a background task
|
||||
- `schedules.task` - Define a scheduled (cron) task
|
||||
- `batch` - Batch trigger operations
|
||||
- `runs` - Run management and polling
|
||||
- `wait` - Wait for events, delays, or other tasks
|
||||
- `retry` - Retry utilities
|
||||
- `queue` - Queue configuration
|
||||
- `metadata` - Run metadata access
|
||||
- `logger` - Structured logging
|
||||
|
||||
## When Adding Features
|
||||
|
||||
1. Implement the feature in the SDK
|
||||
2. Test with `references/hello-world` reference project
|
||||
3. Docs updates (`docs/`) are usually done in a separate PR
|
||||
|
||||
Do NOT update `rules/` or `.claude/skills/trigger-dev-tasks/` unless explicitly asked. These are maintained in separate dedicated passes.
|
||||
Reference in New Issue
Block a user