## Summary
The container entrypoint runs under `set -x`, which echoes every command
to the logs with its variables expanded. Several startup guards
reference full database connection strings, so the DSN (including the
password) was printed to the container logs on every boot. This turns
tracing off around those lines so connection strings are never traced,
while leaving migration behavior and ordinary startup logging unchanged.
## Fix
The leaking lines are the `[ -n "$RUN_OPS_DATABASE_URL" ]` and `[ -n
"$RUN_OPS_LEGACY_DIRECT_URL" ]` guards, and the ClickHouse block (its `[
-n "$CLICKHOUSE_URL" ]` guard plus the lines that build `GOOSE_DBSTRING`
from `CLICKHOUSE_URL`). `set -x` prints each of these with the
credential expanded. Tracing is now disabled around each region and
restored afterward, so non-secret tracing is preserved everywhere else.
The existing legacy-migration subshell already protected its own command
body; this adds the missing protection for the guards and the ClickHouse
block.
```sh
{ set +x; } 2>/dev/null
if [ -n "$RUN_OPS_DATABASE_URL" ]; then
set -x
...
```
## Verification
Built the webapp image and ran it with dummy sentinel connection strings
whose password token is `S3NTINEL_PW_DoNotLog`, then grepped the boot
logs.
Before (unmodified), the token appears in the traced guards:
```
+ [ -n postgresql://user:S3NTINEL_PW_DoNotLog@fake-host:6432/run-ops ]
+ [ -n postgresql://user:S3NTINEL_PW_DoNotLog@fake-host:5432/legacy ]
+ [ -n https://default:S3NTINEL_PW_DoNotLog@fake-host:8443 ]
```
After, `grep S3NTINEL_PW_DoNotLog` on the same run returns nothing, and
the normal "skipping ... migrations" lines still log.
## Summary
Run-graph data (runs, batches, waitpoints, and their related tables) can
now live in a database separate from the control plane, with every read
and write routed to the correct database by each run's residency. This
makes reading and writing run data more reliable once the two are split,
and is a no-op for single-database installs.
## Design
- Run-graph table access goes through the run-store router, which
selects the legacy or the new run-ops store per run instead of assuming
one shared client.
- The legacy run-ops client is now independently pointable, so legacy
run data can be served from its own database (and replica) rather than
the control-plane connection.
- Run-graph writes go straight to the run-graph database instead of
being forwarded through the control plane, and replication targets are
split so runs in the new database still replicate to analytics without
under-counting.
- Read-through slots refuse the control-plane client, so a missing
residency fails loudly instead of silently reading the wrong database.
- Migration `20260710120000_drop_remaining_run_graph_seam_foreign_keys`
drops the foreign keys that still crossed the run-graph / control-plane
seam, which is what lets the two live in separate databases.
The split stays off unless explicitly enabled and the two databases are
confirmed physically distinct; startup fails closed otherwise.
Verified by running the full dashboard end-to-end suite against both a
single-database configuration and a three-database configuration
(control plane, the new database, and a physically separate legacy
database), with runs on both residencies. No misrouted reads in either
configuration.
## What
Adds the ability to **automatically migrate the dedicated run-ops
database** (the NEW DB in the run-ops split), matching how every other
database in the system is migrated. Follow-up to the run-ops split
activation.
## Changes
- **Migrate runner** — new
`internal-packages/run-ops-database/scripts/migrate.mjs`, exposed as
`db:migrate:deploy` / `db:migrate:status`. Connects via
`RUN_OPS_DATABASE_URL` (the same var the app uses) and expands `${VAR}`
refs like Prisma's dotenv.
- **Self-host** — `docker/scripts/entrypoint.sh` runs the run-ops
migration on boot when the DB is configured, gated by
`SKIP_RUN_OPS_MIGRATIONS`. Single-DB installs never set the URL, so it's
a clean no-op.
- **Single env-var family** — the run-ops DB is now addressed by one
canonical `RUN_OPS_*` family, connect path and migrations resolving the
identical URL:
- `RUN_OPS_DATABASE_URL` (writer) — replaces `TASK_RUN_DATABASE_URL`
- `RUN_OPS_LEGACY_DATABASE_URL` — replaces
`TASK_RUN_LEGACY_DATABASE_URL`
- `RUN_OPS_DATABASE_READ_REPLICA_URL` — replaces
`TASK_RUN_DATABASE_READ_REPLICA_URL`
- the old `TASK_RUN_*` aliases, the `??` coalesce, the
`runOpsNewDatabaseUrl` indirection, and the migrate-only `directUrl` are
all removed (consumers read `env.RUN_OPS_DATABASE_URL` directly).
`directUrl` was dropped because it was only ever used by `prisma
migrate` (never the app runtime) to bypass a pooler for advisory locks —
premature here since the run-ops connection isn't wired to the app yet.
If a pooler is later introduced for the app, a direct URL can be
reintroduced then.
## Safety
- **Pure rename** — nothing deployed sets any `TASK_RUN_*` var (the
split isn't activated anywhere yet; `.env.example`, docker-compose, and
cloud already use `RUN_OPS_*`), so there is no config migration.
- **Single-DB / self-host** — no new required env var; entrypoint and
migrate are no-ops when `RUN_OPS_DATABASE_URL` is unset.
- **Cloud** — runs migrations as pre-deploy ECS tasks (companion cloud
PR), calling these same `db:migrate:deploy` / `db:migrate:status`
commands.
## Verification
- Live migration against a fresh scratch DB with only
`RUN_OPS_DATABASE_URL` set: both migrations applied, no `P1012`/`P1013`;
`${VAR}` expansion, idempotent re-run, `status`, and no-op skip all
pass.
- Schema parity 4/4; `typecheck --filter webapp` 18/18; affected
split/replication tests 34/34.
## Scope
This delivers automatic migrations only. Enabling the app to *use* the
new DB (setting `RUN_OPS_DATABASE_URL` + `RUN_OPS_SPLIT_ENABLED` on the
service) is a separate activation step.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
## Summary
Adds an in-dashboard AI agent: a chat panel, reachable from any
environment
page, that answers questions about your runs, errors, tasks, and
analytics,
diagnoses why a run failed, charts your data, reads your connected
repo's
source, and answers product and how-to questions. It is gated behind the
`hasDashboardAgentAccess` feature flag (global or per-org, default off),
so
this PR ships disabled: the launcher is hidden unless the flag is
enabled.
## Design
The agent runs as a standalone `chat.agent` Trigger task in its own
internal
package, with no access to the webapp database, Prisma, or ClickHouse.
It reads
the user's data over the public API, acting as the user via a
short-lived
delegated user-actor token minted server-side each turn (never in the
browser),
building on
[#3997](https://github.com/triggerdotdev/trigger.dev/pull/3997). The
error and analytics tools use
[#4005](https://github.com/triggerdotdev/trigger.dev/pull/4005)
and the TRQL query API.
The first turn of a new chat streams from a warm webapp route (Head
Start) while
the durable agent boots in parallel. Structured answers (a run-failure
diagnosis
card, a live chart) render through a small typed view catalog rather
than
arbitrary markup. A knowledge lane forwards product and how-to questions
to the
support assistant.
Conversation history lives in a separate Drizzle-backed store on its own
Postgres schema, kept as a display read-model so it can never corrupt
the
agent's model context.
The SDK changes add an `apiClient` option to
`chat.createStartSessionAction` and
`chat.headStart`, and keep the Head Start tool-approval tail intact
across a
custom `prepareMessages` hook so prompt caching and Head Start compose.
Adds support for skipping Postgres migrations on container startup via
the new `SKIP_POSTGRES_MIGRATIONS` environment variable.
Set `SKIP_POSTGRES_MIGRATIONS=1` to skip migrations, matching the
existing behavior of `SKIP_CLICKHOUSE_MIGRATIONS`.
* runs replication leader lock expiration fix
* Allow configuring the container image --max-old-space-size using NODE_MAX_OLD_SPACE_SIZE
* Ability to configure the clickhouse keep alive settings
* Add some logging because we might not be able to do telemetry
* WIP clickhouse package with test containers setup
* More clickhouse client setup now with otel and real tests, and the v1 of raw run events
* Add some additional columns to raw_run_events_v1
* WIP runs dashboard service
* Create a new run engine event bus event for the runs dashboard to hook into
* Track run events in the run engine
* make sure engine v1 runs get synced to CH
* Update the attemptNumber of v3 task runs
* Restructure the run events to be more sparse
* emit more stuff
* Setup replication package
* scaffold the replication package
* replication wip
* resolve conflicts
* more replication stuff
* Add ability to drop the replication slot completely on teardown
* Use the new single replacingmergetree task events table for replication
* get it working
* insert payloads into their own table only on insert and then join
* prepare for using clickhouse cloud and now running ch migrations during boot in the entrypoint.sh
* Handover WIP and tests
* Testing the replication service
* Remove the runs dashboard stuff that we aren't using anymore
* Added a test for large payloads
* hacky typecheck fix
* Fix new internal package typecheck issues and start adding telemetry to the replication service
* tracing over spans, some other improvements
* Improvements to the runs replication service, now ready for testing
* Some fixes and cleanups
* Don't need this code anymore
* move transaction types into the runs replication service
* only send spans where there are transaction events
* A couple of suggested tweaks
* WIP
* Report ECS task info on startup and shutdown
* Fixed lifecycle name
* Re-add terminus
* Require the build dir when http server is disabled
* Remove unnecessary logs
* Implement graceful shutdown in ZodWorker
* Re-order some code
* Increase the keepAliveTimeout to 65 seconds to prevent LB 502 errors
* fix: use NODE_ENV when starting from docker image
When starting from `pnpm run start`, NODE_ENV is fixed to production, causing problems like #186.
So, I added `start_docker`, a process dedicated to docker, and started from it.
The process is exactly the same as `start` except that NODE_ENV is not overwritten.
* fix: align execution name with others
* Change: command name to match review
Eric Allam