Commit Graph

6750 Commits

Author SHA1 Message Date
Saadi Myftija 7f25e82299 feat(deployments): support local builds in cloud (#2628)
* Enable skipping image push during deployment finalization step

* Add endpoint to generate registry credentials for a deployment

* Add a --force-local-build flag to the deployment command to skip remote build

* Do not show the new flag in the help output

* Add changeset

* Remove registry login logs from onLog, not useful

* Rename var

* Update platform package to the latest version
2025-10-23 10:57:36 +02:00
Eric Allam d90da7abf7 fix(replication): allow disabling of task run payload inserts via env var (#2626) 2025-10-22 11:22:14 +01:00
Matt Aitken 2affe541e8 Dev concurrency limit by env (with optional global limit) (#2625)
* Limit local dev concurrency using the dev environment concurrency limit

Previously it was limited to max of 25, no matter the environment limit

* Have global dev limit
2025-10-22 10:55:05 +01:00
Matt Aitken 3157b657c7 Test page recent payload: use ClickHouse to get the latest run ids (#2614)
The Postgres query to get the latest run ids for the test page was very 
slow when there were a lot of runs and/or versions.

This now uses the standard runs list we use everywhere else.
2025-10-17 17:17:36 +01:00
Matt Aitken 41bdab58d5 Document the dev --analyze command (#2613) 2025-10-17 13:00:30 +01:00
James Ritchie fe3fe01fe8 feat(queues): Override queue concurrency limits from the dashboard or API (#2609)
* feat(queues): add ability to override concurrency limit via API and dashboard

* Updates the modal layout and tweaks copy

* Improves the dropdown menu item

* Popover supports both Button and LinkButton

* Right align the columns and fix the dropdown menu item styles

* Organize imports,

* Fix spinner icon in dropdown menu

* Remove unused props

* Adds a tooltip to the Concurrency override badge

* Fixes console error with popover menu

* typo

* Fixes incorrect className

* Minimal buttons to view runs

---------

Co-authored-by: Eric Allam <eallam@icloud.com>
2025-10-17 12:58:25 +01:00
Matt Aitken 68d0037e60 fix(dev): dotenv issues when getting setup with the repo (#2612)
- The .env.example was missing required Clickhouse values.
- A symlink was needed for .env from the root to the apps/webapp folder
2025-10-17 11:49:40 +01:00
Matt Aitken 885d2d3560 Tags listing now uses ClickHouse (#2576)
* WIP using ClickHouse for the tags filter list

* WIP on tags listing

* Webapp: exclude test files when typechecking

* Tags filtering working with CH

* Remove unused import

* The AI filter should only look at the last past 30d of tags

* Do the text query in ClickHouse

* Deal with encoded characters better

* More encoding fixes

* Fix for wrong items being checked

* Put applied tags back

* Add the env.id to the dependencies array
2025-10-15 12:56:08 +01:00
Eric Allam a445af1b79 feat(otel): allow clickhouse task events to be inserted using async_insert via env vars (#2608) 2025-10-15 10:37:17 +01:00
nicktrn 5781783c74 fix(engine): default to paid placement on billing errors (#2604)
* chore(billing): improve logs to distinguish between failure modes

* fix(engine): default to paid placement on billing errors

* chore(engine): set plan type according to paying field when missing
2025-10-15 10:32:27 +01:00
nicktrn cca10c22d3 chore(supervisor): add machine label (#2603) 2025-10-15 10:31:37 +01:00
Leonardo Kaynan 63b6fc93fa fix(helm): align values-production-example.yaml with values.yaml (#2606)
- Move S3 credentials from secrets.objectStore to s3.auth
- Update external PostgreSQL config to use databaseUrl/directUrl approach
- Add existingSecret support for PostgreSQL with secretKeys
- Add TLS configuration for external Redis
- Add existingSecret support for Redis, ClickHouse and S3
- Add complete external S3 configuration example
- Improve secure credential management documentation

These changes align the production example file with the current values.yaml
structure, making it easier to configure external services with better
support for secret management.
2025-10-14 23:16:58 +01:00
Saadi Myftija aa66462971 docs(build-server): adjust build env vars section (#2605)
Adds a clarification about the env var prefix stripping.
2025-10-14 17:29:42 +02:00
James Ritchie f6461684ad chore(webapp): adds shortcut key for admin area (#2570)
* Allow shortcuts hook to work if undefined

* Conditionally show shortcut button if only 1 result

* LinkButton can accept conditionally shown shortcuts
2025-10-13 13:26:00 +01:00
James Ritchie a6896b411a Adds a link to edit the profile icon from the main menu (#2572) 2025-10-13 13:25:06 +01:00
Saadi Myftija 0cabbdd31f docs: deploying using the github integration (#2598)
* docs: deploying using the github integration

* Add hint in the gh actions docs page

* Remove extra space

* Add a couple of hints to the build config fields
2025-10-13 14:15:48 +02:00
Eric Allam f8977a7b70 chore(db): remove unnecessary FK constraints on TaskRunExecutionSnapshot (#2533) 2025-10-09 14:07:45 +01:00
Saadi Myftija f0643f76f5 feat(build-server): add option to specify pre-build command (#2596)
* feat(build-server): add option to specify pre-build command

Adds an option to specify a pre-build command in the build settings. Can
be useful for projects that need a step before the build, e.g., to
generate a prisma client.

Also, remove the install directory in favor of simplicity. Both
pre-build and install commands are run from the root of the repo. Users
that need to run the commands in a different dir can just prepend to the
command, e.g., `cd apps/web && pnpm run primsa:migrate`

* Fix spelling
2025-10-08 17:25:44 +02:00
Saadi Myftija 416dbcd536 fix(webapp): disable gh-triggered preview deployments if the preview env is disabled (#2595)
* Show hint if preview branches are disabled in the project

* Enable preview deployments only if the preview environemtn is enabled

* Fix prisma reference
2025-10-08 16:59:25 +02:00
Eric Allam 679b41dc7e chore(electric): upgrade server to 1.1.14 (#2590) 2025-10-08 14:33:10 +01:00
Eric Allam be98aecbfd fix(otel): prevent unpaired unicode surrogate pairs from causing insert errors (#2594)
* fix(otel): prevent unpaired unicode surrogate pairs from causing insert errors

* only check parts of the string that are not going to get truncated
remove unnecessary taks
2025-10-08 14:32:35 +01:00
nicktrn 129dc02f2a chore: add deepwiki badge to main readme (#2587)
This means DeepWiki will automatically re-index our repo and keep our page fresh
2025-10-08 13:15:06 +01:00
nicktrn 8917478d3c fix(runner): SIGTERM handling during warm start long poll (#2593) 2025-10-08 12:51:10 +01:00
Leonardo Kaynan f5caa66348 fix(helm): use bitnami legacy repo for minio console (#2592)
🧭 Helm Chart Release / lint-and-test (push) Has been cancelled
🧭 Helm Chart Release / release (push) Has been cancelled
* fix(charts): use bitnamilegacy repo for MinIO Object Browser

The Bitnami `minio-object-browser` image has been removed from Docker
Hub. This patch updates the Trigger Helm chart to reference the
`bitnamilegacy/minio-object-browser` repository under the
`s3.console.image` section, ensuring that the legacy MinIO browser pod
can still be deployed without image pull errors.

All other MinIO components continue using the `bitnamilegacy/minio`
image for consistency across Bitnami Legacy dependencies.

* chore(helm): bump Helm chart version to 4.0.4
helm-v4.0.4
2025-10-07 22:17:37 +01:00
Eric Allam 64fcc88fa7 fix(otel): spans with entities (like waitpoints) now correctly returned from clickhouse repo (#2591) 2025-10-07 16:55:00 +01:00
nicktrn 23dbe282ed security: upgrade transitive axios dep (CVE-2025-58754) (#2589) 2025-10-07 14:56:16 +01:00
Eric Allam 692316e82a fix(realtime): Upgrade to @electric-sql/client@1.0.14 to prevent cached 409 Conflict errors from breaking realtime updates (#2588) 2025-10-07 14:26:03 +01:00
Saadi Myftija 107f4dc87c fix(deployments): retry transient depot build init failures (#2586)
The Depot build init with `depot.build.v1.BuildService.createBuild` fails surprisingly often due to transient errors, causing the whole deployment to fail. This PR adds a simple retry mechanism with backoff using p-retry. This should improve the failure rate.
2025-10-06 11:33:31 +02:00
Eric Allam b90f3e2173 fix(otel): remove clickhouse event repo feature flag support from v3, now v4 only (#2585) 2025-10-06 09:54:23 +01:00
Eric Allam b3b2553651 fix(otel): propagate the task event store to run descendants (#2583) 2025-10-04 07:28:16 -07:00
Eric Allam cdd1a8838c fix(otel): prevent spans with negative durations (#2582) 2025-10-03 06:50:16 -07:00
Eric Allam 200b7354d0 fix(otel): clickhouse logs/span metrics now exclude partials and debug events (#2581) 2025-10-02 13:51:50 -07:00
Eric Allam eeed38d223 fix(clickhouse): correctly format datetime64(9) input format (#2580) 2025-10-02 11:32:32 -07:00
nicktrn 0ca092651b feat(supervisor): optional custom scheduler (#2579) 2025-10-02 16:22:53 +01:00
Eric Allam 53acdf8ef5 fix(otel): don't pass isDebug when creating postgresql task events (#2578) 2025-10-01 21:57:35 -07:00
Eric Allam 128bc437f6 feat(otel): Add support for storing run spans and log data in Clickhouse (#2567) 2025-10-01 12:41:18 -07:00
James Ritchie 0597691001 Adds 200 and 500 % billing alert options (#2571) 2025-09-30 13:59:44 -07:00
mintlify[bot] dae84a0d29 Update docs/idempotency.mdx (#2575)
Co-authored-by: mintlify[bot] <109931778+mintlify[bot]@users.noreply.github.com>
2025-09-30 13:50:02 -07:00
nicktrn f72d63aac2 chore(helm): migrate to bitnami legacy registry and add configurable utility images (#2574)
🧭 Helm Chart Release / lint-and-test (push) Has been cancelled
🧭 Helm Chart Release / release (push) Has been cancelled
* chore(docker): use bitnami legacy repo

* chore(helm): use bitnami legacy repo

* Make Helm webapp chart images configurable

Adds configurability for init and token syncer container images through
new values in the Helm chart configuration

* chore(helm): refactor utility image config

* chore(helm): bump chart version to 4.0.3

---------

Co-authored-by: LeoKaynan <leokaynan@hotmail.com>
helm-v4.0.3
2025-09-30 16:02:08 +01:00
nicktrn 12cceaa779 feat(helm): support topology spread constraints for webapp (#2560)
* feat(helm): support topology spread constraints

* chore(helm): update topology env var

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>

* chore(helm): limit spread constraints to webapp for now

---------

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
2025-09-30 15:31:04 +01:00
Saadi Myftija ddebe4dce0 feat(webapp): rate limit magic-link login attempts (#2568)
* feat(webapp): rate limit magic-link login attempts

Adds a simple rate limiter to the login with magic link flow. Similar implementation to the MFA rate limits.

* Fix error message

* Add an env var feature flags for login rate limiting

* Use BoolEnv instead of `0`/`1`

* Parse xff properly
2025-09-29 17:28:36 +02:00
Saadi Myftija 09d51c6d24 fix(webapp): add recommended security headers (#2569)
Sets `Referrer-Policy`, `X-Content-Type-Options` and `Permissions-Policy` headers.
Relevant against certain types of attacks.
2025-09-29 16:55:22 +02:00
Saadi Myftija 3ceea774a8 fix(run-engine): waitpoint update misleading error logs (#2566) 2025-09-26 21:16:12 +02:00
Eric Allam 05b6a26c4f fix(run-engine): pass through engine fair dequeue selection strategy options instead of using defaults (#2565) 2025-09-26 17:12:36 +01:00
Eric Allam 558fb11b89 feat(run-engine): ability to repair runs in QUEUED, SUSPENDED, and FINISHED execution status (#2564)
* feat(server): add two admin endpoints for queue and environment concurrency debugging and repairing
feat(run-engine): ability to repair runs in QUEUED, SUSPENDED, and FINISHED execution status

* Handle FINISHED snapshot in the repair
2025-09-26 15:16:10 +01:00
nicktrn 9aedda23a4 fix(run-engine): carryover batchId after PENDING_EXECUTING stalls (#2563) 2025-09-26 14:14:28 +01:00
Eric Allam 743b8dbe0c chore(run-engine): add additional logging around dequeueing and worker queues (#2562) 2025-09-26 11:50:21 +01:00
Eric Allam eb0263e942 feat(server): add two admin endpoints for queue and environment concurrency debugging and repairing (#2559) 2025-09-25 19:33:10 +01:00
Saadi Myftija 7bf579fa50 fix: use higher entropy invite tokens (#2558)
* fix: use higher entropy invite tokens

We currently use CUIDs for invite tokens, which are generated using
a pattern and are not cryptographically secure. This PR switches to
a higher entropy string generated with `nanoid`.

* Dedupe the invite emails in the application
2025-09-25 19:42:59 +02:00
Saadi Myftija 69d52db856 fix(webapp): org scoping issues in plan selection, alerts, pats and usage (#2549)
* fix: org scoping in the select plan flow

Adds proper org scoping in the loader and action in the plans page.

* Fix billing alerts scope

* Fix org usage page scope

* Fix token revoking flow scope check

* Throw error for failed PAT revokes instead of silent failure
2025-09-25 19:17:51 +02:00