Commit Graph

2 Commits

Author SHA1 Message Date
Matt Aitken 71d98b4e6b Support for org-scoped ClickHouse (#3333)
Added `OrganizationDataStore` which allows orgs to have data stored in
specific separate services.

For now this is just used for ClickHouse. When using ClickHouse we get a
client for the factory and pass in the org id.

Particular care has to be made with two hot-insert paths:
1. RunReplicationService
2. OTLPExporter

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
2026-05-22 14:43:03 +01:00
Eric Allam b38405cb88 Realtime and task run performance improvements (#2158)
* Add createdAt filter to realtime subscribing with tags

* Filter realtime colums and expose ability to skip some columns

* Add sharding support for electric

* Use unkey cache for the created at filter caching

* Remove 2 unused indexes on TaskRun

* Run list now filters by a single runtime environment

* Remove project ID indexes

* Use clickhouse in task list aggregation queries instead of pg (keep pg for self-hosters)

* WIP clickhouse powered runs list
stuff

* Improve the query to get the latest tasks for the task list presenter

* Update the usage task list to use clickhouse

* Implement next runs list powered by clickhouse

* Add new index for TaskRun for the runs list, by environment ID

* Add runTags gin index

* Handle possibly malicious inputs

* Ignore claude settings

* Better handling not finding an environment on the schedule page

* Use ms since epoch in test, not seconds

* Remove unused function

* Fix test

* Use an env var for the realtime maximum createdAt filter duration (defaults to 1 day)

* Fixed the query builder to correct the group by / order by order

* Make sure runs.list still works

* Create small-birds-arrive.md
2025-06-10 12:11:01 +01:00