feat(hosting): default self-hosted realtime streams to v2 (s2-lite) (#4185)

## Summary Realtime streams (AI-agent token streaming and run streams) now default to v2 for self-hosters, backed by a bundled [s2-lite](https://s2.dev) service. Self-hosting previously shipped no S2 configuration, so streams ran on the Redis-backed v1 path and there were no docs for wiring up v2. Both the Docker Compose stack and the Helm chart now provision s2-lite with persistent storage and set the stream env vars out of the box. ## What's included - **Docker Compose**: a persistent `s2` service (s2-lite), a basin init spec, and the `REALTIME_STREAMS_S2_*` plus `REALTIME_STREAMS_DEFAULT_VERSION=v2` env on the webapp. `.env.example` documents the v1 fallback and hosted-S2 options. - **Helm**: an `s2` StatefulSet, PVC, Service and ConfigMap (runs as the non-root image user via `fsGroup`), an `s2` values block, and webapp env wiring with an existing-secret path for hosted S2. - **Docs**: the `REALTIME_STREAMS_S2_*` and `REALTIME_STREAMS_DEFAULT_VERSION` vars in the webapp env reference, plus a "Realtime streams" section in the Docker and Kubernetes self-hosting guides. ## Notes - The OSS code default stays `v1`; v2 becomes the default purely through the self-hosting artifacts, so non-self-host deployments are unaffected. Disabling s2, or setting the version back to `v1`, cleanly reverts to Redis-backed v1. - With v2 enabled, the bundled s2 service is a required dependency for streaming: if it is down, streams error while the task itself still runs. That is the intended trade for the better v2 path. - You can point at a hosted S2 at s2.dev instead of the bundled server.
2026-07-08 14:58:37 +01:00
parent 6e827f1da3
commit a682f1d171
10 changed files with 399 additions and 1 deletions
@@ -354,6 +354,27 @@ EVENT_REPOSITORY_DEFAULT_STORE=clickhouse_v2

 This only affects new runs; existing runs continue to read from wherever their events were originally stored.

+## Realtime streams
+
+Realtime streams power AI-agent token streaming and run streams. They default to **v2**, backed by the bundled `s2` service — [s2-lite](https://s2.dev), the open-source, self-hostable S2 server. It stores stream data in a persistent volume and is preconfigured in the webapp compose file, so no setup is required.
+
+To fall back to the Redis-backed **v1** streams, set on the webapp in your `.env`:
+
+```bash
+REALTIME_STREAMS_DEFAULT_VERSION=v1
+```
+
+To use a hosted S2 at [s2.dev](https://s2.dev) instead of the bundled s2-lite, point the endpoint at your basin and supply an access token:
+
+```bash
+REALTIME_STREAMS_S2_BASIN=your-basin
+REALTIME_STREAMS_S2_ENDPOINT=https://your-basin.b.aws.s2.dev/v1
+REALTIME_STREAMS_S2_SKIP_ACCESS_TOKENS=false
+REALTIME_STREAMS_S2_ACCESS_TOKEN=your-access-token
+```
+
+See the [webapp environment variables](/self-hosting/env/webapp) for the full list of realtime stream settings.
+
 ## Troubleshooting

 - **Deployment fails at the push step.** The machine running `deploy` needs registry access. See the [registry setup](#registry-setup) section for more details.
@@ -139,9 +139,14 @@ mode: "wide"
 | **Task events**                                  |          |                       |                                                                                                                    |
 | `EVENT_REPOSITORY_DEFAULT_STORE`                 | No       | postgres              | Where to store task events. Set to `clickhouse_v2` to store in ClickHouse (recommended for production).            |
 | **Realtime**                                     |          |                       |                                                                                                                    |
-| `REALTIME_STREAM_VERSION`                        | No       | v1                    | Realtime stream protocol version. One of `v1`, `v2`.                                                               |
+| `REALTIME_STREAM_VERSION`                        | No       | v1                    | Stream version exposed to tasks via the `TRIGGER_REALTIME_STREAM_VERSION` variable. Distinct from `REALTIME_STREAMS_DEFAULT_VERSION`. One of `v1`, `v2`. |
 | `REALTIME_STREAM_MAX_LENGTH`                     | No       | 1000                  | Realtime stream max length.                                                                                        |
 | `REALTIME_STREAM_TTL`                            | No       | 86400 (1d)            | Realtime stream TTL (s).                                                                                           |
+| `REALTIME_STREAMS_DEFAULT_VERSION`               | No       | v1                    | Server-side default the webapp uses when a stream request doesn't pin a version (modern SDKs request `v2`). The Docker and Helm self-hosting defaults set this to `v2`. One of `v1`, `v2`. |
+| `REALTIME_STREAMS_S2_BASIN`                       | No       | —                     | S2 basin that holds v2 realtime streams. Required for `v2`. Must be at least 8 characters.                          |
+| `REALTIME_STREAMS_S2_ENDPOINT`                    | No       | —                     | Custom S2 API endpoint, including the `/v1` suffix (e.g. `http://s2/v1` for the bundled s2-lite). Omit to use hosted S2 at s2.dev. |
+| `REALTIME_STREAMS_S2_SKIP_ACCESS_TOKENS`         | No       | false                 | Skip minting per-stream access tokens. Set to `true` for s2-lite, which needs no authentication.                    |
+| `REALTIME_STREAMS_S2_ACCESS_TOKEN`               | No       | —                     | S2 access token. Required for hosted S2 unless `REALTIME_STREAMS_S2_SKIP_ACCESS_TOKENS` is `true`.                  |
 | **Bootstrap**                                    |          |                       |                                                                                                                    |
 | `TRIGGER_BOOTSTRAP_ENABLED`                      | No       | 0                     | Trigger bootstrap enabled.                                                                                         |
 | `TRIGGER_BOOTSTRAP_WORKER_GROUP_NAME`            | No       | —                     | Trigger bootstrap worker group name.                                                                               |
@@ -375,6 +375,32 @@ webapp:

 This only affects new runs; existing runs continue to read from wherever their events were originally stored.

+## Realtime streams
+
+Realtime streams power AI-agent token streaming and run streams. They default to **v2**, backed by the bundled `s2` deployment — [s2-lite](https://s2.dev), the open-source, self-hostable S2 server. The chart deploys it with a persistent volume, so no extra services are required.
+
+To fall back to the Redis-backed **v1** streams, set the default version to `v1`:
+
+```yaml
+s2:
+  defaultStreamVersion: "v1"
+```
+
+To use a hosted S2 at [s2.dev](https://s2.dev) instead of the bundled s2-lite, disable the bundled deployment and point at your basin. Supply the access token via an existing secret:
+
+```yaml
+s2:
+  deploy: false
+  skipAccessTokens: false
+  external:
+    endpoint: "https://your-basin.b.aws.s2.dev/v1"
+    existingSecret: "s2-credentials"
+    existingSecretAccessTokenKey: "access-token"
+  basin: "your-basin"
+```
+
+To disable realtime streams v2 entirely and use v1, set `s2.deploy: false` with no external endpoint. See `helm show values` for all `s2` options.
+
 ## Worker token

 When using the default bootstrap configuration, worker creation and authentication is handled automatically. The webapp generates a worker token and makes it available to the supervisor via a shared volume.
@@ -106,6 +106,22 @@ OBJECT_STORE_SECRET_ACCESS_KEY=very-safe-password
 # MINIO_ROOT_USER=admin
 # MINIO_ROOT_PASSWORD=very-safe-password

+# Realtime streams
+# - Realtime streams power AI-agent token streaming and run streams
+# - They default to v2, backed by the bundled s2-lite service (open-source S2, https://s2.dev)
+# - To fall back to the Redis-backed v1 streams, set REALTIME_STREAMS_DEFAULT_VERSION=v1
+# REALTIME_STREAMS_DEFAULT_VERSION=v2
+# The bundled s2-lite creates whatever basin you set here (min 8 characters)
+# REALTIME_STREAMS_S2_BASIN=trigger-realtime
+# REALTIME_STREAMS_S2_ENDPOINT=http://s2/v1
+# REALTIME_STREAMS_S2_SKIP_ACCESS_TOKENS=true
+# To use a hosted S2 (https://s2.dev) instead of the bundled s2-lite:
+# - point the endpoint at your basin, disable token-skipping, and set an access token
+# REALTIME_STREAMS_S2_SKIP_ACCESS_TOKENS=false
+# REALTIME_STREAMS_S2_ACCESS_TOKEN=
+# Pin the s2-lite image in production (full image reference, digest recommended)
+# S2_IMAGE=ghcr.io/s2-streamstore/s2:latest@sha256:d6ded5ca7dd619fa7c946f06e39a98f9c95c6883c8bb884e5eaa129f232c920c
+
 # Other image tags
 # - These are the versions of the other images to use
 # - You should lock these to a specific version in production
@@ -18,6 +18,7 @@ services:
      - postgres
      - redis
      - clickhouse
+      - s2
    networks:
      - webapp
      - supervisor
@@ -44,6 +45,15 @@ services:
      LOGIN_ORIGIN: ${LOGIN_ORIGIN:-http://localhost:8030}
      API_ORIGIN: ${API_ORIGIN:-http://localhost:8030}
      ELECTRIC_ORIGIN: http://electric:3000
+      # Realtime streams v2, backed by the bundled s2-lite service. This powers
+      # AI-agent token streaming and run streams. Point the endpoint at a hosted
+      # S2 (https://s2.dev) instead by overriding these and setting an access token.
+      # To fall back to the Redis-backed v1 streams, set the version to `v1`.
+      REALTIME_STREAMS_DEFAULT_VERSION: ${REALTIME_STREAMS_DEFAULT_VERSION:-v2}
+      REALTIME_STREAMS_S2_BASIN: ${REALTIME_STREAMS_S2_BASIN:-trigger-realtime}
+      REALTIME_STREAMS_S2_ENDPOINT: ${REALTIME_STREAMS_S2_ENDPOINT:-http://s2/v1}
+      REALTIME_STREAMS_S2_SKIP_ACCESS_TOKENS: ${REALTIME_STREAMS_S2_SKIP_ACCESS_TOKENS:-true}
+      REALTIME_STREAMS_S2_ACCESS_TOKEN: ${REALTIME_STREAMS_S2_ACCESS_TOKEN:-}
      DATABASE_URL: ${DATABASE_URL:-postgresql://postgres:postgres@postgres:5432/main?schema=public&sslmode=disable}
      DIRECT_URL: ${DIRECT_URL:-postgresql://postgres:postgres@postgres:5432/main?schema=public&sslmode=disable}
      SESSION_SECRET: ${SESSION_SECRET}
@@ -225,12 +235,62 @@ services:
      retries: 5
      start_period: 10s

+  # Generates the s2-lite basin spec from REALTIME_STREAMS_S2_BASIN so the bundled
+  # basin always matches the basin the webapp targets. s2-lite is distroless (no
+  # shell), so a tiny init container writes the spec into a shared volume first.
+  s2-init:
+    image: busybox:${BUSYBOX_IMAGE_TAG:-1.37}
+    restart: "no"
+    logging: *logging-config
+    environment:
+      REALTIME_STREAMS_S2_BASIN: ${REALTIME_STREAMS_S2_BASIN:-trigger-realtime}
+    command:
+      - sh
+      - -c
+      - |
+        cat > /config/s2-spec.json <<JSON
+        {
+          "basins": [
+            {
+              "name": "$$REALTIME_STREAMS_S2_BASIN",
+              "config": { "create_stream_on_append": true, "create_stream_on_read": true }
+            }
+          ]
+        }
+        JSON
+    volumes:
+      - s2-config:/config
+    networks:
+      - webapp
+
+  # s2-lite: the open-source, self-hostable S2 (https://s2.dev) server that backs
+  # realtime streams v2. It's distroless (no shell), so a wget/curl healthcheck
+  # always reports unhealthy even when the API is up — hence no healthcheck.
+  # Runs as root so it can write to the named volume; storage lives in SlateDB
+  # under /data, and the basin is created on startup from the generated spec.
+  s2:
+    image: ${S2_IMAGE:-ghcr.io/s2-streamstore/s2:latest@sha256:d6ded5ca7dd619fa7c946f06e39a98f9c95c6883c8bb884e5eaa129f232c920c}
+    restart: ${RESTART_POLICY:-unless-stopped}
+    logging: *logging-config
+    depends_on:
+      s2-init:
+        condition: service_completed_successfully
+    command: ["lite", "--init-file", "/config/s2-spec.json", "--local-root", "/data"]
+    user: "0:0"
+    volumes:
+      - s2-config:/config
+      - s2:/data
+    networks:
+      - webapp
+
 volumes:
  clickhouse:
  postgres:
  redis:
  shared:
  minio:
+  s2:
+  s2-config:

 networks:
  docker-proxy:
@@ -387,6 +387,27 @@ http://{{ include "trigger-v4.fullname" . }}-electric:{{ .Values.electric.servic
 {{- end -}}
 {{- end }}

+{{/*
+Whether realtime streams v2 (S2) is wired up: either the bundled s2-lite is
+deployed, or an external S2 endpoint has been configured.
+*/}}
+{{- define "trigger-v4.s2.enabled" -}}
+{{- if or .Values.s2.deploy .Values.s2.external.endpoint -}}
+true
+{{- end -}}
+{{- end }}
+
+{{/*
+S2 API endpoint URL, including the /v1 suffix expected by the client.
+*/}}
+{{- define "trigger-v4.s2.url" -}}
+{{- if .Values.s2.deploy -}}
+http://{{ include "trigger-v4.fullname" . }}-s2:{{ .Values.s2.service.port }}/v1
+{{- else -}}
+{{ .Values.s2.external.endpoint }}
+{{- end -}}
+{{- end }}
+
 {{/*
 ClickHouse hostname
 */}}
@@ -0,0 +1,142 @@
+{{- if .Values.s2.deploy }}
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: {{ include "trigger-v4.fullname" . }}-s2-spec
+  labels:
+    {{- $component := "s2" }}
+    {{- include "trigger-v4.componentLabels" (dict "Chart" .Chart "Release" .Release "Values" .Values "component" $component) | nindent 4 }}
+data:
+  s2-spec.json: |
+    {
+      "$schema": "https://raw.githubusercontent.com/s2-streamstore/s2/main/cli/schema.json",
+      "basins": [
+        {
+          "name": {{ .Values.s2.basin | quote }},
+          "config": {
+            "create_stream_on_append": true,
+            "create_stream_on_read": true
+          }
+        }
+      ]
+    }
+---
+apiVersion: apps/v1
+kind: StatefulSet
+metadata:
+  name: {{ include "trigger-v4.fullname" . }}-s2
+  labels:
+    {{- $component := "s2" }}
+    {{- include "trigger-v4.componentLabels" (dict "Chart" .Chart "Release" .Release "Values" .Values "component" $component) | nindent 4 }}
+spec:
+  replicas: 1
+  serviceName: {{ include "trigger-v4.fullname" . }}-s2
+  selector:
+    matchLabels:
+      {{- include "trigger-v4.componentSelectorLabels" (dict "Chart" .Chart "Release" .Release "Values" .Values "component" $component) | nindent 6 }}
+  template:
+    metadata:
+      annotations:
+        checksum/spec: {{ .Values.s2.basin | sha256sum }}
+        {{- with .Values.s2.podAnnotations }}
+        {{- toYaml . | nindent 8 }}
+        {{- end }}
+      labels:
+        {{- include "trigger-v4.componentSelectorLabels" (dict "Chart" .Chart "Release" .Release "Values" .Values "component" $component) | nindent 8 }}
+    spec:
+      {{- with .Values.s2.podSecurityContext }}
+      securityContext:
+        {{- toYaml . | nindent 8 }}
+      {{- end }}
+      containers:
+        - name: s2
+          {{- with .Values.s2.securityContext }}
+          securityContext:
+            {{- toYaml . | nindent 12 }}
+          {{- end }}
+          image: "{{ .Values.s2.image.registry }}/{{ .Values.s2.image.repository }}:{{ .Values.s2.image.tag }}{{ with .Values.s2.image.digest }}@{{ . }}{{ end }}"
+          imagePullPolicy: {{ .Values.s2.image.pullPolicy }}
+          command:
+            - "lite"
+            - "--init-file"
+            - "/config/s2-spec.json"
+            - "--local-root"
+            - "/data"
+            - "--port"
+            - {{ .Values.s2.service.targetPort | quote }}
+          ports:
+            - name: http
+              containerPort: {{ .Values.s2.service.targetPort }}
+              protocol: TCP
+          {{- if .Values.s2.livenessProbe.enabled }}
+          livenessProbe:
+            tcpSocket:
+              port: http
+            initialDelaySeconds: {{ .Values.s2.livenessProbe.initialDelaySeconds }}
+            periodSeconds: {{ .Values.s2.livenessProbe.periodSeconds }}
+            timeoutSeconds: {{ .Values.s2.livenessProbe.timeoutSeconds }}
+            failureThreshold: {{ .Values.s2.livenessProbe.failureThreshold }}
+            successThreshold: {{ .Values.s2.livenessProbe.successThreshold }}
+          {{- end }}
+          {{- if .Values.s2.readinessProbe.enabled }}
+          readinessProbe:
+            tcpSocket:
+              port: http
+            initialDelaySeconds: {{ .Values.s2.readinessProbe.initialDelaySeconds }}
+            periodSeconds: {{ .Values.s2.readinessProbe.periodSeconds }}
+            timeoutSeconds: {{ .Values.s2.readinessProbe.timeoutSeconds }}
+            failureThreshold: {{ .Values.s2.readinessProbe.failureThreshold }}
+            successThreshold: {{ .Values.s2.readinessProbe.successThreshold }}
+          {{- end }}
+          resources:
+            {{- toYaml .Values.s2.resources | nindent 12 }}
+          volumeMounts:
+            - name: spec
+              mountPath: /config
+              readOnly: true
+            - name: data
+              mountPath: /data
+      volumes:
+        - name: spec
+          configMap:
+            name: {{ include "trigger-v4.fullname" . }}-s2-spec
+        {{- if not .Values.s2.persistence.enabled }}
+        - name: data
+          emptyDir: {}
+        {{- end }}
+  {{- if .Values.s2.persistence.enabled }}
+  volumeClaimTemplates:
+    - metadata:
+        name: data
+        {{- if .Values.s2.persistence.retain }}
+        annotations:
+          helm.sh/resource-policy: keep
+        {{- end }}
+      spec:
+        accessModes:
+          - {{ .Values.s2.persistence.accessMode }}
+        resources:
+          requests:
+            storage: {{ .Values.s2.persistence.size }}
+        {{- if .Values.s2.persistence.storageClass }}
+        storageClassName: {{ .Values.s2.persistence.storageClass }}
+        {{- end }}
+  {{- end }}
+---
+apiVersion: v1
+kind: Service
+metadata:
+  name: {{ include "trigger-v4.fullname" . }}-s2
+  labels:
+    {{- $component := "s2" }}
+    {{- include "trigger-v4.componentLabels" (dict "Chart" .Chart "Release" .Release "Values" .Values "component" $component) | nindent 4 }}
+spec:
+  type: {{ .Values.s2.service.type }}
+  ports:
+    - port: {{ .Values.s2.service.port }}
+      targetPort: http
+      protocol: TCP
+      name: http
+  selector:
+    {{- include "trigger-v4.componentSelectorLabels" (dict "Chart" .Chart "Release" .Release "Values" .Values "component" $component) | nindent 4 }}
+{{- end }}
@@ -39,6 +39,10 @@ This template will fail the Helm deployment if external config is missing for re
 {{- end }}
 {{- end }}

+{{- if and .Values.s2.deploy .Values.s2.external.endpoint }}
+{{- fail "S2 configuration conflict: s2.external.endpoint is set but s2.deploy=true, so the external endpoint would be ignored. Set s2.deploy=false to use an external S2 endpoint, or clear s2.external.endpoint to use the bundled s2-lite." }}
+{{- end }}
+
 {{- if not .Values.registry.deploy }}
 {{- if or (not .Values.registry.external.host) }}
 {{- fail "Registry external configuration is required when registry.deploy=false. Please provide registry.external.host" }}
@@ -188,6 +188,27 @@ spec:
              value: {{ .Values.webapp.apiOrigin | quote }}
            - name: ELECTRIC_ORIGIN
              value: {{ include "trigger-v4.electric.url" . | quote }}
+            {{- if include "trigger-v4.s2.enabled" . }}
+            # Realtime streams v2, backed by S2 (bundled s2-lite or an external endpoint)
+            - name: REALTIME_STREAMS_DEFAULT_VERSION
+              value: {{ .Values.s2.defaultStreamVersion | quote }}
+            - name: REALTIME_STREAMS_S2_ENDPOINT
+              value: {{ include "trigger-v4.s2.url" . | quote }}
+            - name: REALTIME_STREAMS_S2_BASIN
+              value: {{ .Values.s2.basin | quote }}
+            - name: REALTIME_STREAMS_S2_SKIP_ACCESS_TOKENS
+              value: {{ .Values.s2.skipAccessTokens | quote }}
+            {{- if and .Values.s2.external.endpoint .Values.s2.external.existingSecret }}
+            - name: REALTIME_STREAMS_S2_ACCESS_TOKEN
+              valueFrom:
+                secretKeyRef:
+                  name: {{ .Values.s2.external.existingSecret }}
+                  key: {{ .Values.s2.external.existingSecretAccessTokenKey }}
+            {{- else if .Values.s2.external.accessToken }}
+            - name: REALTIME_STREAMS_S2_ACCESS_TOKEN
+              value: {{ .Values.s2.external.accessToken | quote }}
+            {{- end }}
+            {{- end }}
            {{- if include "trigger-v4.postgres.useSecretUrl" . }}
            - name: DATABASE_URL
              valueFrom:
@@ -534,6 +534,88 @@ electric:
    # - name: CUSTOM_VAR
    #   value: "custom-value"

+# S2 (realtime streams v2)
+# s2-lite is the open-source, self-hostable S2 server (https://s2.dev). It backs
+# realtime streams v2, which power AI-agent token streaming and run streams. When
+# enabled, the webapp defaults to v2; set `defaultStreamVersion: v1` to fall back
+# to the Redis-backed v1 streams. Disable entirely with `deploy: false` and no
+# external endpoint.
+s2:
+  deploy: true
+  image:
+    registry: ghcr.io
+    repository: s2-streamstore/s2
+    tag: "latest"
+    # Pinning by digest is strongly recommended for reproducible deployments
+    digest: "sha256:d6ded5ca7dd619fa7c946f06e39a98f9c95c6883c8bb884e5eaa129f232c920c"
+    pullPolicy: IfNotPresent
+
+  # Which stream protocol version the webapp defaults to when a client doesn't pin one.
+  # Only takes effect while S2 is enabled (deploy: true or an external endpoint set);
+  # with S2 fully disabled the webapp always falls back to v1.
+  defaultStreamVersion: "v2"
+
+  # Basin that holds realtime streams. Created on startup from the init config.
+  # Must be at least 8 characters.
+  basin: "trigger-realtime"
+
+  # s2-lite needs no authentication; skip minting per-stream access tokens.
+  # Set to false when pointing at a hosted S2 that requires a token.
+  skipAccessTokens: true
+
+  podAnnotations: {}
+
+  # s2-lite is distroless and runs as uid 65532. fsGroup makes the persistent
+  # volume writable by that user without running the container as root.
+  podSecurityContext:
+    fsGroup: 65532
+  securityContext:
+    runAsNonRoot: true
+    runAsUser: 65532
+    runAsGroup: 65532
+
+  service:
+    type: ClusterIP
+    port: 80
+    # High container port so s2-lite can bind as the non-root image user (65532)
+    # without needing CAP_NET_BIND_SERVICE for a privileged port
+    targetPort: 8080
+
+  # Persistent storage for the SlateDB-backed stream data
+  persistence:
+    enabled: true
+    size: 5Gi
+    accessMode: ReadWriteOnce
+    storageClass: ""
+    retain: false
+
+  resources: {}
+
+  # Health probe configuration (distroless image → TCP probes, no shell/HTTP tooling)
+  livenessProbe:
+    enabled: true
+    initialDelaySeconds: 10
+    periodSeconds: 10
+    timeoutSeconds: 5
+    failureThreshold: 5
+    successThreshold: 1
+  readinessProbe:
+    enabled: true
+    initialDelaySeconds: 5
+    periodSeconds: 10
+    timeoutSeconds: 5
+    failureThreshold: 5
+    successThreshold: 1
+
+  # External S2 connection (when deploy: false). Point at a hosted S2 basin, e.g.
+  # "https://<basin>.b.aws.s2.dev/v1", set skipAccessTokens: false above, and
+  # supply an access token either inline or via an existing secret.
+  external:
+    endpoint: ""
+    accessToken: ""
+    existingSecret: ""
+    existingSecretAccessTokenKey: "access-token"
+
 # ClickHouse configuration
 # Subchart: https://github.com/bitnami/charts/tree/main/bitnami/clickhouse
 clickhouse: