Files
triggerdotdev--trigger.dev/internal-packages
Matt Aitken 536731a5d6 feat(clickhouse): infer mixed-type JSON arrays as Array(Dynamic) on insert (#4095)
##  Checklist

- [ ] I have followed every step in the [contributing
guide](https://github.com/triggerdotdev/trigger.dev/blob/main/CONTRIBUTING.md)
- [x] The PR title follows the convention.
- [ ] I ran and tested the code works

---

## Testing

`pnpm run typecheck --filter @internal/clickhouse` passes. This only
adds a ClickHouse input-format setting to existing insert calls; the
setting affects type inference for newly-inserted/merged data and is
non-destructive to existing rows.

---

## Changelog

Sets
`input_format_json_infer_array_of_dynamic_from_array_of_different_types
= 1` on every native-JSON insert path:

- `task_runs_v2` (`output`, `error`) — `insertTaskRuns`,
`insertTaskRunsCompactArrays`, and the async-insert variants
- `task_events_v1` / `task_events_v2` (`attributes`)
- `metrics_v1`
- `sessions_v1`

### Why

Our JSON columns contain arrays with mixed element types (e.g.
`[{"key":"value"}, "string", "string"]`). With this setting off — which
is the effective default under `24.12` compatibility — ClickHouse infers
those as deeply nested unnamed `Tuple(JSON, Nullable(String), …)` types.
ClickHouse 26.2 introduced `input_format_binary_max_type_complexity`
(default 1000), and those tuple type trees exceeded the limit, causing
background merges to fail with **Code 117**.

With the setting on (the default since 25.8), mixed-type arrays are
inferred as a single `Array(Dynamic)` — a simpler, flatter type
representation that never approaches the complexity limit, even once the
upstream default limit is restored.

Setting this explicitly at insert time keeps behavior deterministic and
version-controlled, so it does not depend on the server profile or a
future compatibility bump. This is a forward-only change: it only
affects newly inserted/merged data and does not rewrite existing parts.
Our read path re-serializes these columns to strings (`toJSONString` via
the materialized `*_text` columns), so the internal Tuple →
Array(Dynamic) representation change is transparent to the application.

### Companion server-side setting

To also apply this on the ClickHouse side (covers merges and any writes
not going through these code paths), set it on the default user:

```sql
ALTER USER default SETTINGS input_format_json_infer_array_of_dynamic_from_array_of_different_types = 1;
```

---

## Screenshots

_N/A_

💯

🤖 Generated with [Claude Code](https://claude.com/claude-code)

https://claude.ai/code/session_01AaChyhestFMBYBWh6bgcCF

---
_Generated by [Claude
Code](https://claude.ai/code/session_01AaChyhestFMBYBWh6bgcCF)_

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-07-01 18:54:53 +01:00
..
2026-07-01 08:43:12 +01:00