Compare commits
1 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 27b81c7449 |
+273
@@ -0,0 +1,273 @@
|
||||
# MySQL / MariaDB Compatibility
|
||||
|
||||
## What this is
|
||||
|
||||
Omnigent was built for SQLite (local dev) and PostgreSQL (production). This document covers everything done to add MySQL/MariaDB as a third supported database backend, why each change was needed, and how to run the stack locally.
|
||||
|
||||
---
|
||||
|
||||
## Local dev setup (no Docker Desktop required)
|
||||
|
||||
We use **Colima** — a lightweight Linux VM that runs Docker without Docker Desktop — to host MariaDB 11. The omnigent server runs directly on the Mac.
|
||||
|
||||
### What's running
|
||||
|
||||
| Process | Where | Port |
|
||||
|---------|-------|------|
|
||||
| MariaDB 11 | Colima Docker container | 3306 |
|
||||
| Omnigent server | Mac host (Python 3.12 venv) | 6767 |
|
||||
|
||||
### Start everything
|
||||
|
||||
```bash
|
||||
# Start the MariaDB container (if not already running)
|
||||
DOCKER_HOST="unix://${HOME}/.colima/default/docker.sock" docker start omnigent-mariadb
|
||||
|
||||
# Start the omnigent server
|
||||
MARIADB_PASSWORD=$(grep ^MARIADB_PASSWORD ~/omnigent/deploy/docker/.env | cut -d= -f2)
|
||||
cd ~/omnigent
|
||||
nohup .venv-omnigent/bin/python -m omnigent server \
|
||||
--database-uri "mysql+pymysql://omnigent:${MARIADB_PASSWORD}@127.0.0.1:3306/omnigent?charset=utf8mb4" \
|
||||
> /tmp/omnigent-server.log 2>&1 &
|
||||
```
|
||||
|
||||
Then open http://localhost:6767 in your browser.
|
||||
|
||||
### Connect a runner
|
||||
|
||||
After the server is up, open a new terminal and run:
|
||||
|
||||
```bash
|
||||
omnigent
|
||||
```
|
||||
|
||||
The CLI will prompt you to connect to a server — point it at `http://localhost:6767`. The runner handles actual agent execution; without it, sessions show "internal error" when you try to use them.
|
||||
|
||||
### Query the database interactively
|
||||
|
||||
```bash
|
||||
MARIADB_PASSWORD=$(grep ^MARIADB_PASSWORD ~/omnigent/deploy/docker/.env | cut -d= -f2)
|
||||
DOCKER_HOST="unix://${HOME}/.colima/default/docker.sock" \
|
||||
docker exec -it -e MYSQL_PWD="${MARIADB_PASSWORD}" omnigent-mariadb \
|
||||
mariadb -u omnigent omnigent
|
||||
```
|
||||
|
||||
### Stop everything
|
||||
|
||||
```bash
|
||||
kill $(cat /tmp/omnigent-server.pid 2>/dev/null)
|
||||
DOCKER_HOST="unix://${HOME}/.colima/default/docker.sock" docker stop omnigent-mariadb
|
||||
colima stop
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Code changes made
|
||||
|
||||
### 1. Driver dependency (`pyproject.toml`)
|
||||
|
||||
Added `PyMySQL` as an optional dependency. PyMySQL is a pure-Python MySQL/MariaDB driver — no system libraries needed.
|
||||
|
||||
```toml
|
||||
mysql = ["PyMySQL>=1.1,<2"]
|
||||
```
|
||||
|
||||
Also added it explicitly to the Docker server image (`deploy/docker/Dockerfile`) so the container works with both PostgreSQL and MariaDB without a separate build.
|
||||
|
||||
---
|
||||
|
||||
### 2. Upsert syntax
|
||||
|
||||
**The biggest change.** PostgreSQL uses `ON CONFLICT DO UPDATE` for upserts (insert-or-update atomically). MySQL/MariaDB uses `ON DUPLICATE KEY UPDATE`. These are not interchangeable — you have to use the dialect-specific SQLAlchemy insert object.
|
||||
|
||||
**5 places were affected:**
|
||||
|
||||
| File | Method | What it upserts |
|
||||
|------|--------|----------------|
|
||||
| `permission_store/sqlalchemy_store.py` | `grant()` | User→session permission level |
|
||||
| `permission_store/sqlalchemy_store.py` | `ensure_user()` | User row (insert if not exists) |
|
||||
| `conversation_store/sqlalchemy_store.py` | `_dialect_upsert_labels()` | Conversation policy labels |
|
||||
| `conversation_store/sqlalchemy_store.py` | `_upsert_daily_cost_dialect()` | Per-user daily LLM spend (atomic increment) |
|
||||
| `conversation_store/sqlalchemy_store.py` | `set_daily_ask_approved()` | Cost approval checkpoint |
|
||||
|
||||
Each had a SQLite branch and a PostgreSQL branch. A MySQL/MariaDB branch was added to each using `sqlalchemy.dialects.mysql.insert` with `.on_duplicate_key_update()`.
|
||||
|
||||
The callers that gate which dialect gets the fast path (e.g. `if dialect in ("sqlite", "postgresql"):`) were also updated to include `"mysql"` and `"mariadb"`.
|
||||
|
||||
---
|
||||
|
||||
### 3. Search query (`::text` cast and `ILIKE`)
|
||||
|
||||
The full-text search fallback for PostgreSQL used two PostgreSQL-specific SQL features:
|
||||
|
||||
- `ci.data::text` — PostgreSQL shorthand for casting to text. In MySQL the `data` column is already `TEXT`, so no cast is needed.
|
||||
- `ILIKE` — PostgreSQL case-insensitive LIKE. MySQL doesn't have it, but `LIKE` is case-insensitive by default with `utf8mb4_general_ci` collation.
|
||||
|
||||
A MySQL/MariaDB branch was added in `conversation_store/sqlalchemy_store.py` `search()` that uses plain `LIKE` with no cast.
|
||||
|
||||
---
|
||||
|
||||
### 4. Transaction isolation (`db/utils.py`)
|
||||
|
||||
MySQL and MariaDB default to `REPEATABLE READ` transaction isolation. This caused error 1020 ("Record has changed since last read") when two concurrent transactions touched the same conversation row — for example, the server auto-generating a title while the session was also being updated.
|
||||
|
||||
PostgreSQL defaults to `READ COMMITTED`, which doesn't have this problem. We explicitly set `READ COMMITTED` for MySQL/MariaDB in `_create_engine()`:
|
||||
|
||||
```python
|
||||
**({"isolation_level": "READ COMMITTED"} if is_mysql else {})
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Migration changes
|
||||
|
||||
Alembic migrations were written and tested only against SQLite and PostgreSQL. Running them fresh on MySQL/MariaDB exposed four categories of incompatibility:
|
||||
|
||||
---
|
||||
|
||||
### Migration issue 1: Drop index before drop table (FK constraint)
|
||||
|
||||
**Affected migrations:**
|
||||
- `e3b1f2a4c9d7_drop_pending_tool_calls_table.py`
|
||||
- `b9c1d2e3f4a5_drop_tasks_table.py`
|
||||
|
||||
**What failed:** MySQL/MariaDB refuses to drop an index that a foreign key constraint depends on while the constraint still exists. SQLite and PostgreSQL handle this automatically.
|
||||
|
||||
**Example error:**
|
||||
```
|
||||
(1553, "Cannot drop index 'ix_pending_tool_calls_task_id':
|
||||
needed in a foreign key constraint")
|
||||
```
|
||||
|
||||
**Fix:** For MySQL/MariaDB, skip the index drops and just call `op.drop_table()` directly. MySQL removes all indexes and FK constraints when a table is dropped — so the separate `drop_index` calls are redundant anyway.
|
||||
|
||||
```python
|
||||
def upgrade() -> None:
|
||||
bind = op.get_bind()
|
||||
if bind.dialect.name in ("mysql", "mariadb"):
|
||||
op.drop_table("pending_tool_calls") # drops indexes + FKs automatically
|
||||
return
|
||||
op.drop_index("ix_pending_tool_calls_task_id", ...)
|
||||
op.drop_table("pending_tool_calls")
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Migration issue 2: FK constraint blocks index drop during ALTER TABLE
|
||||
|
||||
**Affected migration:** `a3b4c5d6e7f8_add_session_policy_columns.py`
|
||||
|
||||
**What failed:** This migration restructures the `policies` table — it drops the `agent_id` column (which had a FK to `agents.id`) and adds a `session_id` column. Part of the restructure drops `ix_policies_agent_id`, but MySQL won't drop that index while the FK on `agent_id` is still alive.
|
||||
|
||||
**Fix:** For MySQL/MariaDB, dynamically discover the auto-generated FK constraint name using SQLAlchemy's inspector and drop it first before the main batch alter:
|
||||
|
||||
```python
|
||||
if bind.dialect.name in ("mysql", "mariadb"):
|
||||
from sqlalchemy import inspect as sa_inspect
|
||||
fks = sa_inspect(bind).get_foreign_keys("policies")
|
||||
agent_fk = next((fk for fk in fks if "agent_id" in fk["constrained_columns"]), None)
|
||||
if agent_fk and agent_fk.get("name"):
|
||||
with op.batch_alter_table("policies") as pre_op:
|
||||
pre_op.drop_constraint(agent_fk["name"], type_="foreignkey")
|
||||
```
|
||||
|
||||
MySQL auto-names FK constraints (e.g. `policies_ibfk_1`) so the name can't be hardcoded — it has to be discovered at migration time.
|
||||
|
||||
---
|
||||
|
||||
### Migration issue 3: `CAST(x AS BIGINT)` syntax
|
||||
|
||||
**Affected migration:** `ecc0e25727b0_add_updated_at_to_comments.py`
|
||||
|
||||
**What failed:** A raw SQL string used PostgreSQL/SQLite syntax for integer casting:
|
||||
|
||||
```sql
|
||||
UPDATE comments SET updated_at = CAST(created_at AS BIGINT) * 1000000
|
||||
```
|
||||
|
||||
MySQL/MariaDB uses `SIGNED` or `UNSIGNED` instead of `BIGINT` inside `CAST()`.
|
||||
|
||||
**Fix:** Detect the dialect and use the right keyword:
|
||||
|
||||
```python
|
||||
cast_expr = "CAST(created_at AS SIGNED)" if mysql else "CAST(created_at AS BIGINT)"
|
||||
op.execute(f"UPDATE comments SET updated_at = {cast_expr} * 1000000 WHERE updated_at IS NULL")
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Migration issue 4: `CHECK` constraint syntax in batch mode
|
||||
|
||||
**Affected migration:** `b8c4f2e7a9d1_add_workspace_to_conversations.py`
|
||||
|
||||
**What failed:** This migration adds a `CHECK` constraint to enforce that `workspace` must be set when `host_id` is set. Alembic's `batch_alter_table` emits the CHECK in a form that MariaDB 11 rejects:
|
||||
|
||||
```
|
||||
(1901, "Function or expression 'host_id' cannot be used in the CHECK clause")
|
||||
```
|
||||
|
||||
**Fix:** Skip the CHECK constraint entirely for MySQL/MariaDB. The application already enforces this rule before writing — the DB constraint is a belt-and-suspenders guard that MySQL/MariaDB can't provide here.
|
||||
|
||||
```python
|
||||
if bind.dialect.name not in ("mysql", "mariadb"):
|
||||
batch_op.create_check_constraint(
|
||||
"ck_conversations_workspace_required_for_host",
|
||||
"host_id IS NULL OR workspace IS NOT NULL",
|
||||
)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### New migration: `o1a2b3c4d5e6_mysql_mariadb_partial_index.py`
|
||||
|
||||
**Why it exists:** The `agents` table has a partial unique index on `name WHERE session_id IS NULL` — meaning only template agents must have unique names. Session-scoped agents (one per session, e.g. "claude-native") can share names across sessions freely.
|
||||
|
||||
MySQL/MariaDB ignores the `WHERE` clause on `Index()` and creates a full unique index on `name` instead. This would block every second session from loading a built-in agent, since two session agents can't share the name "claude-native" under a full unique index.
|
||||
|
||||
**What MySQL/MariaDB can't do:** Neither supports partial/filtered indexes (`CREATE INDEX ... WHERE ...`). This is a hard limitation of the database engine.
|
||||
|
||||
**Fix:** For MySQL/MariaDB, the migration drops the over-restrictive full unique index entirely. Template-agent name uniqueness is enforced at the application layer instead.
|
||||
|
||||
---
|
||||
|
||||
## What MySQL/MariaDB doesn't get vs PostgreSQL
|
||||
|
||||
These are MySQL/MariaDB limitations — nothing is regressed for PostgreSQL users.
|
||||
|
||||
| | PostgreSQL | MySQL/MariaDB |
|
||||
|--|-----------|---------------|
|
||||
| Partial unique index on `agents.name` | ✓ DB-level | App-level only |
|
||||
| `CHECK` constraint on workspace/host_id | ✓ DB-level | App-level only |
|
||||
| Full-text search | ILIKE fallback | LIKE fallback (same result) |
|
||||
|
||||
---
|
||||
|
||||
## Files changed
|
||||
|
||||
| File | What changed |
|
||||
|------|-------------|
|
||||
| `pyproject.toml` | Added `mysql` optional extra with PyMySQL |
|
||||
| `deploy/docker/Dockerfile` | Added PyMySQL install in server-builder stage |
|
||||
| `deploy/docker/docker-compose.mariadb.yaml` | New compose file for MariaDB + omnigent |
|
||||
| `deploy/docker/MARIADB.md` | Quickstart docs for MariaDB setup |
|
||||
| `omnigent/db/utils.py` | `READ COMMITTED` isolation for MySQL; `_create_engine()` |
|
||||
| `omnigent/stores/permission_store/sqlalchemy_store.py` | `grant()`, `ensure_user()` — MySQL upsert branches |
|
||||
| `omnigent/stores/conversation_store/sqlalchemy_store.py` | 3 upsert sites, LIKE search, dialect gate expansions |
|
||||
| `omnigent/db/migrations/versions/e3b1f2a4c9d7_*` | Drop table directly on MySQL instead of dropping indexes first |
|
||||
| `omnigent/db/migrations/versions/b9c1d2e3f4a5_*` | Same fix for tasks table |
|
||||
| `omnigent/db/migrations/versions/a3b4c5d6e7f8_*` | Drop FK before index drop on MySQL |
|
||||
| `omnigent/db/migrations/versions/b8c4f2e7a9d1_*` | Skip CHECK constraint on MySQL/MariaDB |
|
||||
| `omnigent/db/migrations/versions/ecc0e25727b0_*` | `CAST(x AS SIGNED)` instead of `CAST(x AS BIGINT)` |
|
||||
| `omnigent/db/migrations/versions/o1a2b3c4d5e6_*` | New migration: drop over-restrictive unique index on MySQL |
|
||||
|
||||
---
|
||||
|
||||
## Rule of thumb for future migrations
|
||||
|
||||
Any time a new migration does one of these things, add a MySQL/MariaDB check:
|
||||
|
||||
1. **`drop_index` followed by `drop_table`** → just `drop_table` for MySQL (it drops everything)
|
||||
2. **`drop_index` on a column that has a FK** → drop the FK first on MySQL using `sa_inspect`
|
||||
3. **Raw SQL with `CAST(x AS BIGINT)`** → use `SIGNED` for MySQL
|
||||
4. **`create_check_constraint` in `batch_alter_table`** → guard with `if dialect not in ("mysql", "mariadb")`
|
||||
5. **`op.execute()` with PostgreSQL-specific syntax** (`::type`, `ILIKE`, `RETURNING`, `ON CONFLICT`) → add a MySQL branch
|
||||
Generated
+23
-8
@@ -6746,7 +6746,6 @@
|
||||
"version": "19.2.17",
|
||||
"resolved": "https://registry.npmjs.org/@types/react/-/react-19.2.17.tgz",
|
||||
"integrity": "sha512-MXfmqaVPEVgkBT/aY0aGCkRWWtByiYQXo3xdQ8r5RzuFrPiRn8Gar2tQdXSUQ2GKV3bkXckek89V8wQBY2Q/Aw==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"csstype": "^3.2.2"
|
||||
@@ -6756,7 +6755,6 @@
|
||||
"version": "19.2.3",
|
||||
"resolved": "https://registry.npmjs.org/@types/react-dom/-/react-dom-19.2.3.tgz",
|
||||
"integrity": "sha512-jp2L/eY6fn+KgVVQAOqYItbF0VY/YApe5Mz2F0aykSO8gx31bYCZyvSeYxCHKvzHG5eZjc+zyaS5BrBWya2+kQ==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"peerDependencies": {
|
||||
"@types/react": "^19.2.0"
|
||||
@@ -8097,6 +8095,15 @@
|
||||
"node": ">=10"
|
||||
}
|
||||
},
|
||||
"node_modules/cosmiconfig/node_modules/yaml": {
|
||||
"version": "1.10.3",
|
||||
"resolved": "https://npm-proxy.cloud.databricks.com/yaml/-/yaml-1.10.3.tgz",
|
||||
"integrity": "sha512-vIYeF1u3CjlhAFekPPAk2h/Kv4T3mAkMox5OymRiJQB0spDP10LHvt+K7G9Ny6NuuMAb25/6n1qyUjAcGNf/AA==",
|
||||
"license": "ISC",
|
||||
"engines": {
|
||||
"node": ">= 6"
|
||||
}
|
||||
},
|
||||
"node_modules/cross-spawn": {
|
||||
"version": "7.0.6",
|
||||
"resolved": "https://registry.npmjs.org/cross-spawn/-/cross-spawn-7.0.6.tgz",
|
||||
@@ -16253,7 +16260,6 @@
|
||||
"version": "4.3.0",
|
||||
"resolved": "https://registry.npmjs.org/tailwindcss/-/tailwindcss-4.3.0.tgz",
|
||||
"integrity": "sha512-y6nxMGB1nMW9R6k96e5gdIFzcfL/gTJRNaqGes1YvkLnPVXzWgbqFF2yLC0T8G774n24cx3Pe8XrKoniCOAH+Q==",
|
||||
"dev": true,
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/tapable": {
|
||||
@@ -16544,7 +16550,7 @@
|
||||
"version": "6.0.3",
|
||||
"resolved": "https://registry.npmjs.org/typescript/-/typescript-6.0.3.tgz",
|
||||
"integrity": "sha512-y2TvuxSZPDyQakkFRPZHKFm+KKVqIisdg9/CZwm9ftvKXLP8NRWj38/ODjNbr43SsoXqNuAisEf1GdCxqWcdBw==",
|
||||
"dev": true,
|
||||
"devOptional": true,
|
||||
"license": "Apache-2.0",
|
||||
"bin": {
|
||||
"tsc": "bin/tsc",
|
||||
@@ -17353,12 +17359,21 @@
|
||||
"license": "ISC"
|
||||
},
|
||||
"node_modules/yaml": {
|
||||
"version": "1.10.3",
|
||||
"resolved": "https://registry.npmjs.org/yaml/-/yaml-1.10.3.tgz",
|
||||
"integrity": "sha512-vIYeF1u3CjlhAFekPPAk2h/Kv4T3mAkMox5OymRiJQB0spDP10LHvt+K7G9Ny6NuuMAb25/6n1qyUjAcGNf/AA==",
|
||||
"version": "2.9.0",
|
||||
"resolved": "https://npm-proxy.cloud.databricks.com/yaml/-/yaml-2.9.0.tgz",
|
||||
"integrity": "sha512-2AvhNX3mb8zd6Zy7INTtSpl1F15HW6Wnqj0srWlkKLcpYl/gMIMJiyuGq2KeI2YFxUPjdlB+3Lc10seMLtL4cA==",
|
||||
"dev": true,
|
||||
"license": "ISC",
|
||||
"optional": true,
|
||||
"peer": true,
|
||||
"bin": {
|
||||
"yaml": "bin.mjs"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">= 6"
|
||||
"node": ">= 14.6"
|
||||
},
|
||||
"funding": {
|
||||
"url": "https://github.com/sponsors/eemeli"
|
||||
}
|
||||
},
|
||||
"node_modules/yargs": {
|
||||
|
||||
@@ -145,6 +145,11 @@ RUN test -f ./omnigent/server/static/web-ui/index.html \
|
||||
# [databricks] extra in pyproject — so add it explicitly here.
|
||||
RUN uv pip install --no-cache-dir --index-url ${PYPI_INDEX_URL} 'psycopg[binary]>=3.1,<4'
|
||||
|
||||
# PyMySQL: pure-Python MySQL/MariaDB driver. Installed unconditionally so
|
||||
# the server image supports both Postgres (psycopg above) and MariaDB
|
||||
# (deploy/docker/docker-compose.mariadb.yaml) without a separate build.
|
||||
RUN uv pip install --no-cache-dir --index-url ${PYPI_INDEX_URL} 'PyMySQL>=1.1,<2'
|
||||
|
||||
# Optional managed-sandbox provider extras for the SERVER (the launcher imports
|
||||
# the provider SDK — e.g. the kubernetes client for `sandbox.provider:
|
||||
# kubernetes`). Off by default; the runner host image needs none of these. Build
|
||||
|
||||
@@ -0,0 +1,75 @@
|
||||
# Running Omnigent with MariaDB
|
||||
|
||||
This directory contains `docker-compose.mariadb.yaml` for running the omnigent server locally against **MariaDB 11** instead of PostgreSQL.
|
||||
|
||||
## Why MariaDB (not vanilla MySQL)?
|
||||
|
||||
- **Partial indexes**: MariaDB 10.5.2+ supports `CREATE UNIQUE INDEX ... WHERE ...` on InnoDB, which the omnigent schema requires for the agents table. MariaDB 11 satisfies this.
|
||||
- **Open-source**: MariaDB is GPL-licensed with no commercial restrictions.
|
||||
- **Drop-in compatible**: Uses the same `mysql+pymysql://` SQLAlchemy driver as MySQL 8.
|
||||
|
||||
## Quickstart
|
||||
|
||||
```bash
|
||||
cd deploy/docker
|
||||
|
||||
# 1. Set required env vars (copy from example if you haven't already)
|
||||
cp .env.example .env
|
||||
# Edit .env and add:
|
||||
# MARIADB_PASSWORD=<choose a password>
|
||||
# MARIADB_ROOT_PASSWORD=<choose a root password>
|
||||
|
||||
# 2. Start MariaDB + omnigent server (builds the image on first run)
|
||||
docker compose -f docker-compose.mariadb.yaml up -d --build
|
||||
|
||||
# 3. Check logs — first boot prints the admin password
|
||||
docker compose -f docker-compose.mariadb.yaml logs omnigent
|
||||
|
||||
# 4. Open the UI
|
||||
open http://localhost:8000
|
||||
```
|
||||
|
||||
## Connecting a Runner
|
||||
|
||||
After the server is up and you've logged in, the UI will display a command to start a local runner. The runner runs **on your local machine** (not in Docker) so it has access to your filesystem, terminal, and tools. It connects back to the server via WebSocket.
|
||||
|
||||
## Connection String
|
||||
|
||||
```
|
||||
mysql+pymysql://omnigent:<password>@localhost:3306/omnigent?charset=utf8mb4
|
||||
```
|
||||
|
||||
- `mysql+pymysql://` — SQLAlchemy dialect using the PyMySQL driver
|
||||
- `charset=utf8mb4` — required for full Unicode + emoji support in MariaDB
|
||||
|
||||
## Known Limitations vs PostgreSQL
|
||||
|
||||
| Feature | PostgreSQL | MariaDB |
|
||||
|---------|-----------|---------|
|
||||
| Full-text search | ILIKE fallback | LIKE fallback (utf8mb4 case-insensitive) |
|
||||
| Upsert | `ON CONFLICT DO UPDATE` | `ON DUPLICATE KEY UPDATE` |
|
||||
| Partial indexes | Native | Requires MariaDB 10.5.2+ (MariaDB 11 ✓) |
|
||||
|
||||
**No FTS**: MariaDB does not use the SQLite FTS5 virtual table. Search falls back to a `LIKE %query%` scan on the `data` column. This is slower on large conversation histories but functionally correct. Adding native `FULLTEXT` index support is a future improvement.
|
||||
|
||||
## Code Changes Made
|
||||
|
||||
These files were modified to add MariaDB dialect support:
|
||||
|
||||
| File | Change |
|
||||
|------|--------|
|
||||
| `pyproject.toml` | Added `mysql` optional extra with `PyMySQL>=1.1,<2` |
|
||||
| `deploy/docker/Dockerfile` | Added `PyMySQL` install in server-builder stage |
|
||||
| `omnigent/stores/permission_store/sqlalchemy_store.py` | `grant()` and `ensure_user()`: added `mysql`/`mariadb` upsert branch |
|
||||
| `omnigent/stores/conversation_store/sqlalchemy_store.py` | 3 upsert sites + search query + dialect gate expansions |
|
||||
| `omnigent/db/migrations/versions/o1a2b3c4d5e6_mysql_mariadb_partial_index.py` | Fixes agents.name unique partial index for MariaDB |
|
||||
|
||||
## Stopping / Resetting
|
||||
|
||||
```bash
|
||||
# Stop containers (data persists in volumes)
|
||||
docker compose -f docker-compose.mariadb.yaml down
|
||||
|
||||
# Full reset including database volume
|
||||
docker compose -f docker-compose.mariadb.yaml down -v
|
||||
```
|
||||
@@ -0,0 +1,100 @@
|
||||
# Omnigent server + MariaDB (external-runner mode).
|
||||
#
|
||||
# Quickstart (single-user dev):
|
||||
#
|
||||
# cd deploy/docker
|
||||
# cp .env.example .env
|
||||
# # Edit .env — set at minimum:
|
||||
# # MARIADB_PASSWORD=<choose a password>
|
||||
# # MARIADB_ROOT_PASSWORD=<choose a root password>
|
||||
# docker compose -f docker-compose.mariadb.yaml up -d --build
|
||||
# open http://localhost:8000 # first boot prints admin credentials to logs
|
||||
#
|
||||
# Auth: built-in accounts mode by default (same as the Postgres compose).
|
||||
# See docker-compose.yaml comments for OIDC / header / single-user options.
|
||||
#
|
||||
# Runner: after the server is up, open http://localhost:8000 — the UI
|
||||
# will prompt you with a command to start a local runner that connects
|
||||
# back to this server via WebSocket.
|
||||
#
|
||||
# See MARIADB.md in this directory for full details and known limitations.
|
||||
|
||||
name: omnigent-mariadb
|
||||
|
||||
services:
|
||||
mariadb:
|
||||
image: mariadb:11
|
||||
restart: unless-stopped
|
||||
environment:
|
||||
MARIADB_DATABASE: ${MARIADB_DB:-omnigent}
|
||||
MARIADB_USER: ${MARIADB_USER:-omnigent}
|
||||
MARIADB_PASSWORD: ${MARIADB_PASSWORD:?set MARIADB_PASSWORD in .env}
|
||||
MARIADB_ROOT_PASSWORD: ${MARIADB_ROOT_PASSWORD:?set MARIADB_ROOT_PASSWORD in .env}
|
||||
volumes:
|
||||
- mariadb-data:/var/lib/mysql
|
||||
healthcheck:
|
||||
test: ["CMD", "mariadb-admin", "ping", "-h", "localhost",
|
||||
"-u", "${MARIADB_USER:-omnigent}",
|
||||
"-p${MARIADB_PASSWORD}"]
|
||||
interval: 10s
|
||||
timeout: 5s
|
||||
retries: 5
|
||||
|
||||
omnigent:
|
||||
# Pre-built image published to GHCR on every main-branch merge.
|
||||
# `docker compose pull` fetches the latest; pin OMNIGENT_IMAGE_TAG
|
||||
# to a sha-<short> or vX.Y.Z tag for reproducible deploys.
|
||||
image: ${OMNIGENT_IMAGE:-ghcr.io/omnigent-ai/omnigent-server}:${OMNIGENT_IMAGE_TAG:-latest}
|
||||
# Local-build fallback: used when the image isn't already pulled AND
|
||||
# you run `docker compose up --build` explicitly.
|
||||
build:
|
||||
context: ../..
|
||||
dockerfile: deploy/docker/Dockerfile
|
||||
args:
|
||||
PYPI_INDEX_URL: ${PYPI_INDEX_URL:-https://pypi.org/simple}
|
||||
restart: unless-stopped
|
||||
depends_on:
|
||||
mariadb:
|
||||
condition: service_healthy
|
||||
environment:
|
||||
# mysql+pymysql:// tells SQLAlchemy to use the PyMySQL driver.
|
||||
# charset=utf8mb4 is required for full Unicode + emoji support.
|
||||
DATABASE_URL: mysql+pymysql://${MARIADB_USER:-omnigent}:${MARIADB_PASSWORD}@mariadb:3306/${MARIADB_DB:-omnigent}?charset=utf8mb4
|
||||
ARTIFACT_DIR: /data/artifacts
|
||||
HOST: 0.0.0.0
|
||||
PORT: "8000"
|
||||
# Pin to the persistent volume so admin credentials survive restarts.
|
||||
OMNIGENT_ADMIN_CREDENTIALS_PATH: /data/admin-credentials
|
||||
|
||||
# ── Auth ─────────────────────────────────────────
|
||||
OMNIGENT_AUTH_ENABLED: "${OMNIGENT_AUTH_ENABLED:-1}"
|
||||
OMNIGENT_AUTH_PROVIDER: "${OMNIGENT_AUTH_PROVIDER:-}"
|
||||
|
||||
# OIDC (leave unset to use built-in accounts mode)
|
||||
OMNIGENT_OIDC_ISSUER: "${OMNIGENT_OIDC_ISSUER:-}"
|
||||
OMNIGENT_OIDC_CLIENT_ID: "${OMNIGENT_OIDC_CLIENT_ID:-}"
|
||||
OMNIGENT_OIDC_CLIENT_SECRET: "${OMNIGENT_OIDC_CLIENT_SECRET:-}"
|
||||
OMNIGENT_OIDC_COOKIE_SECRET: "${OMNIGENT_OIDC_COOKIE_SECRET:-}"
|
||||
OMNIGENT_OIDC_SCOPES: "${OMNIGENT_OIDC_SCOPES:-}"
|
||||
OMNIGENT_OIDC_SESSION_TTL_HOURS: "${OMNIGENT_OIDC_SESSION_TTL_HOURS:-8}"
|
||||
OMNIGENT_OIDC_ALLOWED_DOMAINS: "${OMNIGENT_OIDC_ALLOWED_DOMAINS:-}"
|
||||
OMNIGENT_OIDC_LOGOUT_REDIRECT_URI: "${OMNIGENT_OIDC_LOGOUT_REDIRECT_URI:-}"
|
||||
OMNIGENT_OIDC_ALLOW_INVITES: "${OMNIGENT_OIDC_ALLOW_INVITES:-}"
|
||||
OMNIGENT_DOMAIN: "${OMNIGENT_DOMAIN:-}"
|
||||
|
||||
# Built-in accounts (default when no OIDC issuer is set)
|
||||
OMNIGENT_ACCOUNTS_COOKIE_SECRET: "${OMNIGENT_ACCOUNTS_COOKIE_SECRET:-}"
|
||||
OMNIGENT_ACCOUNTS_BASE_URL: "${OMNIGENT_ACCOUNTS_BASE_URL:-}"
|
||||
OMNIGENT_ACCOUNTS_INIT_ADMIN_PASSWORD: "${OMNIGENT_ACCOUNTS_INIT_ADMIN_PASSWORD:-}"
|
||||
OMNIGENT_ACCOUNTS_SESSION_TTL_HOURS: "${OMNIGENT_ACCOUNTS_SESSION_TTL_HOURS:-8}"
|
||||
OMNIGENT_ACCOUNTS_INVITE_TTL_HOURS: "${OMNIGENT_ACCOUNTS_INVITE_TTL_HOURS:-72}"
|
||||
OMNIGENT_ACCOUNTS_MAGIC_TTL_MINUTES: "${OMNIGENT_ACCOUNTS_MAGIC_TTL_MINUTES:-10}"
|
||||
OMNIGENT_ACCOUNTS_AUTO_OPEN: "${OMNIGENT_ACCOUNTS_AUTO_OPEN:-0}"
|
||||
volumes:
|
||||
- artifact-data:/data
|
||||
ports:
|
||||
- "${OMNIGENT_PORT:-8000}:8000"
|
||||
|
||||
volumes:
|
||||
mariadb-data:
|
||||
artifact-data:
|
||||
@@ -21,6 +21,21 @@ down_revision: str | None = "b2c3d4e5f6a7"
|
||||
|
||||
def upgrade() -> None:
|
||||
"""Restructure policies table for session-scoped handler policies."""
|
||||
bind = op.get_bind()
|
||||
if bind.dialect.name in ("mysql", "mariadb"):
|
||||
# MySQL/MariaDB: cannot drop ix_policies_agent_id while the FK
|
||||
# constraint on agent_id (auto-named by MySQL, e.g. policies_ibfk_1)
|
||||
# still exists. Discover and drop the FK first, then proceed normally.
|
||||
from sqlalchemy import inspect as sa_inspect
|
||||
|
||||
fks = sa_inspect(bind).get_foreign_keys("policies")
|
||||
agent_fk = next(
|
||||
(fk for fk in fks if "agent_id" in fk["constrained_columns"]), None
|
||||
)
|
||||
if agent_fk and agent_fk.get("name"):
|
||||
with op.batch_alter_table("policies") as pre_op:
|
||||
pre_op.drop_constraint(agent_fk["name"], type_="foreignkey")
|
||||
|
||||
with op.batch_alter_table("policies") as batch_op:
|
||||
batch_op.add_column(sa.Column("session_id", sa.String(64), nullable=True))
|
||||
batch_op.add_column(sa.Column("handler", sa.Text(), nullable=True))
|
||||
|
||||
@@ -43,12 +43,18 @@ def upgrade() -> None:
|
||||
in place — alembic copies the table, applies the column + check
|
||||
in one shot, and renames it back.
|
||||
"""
|
||||
bind = op.get_bind()
|
||||
with op.batch_alter_table("conversations") as batch_op:
|
||||
batch_op.add_column(sa.Column("workspace", sa.String(length=2048), nullable=True))
|
||||
batch_op.create_check_constraint(
|
||||
"ck_conversations_workspace_required_for_host",
|
||||
"host_id IS NULL OR workspace IS NOT NULL",
|
||||
)
|
||||
if bind.dialect.name not in ("mysql", "mariadb"):
|
||||
# MySQL/MariaDB: Alembic's batch_alter_table emits a CHECK
|
||||
# constraint syntax that MariaDB 11 rejects (error 1901).
|
||||
# The constraint is a data-integrity guard; the application
|
||||
# enforces it, so skipping it here is safe.
|
||||
batch_op.create_check_constraint(
|
||||
"ck_conversations_workspace_required_for_host",
|
||||
"host_id IS NULL OR workspace IS NOT NULL",
|
||||
)
|
||||
# Index + FK on host_id, folded into this batch since it already
|
||||
# recreates the table (avoids a second rebuild). FK targets
|
||||
# hosts.host_id (its uq_hosts_host_id unique column); ON DELETE
|
||||
|
||||
@@ -32,6 +32,12 @@ depends_on: str | Sequence[str] | None = None
|
||||
|
||||
def upgrade() -> None:
|
||||
"""Drop the tasks table and all of its indexes."""
|
||||
bind = op.get_bind()
|
||||
if bind.dialect.name in ("mysql", "mariadb"):
|
||||
# MySQL/MariaDB: cannot drop indexes that support FK constraints while
|
||||
# the constraints still exist. DROP TABLE removes everything atomically.
|
||||
op.drop_table("tasks")
|
||||
return
|
||||
with op.batch_alter_table("tasks") as batch_op:
|
||||
batch_op.drop_index("ix_tasks_conversation_id")
|
||||
batch_op.drop_index("ix_tasks_agent_id")
|
||||
|
||||
@@ -25,6 +25,14 @@ depends_on: str | Sequence[str] | None = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
bind = op.get_bind()
|
||||
if bind.dialect.name in ("mysql", "mariadb"):
|
||||
# MySQL/MariaDB: cannot drop an index that supports a foreign key
|
||||
# constraint while the constraint still exists. Since we're dropping
|
||||
# the whole table anyway, skip straight to DROP TABLE — MySQL removes
|
||||
# all indexes and FK constraints with the table automatically.
|
||||
op.drop_table("pending_tool_calls")
|
||||
return
|
||||
op.drop_index("ix_pending_tool_calls_task_id", table_name="pending_tool_calls")
|
||||
op.drop_index("ix_pending_tool_calls_root_task_id", table_name="pending_tool_calls")
|
||||
op.drop_table("pending_tool_calls")
|
||||
|
||||
@@ -39,8 +39,14 @@ def upgrade() -> None:
|
||||
op.add_column("comments", sa.Column("updated_at", sa.BigInteger(), nullable=True))
|
||||
# CAST first: created_at is int4 on PostgreSQL and int4 * int4 stays
|
||||
# int4, so epoch-seconds * 1e6 overflows on any table with rows.
|
||||
# MySQL/MariaDB uses CAST(x AS SIGNED) not CAST(x AS BIGINT).
|
||||
bind = op.get_bind()
|
||||
if bind.dialect.name in ("mysql", "mariadb"):
|
||||
cast_expr = "CAST(created_at AS SIGNED)"
|
||||
else:
|
||||
cast_expr = "CAST(created_at AS BIGINT)"
|
||||
op.execute(
|
||||
"UPDATE comments SET updated_at = CAST(created_at AS BIGINT) * 1000000 "
|
||||
f"UPDATE comments SET updated_at = {cast_expr} * 1000000 "
|
||||
"WHERE updated_at IS NULL"
|
||||
)
|
||||
with op.batch_alter_table("comments") as batch_op:
|
||||
|
||||
@@ -0,0 +1,52 @@
|
||||
"""mysql/mariadb: fix agents.name unique index
|
||||
|
||||
Revision ID: o1a2b3c4d5e6
|
||||
Revises: n1a2b3c4d5e6
|
||||
Create Date: 2026-06-25 00:00:00.000000
|
||||
|
||||
On SQLite and PostgreSQL the ``ix_agents_template_name`` index is created
|
||||
as a *partial* unique index (``WHERE session_id IS NULL``), meaning only
|
||||
template agents (session_id IS NULL) must have unique names. Session-scoped
|
||||
agents share names across sessions freely (e.g. every session loads a
|
||||
"claude-native" agent).
|
||||
|
||||
MySQL and MariaDB silently ignore the ``postgresql_where`` / ``sqlite_where``
|
||||
kwargs on ``Index()`` and create a *full* unique index on ``name`` instead.
|
||||
That would block multiple sessions from loading the same built-in agent name
|
||||
(e.g. "claude-native"), crashing session startup from the second session on.
|
||||
|
||||
Neither MySQL nor MariaDB support partial/filtered indexes (WHERE clause on
|
||||
CREATE INDEX). This migration therefore drops the over-restrictive full
|
||||
unique index on MySQL/MariaDB. Template-agent name uniqueness is enforced
|
||||
at the application level (the agent store checks for name conflicts before
|
||||
inserting a template agent).
|
||||
|
||||
SQLite and PostgreSQL already have the correct partial index from the
|
||||
original migration (``d7a6b3c91f48``); this migration is a no-op for them.
|
||||
"""
|
||||
|
||||
from alembic import op
|
||||
|
||||
revision = "o1a2b3c4d5e6"
|
||||
down_revision = "n1a2b3c4d5e6"
|
||||
branch_labels = None
|
||||
depends_on = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
bind = op.get_bind()
|
||||
if bind.dialect.name not in ("mysql", "mariadb"):
|
||||
# SQLite and PostgreSQL already have the correct partial index.
|
||||
return
|
||||
|
||||
# MySQL/MariaDB created a full unique index on agents.name (ignoring the
|
||||
# WHERE clause). Drop it — partial indexes are unsupported on these
|
||||
# dialects. Template-agent name uniqueness is enforced by the application
|
||||
# layer instead.
|
||||
op.drop_index("ix_agents_template_name", table_name="agents", if_exists=True)
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
# The original migration (d7a6b3c91f48) owns the canonical definition;
|
||||
# rolling back this migration is a no-op.
|
||||
pass
|
||||
@@ -261,6 +261,7 @@ def _create_engine(db_uri: str) -> Engine:
|
||||
pool_recycle = (
|
||||
_LAKEBASE_POOL_RECYCLE_SECONDS if token_provider else _SERVER_POOL_RECYCLE_SECONDS
|
||||
)
|
||||
is_mysql = db_uri.startswith("mysql")
|
||||
engine = create_engine(
|
||||
db_uri,
|
||||
# Verify connections are alive before checking them out
|
||||
@@ -283,6 +284,11 @@ def _create_engine(db_uri: str) -> Engine:
|
||||
# blocking indefinitely; surfaces real saturation as an
|
||||
# error rather than a hang.
|
||||
pool_timeout=10,
|
||||
# MySQL/MariaDB default isolation is REPEATABLE READ, which raises
|
||||
# error 1020 ("Record has changed since last read") when concurrent
|
||||
# transactions update the same row. READ COMMITTED matches
|
||||
# PostgreSQL's default and eliminates this false conflict.
|
||||
**({"isolation_level": "READ COMMITTED"} if is_mysql else {}),
|
||||
)
|
||||
if token_provider:
|
||||
_install_lakebase_token_refresh(engine, token_provider)
|
||||
|
||||
@@ -271,13 +271,13 @@ def _upsert_labels(
|
||||
}
|
||||
for key, value in updates.items()
|
||||
]
|
||||
if dialect in ("sqlite", "postgresql"):
|
||||
if dialect in ("sqlite", "postgresql", "mysql", "mariadb"):
|
||||
_dialect_upsert_labels(session, dialect, rows)
|
||||
return
|
||||
# Generic dialect fallback — SELECT-then-INSERT/UPDATE in
|
||||
# one transaction. Safe for the v1 "one active workflow
|
||||
# per conversation" invariant (POLICIES.md §10); the
|
||||
# SQLite / Postgres dialect-specific paths above give
|
||||
# SQLite / Postgres / MySQL dialect-specific paths above give
|
||||
# true atomic UPSERT for the supported production dbs.
|
||||
for row in rows:
|
||||
existing = session.get(
|
||||
@@ -302,39 +302,57 @@ def _dialect_upsert_labels(
|
||||
rows: list[dict[str, Any]],
|
||||
) -> None:
|
||||
"""
|
||||
Dialect-specific UPSERT path for SQLite / PostgreSQL.
|
||||
Dialect-specific UPSERT path for SQLite / PostgreSQL / MySQL / MariaDB.
|
||||
|
||||
Extracted from ``_upsert_labels`` so the two branches
|
||||
Extracted from ``_upsert_labels`` so the branches
|
||||
(which use different ``insert`` builders producing
|
||||
incompatible type variances at the mypy level) each live
|
||||
in their own narrow scope. The outer function selects the
|
||||
branch; this one executes it.
|
||||
|
||||
:param session: Active SQLAlchemy session.
|
||||
:param dialect: ``"sqlite"`` or ``"postgresql"`` (the
|
||||
outer function gates all other dialects onto the
|
||||
generic fallback path).
|
||||
:param dialect: ``"sqlite"``, ``"postgresql"``, ``"mysql"``, or
|
||||
``"mariadb"`` (the outer function gates all other dialects
|
||||
onto the generic fallback path).
|
||||
:param rows: Pre-built row dicts to upsert.
|
||||
"""
|
||||
# Typed as Any to sidestep the mypy variance issue between
|
||||
# the two dialect-specific ``Insert`` classes; the runtime
|
||||
# shape of both classes is identical for our use.
|
||||
# the dialect-specific ``Insert`` classes; the runtime
|
||||
# shape of all classes is identical for our use.
|
||||
stmt: Any
|
||||
if dialect == "sqlite":
|
||||
from sqlalchemy.dialects.sqlite import insert as sqlite_insert
|
||||
|
||||
stmt = sqlite_insert(SqlConversationLabel).values(rows)
|
||||
stmt = stmt.on_conflict_do_update(
|
||||
index_elements=["conversation_id", "key"],
|
||||
set_={
|
||||
"value": stmt.excluded.value,
|
||||
"updated_at": stmt.excluded.updated_at,
|
||||
},
|
||||
)
|
||||
elif dialect in ("mysql", "mariadb"):
|
||||
# MySQL/MariaDB: ON DUPLICATE KEY UPDATE instead of PostgreSQL's
|
||||
# ON CONFLICT DO UPDATE. Both are atomic upserts; the SQLAlchemy
|
||||
# dialect objects differ but the intent is identical.
|
||||
from sqlalchemy.dialects.mysql import insert as mysql_insert
|
||||
|
||||
stmt = mysql_insert(SqlConversationLabel).values(rows)
|
||||
stmt = stmt.on_duplicate_key_update(
|
||||
value=stmt.inserted.value,
|
||||
updated_at=stmt.inserted.updated_at,
|
||||
)
|
||||
else:
|
||||
from sqlalchemy.dialects.postgresql import insert as pg_insert
|
||||
|
||||
stmt = pg_insert(SqlConversationLabel).values(rows)
|
||||
stmt = stmt.on_conflict_do_update(
|
||||
index_elements=["conversation_id", "key"],
|
||||
set_={
|
||||
"value": stmt.excluded.value,
|
||||
"updated_at": stmt.excluded.updated_at,
|
||||
},
|
||||
)
|
||||
stmt = stmt.on_conflict_do_update(
|
||||
index_elements=["conversation_id", "key"],
|
||||
set_={
|
||||
"value": stmt.excluded.value,
|
||||
"updated_at": stmt.excluded.updated_at,
|
||||
},
|
||||
)
|
||||
session.execute(stmt)
|
||||
|
||||
|
||||
@@ -946,7 +964,7 @@ class SqlAlchemyConversationStore(ConversationStore):
|
||||
now = now_epoch()
|
||||
with self._session() as session:
|
||||
dialect = session.bind.dialect.name if session.bind is not None else ""
|
||||
if dialect in ("sqlite", "postgresql"):
|
||||
if dialect in ("sqlite", "postgresql", "mysql", "mariadb"):
|
||||
self._upsert_daily_cost_dialect(session, dialect, user_id, day_utc, delta_usd, now)
|
||||
return
|
||||
# Generic dialect fallback — SELECT-then-INSERT/UPDATE in one
|
||||
@@ -1005,18 +1023,45 @@ class SqlAlchemyConversationStore(ConversationStore):
|
||||
from sqlalchemy.dialects.sqlite import insert as sqlite_insert
|
||||
|
||||
stmt = sqlite_insert(SqlUserDailyCost)
|
||||
stmt = stmt.values(
|
||||
user_id=user_id, day_utc=day_utc, cost_usd=delta_usd, updated_at=now
|
||||
)
|
||||
stmt = stmt.on_conflict_do_update(
|
||||
index_elements=["user_id", "day_utc"],
|
||||
set_={
|
||||
"cost_usd": SqlUserDailyCost.cost_usd + stmt.excluded.cost_usd,
|
||||
"updated_at": stmt.excluded.updated_at,
|
||||
},
|
||||
)
|
||||
elif dialect in ("mysql", "mariadb"):
|
||||
# MySQL/MariaDB: ON DUPLICATE KEY UPDATE instead of PostgreSQL's
|
||||
# ON CONFLICT DO UPDATE. stmt.inserted.cost_usd is the MySQL
|
||||
# equivalent of stmt.excluded.cost_usd — the value from the
|
||||
# attempted INSERT row. Both produce an atomic increment.
|
||||
from sqlalchemy.dialects.mysql import insert as mysql_insert
|
||||
|
||||
stmt = mysql_insert(SqlUserDailyCost)
|
||||
stmt = stmt.values(
|
||||
user_id=user_id, day_utc=day_utc, cost_usd=delta_usd, updated_at=now
|
||||
)
|
||||
stmt = stmt.on_duplicate_key_update(
|
||||
cost_usd=SqlUserDailyCost.cost_usd + stmt.inserted.cost_usd,
|
||||
updated_at=stmt.inserted.updated_at,
|
||||
)
|
||||
else:
|
||||
from sqlalchemy.dialects.postgresql import insert as pg_insert
|
||||
|
||||
stmt = pg_insert(SqlUserDailyCost)
|
||||
stmt = stmt.values(user_id=user_id, day_utc=day_utc, cost_usd=delta_usd, updated_at=now)
|
||||
stmt = stmt.on_conflict_do_update(
|
||||
index_elements=["user_id", "day_utc"],
|
||||
set_={
|
||||
"cost_usd": SqlUserDailyCost.cost_usd + stmt.excluded.cost_usd,
|
||||
"updated_at": stmt.excluded.updated_at,
|
||||
},
|
||||
)
|
||||
stmt = stmt.values(
|
||||
user_id=user_id, day_utc=day_utc, cost_usd=delta_usd, updated_at=now
|
||||
)
|
||||
stmt = stmt.on_conflict_do_update(
|
||||
index_elements=["user_id", "day_utc"],
|
||||
set_={
|
||||
"cost_usd": SqlUserDailyCost.cost_usd + stmt.excluded.cost_usd,
|
||||
"updated_at": stmt.excluded.updated_at,
|
||||
},
|
||||
)
|
||||
session.execute(stmt)
|
||||
|
||||
def get_daily_cost(self, user_id: str, day_utc: str) -> float:
|
||||
@@ -1077,34 +1122,68 @@ class SqlAlchemyConversationStore(ConversationStore):
|
||||
now = now_epoch()
|
||||
with self._session() as session:
|
||||
dialect = session.bind.dialect.name if session.bind is not None else ""
|
||||
if dialect in ("sqlite", "postgresql"):
|
||||
if dialect in ("sqlite", "postgresql", "mysql", "mariadb"):
|
||||
# Typed as Any to sidestep the mypy variance between the
|
||||
# two dialect-specific ``Insert`` classes.
|
||||
# dialect-specific ``Insert`` classes.
|
||||
stmt: Any
|
||||
if dialect == "sqlite":
|
||||
from sqlalchemy.dialects.sqlite import insert as sqlite_insert
|
||||
|
||||
stmt = sqlite_insert(SqlUserDailyCost)
|
||||
stmt = stmt.values(
|
||||
user_id=user_id,
|
||||
day_utc=day_utc,
|
||||
cost_usd=0.0,
|
||||
ask_approved_usd=ask_approved_usd,
|
||||
updated_at=now,
|
||||
)
|
||||
# On conflict touch only the approval (+ stamp) — never
|
||||
# the accumulated cost.
|
||||
stmt = stmt.on_conflict_do_update(
|
||||
index_elements=["user_id", "day_utc"],
|
||||
set_={
|
||||
"ask_approved_usd": stmt.excluded.ask_approved_usd,
|
||||
"updated_at": stmt.excluded.updated_at,
|
||||
},
|
||||
)
|
||||
elif dialect in ("mysql", "mariadb"):
|
||||
# MySQL/MariaDB: ON DUPLICATE KEY UPDATE instead of
|
||||
# PostgreSQL's ON CONFLICT DO UPDATE. Only touches the
|
||||
# approval field, never the accumulated cost.
|
||||
from sqlalchemy.dialects.mysql import insert as mysql_insert
|
||||
|
||||
stmt = mysql_insert(SqlUserDailyCost)
|
||||
stmt = stmt.values(
|
||||
user_id=user_id,
|
||||
day_utc=day_utc,
|
||||
cost_usd=0.0,
|
||||
ask_approved_usd=ask_approved_usd,
|
||||
updated_at=now,
|
||||
)
|
||||
stmt = stmt.on_duplicate_key_update(
|
||||
ask_approved_usd=stmt.inserted.ask_approved_usd,
|
||||
updated_at=stmt.inserted.updated_at,
|
||||
)
|
||||
else:
|
||||
from sqlalchemy.dialects.postgresql import insert as pg_insert
|
||||
|
||||
stmt = pg_insert(SqlUserDailyCost)
|
||||
stmt = stmt.values(
|
||||
user_id=user_id,
|
||||
day_utc=day_utc,
|
||||
cost_usd=0.0,
|
||||
ask_approved_usd=ask_approved_usd,
|
||||
updated_at=now,
|
||||
)
|
||||
# On conflict touch only the approval (+ stamp) — never
|
||||
# the accumulated cost.
|
||||
stmt = stmt.on_conflict_do_update(
|
||||
index_elements=["user_id", "day_utc"],
|
||||
set_={
|
||||
"ask_approved_usd": stmt.excluded.ask_approved_usd,
|
||||
"updated_at": stmt.excluded.updated_at,
|
||||
},
|
||||
)
|
||||
stmt = stmt.values(
|
||||
user_id=user_id,
|
||||
day_utc=day_utc,
|
||||
cost_usd=0.0,
|
||||
ask_approved_usd=ask_approved_usd,
|
||||
updated_at=now,
|
||||
)
|
||||
# On conflict touch only the approval (+ stamp) — never
|
||||
# the accumulated cost.
|
||||
stmt = stmt.on_conflict_do_update(
|
||||
index_elements=["user_id", "day_utc"],
|
||||
set_={
|
||||
"ask_approved_usd": stmt.excluded.ask_approved_usd,
|
||||
"updated_at": stmt.excluded.updated_at,
|
||||
},
|
||||
)
|
||||
session.execute(stmt)
|
||||
return
|
||||
# Generic dialect fallback — SELECT-then-INSERT/UPDATE.
|
||||
@@ -1177,9 +1256,11 @@ class SqlAlchemyConversationStore(ConversationStore):
|
||||
with self._session() as session:
|
||||
# Dialect-specific search: the SQLite family (SQLite + D1) has
|
||||
# FTS5 virtual tables (MATCH + rank); PostgreSQL doesn't. ILIKE on
|
||||
# the JSON data column is a functional fallback there. Proper
|
||||
# tsvector indexing is a future optimization (tracked in GAPS.md).
|
||||
use_fts = _supports_fts5(self._engine.dialect.name)
|
||||
# the JSON data column is a functional fallback there. MySQL/MariaDB
|
||||
# uses LIKE (case-insensitive by default with utf8mb4_general_ci).
|
||||
# Proper tsvector/FULLTEXT indexing is a future optimization.
|
||||
dialect = self._engine.dialect.name
|
||||
use_fts = _supports_fts5(dialect)
|
||||
if use_fts:
|
||||
if conversation_id is not None:
|
||||
stmt = text(
|
||||
@@ -1194,6 +1275,25 @@ class SqlAlchemyConversationStore(ConversationStore):
|
||||
"WHERE search_text MATCH :query "
|
||||
"ORDER BY rank LIMIT :limit"
|
||||
)
|
||||
elif dialect in ("mysql", "mariadb"):
|
||||
# MySQL/MariaDB: `data` is a TEXT column (no ::text cast needed),
|
||||
# and LIKE is case-insensitive by default with utf8mb4_general_ci
|
||||
# collation (no ILIKE needed).
|
||||
like_pattern = f"%{query}%"
|
||||
if conversation_id is not None:
|
||||
stmt = text(
|
||||
"SELECT ci.id FROM conversation_items ci "
|
||||
"WHERE ci.conversation_id = :cid "
|
||||
"AND ci.data LIKE :query "
|
||||
"ORDER BY ci.created_at DESC LIMIT :limit"
|
||||
)
|
||||
else:
|
||||
stmt = text(
|
||||
"SELECT ci.id FROM conversation_items ci "
|
||||
"WHERE ci.data LIKE :query "
|
||||
"ORDER BY ci.created_at DESC LIMIT :limit"
|
||||
)
|
||||
query = like_pattern
|
||||
else:
|
||||
# PostgreSQL: ILIKE fallback (no FTS5 virtual table).
|
||||
# Full tsvector/tsquery indexing can be added later.
|
||||
|
||||
@@ -53,13 +53,13 @@ class SqlAlchemyPermissionStore(PermissionStore):
|
||||
) -> SessionPermission:
|
||||
"""Upsert a permission grant. See base class for contract."""
|
||||
with self._session() as session:
|
||||
is_sqlite = self._engine.dialect.name == "sqlite"
|
||||
dialect = self._engine.dialect.name
|
||||
values = {
|
||||
"user_id": user_id,
|
||||
"conversation_id": conversation_id,
|
||||
"level": level,
|
||||
}
|
||||
if is_sqlite:
|
||||
if dialect == "sqlite":
|
||||
stmt = (
|
||||
sqlite_insert(SqlSessionPermission)
|
||||
.values(**values)
|
||||
@@ -68,6 +68,14 @@ class SqlAlchemyPermissionStore(PermissionStore):
|
||||
set_={"level": level},
|
||||
)
|
||||
)
|
||||
elif dialect in ("mysql", "mariadb"):
|
||||
# MySQL/MariaDB: ON DUPLICATE KEY UPDATE instead of PostgreSQL's
|
||||
# ON CONFLICT DO UPDATE. Both are atomic upserts; the SQLAlchemy
|
||||
# dialect objects differ but the intent is identical.
|
||||
from sqlalchemy.dialects.mysql import insert as mysql_insert
|
||||
|
||||
stmt = mysql_insert(SqlSessionPermission).values(**values)
|
||||
stmt = stmt.on_duplicate_key_update(level=stmt.inserted.level)
|
||||
else:
|
||||
stmt = (
|
||||
pg_insert(SqlSessionPermission)
|
||||
@@ -208,14 +216,21 @@ class SqlAlchemyPermissionStore(PermissionStore):
|
||||
def ensure_user(self, user_id: str, *, is_admin: bool = False) -> None:
|
||||
"""Upsert a user row. See base class for contract."""
|
||||
with self._session() as session:
|
||||
is_sqlite = self._engine.dialect.name == "sqlite"
|
||||
dialect = self._engine.dialect.name
|
||||
values = {"id": user_id, "is_admin": is_admin}
|
||||
if is_sqlite:
|
||||
if dialect == "sqlite":
|
||||
stmt = (
|
||||
sqlite_insert(SqlUser)
|
||||
.values(**values)
|
||||
.on_conflict_do_nothing(index_elements=["id"])
|
||||
)
|
||||
elif dialect in ("mysql", "mariadb"):
|
||||
# MySQL/MariaDB: no ON CONFLICT DO NOTHING; use a no-op
|
||||
# ON DUPLICATE KEY UPDATE instead (update id to itself).
|
||||
from sqlalchemy.dialects.mysql import insert as mysql_insert
|
||||
|
||||
stmt = mysql_insert(SqlUser).values(**values)
|
||||
stmt = stmt.on_duplicate_key_update(id=stmt.inserted.id)
|
||||
else:
|
||||
stmt = (
|
||||
pg_insert(SqlUser)
|
||||
|
||||
@@ -177,6 +177,9 @@ databricks = [
|
||||
# (bare OSS installs get it via the `tracing` extra instead).
|
||||
"mlflow>=3,<4",
|
||||
]
|
||||
# MySQL/MariaDB driver. Pure-Python; works with MySQL 8+ and MariaDB 10+.
|
||||
# Use with DATABASE_URL = mysql+pymysql://user:pass@host:3306/db?charset=utf8mb4
|
||||
mysql = ["PyMySQL>=1.1,<2"]
|
||||
dev = [
|
||||
"pytest>=7.0",
|
||||
# mlflow is now opt-in (the `tracing` / `databricks` extras), but the
|
||||
|
||||
Reference in New Issue
Block a user