fix(proxy): keep internal SurrealDB websocket out of HTTP proxy (#1185)

* fix(proxy): keep internal SurrealDB websocket out of HTTP proxy (#1160)

websockets 15.0 auto-detects HTTP_PROXY/HTTPS_PROXY and tunnels even
ws:// connections through the proxy. The SurrealDB SDK connects over a
websocket, so with a proxy set the internal DB connection was routed
through the external proxy, which rejected the internal host with HTTP
403 and killed the worker/API on startup.

- Add ensure_internal_no_proxy() helper that merges host.docker.internal,
  surrealdb, localhost, 127.0.0.1 into no_proxy/NO_PROXY (never clobbering
  a user value) and call it at API, worker and DB-module startup.
- Add host.docker.internal and surrealdb to the .env.example and docs
  NO_PROXY examples.
- Add unit tests for the injection helper.

* fix(proxy): preserve NO_PROXY wildcard and include custom SurrealDB host (#1160)

Address review findings on the no_proxy injection:

- NO_PROXY=* (bypass all hosts) is now treated as terminal: leave the
  user's config untouched instead of narrowing the wildcard to a finite
  list by appending the internal hosts.
- Parse the SurrealDB host from SURREAL_URL (falling back to
  SURREAL_ADDRESS) and add it to the bypass list, so deployments with a
  custom DB host/IP no longer route DB traffic through the proxy. Unset
  or malformed values fall back to the four defaults gracefully.
- Drop the inaccurate getproxies() caching remark in the test.

.env.example

@@ -74,5 +74,15 @@ SURREAL_DATABASE=open_notebook
 # BASIC_AUTH_USERNAME=admin
 # BASIC_AUTH_PASSWORD=secret
 
+# Network / Proxy (corporate / firewalled environments)
+# Route outbound HTTP(S) through a proxy. NO_PROXY MUST list the internal
+# SurrealDB hosts (host.docker.internal, surrealdb) — otherwise the DB
+# websocket gets tunnelled through the proxy and the worker/API fail to start
+# with HTTP 403. The app also injects these hosts automatically as a safety
+# net, but keep them here so your config is explicit.
+# HTTP_PROXY=http://proxy.corp.com:8080
+# HTTPS_PROXY=http://proxy.corp.com:8080
+# NO_PROXY=localhost,127.0.0.1,host.docker.internal,surrealdb,.local
+
 # For more configuration options, see:
 # https://github.com/lfnovo/open-notebook/blob/main/docs/5-CONFIGURATION/environment-reference.md

CHANGELOG.md

@@ -15,6 +15,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - **Four new AI providers** (surfaced by esperanto 2.25.1) are now in the provider matrix. **Cohere** (`COHERE_API_KEY`) adds language and embedding models via Cohere's native v2 API (`command-*` chat, `embed-*`); model discovery is bespoke (esperanto's `AIFactory.get_provider_models`) since Cohere is not OpenAI-compatible, and Cohere reranking is intentionally left out of scope (tracked at #1087). **PayPerQ / PPQ** (`PPQ_API_KEY`, `https://api.ppq.ai/v1`) is a multi-modality OpenAI-compatible gateway offering language, embedding, speech-to-text and text-to-speech. **Novita** (`NOVITA_API_KEY`, `https://api.novita.ai/openai`) is an OpenAI-compatible LLM gateway. PPQ and Novita auto-discover their models through the standard `/models` endpoint. In addition, the existing **Deepgram** provider now also supports **speech-to-text** (Nova/Whisper transcription models, e.g. `nova-3`) alongside its Aura text-to-speech voices. All four are configured from Settings → Models exactly like the existing providers (#1170)
 
 ### Fixed
+- **HTTP proxy no longer breaks worker/API startup.** `websockets` 15.0 began auto-detecting `HTTP_PROXY`/`HTTPS_PROXY` and tunnels even `ws://` connections through the proxy, so with a proxy set the internal SurrealDB websocket (`ws://host.docker.internal:8018/rpc` or `ws://surrealdb:8000/rpc`) was routed through the external proxy, which rejected the internal host with HTTP 403 and killed the worker on startup. Open Notebook now injects `host.docker.internal,surrealdb,localhost,127.0.0.1` into `no_proxy`/`NO_PROXY` at startup (merged with any user value, never clobbering it), and the `.env.example` / docs `NO_PROXY` examples now include the internal DB hosts (#1160)
 - Auto-assign no longer silently re-populates optional model defaults that were deliberately cleared. Previously `POST /models/auto-assign` treated every empty slot as "missing" and filled it, so an optional slot a user intentionally cleared (to fall back to the chat model) got re-assigned on the next run. Auto-assign now fills only the two required slots (chat, embedding); the optional slots (transformation, tools, large context, TTS, STT) are left untouched. `large_context` now also falls back to the chat model when unset (matching transformation/tools) instead of returning nothing, and the Settings UI shows an inline hint on each empty optional slot — "using chat model (…)" for the text slots, "not configured" for TTS/STT (#1098)
 - PPQ model discovery now lists all modalities, not just chat. PPQ's `/v1/models` returns only chat/language models by default; the discovery URL now requests `?type=all` so the embedding, speech-to-text and text-to-speech models this multi-modality gateway advertises actually surface in the provider matrix (`PPQ_MODEL_TYPES` classifies them into the right slots) (#1180)
 - Vertex credentials now configure text-to-speech (and other Vertex providers) correctly when used through a stored credential rather than environment variables. `Credential.to_esperanto_config()` emitted the generic `project`/`location` keys, but esperanto's Vertex providers expect `vertex_project`/`vertex_location` — so credential-linked Vertex TTS crashed with `__init__() got an unexpected keyword argument 'project'`. The config now maps to the Vertex-specific key names for the `vertex` provider only; the `Credential` schema/API fields stay `project`/`location`, and non-Vertex providers are unaffected (#1151)

api/main.py

@@ -3,6 +3,13 @@ from dotenv import load_dotenv
 
 load_dotenv()
 
+# Keep the internal SurrealDB websocket out of any configured HTTP proxy
+# (issue #1160). Must run after load_dotenv() (so a proxy set in .env is
+# already visible) and before the DB is touched.
+from open_notebook.utils.proxy import ensure_internal_no_proxy
+
+ensure_internal_no_proxy()
+
 import asyncio
 import os
 from contextlib import asynccontextmanager

commands/__init__.py

@@ -1,5 +1,13 @@
 """Surreal-commands integration for Open Notebook"""
 
+# The worker starts via `surreal-commands-worker --import-modules commands`,
+# so this package is imported before the worker connects to SurrealDB. Inject
+# the internal DB hosts into no_proxy first so the DB websocket is never
+# tunnelled through a configured HTTP proxy (issue #1160).
+from open_notebook.utils.proxy import ensure_internal_no_proxy
+
+ensure_internal_no_proxy()
+
 from .embedding_commands import (
     embed_insight_command,
     embed_note_command,

docs/5-CONFIGURATION/environment-reference.md

@@ -137,10 +137,12 @@ The `CCORE_FIRECRAWL_*` variables are passed straight through to the content-cor
 |----------|-----------|---------|-------------|
 | `HTTP_PROXY` | No | None | HTTP proxy URL for outbound HTTP requests |
 | `HTTPS_PROXY` | No | None | HTTPS proxy URL for outbound HTTPS requests |
-| `NO_PROXY` | No | None | Comma-separated list of hosts to bypass proxy |
+| `NO_PROXY` | No | None | Comma-separated list of hosts to bypass proxy (must include the internal DB hosts — see below) |
 
 Route all outbound HTTP requests through a proxy server. Useful for corporate/firewalled environments.
 
+> **Important:** `NO_PROXY` must list the internal SurrealDB hosts — `host.docker.internal` (Docker) and `surrealdb` (the compose service name). The SurrealDB SDK connects over a websocket, and `websockets` 15.0+ tunnels even `ws://` connections through a configured proxy, which then rejects the internal host with **HTTP 403** and prevents the API and worker from starting. Open Notebook injects `host.docker.internal,surrealdb,localhost,127.0.0.1` into `NO_PROXY` automatically at startup as a safety net, but you should still set them explicitly.
+
 The underlying libraries (esperanto, content-core, podcast-creator) automatically detect proxy settings from these standard environment variables.
 
 **Affects:**
@@ -160,8 +162,8 @@ HTTPS_PROXY=http://proxy.corp.com:8080
 HTTP_PROXY=http://user:password@proxy.corp.com:8080
 HTTPS_PROXY=http://user:password@proxy.corp.com:8080
 
-# Bypass proxy for local hosts
-NO_PROXY=localhost,127.0.0.1,.local
+# Bypass proxy for local hosts (include the internal DB hosts!)
+NO_PROXY=localhost,127.0.0.1,host.docker.internal,surrealdb,.local
 ```
 
 ---
@@ -212,7 +214,7 @@ API_URL=https://mynotebook.example.com
 OPEN_NOTEBOOK_ENCRYPTION_KEY=your-secret-key
 HTTP_PROXY=http://proxy.corp.com:8080
 HTTPS_PROXY=http://proxy.corp.com:8080
-NO_PROXY=localhost,127.0.0.1
+NO_PROXY=localhost,127.0.0.1,host.docker.internal,surrealdb,.local
 ```
 
 ### High-Performance Deployment

open_notebook/database/repository.py

@@ -8,6 +8,13 @@ from loguru import logger
 from surrealdb import AsyncSurreal, RecordID  # type: ignore
 from surrealdb.data.types.table import Table  # type: ignore
 
+from open_notebook.utils.proxy import ensure_internal_no_proxy
+
+# Keep the internal SurrealDB websocket out of any configured HTTP proxy
+# (issue #1160). Runs at import time - i.e. before any db_connection() can be
+# opened - so it protects every entrypoint (API + worker) that touches the DB.
+ensure_internal_no_proxy()
+
 T = TypeVar("T", Dict[str, Any], List[Dict[str, Any]])
 
 # Bare SurrealDB table/relation identifier: no ':', whitespace, or query

open_notebook/utils/proxy.py

@@ -0,0 +1,118 @@
+"""Proxy environment helpers.
+
+Defensive handling for HTTP proxy setups (issue #1160).
+
+``websockets`` 15.0 started auto-detecting ``HTTP_PROXY`` / ``HTTPS_PROXY``
+from the environment and tunnels *every* connection through the proxy,
+including plain ``ws://`` ones. The SurrealDB SDK connects to the database
+over a websocket, so when a proxy is configured the *internal* DB connection
+(e.g. ``ws://host.docker.internal:8018/rpc`` or ``ws://surrealdb:8000/rpc``)
+gets routed through the external proxy, which rejects the internal host with
+HTTP 403 and kills the API / worker on startup.
+
+To make this robust regardless of user configuration, we inject the internal
+DB hosts into the ``no_proxy`` / ``NO_PROXY`` env vars at startup (merged with
+any user-provided value, never clobbering it). ``urllib.request.proxy_bypass``
+- which ``websockets`` calls to decide whether to tunnel - reads these vars,
+so the internal websocket is left un-proxied.
+"""
+
+import os
+from urllib.parse import urlsplit
+
+# Hosts the DB is reachable at internally. These must never be routed through
+# an external proxy. Covers Docker (host.docker.internal, surrealdb service
+# name) and local (localhost / 127.0.0.1) topologies. Deployments that point
+# at a custom SurrealDB host/IP have it added dynamically - see
+# _configured_db_host().
+INTERNAL_NO_PROXY_HOSTS = (
+    "host.docker.internal",
+    "surrealdb",
+    "localhost",
+    "127.0.0.1",
+)
+
+# urllib.request.proxy_bypass_environment reads the lowercase var first, then
+# falls back to the uppercase one. We keep both in sync so the internal hosts
+# are honored no matter which variant a user set.
+_NO_PROXY_ENV_VARS = ("no_proxy", "NO_PROXY")
+
+
+def _split_hosts(value: str) -> list[str]:
+    return [h.strip() for h in value.split(",") if h.strip()]
+
+
+def _configured_db_host() -> str | None:
+    """Best-effort extraction of the SurrealDB host from the environment.
+
+    Mirrors the resolution order in ``open_notebook.database.repository`` -
+    ``SURREAL_URL`` wins, otherwise the legacy ``SURREAL_ADDRESS``. Returns the
+    bare host (no scheme/port/path), or ``None`` when unset or unparseable.
+
+    Parsed here from the env directly (rather than importing repository) to
+    avoid a circular import - repository imports this module at load time.
+    """
+    candidate = os.environ.get("SURREAL_URL", "").strip()
+    if not candidate:
+        candidate = os.environ.get("SURREAL_ADDRESS", "").strip()
+    if not candidate:
+        return None
+
+    # urlsplit only populates .hostname when a scheme is present; add a dummy
+    # one for bare ``host`` / ``host:port`` values (e.g. SURREAL_ADDRESS).
+    if "://" not in candidate:
+        candidate = "ws://" + candidate
+
+    try:
+        host = urlsplit(candidate).hostname
+    except ValueError:
+        return None
+    return host or None
+
+
+def _internal_hosts() -> list[str]:
+    """The default internal hosts plus the configured SurrealDB host, if any."""
+    hosts = list(INTERNAL_NO_PROXY_HOSTS)
+    db_host = _configured_db_host()
+    if db_host and db_host.lower() not in {h.lower() for h in hosts}:
+        hosts.append(db_host)
+    return hosts
+
+
+def ensure_internal_no_proxy() -> None:
+    """Ensure internal DB hosts bypass any configured proxy.
+
+    Merges the internal hosts (see :func:`_internal_hosts`) into both
+    ``no_proxy`` and ``NO_PROXY`` (preserving existing entries and their order)
+    and writes the combined value back to both env vars. Idempotent - safe to
+    call more than once and from multiple entrypoints (API + worker).
+
+    A wildcard (``*``) already bypasses the proxy for every host, so if the
+    user set ``no_proxy``/``NO_PROXY`` to (or containing) a bare ``*`` we leave
+    their configuration untouched rather than turning it into a finite list.
+    """
+    # Collect existing entries from whichever variants the user set, de-duped
+    # while preserving order.
+    existing: list[str] = []
+    seen: set[str] = set()
+    for var in _NO_PROXY_ENV_VARS:
+        for host in _split_hosts(os.environ.get(var, "")):
+            # Wildcard is terminal: it already bypasses everything (including
+            # the internal DB hosts), so don't narrow it to a finite list.
+            if host == "*":
+                return
+            key = host.lower()
+            if key not in seen:
+                seen.add(key)
+                existing.append(host)
+
+    # Append internal hosts that aren't already present.
+    merged = list(existing)
+    for host in _internal_hosts():
+        if host.lower() not in seen:
+            seen.add(host.lower())
+            merged.append(host)
+
+    combined = ",".join(merged)
+    for var in _NO_PROXY_ENV_VARS:
+        os.environ[var] = combined

tests/test_proxy.py

@@ -0,0 +1,138 @@
+"""Tests for the internal no_proxy injection (issue #1160)."""
+
+import os
+
+import pytest
+
+from open_notebook.utils.proxy import (
+    INTERNAL_NO_PROXY_HOSTS,
+    ensure_internal_no_proxy,
+)
+
+_PROXY_VARS = ("no_proxy", "NO_PROXY")
+_DB_VARS = ("SURREAL_URL", "SURREAL_ADDRESS")
+
+
+@pytest.fixture(autouse=True)
+def _clean_proxy_env(monkeypatch):
+    """Ensure each test starts with the proxy / DB-host vars unset."""
+    for var in (*_PROXY_VARS, *_DB_VARS):
+        monkeypatch.delenv(var, raising=False)
+    yield
+
+
+def test_injects_internal_hosts_when_unset():
+    ensure_internal_no_proxy()
+    for var in _PROXY_VARS:
+        value = os.environ[var]
+        for host in INTERNAL_NO_PROXY_HOSTS:
+            assert host in value.split(",")
+
+
+def test_preserves_user_value(monkeypatch):
+    monkeypatch.setenv("NO_PROXY", "example.com,10.0.0.5")
+    ensure_internal_no_proxy()
+    entries = os.environ["NO_PROXY"].split(",")
+    # User entries kept, in their original order and at the front.
+    assert entries[:2] == ["example.com", "10.0.0.5"]
+    # Internal hosts appended.
+    for host in INTERNAL_NO_PROXY_HOSTS:
+        assert host in entries
+
+
+def test_does_not_duplicate_existing_hosts(monkeypatch):
+    monkeypatch.setenv("NO_PROXY", "surrealdb,example.com")
+    ensure_internal_no_proxy()
+    entries = os.environ["NO_PROXY"].split(",")
+    assert entries.count("surrealdb") == 1
+
+
+def test_lowercase_and_uppercase_kept_in_sync(monkeypatch):
+    monkeypatch.setenv("no_proxy", "example.com")
+    ensure_internal_no_proxy()
+    assert os.environ["no_proxy"] == os.environ["NO_PROXY"]
+
+
+def test_merges_both_case_variants(monkeypatch):
+    monkeypatch.setenv("no_proxy", "lower.example.com")
+    monkeypatch.setenv("NO_PROXY", "UPPER.example.com")
+    ensure_internal_no_proxy()
+    combined = os.environ["no_proxy"]
+    assert "lower.example.com" in combined
+    assert "UPPER.example.com" in combined
+    # De-duped case-insensitively, so no crash and both variants match.
+    assert os.environ["no_proxy"] == os.environ["NO_PROXY"]
+
+
+def test_idempotent(monkeypatch):
+    monkeypatch.setenv("NO_PROXY", "example.com")
+    ensure_internal_no_proxy()
+    first = os.environ["NO_PROXY"]
+    ensure_internal_no_proxy()
+    assert os.environ["NO_PROXY"] == first
+
+
+def test_wildcard_preserved(monkeypatch):
+    """NO_PROXY=* bypasses every host; it must not be narrowed to a list."""
+    monkeypatch.setenv("NO_PROXY", "*")
+    ensure_internal_no_proxy()
+    assert os.environ["NO_PROXY"] == "*"
+
+
+def test_wildcard_among_entries_preserved(monkeypatch):
+    """A bare * among other entries is still terminal - leave config as-is."""
+    monkeypatch.setenv("NO_PROXY", "example.com,*")
+    ensure_internal_no_proxy()
+    assert os.environ["NO_PROXY"] == "example.com,*"
+
+
+def test_custom_surreal_url_host_included(monkeypatch):
+    monkeypatch.setenv("SURREAL_URL", "ws://db.internal.corp:8000/rpc")
+    ensure_internal_no_proxy()
+    entries = os.environ["NO_PROXY"].split(",")
+    assert "db.internal.corp" in entries
+    # Defaults still present.
+    for host in INTERNAL_NO_PROXY_HOSTS:
+        assert host in entries
+
+
+def test_custom_surreal_address_host_included(monkeypatch):
+    # Legacy address form (host:port, no scheme).
+    monkeypatch.setenv("SURREAL_ADDRESS", "10.1.2.3:8000")
+    ensure_internal_no_proxy()
+    assert "10.1.2.3" in os.environ["NO_PROXY"].split(",")
+
+
+def test_surreal_url_takes_precedence_over_address(monkeypatch):
+    monkeypatch.setenv("SURREAL_URL", "ws://from-url:8000/rpc")
+    monkeypatch.setenv("SURREAL_ADDRESS", "from-address")
+    ensure_internal_no_proxy()
+    entries = os.environ["NO_PROXY"].split(",")
+    assert "from-url" in entries
+    assert "from-address" not in entries
+
+
+def test_malformed_surreal_url_falls_back_to_defaults(monkeypatch):
+    monkeypatch.setenv("SURREAL_URL", "::not a url::")
+    ensure_internal_no_proxy()
+    entries = os.environ["NO_PROXY"].split(",")
+    # No crash; the four defaults are still injected.
+    for host in INTERNAL_NO_PROXY_HOSTS:
+        assert host in entries
+
+
+def test_custom_host_not_duplicated_when_default(monkeypatch):
+    monkeypatch.setenv("SURREAL_URL", "ws://surrealdb:8000/rpc")
+    ensure_internal_no_proxy()
+    assert os.environ["NO_PROXY"].split(",").count("surrealdb") == 1
+
+
+def test_bypass_recognized_by_urllib(monkeypatch):
+    """The injected hosts are actually honored by urllib.request.proxy_bypass,
+    which is what websockets calls to decide whether to tunnel."""
+    import urllib.request
+
+    monkeypatch.setenv("HTTP_PROXY", "http://proxy.corp.com:8080")
+    ensure_internal_no_proxy()
+    assert urllib.request.proxy_bypass("surrealdb:8000")
+    assert urllib.request.proxy_bypass("host.docker.internal:8018")