The router-level token_dep was intentionally removed (PR #2063) because
it broke WebSocket upgrades. Update the structural test to assert the
dependency is absent — auth is enforced by AuthGateMiddleware.
Playground Advanced Config always 400'd because /config/dump requires
type but pyConfigToJson only sent code. Include #cfg-type, align stream
fallback with dump shape, and teach shouldUseStream both nestings.
Monitor dashboard WS /monitor/ws 500'd because router-level token_dep is
HTTP Request-only and cannot inject into WebSocket scopes. Auth remains
on AuthGateMiddleware; admin routes keep require_admin.
Fixes#2059Fixes#2060
Add new contributors (fstark96, TobiasWallura-xitaso, harshmathurx,
RajanChavada, bibi-creator), update existing entries (nightcityblade,
hafezparast) with v0.9.1 PRs, and acknowledge security researchers
Y4tacker and Rafael in SECURITY-CREDITS.md.
Bump version to 0.9.1 across all references.
Add release notes, demo verification script, and doc updates.
Key highlights:
- PruningContentFilter preserve_classes/preserve_tags whitelist
- 12 bug fixes across Docker, browser, core, and extraction
- lxml ceiling widened to allow 6.x
Relax lxml~=5.3 (>=5.3,<6) to >=5.3,<7 so crawl4ai can co-install
with packages requiring lxml 6.x (e.g. scrapling). Crawl4ai only uses
stable lxml APIs (etree, html, fromstring, CSSSelector) unchanged in 6.x.
The auth gate fix (cbf5e5c) intentionally makes UI shell pages load
without credentials. Update the posture-gate tests to match: move
/dashboard/ and /playground/ from PROTECTED_ENDPOINTS to a new
PUBLIC_UI_PATHS list with an explicit 200 assertion.
- Replace single Enterprise Sponsors & Partners section with two sections:
Strategic Partners (Massive) and Enterprise Sponsors (Kipo, Kidocode, Aleph Null)
- Add Become a Strategic Partner or Sponsor CTA with contact email
- Remove CapSolver tip blockquote
- Fix Massive description punctuation
The python:3.12-slim-bookworm base image only has main and
non-free-firmware components enabled. nvidia-cuda-toolkit lives in
non-free, which is not included by default in Debian Bookworm slim
images.
Add deb http://deb.debian.org/debian bookworm non-free to sources.list
before the GPU install block so ENABLE_GPU=true builds don't fail with
"Package has no installation candidate".
Also fixes a stray trailing space after apt-get clean.
Fixes#2020
- Add Massive (joinmassive.com) as first entry in Enterprise Sponsors table
- Remove Thordata, NstProxy, Scrapeless, and Capsolver rows
- Add locally hosted light/dark logo variants for Massive
- Normalise all sponsor logo heights to 40px for visual consistency
- Remove stray KidoCode text link from Kidocode row
BrowserConfig.chrome_channel defaults to 'chromium'. When passed to
playwright.chromium.launch() as channel='chromium', Playwright looks
for a system Chrome installation instead of using the bundled
ms-playwright binary. On Windows this causes Chrome to exit
immediately with code 0, resulting in:
TargetClosedError: BrowserType.launch: Target page, context or
browser has been closed
Skip passing channel when it equals the default 'chromium' value.
GitHub's Camo proxy fails to render external SVGs from gist URLs and
nstproxy.com. Move Thor Data and nstproxy logos into the repo and also
fix the nstproxy fallback URL typo (ttps -> https).
- supervisord: write pidfile to /tmp (tmpfs) instead of read-only /app
- redis: set working dir to /var/lib/redis (tmpfs) for RDB snapshots
Fixes "could not write pidfile" and "Failed opening temp RDB file" errors
in read-only root filesystem deployment (docker-compose.yml security hardening).
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Two unreachable functions in utils.py:
- The first `normalize_url` (plain urljoin wrapper) was silently shadowed
by the extended `normalize_url` defined ~20 lines later. Python last-write
wins, so the first definition was never callable.
- `normalize_url_tmp` was a hand-rolled URL joiner (string split on "/")
with no callers outside utils.py itself. `urllib.parse.urljoin` already
covers this correctly.
adaptive_crawler copy.py was an uncommitted editor artifact that ended up
tracked in the repo. It is byte-for-byte identical to adaptive_crawler.py
and is not imported anywhere.
The AuthGateMiddleware blocked UI static pages with 401 because browsers
cannot attach Authorization headers to top-level navigation. The UI shell
serves no data, so it is safe to load without credentials.
- Add public_prefixes to AuthGateMiddleware for prefix-based path bypass
- Register /dashboard, /playground, /static as public prefixes
- Add token input bar to both playground and dashboard UIs
- Replace all bare fetch() calls with authFetch() that attaches Bearer token
- Append ?token= to monitor WebSocket URL (gate already accepts it for WS)
All API/data routes remain fail-closed behind the auth gate.
Closes#2037
- Add a 0.9.0 secure-by-default callout + migration-guide link at the top.
- Mark the inline-Python hooks section as removed/replaced by declarative
hooks (it described an unauthenticated code-execution surface).
- Note the 0.9.0 secure defaults on the embedded security config block.
- Bump docs site_name to v0.9.x.
A fuller per-example rewrite of self-hosting.md will follow.
UncleCode