* fix(cli): prefer puppeteer cache + numeric version sort (staff review)
Two correctness fixes from PR #821 self-review:
1. Cache priority order. Previous order was hyperframes-managed cache →
puppeteer cache. HF cache is pinned to CHROME_VERSION (131-era) which
lags 17+ releases behind upstream; if a user separately installed a
newer chrome-headless-shell via @puppeteer/browsers install, the CLI
would silently hand engine the older HF-cache binary while engine's
own resolveHeadlessShellPath would have picked the newer one. Flip
the priority so puppeteer cache wins, matching engine semantics.
2. Numeric (not lexicographic) version sort. `readdirSync.sort().reverse()`
over names like `linux-148.0.7778.97` and `linux-99.0.6533.123` would
return `linux-99...` first because character '9' outranks '1'. Parse
each name into integer segments and compare them numerically.
Tests: add both-caches-populated and linux-148-beats-linux-99 cases.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* perf(engine): page-side compositing for shader transitions (opt-in spike)
Add an opt-in `--page-side-compositing` flag (CLI) backed by a new engine
config field `enablePageSideCompositing` and env var `HF_PAGE_SIDE_COMPOSITING`.
When set, SDR shader-transition compositions skip the Node-side layered blend
(the hf#677 chain) and instead run the shader inside Chrome via a page-side
WebGL canvas; the engine then captures ONE opaque RGB frame per output frame
via the existing streaming capture path.
This is the strongest non-beginFrame perf lever for Mac users, who cannot
take the beginFrame `~5×` path (Chromium structural limit, crbug.com/40656275).
Stacks on top of the hf#677 1.95× baseline.
Default OFF — existing fixture pins (byte-exact MP4 output) are preserved.
Opt-in path is intentionally PSNR-pinned, not byte-equal (WebGL is f32; Node
is f64). HDR content forces the existing layered path regardless.
Implementation:
- engine: new `EngineConfig.enablePageSideCompositing` (default false).
- producer/fileServer: new `HF_PAGE_SIDE_COMPOSITING_STUB` early-page script
injected into the served HTML head when the flag is on.
- producer/renderOrchestrator: when the flag + no HDR + no png-sequence,
route SDR transitions through the streaming path instead of the layered
HDR stage.
- shader-transitions: new `engineModePageComposite.ts` installs a fullscreen
WebGL compositor overlay and wraps `window.__hf.seek` so each seek inside
a transition window captures both scenes via the Chromium
`drawElementImage` API to GL textures, runs the fragment shader, and
displays the composited result on the overlay canvas. The engine takes
one screenshot per frame and sees the composited overlay.
- cli: new `--page-side-compositing` flag sets `HF_PAGE_SIDE_COMPOSITING=true`
before producer load.
- scripts/page-side-compositing-smoke: bundled-CLI smoke that renders a
representative fixture with and without the flag, validates the canary
strings are in the shipped bundles, and writes a wall-time pair.
Determinism trade documented in the engine config doc-comment. The smoke
script enforces the bundled-CLI validation discipline from prior perf work
(see internal feedback note `validate_bundled_cli_not_dev_path`).
Runtime requirement: Chromium's `CanvasDrawElement` feature (already
enabled by the engine's `--enable-features=CanvasDrawElement` launch flag).
When the runtime feature is unavailable, the page-side installer logs a
warning and falls back to opacity-flip mode — the engine still takes the
streaming path; the transition window degrades to a hard scene swap. Vance
will validate on Mac Chrome where the feature is supported.
Co-Authored-By: Vai <vai@heygen.com>
* fix(shader-transitions): use html2canvas for page-side compositor capture
The original drawElementImage approach fails in engine render mode because
the virtual-time shim prevents Chromium from generating paint records for
cloned elements. drawElementImage requires a cached paint record from the
browser's compositor — clones created at capture time never receive one
because (a) shimmed rAFs deadlock inside the seek wrapper, (b) original
rAFs don't produce real paints under virtual-time control, and
(c) layoutsubtree canvases don't apply CSS stylesheet rules to children.
Switch scene capture to html2canvas (foreignObjectRendering: false), the
same JS-based renderer already used by the preview-mode fallback path in
capture.ts. html2canvas reads computed styles and renders via its own
canvas drawing pipeline with no dependency on the browser paint cycle.
Also fixes:
- Engine seek must return the result so Puppeteer awaits async seek
promises (frameCapture.ts).
- GSAP opacity cache: compositor must restore scene opacity before seek,
not after — GSAP caches inline values and skips re-writes.
- Support check gates on WebGL availability, not drawElementImage.
Perf: 15-scene shader-perf fixture (28s, 14 transitions, 30fps)
Baseline (Node-side layered): 137s
Page-side (html2canvas+WebGL): 33s → 4.1× speedup
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* refactor(shader-transitions): simplify review fixes for page-side compositor
- Use uploadTexture (zeroes canvas backing store after upload) to prevent
~2.2GB transient memory pressure across 280 html2canvas calls per render
- Add ignoreElements + stabilizeTransformedBoxShadows to html2canvas call,
matching the preview-path capture.ts behavior
- Parallelize from/to scene captures with Promise.all
- Wrap post-capture render in try/finally so opacity is always restored
- Fix WebGL context leak in isPageSideCompositingSupported probe
- Remove dead ResolvedTransition.index field
- Export stabilizeTransformedBoxShadows from capture.ts
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(producer): unify page-side compositing gating and Docker forwarding
Addresses three issues from staff review:
1. ignoreElements filter stripped all in-scene canvases (Chart.js, D3,
p5.js) — narrowed to data-no-capture only since the compositor canvas
is a body sibling never in the scene subtree.
2. Docker mode silently dropped --page-side-compositing — thread
pageSideCompositing through DockerRenderOptions/buildDockerRunArgs
with regression tests.
3. Fragmented gating across 4 independent sites could disagree:
- Stub injection gated only on cfg flag (leaked into HDR/alpha)
- Probe-created fileServer never got the stub
- needsAlpha (WebM/MOV) not excluded from the gate
- WebGL-unavailable fallback claimed layered path would run but
orchestrator had already disabled it
Fix: compute stub injection at the same site as the layered-bypass
decision (after hasHdrContent is known), using addPreHeadScript on
the already-running fileServer. Single predicate now gates both
decisions, including !needsAlpha.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* perf(engine): two-phase drawElementImage capture for page-side compositing
Replace html2canvas with native drawElementImage for scene capture in
the page-side compositor. drawElementImage reads from the browser's own
paint cache, giving pixel-identical output to the preview path.
The blocker was that cloned elements inside layoutsubtree canvases have
no cached paint record under virtual time — the compositor only paints
when explicitly triggered. Fix: split the seek+composite into two phases
with an engine-forced paint between them.
Phase 1 (seek wrapper, page-side):
- GSAP seek positions the timeline
- Clone FROM/TO scenes into visible layoutsubtree staging canvases
- Set window.__hf_page_composite_pending flag
Engine paint force (frameCapture.ts):
- Detect pending flag after seek returns
- Fire micro Page.captureScreenshot (1x1 clip) via CDP to force the
browser compositor to paint all visible elements including staging
canvas children
Phase 2 (page.evaluate, page-side):
- drawElementImage reads the now-valid paint records
- Upload textures to WebGL, run shader, show GL overlay
Key insight: staging canvases must be visible (not opacity:0) for the
browser to paint their children. They sit at z-index:-9998, behind
the main DOM and covered by the GL overlay during transitions.
Perf: 15-scene fixture (28s, 14 transitions, 30fps):
Baseline (Node-side layered): 137s
html2canvas + WebGL: 33s (3.7×)
drawElementImage + WebGL: 21s (6.6×)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* perf(engine): optimize two-phase compositor hot path
- uploadTextureSource instead of uploadTexture: eliminates ~2.3GB of
canvas buffer alloc/dealloc churn (persistent staging canvases don't
need the one-shot zeroing behavior)
- Fold hasPending check into seek page.evaluate: eliminates one CDP
round-trip per frame (~700 unnecessary IPC calls on non-transition
frames)
- Fix renderShader error handling: on failure, leave source scenes
visible as fallback instead of hiding both scenes + GL overlay
(which produced black frames)
- Move mutable state declarations above resolveComposite to prevent
TDZ risk on refactor
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(engine): staff review — staging cleanup, pending flag, beginFrame guard
- Clear staging canvas children when leaving transition window (prevents
visible clone bleed-through on transparent compositions)
- Clear __hf_page_composite_pending on all resolveComposite exit paths
- Guard micro-screenshot paint force against beginFrame mode (CDP
Page.captureScreenshot conflicts with beginFrame compositor control)
- Update CLI flag description: document video/canvas limitation, remove
stale PSNR claim
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat(engine): default-on page-side compositing for SDR shader transitions
Page-side compositing is now enabled by default for SDR shader-transition
renders without video content. The 6.6× speedup applies automatically —
no flag needed.
Auto-disables when:
- HDR content detected
- Alpha output (WebM/MOV/PNG-sequence)
- Composition contains <video> elements (cloneNode loses playback state)
- beginFrame capture mode (Linux headless)
Use --no-page-side-compositing to force the Node-side layered path.
Changes:
- Engine config: enablePageSideCompositing defaults to true
- CLI: flag default flipped to true; --no-page-side-compositing disables
- Orchestrator: added composition.videos.length === 0 gate
- Docker: forwards --no-page-side-compositing when explicitly disabled
- Config tests updated for new default
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat(engine): support video elements on page-side compositing fast path
Three-phase capture protocol lets shader transitions render video scenes
without falling back to the slow Node-side layered pipeline:
1. Seek → compositor records transition metadata, sets pending flag
2. onBeforeCapture → video frame injector updates <img> replacements
3. prepare → cloneNode picks up current video frames, img.decode() awaits
4. micro-screenshot → forces browser to paint cloned elements
5. resolve → drawElementImage reads paint records, shader composites
Key changes:
- Remove `composition.videos.length === 0` gate from orchestrator
- Split compositor resolve into prepare (clone) + resolve (shader)
- Move onBeforeCapture before compositor prepare in frameCapture.ts
- Await img.decode() on cloned data-URI images to prevent stale frames
- Stop manipulating scene opacity in compositor (GL canvas overlay suffices)
- Add gsap.set declaration for shader-transitions ambient types
- Add video_missing_timing_attrs lint rule for <video> without id/data-start/data-end
Performance: compositions with video now render at 7.5s (6 workers) instead
of 2m38s on the layered path.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(core): auto-inject data-start on video/audio so frame extraction works without explicit attrs
The timing compiler now injects data-start="0" on <video> and <audio>
elements that lack it. This makes discoverMediaFromBrowser() find the
element (it queries video[data-start]), so the frame extraction pipeline
activates automatically. Videos "just work" without requiring authors to
add data-start, data-end, or id attributes.
Also removes the video_missing_timing_attrs lint rule — the compiler
handles the missing attributes automatically, so the lint rule would
only false-positive.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat(core): add data-hf-auto-start sentinel on auto-injected video timing
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat(producer): add discoverVideoVisibilityFromTimeline for runtime video discovery
Seeks the GSAP timeline in Puppeteer to discover when each video's parent
scene is visible (opacity > 0). Uses coarse sampling at 100ms steps followed
by binary search refinement to frame-level precision (1/60s). Only processes
videos with the data-hf-auto-start sentinel so author-specified timing is
never overridden.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat(producer): integrate runtime video visibility discovery into probe stage
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(producer): trigger browser probe for auto-start videos, remove debug logging
The probe stage was skipping browser launch when composition duration was
already known, which meant discoverVideoVisibilityFromTimeline never ran.
Now needsBrowser also checks for data-hf-auto-start sentinel in compiled HTML.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(scripts): use mkdtempSync for smoke test work directory
Replaces hardcoded /tmp/hf-page-side-smoke with a unique temp directory
via mkdtempSync to resolve CodeQL "insecure temporary file" alert.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* style: format smoke test script
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Vai <vai@heygen.com>
Vance Ingalls
f84cc492de