-
Screen pointer during share, unified Skipper companion, permissions panel (#847)
frostbyte_neo 发布于
2026-08-11 17:54:56 +00:00 - feat(x): assistant screen pointer during screen share + permissions panel
On calls with a live screen share, the assistant can now POINT at the
user's real screen — an animated laser-dot with ping rings and an
optional label, placed at fractional coordinates estimated from the
latest screen-share frame.- screen-pointer builtin (permission: none): transparent click-through
screen-saver-level panel over the shared (primary) display, hash route
#screen-pointer. Auto-hides; torn down the instant the share ends.
State is pushed AND pulled on mount (the pull fixes the load race that
made the first pointer invisible). Executes via a main-process DI
service (IScreenPointerService), share-gated by the renderer. - Prompt surface: video-mode fragment + app-navigation skill teach
point-while-explaining, with worked example; pointing is explicitly
the LIMIT — acting on web pages routes to the embedded browser. - Auto-permission: voice/PTT submissions no longer fall back to manual
mode (composer default is auto), so gated tools stop carding every
spoken request mid-call. - Settings → Permissions panel: OS permissions grouped by the feature
they serve (voice calls / video / screen sharing / notifications) with
honest statuses (unknowns stay unknown), native grant buttons where
macOS allows prompting, System Settings deep links elsewhere,
focus-triggered refresh. Platform-aware (Windows collapses to
mic/camera/notifications). - Companion pill/tucked mascot: pulsing green listening halo + loud
status chip while the mic gate is open — PTT feedback was too subtle
to trust.
A click/type screen-control tool was explored and deliberately removed
before merge (blind-between-frames sequencing made it overclaim); design
notes and the CGEvent/TCC lessons are recorded in VIDEO_MODE.md.Co-Authored-By: Claude Fable 5 noreply@anthropic.com
- feat(x): unify the Skipper — mascot with a foldable text panel, one surface
Voice sessions no longer flip between two presentations (tucked mascot vs
expanded text card). The Skipper is ONE corner-anchored unit: the mascot
with its text panel attached.- Text is the DEFAULT: every Skipper landing (shortcut summon, tuck,
pop-out) arrives with the panel open, bottom-right of the cursor's
display, focused and ready to speak or type. - Fold, don't switch: collapsing hides the panel and the window shrinks
toward the mascot's corner; expanding unfolds from it. The corner is
wherever the user last DRAGGED the Skipper — the mascot is the drag
handle ("carry it around") and it never jumps on fold/unfold. - Blur no longer steals the text away: pinned surfaces survive focus
loss like the pill always did. Tucking is explicit (handle, Esc, or a
click on the transparent stage). - Right ⌘ works in BOTH states: the PTT key listeners moved to capture
phase so the embedded composer can't swallow the key, and the card's
mascot gets the same green listening halo as the tucked mascot. - The pill (camera calls) keeps its geometry; summoned ⌥⇧Space bar
unchanged.
Co-Authored-By: Claude Fable 5 noreply@anthropic.com
- feat(x): modality-based speech + Skipper controls live on the mascot
Speech now follows the QUESTION, not the surface: a spoken question (PTT)
gets a spoken reply even with the text panel open; a typed question
renders silently. The old surface rule (expanded card = all speech off)
and its quickAsk:setTextMode / quick-ask:text-mode channels are gone —
the per-turn choice is stamped at submit from pendingVoiceInputRef, so
tucking or typing mid-reply never flips an in-flight answer.- Speaker mute (output): new toggle-speaker popout action +
speakerMuted in the mirrored call state. Muting hushes in-flight
speech and drops the queued backlog; independent of micMuted (input).
Resets at call start. The toggle lives on the TEXT PANEL's strip (and
the pill's control bar), deliberately NOT on the mascot: it's a "read
instead of listen" choice that only exists while there is text to
read — with the panel folded, voice is the only output channel, so
folding auto-unmutes. - One control surface: the folded mascot's pin cluster is extracted
(SkipperPins) and now also rides the expanded card's mascot — hold-to-
talk/Stop on the hat, share bow light, text fold/unfold, end — with
the same status line (SkipperStatusChip, "Hold the mic — or right ⌘")
beneath it in both presentations. The card's action strip keeps only
chat-destination affordances plus the speaker mute; its duplicated
call controls are gone.
Co-Authored-By: Claude Fable 5 noreply@anthropic.com
- fix(x): Skipper mascot stays pixel-fixed across text fold/unfold
Folding the text used to swap the whole surface to a different component
tree (TuckedMascot) with different geometry — mascot size 132 vs 124,
different paddings and alignment — so the mascot visibly jumped and
replayed its entry animation on every fold/unfold.The card surface no longer branches to TuckedMascot when folded: ONE
Skipper layout renders both states, and the mascot column is the same
mounted node throughout — same 132px head, pins, fixed-height caption +
status-chip slots, same px-6/pb-5 offsets from the window's bottom-right
corner. Since the window is corner-anchored, fold/unfold changes exactly
one thing: whether the card exists beside the mascot. The folded stage
doubles as a drag area. TuckedMascot remains for the pill's tuck (camera
surface, different geometry).Co-Authored-By: Claude Fable 5 noreply@anthropic.com
- fix(x): history peek no longer repeats the current exchange
The peek fetches the chat's latest messages — which include the exchange
already rendered below the "earlier" divider, so the last question and
answer appeared twice. The render now trims the current exchange off the
history tail, keyed on the question text (the reliable key — a streaming
reply can differ from its stored copy): the newest user message and
everything after it are dropped iff it is the question on display.Co-Authored-By: Claude Fable 5 noreply@anthropic.com
- fix(x): Skipper status chip stays one centered line
The chip lives in the 132px mascot column now — "Hold the mic — or
right ⌘" wrapped into a left-ragged two-liner. Single line, centered on
the mascot: wider than the column it overflows symmetrically (the column
doesn't clip), reading as a caption under the head. The caption slot
gets the same treatment with a truncation cap sized to the folded
window.Co-Authored-By: Claude Fable 5 noreply@anthropic.com
- fix(x): Skipper fold state can't wedge + ⌥⇧Space toggles the text panel
The dead-controls bug after popping a chat out was a renderer/main state
desync: the renderer flippedcollapsedoptimistically while main owns
the window geometry, and main's same-state short-circuits refused to
repair a divergence — leaving the expanded card squeezed inside the
folded 250px window (the chip-over-composer artifact), where every
control appears dead and fold/unfold requests no-op forever.Three changes make the wedge structurally impossible:
- Main is the single source of truth: requestCollapsed no longer flips
local state; the quick-ask:mode push round-trip does. - setPinnedCollapsed drops its same-state short-circuit — geometry is
re-applied and mode re-pushed idempotently, so ANY divergence
self-heals on the next request instead of sticking. - setCompanionPinned(true) while already pinned re-asserts the current
presentation (callSurface flaps in the app could leave the window
sized for the wrong state) instead of bailing.
And the requested shortcut: ⌥⇧Space now TOGGLES the Skipper's text
panel during a session — folded → unfold + focus, open → fold away —
handled entirely in main, so the keyboard works even from states where
window controls don't.Co-Authored-By: Claude Fable 5 noreply@anthropic.com
- feat(x): live activity label on the Skipper while a turn runs
"Thinking…" for thirty seconds reads as wedged. The Skipper now shows
WHAT is happening, at tool-name level only ("Searching the web…",
"Reasoning…" — never arguments; the window floats over other apps):- App-side, the summoned bar's existing status derivation (newest
pending/running tool via getToolDisplayName, else reasoning, else
thinking) is hoisted into one liveActivityText memo feeding BOTH the
quick-ask state push and the call mirror (new activityText field). - Rendered everywhere the companion shows status: the Skipper's chip
(both fold states) and text-panel status line, the tucked chip, and
the pill's corner chip — replacing the generic "Thinking…" whenever
an activity is known, falling back to it otherwise. - Anti-flicker: tool bursts would strobe the chip, so labels display
through a minimum ~800ms hold (useHeldLabel) with the newest value
winning after each beat.
Co-Authored-By: Claude Fable 5 noreply@anthropic.com
- chore(x): drop stale comment referencing removed permission sections
Co-Authored-By: Claude Fable 5 noreply@anthropic.com
Co-authored-by: Claude Fable 5 noreply@anthropic.com
下载附件