feat(voice): harden Gemini Live provider integration

docs/VOICE-OWNERSHIP.md

@@ -5,22 +5,22 @@ overall lifetime. It coordinates startup and shutdown. `realtimeFacade.js`
 preserves existing methods and properties by delegation, without copying state.
 The session adapter, action runner and backend interfaces remain unchanged.
 
-| Owner | State and responsibility |
-| --- | --- |
-| `realtimeConnection.js` | Start generation, request cancellation, peer/data channel, acquired microphone stream and playback element |
-| `realtimeBackend.js` | Token and SDP requests, their separate transports, the abort/timeout scope around both, and client-secret expiry |
-| `realtimeTurns.js` | Active and superseded responses, pending confirmations, deduplicated calls and action cancellation |
-| `realtimeRadio.js` | Speaker ducking, prepared playback, stronger-action reservations and playback observers |
-| `realtimeInput.js` | Physical Space gesture, microphone mute, input/output meters and their frame/listener lifetime |
-| `realtimeCost.js` | Next-session preferences and the current session's model-bound meter |
-| `realtimeViewport.js` | One retained image, bounded deletion identities and capture generation |
-| `realtimeDiagnostics.js` | Bounded error history and sanitized optional diagnostics |
-| `turnMetrics.js` | Per-turn latency and speech spans written to the debug log |
-| `narration.js` | Progress lines and the earcon while tools run; spoken through the turns owner |
-| `pointerContext.js` | Pointer position, dwell pick and the current turn's pointer snapshot (session-scoped) |
-| `referents.js` | Numbered items from the latest result for `referent:n` (session-scoped) |
-| `resultDisplay.js` | What the voice card shows for a result and which items it numbers (read by the card and `referents.js`) |
-| `gevActions.js` runner | Analyst follow-up memory (session: `resetConversation()` on stop, `dispose()` on removal) |
+| Owner                    | State and responsibility                                                                                         |
+| ------------------------ | ---------------------------------------------------------------------------------------------------------------- |
+| `realtimeConnection.js`  | Start generation, request cancellation, peer/data channel, acquired microphone stream and playback element       |
+| `realtimeBackend.js`     | Token and SDP requests, their separate transports, the abort/timeout scope around both, and client-secret expiry |
+| `realtimeTurns.js`       | Active and superseded responses, pending confirmations, deduplicated calls and action cancellation               |
+| `realtimeRadio.js`       | Speaker ducking, prepared playback, stronger-action reservations and playback observers                          |
+| `realtimeInput.js`       | Physical Space gesture, microphone mute, input/output meters and their frame/listener lifetime                   |
+| `realtimeCost.js`        | Next-session preferences and the current session's model-bound meter                                             |
+| `realtimeViewport.js`    | One retained image, bounded deletion identities and capture generation                                           |
+| `realtimeDiagnostics.js` | Bounded error history and sanitized optional diagnostics                                                         |
+| `turnMetrics.js`         | Per-turn latency and speech spans written to the debug log                                                       |
+| `narration.js`           | Progress lines and the earcon while tools run; spoken through the turns owner                                    |
+| `pointerContext.js`      | Pointer position, dwell pick and the current turn's pointer snapshot (session-scoped)                            |
+| `referents.js`           | Numbered items from the latest result for `referent:n` (session-scoped)                                          |
+| `resultDisplay.js`       | What the voice card shows for a result and which items it numbers (read by the card and `referents.js`)          |
+| `gevActions.js` runner   | Analyst follow-up memory (session: `resetConversation()` on stop, `dispose()` on removal)                        |
 
 The backend is replaceable per deployment: `realtimeController.js` accepts it as a
 `backend` option and falls back to `createRealtimeBackend()`. Its two method
@@ -45,11 +45,122 @@ Action and post-capture continuations retain their conversation identity before
 publishing output, queueing a response or changing status.
 
 Behavioral invariants remain in force: a 500 ms Space hold claims push-to-talk;
-short control taps and text entry remain native; a claimed hold while the assistant speaks interrupts it (never during a Radio handoff); a click-started session stays
-open-mic. Every accepted or refused tool call receives its terminal output once.
+short control taps and text entry remain native. A claimed hold silences local playback when the Radio speaker guard permits it;
+admitted speech retires the provider turn and supersedes its tool work. A denied local claim
+preserves queued confirmation audio; a later native Gemini interruption retires
+the old reply and cancels its handoff. A click-started session stays open-mic.
+Gemini PTT waits for two consecutive 10 ms PCM windows above a low energy floor
+before submitting pointer context, activity markers or audio. Up to 400 ms of
+bounded pre-roll preserves speech onset; after admission, tails and pauses pass
+through until release. An empty or near-silent hold submits no new provider turn
+and clears its local pending-input state. It still silences existing local
+playback as an explicit Space hold and revokes a pending Radio handoff. A silent
+hold alone preserves the current provider/tool turn until its natural completion. This energy check is not a speech classifier:
+very quiet input below the floor is suppressed and sustained loud noise can pass.
+Open-mic capture and native automatic activity detection are unchanged.
+Current, uncancelled tool calls receive one terminal output within their owning
+provider conversation. Gemini suppresses calls retired by a session or local-turn
+boundary, or cancelled by the server. OpenAI answers superseded calls with a
+terminal refusal while the same conversation remains owned; callbacks from a
+replaced conversation cannot publish into the new session.
 New typed input supersedes old intent. Radio waits for the spoken confirmation and
-verified muted playback. A live cost meter stays bound to its negotiated model,
-even when the next-session preference changes.
+verified muted playback. Eligible stronger Radio controls begin without waiting
+for an older station lookup; a successful control cancels covered pending work.
+Already committed selection and unrelated sibling calls retain their authority.
+“Turn Radio on” enables the layer without selecting or playing a station;
+playback requires an explicit play, listen or resume request. Radio playback
+suspends an active voice session and remembers it until Radio is paused or
+disabled. This also applies to manual playback; merely enabling the layer does
+not suspend voice. Pausing or disabling Radio leaves a previously inactive
+voice session off. Provider replacement or explicit voice Stop cancels the
+saved resume; manually starting voice pauses Radio. A live cost meter stays
+bound to its negotiated model,
+even when the next-session preference changes. Clear standalone first-party
+provider requests remain ordinary provider turns, so the active OpenAI or Gemini
+model says in its normal voice, “I can’t switch voice providers directly. Hover
+over the Voice panel and use the provider dropdown.” Hosts without at least two
+distinct usable chooser options hide that control and
+instead explain that provider selection is managed by the app. The built-in
+endpoint defaults to this neutral guidance; the standalone host opts into chooser
+guidance. No tool can change the selected provider. Authoritative typed requests and
+recognized OpenAI speech block app actions; Gemini native-call refusal is
+described below. These refusals never change or save provider state, reconnect
+voice or remount a session. Ambiguous, quoted, negated, conditional and compound text
+remains ordinary conversation. Manual selection is the only provider switching
+path.
+
+Gemini native tool calls are bound to the current socket, session epoch, local
+turn and unique call ID. Native input captions are unordered advisory segments:
+they are shown without authoritative request attribution and cannot grant or
+revoke a tool's lifetime. First-party typed input retains its authenticated
+request token through transport pressure, deferred delivery and same-turn
+continuation, until real new input or Stop revokes it. An accepted fresh
+`set_voice_provider` call refuses every not-yet-started tool for the rest of its
+local turn; malformed or replayed calls cannot quarantine a new turn. No tool
+changes or saves the selected provider, reconnects voice or remounts a session.
+
+Gemini's sentence-level `ACTIVITY_START`/`ACTIVITY_END` signals track detected
+speech, not proof that a reply generation exists. Automatic activity alone does
+not retire an unanswered generation; observed server output or an explicit
+submitted push-to-talk/typed request retains the reply fence. Local activity
+owners preserve expiry liveness across an older retired completion, without
+identifying server generations or granting tools authority. The old boundary
+cannot clear a newer submitted request's pending reply. If expiry discards a
+newest deferred request, the ordered retired boundary closes the session rather
+than allowing its superseded forwarded predecessor to run under that request.
+Shorter repeated expiry notices retain that supersession. A native terminal
+completion settles even a silent user turn; pending tools, queued controls and
+audio still delay presentation completion, and no speech or captions are invented.
+
+Noise-only automatic activity may receive no generation. During expiry it can
+wait with the microphone muted until the supplied deadline, capped at ten
+minutes; no shorter no-evidence timer assigns reply identity. Native Live events
+do not provide a VAD-sentence generation ID or guarantee their ordering relative
+to tools. Local activity ownership cannot prove that arbitrary undelimited late
+output belongs to an old generation; tools retain the socket/session/local-turn
+and call-ID policy above.
+
+There is no first-party typed-command UI. Embedders and programmatic `sendTextCommand`
+callers can issue rapid typed requests while open-mic input is active. While a
+typed request is deferred behind an older reply boundary, concurrent microphone
+PCM may be discarded for up to that boundary's deadline. It is never replayed
+under the typed owner's identity. `deferredAudioDrops.nonPttOwner` and
+`deferredAudioDrops.closedActivity` report reason-specific lifetime-cumulative
+counts alongside `droppedAudioChunks`; they do not change admission or recovery.
+
+`sendTextCommand` return values are provider-dependent. Gemini returns `true`
+when it admits the request locally (sent or queued) and `false` for empty input,
+an expiring session or failed local admission. Its empty-input and expiry checks
+precede connection validation; ordinary nonempty input requires an open socket.
+Oversized text and some context or queue-limit violations throw. A queued request
+can still be superseded or fail its bounded wait, which status events report.
+OpenAI returns `undefined` for successful submission or connected empty input,
+`false` when sending the user conversation item fails, and throws when it is not
+connected. Local admission does not guarantee provider delivery or execution.
+
+For authoritative typed input and OpenAI speech, provider classification
+normalizes sentence punctuation at word boundaries, so "Switch to. Gemini."
+still identifies the direct request. Later authoritative segments can cancel
+cooperative pending resources, including generic and CCTV layer initialization.
+Pending annotations and their deferred outlines use the calling voice operation's
+cancellation signal. Stop, interruption and replacement cancel only that work;
+already committed marks and unrelated annotation owners remain. Completed actions
+and map-stack changes without caller cancellation cannot be undone. Gemini
+captions do not supply cancellation authority.
+Gemini retires old replies before forwarding deferred input; continued old output
+refreshes its five-second inactivity bound while deferred-input deadlines stay
+absolute. Stop and replacement invalidate both waits and their continuations.
+OpenAI input with explicitly disabled or failed optional transcription retains
+its authentic speech identity for ordinary tools. Missing, empty or untracked
+input cannot supply tool authority, and provider switching remains manual.
+
+OpenAI response reservations remain ordered: a delayed response must consume its
+own slot. More than 16 unmatched reservations ends the owned session and shows
+"Voice lost turn tracking. Start voice again." A deliberate restart restores a
+fresh transport and turn tracker; it never automatically reconnects or assigns
+old responses to new input. A missing response before overflow remains ambiguous
+with a delayed one, so this does not establish native leak frequency or repair
+every possible FIFO mismatch.
 
 Run the unit suite and both live recorded-audio modes against a configured local
 server: