fix(voice): preserve request ownership across callbacks

keep typed and tool-follow-up responses attached to their original requests
reject duplicate events before they affect newer work
preserve newer input and Radio handoffs during old callback cleanup
integrate the tested main snapshot without dropping existing history
This commit is contained in:
manjunath committed 2026-10-16 22:29:00 +05:30
commit 8abc99aa0a
26 files changed
+4010 -132

No files matched your search

+42
View File
@@ -7,6 +7,29 @@ deferred Radio playback; completed layer changes or station selection remain
reported truthfully. This limit refusal does not infer a provider request or
change providers.
OpenAI admits each nonempty provider speech-input ID once per owned connection
lifetime, before input, tool, narration or response-queue effects. Stop/reset
clears that exact admission history; storage grows with admitted inputs and ID
bytes. A newer local input or Stop from an observer prevents the older onset
from overwriting it. An admitted onset superseded before becoming current is
remembered only as retired for its later real commit: it grants no authority
and reserves no response early. Existing input-record and response-queue limits
are unchanged.
OpenAI explicitly requested typed and tool-follow-up responses carry a client
request ID in response metadata. Their returned IDs select only that original
request; untagged automatic responses use speech-commit slots and cannot borrow
typed authority. Unknown or malformed client IDs fail stale. Deferred requests
retain their input identity, and Stop or newer input invalidates old captures.
A tool follow-up retains the requesting response's record rather than borrowing
another current input. Gemini keeps its separate existing response binding.
OpenAI response-creation IDs are also admitted once per owned lifetime, before
binding, narration, metrics or response bookkeeping. Other distinct events for
the response continue normally. This exact history grows with response count
and ID bytes and is cleared on Stop/reset. Rejection cleanup completes before
callbacks that may submit newer input, preserving that input without retrying
the rejected request.
Voice retains authentic user-turn identity when OpenAI input transcription is
explicitly disabled or fails. Gemini native tool calls use current socket,
session, local-turn and call ownership; unordered input captions are advisory
@@ -5734,6 +5757,25 @@ when their footprints touch or contain one another, never across a seam toleranc
### Gemini Live provider integration
OpenAI admits each nonempty provider speech item ID once per connection lifetime,
so duplicate or previously admitted starts cannot retire current input, abort its
tools, restart narration or drop a queued typed request. The exact ID history is
retained until Stop/reset rather than evicting older IDs; memory grows with the
number and total size of admitted IDs in that lifetime. New unseen input still
supersedes earlier work. Observer-triggered Stop, replacement or newer typed
input cannot be overwritten when the earlier onset callback resumes.
Typed input keeps its own request identity across an interruption acknowledgment
for a locally known, unobserved Gemini automatic activity. This single-use
predecessor record does not treat captions as authority or make VAD alone wait
for a generated reply. It expires on successor output, completion or newer input;
later unattributed interruption markers retain the existing interruption policy.
A genuine new automatic speech onset also revokes typed input when it arrives
separately during that predecessor wait, without adding another retirement wait.
On OpenAI, a newer speech onset discards a typed response still waiting to be sent;
a deferred typed response with no newer input still answers once. The discarded
request may go unanswered if the superseding speech is noise or gets no response.
Standalone voice supports OpenAI and Gemini; embedded hosts default to OpenAI and opt into Gemini through `voice.supportedProviders`. The provider dropdown uses the existing microphone control and result card. Manual selection reconnects an active session in its prior input mode; push-to-talk stays muted until a fresh Space hold. Voice that was off stays off.
Gemini uses a separate server-side `GEMINI_API_KEY` and one-session ephemeral credentials from `/api/gemini/token`. Device-rate microphone capture is resampled to 16 kHz; playback uses a separate 24 kHz context. Microphone permission precedes token minting. Bounded control queues, retired replies and pending tool actions are invalidated on interruption, Stop, replacement and teardown; input capture identities follow the physical microphone gate and end on Stop, replacement and teardown rather than on every interruption. `goAway` stops new turns and drains the current reply until completion or its supplied deadline, then requires an explicit restart.