mirror of
https://github.com/bilawalsidhu/gods-eye-view.git
synced 2026-11-05 19:04:02 +00:00
fix(voice): preserve request ownership across callbacks
keep typed and tool-follow-up responses attached to their original requests reject duplicate events before they affect newer work preserve newer input and Radio handoffs during old callback cleanup integrate the tested main snapshot without dropping existing history
This commit is contained in:
commit
8abc99aa0a
26 files changed
+4010
-132
No files matched your search
@@ -7,6 +7,29 @@ deferred Radio playback; completed layer changes or station selection remain
|
||||
reported truthfully. This limit refusal does not infer a provider request or
|
||||
change providers.
|
||||
|
||||
OpenAI admits each nonempty provider speech-input ID once per owned connection
|
||||
lifetime, before input, tool, narration or response-queue effects. Stop/reset
|
||||
clears that exact admission history; storage grows with admitted inputs and ID
|
||||
bytes. A newer local input or Stop from an observer prevents the older onset
|
||||
from overwriting it. An admitted onset superseded before becoming current is
|
||||
remembered only as retired for its later real commit: it grants no authority
|
||||
and reserves no response early. Existing input-record and response-queue limits
|
||||
are unchanged.
|
||||
|
||||
OpenAI explicitly requested typed and tool-follow-up responses carry a client
|
||||
request ID in response metadata. Their returned IDs select only that original
|
||||
request; untagged automatic responses use speech-commit slots and cannot borrow
|
||||
typed authority. Unknown or malformed client IDs fail stale. Deferred requests
|
||||
retain their input identity, and Stop or newer input invalidates old captures.
|
||||
A tool follow-up retains the requesting response's record rather than borrowing
|
||||
another current input. Gemini keeps its separate existing response binding.
|
||||
OpenAI response-creation IDs are also admitted once per owned lifetime, before
|
||||
binding, narration, metrics or response bookkeeping. Other distinct events for
|
||||
the response continue normally. This exact history grows with response count
|
||||
and ID bytes and is cleared on Stop/reset. Rejection cleanup completes before
|
||||
callbacks that may submit newer input, preserving that input without retrying
|
||||
the rejected request.
|
||||
|
||||
Voice retains authentic user-turn identity when OpenAI input transcription is
|
||||
explicitly disabled or fails. Gemini native tool calls use current socket,
|
||||
session, local-turn and call ownership; unordered input captions are advisory
|
||||
@@ -5734,6 +5757,25 @@ when their footprints touch or contain one another, never across a seam toleranc
|
||||
|
||||
### Gemini Live provider integration
|
||||
|
||||
OpenAI admits each nonempty provider speech item ID once per connection lifetime,
|
||||
so duplicate or previously admitted starts cannot retire current input, abort its
|
||||
tools, restart narration or drop a queued typed request. The exact ID history is
|
||||
retained until Stop/reset rather than evicting older IDs; memory grows with the
|
||||
number and total size of admitted IDs in that lifetime. New unseen input still
|
||||
supersedes earlier work. Observer-triggered Stop, replacement or newer typed
|
||||
input cannot be overwritten when the earlier onset callback resumes.
|
||||
|
||||
Typed input keeps its own request identity across an interruption acknowledgment
|
||||
for a locally known, unobserved Gemini automatic activity. This single-use
|
||||
predecessor record does not treat captions as authority or make VAD alone wait
|
||||
for a generated reply. It expires on successor output, completion or newer input;
|
||||
later unattributed interruption markers retain the existing interruption policy.
|
||||
A genuine new automatic speech onset also revokes typed input when it arrives
|
||||
separately during that predecessor wait, without adding another retirement wait.
|
||||
On OpenAI, a newer speech onset discards a typed response still waiting to be sent;
|
||||
a deferred typed response with no newer input still answers once. The discarded
|
||||
request may go unanswered if the superseding speech is noise or gets no response.
|
||||
|
||||
Standalone voice supports OpenAI and Gemini; embedded hosts default to OpenAI and opt into Gemini through `voice.supportedProviders`. The provider dropdown uses the existing microphone control and result card. Manual selection reconnects an active session in its prior input mode; push-to-talk stays muted until a fresh Space hold. Voice that was off stays off.
|
||||
|
||||
Gemini uses a separate server-side `GEMINI_API_KEY` and one-session ephemeral credentials from `/api/gemini/token`. Device-rate microphone capture is resampled to 16 kHz; playback uses a separate 24 kHz context. Microphone permission precedes token minting. Bounded control queues, retired replies and pending tool actions are invalidated on interruption, Stop, replacement and teardown; input capture identities follow the physical microphone gate and end on Stop, replacement and teardown rather than on every interruption. `goAway` stops new turns and drains the current reply until completion or its supplied deadline, then requires an explicit restart.
|
||||
|
||||
Reference in new issue
Block a user