mirror of
https://github.com/bilawalsidhu/gods-eye-view.git
synced 2026-11-06 03:14:00 +00:00
feat(voice): harden Gemini Live provider integration
This commit is contained in:
1 parent
d4492d8f8c
commit
4fde920665
102 files changed
+28528
-1581
No files matched your search
@@ -6,6 +6,11 @@
|
||||
MobilityData's systems catalogue; AMBici's is CC0. `gbfs.nextbike.net` joins
|
||||
the proxy allowlist. These are the layer's first stations in Spain.
|
||||
|
||||
- Suppress empty and near-silent Gemini push-to-talk submissions without
|
||||
losing buffered speech onset or release tails. Space holds silence local
|
||||
playback; only admitted speech retires the current provider/tool turn. Keep narrow-screen toast notices clear of
|
||||
the voice provider selector and bottom dock.
|
||||
|
||||
- Add nine Ontario transit feeds: TTC, MiWay, Hamilton HSR, Durham Region,
|
||||
YRT/Viva, Guelph, Burlington, Barrie and Kingston, each carrying its own
|
||||
licence quote and attribution. A feed can now name its own modes, so Toronto's
|
||||
@@ -130,6 +135,56 @@
|
||||
get an empty body; anything else that cannot become a response fails that
|
||||
mirror so the next one is tried.
|
||||
|
||||
- Cancel pending voice-owned annotation lookups and deferred outlines on Stop,
|
||||
interruption or replacement without clearing committed or unrelated marks.
|
||||
Let stronger Gemini Radio controls start without waiting for older station
|
||||
lookups; successful controls cancel covered pending work while preserving
|
||||
committed selection and unrelated sibling calls. “Turn Radio on” enables the
|
||||
layer without audio; playback requires an explicit play, listen or resume request.
|
||||
Resume voice after Radio pause/off only when playback interrupted active voice;
|
||||
manual playback follows the same rule. Provider changes and explicit voice
|
||||
Stop cancel automatic resume, and manual voice startup pauses Radio.
|
||||
Show a toast when Radio playback stops active voice. A tuner drag that ends
|
||||
paused without a new station also resumes the voice Radio interrupted. An
|
||||
older queued Gemini Radio request cannot start after a later successful
|
||||
Stop, and deferred annotation outlines outlive their tool call for both
|
||||
providers until the next input or session boundary.
|
||||
|
||||
- Keep Gemini sentence-level voice activity separate from observed reply
|
||||
generation, so a pause before the first answer does not retire that answer.
|
||||
Preserve newer submitted input across an older reply's completion and finish
|
||||
silent native replies without inventing audio or captions. Expiry activity
|
||||
bookkeeping does not identify server generations or authorize tools.
|
||||
|
||||
- Reject malformed Gemini tool-call identities before they can affect turn
|
||||
policy. Retain accepted and cancelled identities for the socket lifetime;
|
||||
require an explicit restart before a new batch exceeds the 256-identity budget.
|
||||
|
||||
- Refuse voice tool dispatch when authoritative typed input or OpenAI speech
|
||||
exceeds the normalized 500-character policy limit. Ask for a shorter command
|
||||
without inferring a provider change; cancel cooperative preparation and withhold
|
||||
deferred Radio playback while preserving changes that already completed.
|
||||
|
||||
- Preserve OpenAI speech-command identity with disabled or failed optional
|
||||
transcription. Gemini native tool calls use current socket, session, local-turn
|
||||
and call ownership; unordered input captions are advisory. First-party typed
|
||||
input retains its authority through deferred delivery and same-turn
|
||||
continuation until new input or Stop supersedes it. An accepted fresh
|
||||
`set_voice_provider` call refuses tools that have not started in that local
|
||||
turn; provider selection remains manual. Narrow provider interception, make
|
||||
chooser guidance host-aware, hide single-option choosers, preserve useful
|
||||
bounded failure reasons, and use injected Gemini boundary timers.
|
||||
Later authoritative input can cancel cooperative generic/CCTV preparation;
|
||||
committed changes and non-cooperative map-stack work are not undone.
|
||||
|
||||
- End OpenAI sessions with an explicit restart message when unmatched response
|
||||
reservations overflow. Preserve FIFO ownership and reject stale callbacks;
|
||||
recovery never reconnects automatically or guesses a response's user item.
|
||||
|
||||
- Recognize direct voice-provider requests whose finalized input segments carry
|
||||
sentence punctuation between command and target, such as "Switch to. Gemini."
|
||||
Provider selection remains manual.
|
||||
|
||||
- Snow flakes sway within a fixed range instead of drifting sideways faster the
|
||||
longer the app runs; the Wind setting still scales the sway.
|
||||
|
||||
@@ -372,6 +427,14 @@
|
||||
Mapillary only and hide while GOOGLE STREET VIEW is lit; its globe marker
|
||||
is a Google-blue view cone.
|
||||
|
||||
- Add Gemini Live as an alternative voice provider using a separate server-side
|
||||
API key and short-lived browser credentials. It shares the existing microphone,
|
||||
push-to-talk, result card, map tools and Radio handoff with OpenAI. The manual
|
||||
provider dropdown reconnects active voice in its prior input mode; spoken or
|
||||
typed provider-switch requests give fixed dropdown guidance. Gemini owns
|
||||
device-rate capture, 16 kHz resampling, 24 kHz playback, bounded queues,
|
||||
interruption cleanup and explicit restart after its connection expires.
|
||||
|
||||
- Add an experimental ChatGPT/Codex OAuth option for cloud voice in Provider
|
||||
Settings. API-key voice remains the default. Local sign-in can start from
|
||||
the auth button and reports completion, failure, timeout, or expired
|
||||
|
||||
Reference in new issue
Block a user