mirror of
https://github.com/bilawalsidhu/gods-eye-view.git
synced 2026-11-05 19:04:02 +00:00
feat(voice): harden Gemini Live provider integration
This commit is contained in:
1 parent
d4492d8f8c
commit
4fde920665
102 files changed
+28528
-1581
No files matched your search
@@ -662,3 +662,14 @@ One heads-up from the inside: build in this space for a week and you learn that
|
||||
**🌐 God's Eye View. No place left behind.**
|
||||
|
||||
</div>
|
||||
|
||||
|
||||
### Gemini voice provider
|
||||
|
||||
For Gemini, create a separate API key in [Google AI Studio](https://aistudio.google.com/apikey) and add it as `GEMINI_API_KEY` in **Provider Settings**. The Google Maps key does not configure Gemini voice. See Google's [API key setup and security guidance](https://ai.google.dev/gemini-api/docs/api-key). The rest of the app works without either voice key.
|
||||
|
||||
Gemini uses the Developer API's Live API with `gemini-3.8-live` by default (`GEMINI_LIVE_MODEL` overrides it on the server). AI Studio is where you create its key. Free-tier availability depends on the model and project quotas; see the [Gemini pricing](https://ai.google.dev/gemini-api/docs/pricing) and [rate limits](https://ai.google.dev/gemini-api/docs/rate-limits). A connection or quota failure appears in the voice error tray. The [ephemeral token](https://ai.google.dev/gemini-api/docs/live-api/ephemeral-tokens) must start its one new session within 60 seconds and is configured to expire after 30 minutes, but that token lifetime does not promise an uninterrupted 30-minute conversation: Google limits a Live connection to [around 10 minutes](https://ai.google.dev/gemini-api/docs/live-api/session-management). This version intentionally does not resume the session. When Gemini sends `goAway`, GEV stops admitting new turns and drains the current turn until its reply finishes or the supplied deadline expires, then releases local resources. Start voice again to request a fresh token and a new conversation. The same explicit restart is required before a session exceeds 256 distinct tool-call identities; old identities remain blocked for that connection.
|
||||
|
||||
For a clear standalone spoken or typed request to switch voice providers, the active model says in its normal voice, “I can’t switch voice providers directly. Hover over the Voice panel and use the provider dropdown.” No tool can change the selected provider. Authoritative typed requests and recognized OpenAI speech block app actions; Gemini native speech uses current socket, session, local-turn and call ownership, with captions treated as advisory. An accepted fresh `set_voice_provider` call refuses not-yet-started tools in that Gemini turn without undoing completed work. These refusals do not write the saved provider preference, reconnect voice or remount a session. Ambiguous, quoted, negated, conditional and compound text remains ordinary conversation. The manual dropdown is the only provider-switching path. Gemini captures at the device rate, resamples input to 16 kHz and plays output through a separate 24 kHz context. Microphone permission is obtained before minting its short-admission token. Congestion may drop obsolete microphone or image frames; reliable control and tool results have bounded queue and recovery limits.
|
||||
|
||||
Gemini Live is available alongside OpenAI in the standalone app. Choose a supported provider, then use the existing microphone button or hold Space. The API-key provider setup remains server-owned. Gemini has no OpenAI cost estimate or spend cap; OpenAI retains its configured cost controls.
|
||||
Reference in new issue
Block a user