Commit Graph
9 Commits
Author SHA1 Message Date
Sameh Khamis b1c5de0ed1 fix(voice): Gemini voice messages follow a language change
The Gemini session translated each status and error message once, when it
happened, so after a language change the voice control kept showing it in
the old language. The state event now also carries the English catalog text
as sourceDetail, and the control words it when shown. The local server's
known token errors stay catalog text until then instead of being translated
on arrival.
2026-11-10 18:04:52 -08:00
Sameh Khamis fdbff75e1e Format the keyless Gemini voice tests (#1660) 2026-11-10 17:10:45 -08:00
Sameh Khamis b89f81b729 fix(voice): keyless Gemini voice drops the microphone hint
The Gemini token route now answers a missing key with code API_KEY_MISSING,
the session carries it to the error state as missingKey, and the voice
control hides "Check microphone permission" for that error, as it already
does for OpenAI. The OpenAI controller sends the same flag, so the shared
control and the controller agree whichever updates last.
2026-11-10 17:06:50 -08:00
Sameh Khamis dae95e1c7f Refuse a motion after its reply's fly-to found nothing, also on Gemini
Gemini runs one turn's calls one after another. By the time "pan left" ran beside "fly to Xyzzyville", the lookup had already found nothing, so there was no flight to wait on and the pan moved the old view with ok: true. OpenAI's parallel calls refused the same reply.

The runner now remembers, for the latest reply that sent a voice fly-to, whether one moved the camera and whether one found nothing. A tilt, orbit, pan or rotate of that reply with no flight left to wait on is refused when its fly-to found nothing and none moved. The reply is Gemini's turn, passed as a per-turn reply object, or OpenAI's authorityGroup. A motion in a later reply, or beside a fly-to of the same reply that did land, runs as before.
2026-11-05 15:51:14 -08:00
manjunath22466 b2fb4a261e feat(camera): add globe rotation (#1225)
Add continuous globe rotation with direction and speed controls, viewport fit checks, camera ownership handoff, and voice refusal guards.
2026-10-23 14:51:42 -07:00
Sameh Khamis 1c727842df fix(voice): refuse unrequested Gemini tool calls and share pointer turns
in push-to-talk, answer a Gemini tool call that arrives with no input, reply
or tool result outstanding with a failure instead of running it, and refuse
every other call in the reply it starts, including one after unprompted
speech; the reply stays tracked so the next input retires it and its late
boundary cannot close the user's turn

in push-to-talk, a caption that arrives after its turn ended no longer opens
a reply, so the next request is not retired with its tool calls discarded
open mic cannot tell either case from a new reply and stays unchanged

check packet ownership after the narration callback before queuing tools,
and raise the per-connection call history to 4096 IDs

move the pointer turn event, context item and announcement into
pointerContext so the OpenAI and Gemini controllers share them, and drop the
Gemini token route's copy of the shared exact-Origin comparison
2026-10-19 12:30:54 -07:00
manjunath bd2342dc81 test(voice): fix resize fixture overlap and trim duplicate coverage 2026-10-19 22:47:25 +05:30
manjunath 10916227cb refactor(voice): isolate gemini live provider core 2026-10-19 21:40:40 +05:30
manjunath 4fde920665 feat(voice): harden Gemini Live provider integration 2026-10-16 17:09:03 +05:30