fix(voice): refuse unrequested Gemini tool calls and share pointer turns

in push-to-talk, answer a Gemini tool call that arrives with no input, reply
or tool result outstanding with a failure instead of running it, and refuse
every other call in the reply it starts, including one after unprompted
speech; the reply stays tracked so the next input retires it and its late
boundary cannot close the user's turn

in push-to-talk, a caption that arrives after its turn ended no longer opens
a reply, so the next request is not retired with its tool calls discarded
open mic cannot tell either case from a new reply and stays unchanged

check packet ownership after the narration callback before queuing tools,
and raise the per-connection call history to 4096 IDs

move the pointer turn event, context item and announcement into
pointerContext so the OpenAI and Gemini controllers share them, and drop the
Gemini token route's copy of the shared exact-Origin comparison
This commit is contained in:
Sameh Khamis committed 2026-10-19 12:30:54 -07:00
1 parent bd2342dc81
commit 1c727842df
7 files changed
+448 -91

No files matched your search

+57
View File
@@ -182,6 +182,63 @@ export function pointerChip(snapshot) {
};
}
/** Dedupe key of the clearing pointer_context item. */
export const POINTER_NONE_KEY = 'none';
/**
* The voice-card event for a turn's pointer: a gesture reveals its target at
* once; open-mic and typed turns reveal it only through announcePointer.
* @param {object|null} snapshot
* @param {boolean} gesture
* @returns {{announced: object|null, pointer: object|null}}
*/
export function pointerTurnEvent(snapshot, gesture) {
const chip = gesture ? pointerChip(snapshot) : null;
return chip
? { announced: snapshot, pointer: { ...chip, screenPx: snapshot.screenPx } }
: { announced: null, pointer: null };
}
/**
* The pointer_context item a turn should send, or null when there is nothing
* new. Without a gesture the model learns only what kind of thing is pointed
* at, never which. Losing a sent pointer produces one clearing item.
* @param {object|null} snapshot
* @param {{at: string, gesture: boolean, sentKey: string|null}} turn
* @returns {{item: object, key: string}|null}
*/
export function pointerTurnContext(snapshot, { at, gesture, sentKey }) {
const chip = pointerChip(snapshot);
const item = !chip
? sentKey && sentKey !== POINTER_NONE_KEY
? { type: 'pointer_context', at, target: 'none' }
: null
: gesture
? pointerContextItem(snapshot)
: {
type: 'pointer_context',
at,
pointing: snapshot.entity?.kind || 'ground',
};
if (!item) return null;
const { at: _at, ...content } = item;
const key = chip ? JSON.stringify(content) : POINTER_NONE_KEY;
return key === sentKey ? null : { item, key };
}
/**
* The voice-card event revealing a pointer a tool resolved, once per snapshot.
* @param {object|null} snapshot
* @param {object|null} announced Snapshot already shown.
* @returns {object|null}
*/
export function pointerAnnouncement(snapshot, announced) {
const chip = pointerChip(snapshot);
return chip && snapshot !== announced
? { ...chip, screenPx: snapshot.screenPx }
: null;
}
/**
* Track the pointer over one element (the globe canvas).
* @param {object} options