feat(voice): voice core — honest layer reach, spoken summary with on-screen card, narration, point-and-ask

- Voice layer manifest (src/voice/layerManifest.js) generates the layer enums,
  aliases and analyst field hints; a test fails when a catalog layer has no entry.
- analyst_query validates fields, operators, units, values and scopes, counts up
  to 250,000 loaded records per layer, reports complete/partial/unanswered, tells
  LAYER_OFF / NOT_READY / FEED_UNAVAILABLE apart from zero, refuses mismatched
  follow-ups and never lets a superseded query replace follow-up memory. Seven
  more layers expose analyst records.
- Tool results carry a code-built say line and a display card; the voice card in
  the mic control shows captions, plan steps and the result
  (src/voice/resultDisplay.js is the one adapter). Slow tools narrate their
  current step out of band; holding Space while the assistant talks cuts it off.
  Turn spans go to the debug log; caption transcription is metered; the log
  omits what was said unless GEV_VOICE_LOG_CONTENT=1.
- Point and ask: the Space-down pointer snapshot resolves "this", "that one" and
  "around here" through 'pointer' targets, and "the second one" through
  referent:n from the last numbered list.
- Prompt states each policy once (instructions + tools 48.5k -> 46.7k chars).
- Voice benchmark harness (scripts/qa-voice-bench.mjs, scripts/voice-bench/)
  shares its phrase table and grading with scripts/qa-voice-routing.mjs.
This commit is contained in:
Bilawal Sidhu committed 2026-09-28 21:19:23 -05:00
1 parent e7707d9a0f
commit a19897ad15
97 files changed
+14343 -755

No files matched your search

+21 -3
View File
@@ -2,7 +2,8 @@
* Protocol-independent voice lifetime and action dispatch.
* An adapter receives { emit, runAction, signal } and supplies start, stop,
* sendText and sendMapEvent. It owns its microphone/connection and wire messages.
* Events: state, transcript, action-call, action-result, interruption, completion.
* Events: state, transcript, action-call, progress, action-result, interruption,
* completion, turn-metrics.
*/
export function createVoiceSession({ createAdapter, runner, signal }) {
if (typeof createAdapter !== 'function' || typeof runner !== 'function')
@@ -63,15 +64,32 @@ export function createVoiceSession({ createAdapter, runner, signal }) {
(!options.isCurrent || options.isCurrent());
try {
if (!isCurrent()) throw abortError();
emit({ type: 'action-call', name, arguments: args });
// Calls carry the adapter's call id so observers can match progress
// and results to the right call when a tool runs twice at once.
const callId = options.callId ?? null;
emit({ type: 'action-call', name, callId, arguments: args });
if (!isCurrent()) throw abortError();
// Step reports reach observers only while the action is current, so a
// cancelled tool cannot update the plan or start narration.
const progress = (update) => {
if (!isCurrent() || !update?.step) return;
emit({
type: 'progress',
name,
callId,
step: update.step,
label: update.label,
});
options.progress?.(update);
};
const result = await runner(name, args, {
...options,
signal: actionSignal,
isCurrent,
progress,
});
if (!isCurrent()) throw abortError();
emit({ type: 'action-result', name, result });
emit({ type: 'action-result', name, callId, result });
return result;
} finally {
actions.delete(action);