feat(voice): voice core — honest layer reach, spoken summary with on-screen card, narration, point-and-ask

- Voice layer manifest (src/voice/layerManifest.js) generates the layer enums,
  aliases and analyst field hints; a test fails when a catalog layer has no entry.
- analyst_query validates fields, operators, units, values and scopes, counts up
  to 250,000 loaded records per layer, reports complete/partial/unanswered, tells
  LAYER_OFF / NOT_READY / FEED_UNAVAILABLE apart from zero, refuses mismatched
  follow-ups and never lets a superseded query replace follow-up memory. Seven
  more layers expose analyst records.
- Tool results carry a code-built say line and a display card; the voice card in
  the mic control shows captions, plan steps and the result
  (src/voice/resultDisplay.js is the one adapter). Slow tools narrate their
  current step out of band; holding Space while the assistant talks cuts it off.
  Turn spans go to the debug log; caption transcription is metered; the log
  omits what was said unless GEV_VOICE_LOG_CONTENT=1.
- Point and ask: the Space-down pointer snapshot resolves "this", "that one" and
  "around here" through 'pointer' targets, and "the second one" through
  referent:n from the last numbered list.
- Prompt states each policy once (instructions + tools 48.5k -> 46.7k chars).
- Voice benchmark harness (scripts/qa-voice-bench.mjs, scripts/voice-bench/)
  shares its phrase table and grading with scripts/qa-voice-routing.mjs.
This commit is contained in:
Bilawal Sidhu committed 2026-09-28 21:19:23 -05:00
1 parent e7707d9a0f
commit a19897ad15
97 files changed
+14343 -755

No files matched your search

+48
View File
@@ -1,5 +1,28 @@
# Changelog
- The voice card shows analyst counts ("At least 250,000 aircraft …") with
their scope, feed window and caveats, and marks partial answers.
- Voice answers are shorter and fill slow work instead of going silent. Slow
or multi-step requests may open with one short spoken plan; instant ones
just happen. Tool results carry a code-built `say` line and a `display`
card: sources and notes stay on screen, a caveat is spoken only when it
changes the answer, and "at least" and partial answers are always said. A card above the mic shows captions, the plan's steps and the
result. When a tool that reports steps (layers, place lookups, outlines,
nearest-aircraft) is still running after 1.5 s its current step is spoken
once; others say "still working" once after 8 s. A soft
tick plays in push-to-talk. Holding Space while the assistant talks cuts it
off. Each turn writes latency and speech figures to the debug log
(`node scripts/voice-turn-spans.mjs`). Captions of what you say are
counted in the voice cost meter; the debug log leaves out what was said
unless `GEV_VOICE_LOG_CONTENT=1`.
- Point and ask: hold Space with the cursor on an aircraft, ship or spot on
the map and say "what's that", "track that one" or "how many flights
around here". A reticle marks the point and the voice card shows what
"this" meant. In open mic it works when you have just moved the cursor
there. "The second one" picks from the last numbered list. Asking about
bare ground up close gives the assistant a small crop of the map around the
point.
- Public Overpass instances are no longer used by default. Street Traffic
roads come from TomTom flow tiles, OpenFreeMap vector tiles, or both, chosen
on the layer row (TomTom / OSM / Hybrid) or with `?trafficRoads=`. With a
@@ -149,6 +172,7 @@
geocode and admin-boundary fallback answers `region-timeout` after 3 s
instead of holding the reply.
- Transit and Directions rows repaint as soon as their data lands again:
`refreshLayerStats()` now lives on the layer lifecycle, not only on the
compatibility facade. `scripts/qa-radio.mjs` uses it instead of a private
@@ -182,6 +206,30 @@
(`src/ui/imagerySplit.js`, `src/maps/imageryComparison.js`), and
`MapSourceController.subscribe()` reports every settled map switch.
## Unreleased — voice reaches every layer; honest analyst answers
- Voice can switch on every shipped data layer (the two scene-driven
Bhote Koshi layers stay scene-only), including transit, Recent
Imagery, wind, the weather layers, cyclones and mapped installations, and
understands their spoken names ("buses", "hurricanes", "military bases").
One voice manifest (`src/voice/layerManifest.js`) generates the tool enums
and aliases, and a unit test fails when a new layer has no entry.
- `analyst_query` counts up to 250,000 loaded records per layer instead of the
first 2,000, and says when a layer went past that (`complete: false`). It
also answers over bikeshare stations, transit vehicles, rocket launches,
cyclones, ALPR cameras, mapped installations, local ADS-B aircraft and fire perimeters; fire
records carry their sensor (VIIRS or MODIS).
- Unknown fields, operators, units, value types, scope kinds and invalid
centres are refused with the allowed values instead of answering zero or widening to anywhere. A
layer that is off, still loading or unavailable is reported as such, not
as a count of zero; a partly answerable query is marked `partial`.
Distance sorts honour `sortDir`, a written unit such as `altitudeM(ft)`
converts, and large rankings no longer sort the whole set.
- Results carry `lat`/`lon` for each item and a `display` object with the
scope, the feed window (earthquakes and fires cover the last 24 h) and any
caveat for the screen. The model is asked to answer in one short phrase.
- The weather and Recent Imagery panels can be opened by voice.
## Unreleased — local receiver feeds
- The Local ADS-B layer also reads local 1090 MHz and 978 MHz UAT decoder