Main now offers osm_query, and names it in the prompt, only when Overpass
is configured. Typed chat follows the same switch on both providers: its
prompt, the tools it offers and the calls it accepts.
Typed chat sessions no longer end on the voice inactivity timeout, and a
failed or stopped reply keeps the conversation: Stop cancels only the
reply in progress, and map actions that already ran stay in the model's
context. Each upstream request and map action gets its own deadline, old
tool results shrink instead of the history overflowing, and step limits,
a missing or rejected key, the OAuth lock and throttling each get their
own message.
- Server: OpenAI chat is off under GEV_PREFER_CODEX_OAUTH; chat throttles
default to the provider's throttle; replies carry API_KEY_MISSING,
API_KEY_REJECTED, CODEX_OAUTH_REQUIRED and RATE_LIMITED codes; upstream
Retry-After passes through, and a rejected key (Gemini's invalid-key 400
included) cools down for a minute
- Pinokio forwards the chat throttle settings
- UI: the reply reveal publishes only the message list; the border orbit
runs only while busy; layers attach with a button or Enter; the log is
aria-busy while a reply types out; auto-scroll respects a reader who
scrolled up; Escape closes the attach menu; the chat gear says Settings;
a note says chat bills the OpenAI key when voice uses ChatGPT sign-in
- Layout does less work while panel height animates; unused blurs removed
- Gemini voice start ends chat; the voice preview clears on every new
session
- Tests for the route gates, client limits, attachments and the fixes
Tightening the AREAS rule dropped "to outline it (draw=true)", and on Gemini
"Outline the state of Texas" stopped passing draw: none of 5 runs set it, so
the model drew the state again with annotate_map. With the clause back all 4
runs called resolve_area with draw. The prompt is 55,263 characters with an
operator Overpass, ~240 under the 55,500 ceiling.
From review:
- An answer the view picked ("Springfield" over Illinois) is reused or
shared only within the same ~2° view cell, so asking again elsewhere no
longer gets the first state's area.
- Admin names compare with accents folded on Latin letters only: "Quebec"
matches "Québec" and "Cordoba" asks between Argentina and Spain, while a
Thai or Devanagari mark still tells names apart.
- A Natural Earth name used by several features is one place with every
part when they touch (the Great Rift Valley, the Lau Group across 180°)
and a question when they are far apart (Flores); natural places can now
be candidates.
- The AREAS rule and the area tool descriptions are tighter, leaving a
~270-character margin under the 55,500 prompt ceiling, and resolve_area
no longer names osm_query when that tool is not offered.
One provenanceLine() builds STATE | text | LAYERS for the HUD and
get_hud_caption. hasWord is one lookaround regex, capAtWord one grapheme
pass. Feed-state helpers are imported from layerSnapshot.js directly, and the
kept-whole sanitizer cases are one table.
Both stack parents (#837, #842) are on main, so the voice geometry branch
now builds on main directly. Conflict resolutions keep main's behaviour and
add the geometry work on top: osm_query gating on every provider, restored
admin identity for Nominatim outlines, merged analyst follow-up memory, and
the cancelled Globe View reset instruction ahead of the generic cancellation.
Chat sits next to the mic in the command dock. Replies are silent and never
request the microphone; the first message connects with the selected provider
and runs the same map actions as voice through each provider's text API
(OpenAI OPENAI_CHAT_MODEL, Gemini GEMINI_CHAT_MODEL via generateContent, which
never starts Live). Only one mode runs at a time.
- Chat panel rendered by the voice control presenter: conversation, recent and
quick-command suggestions, Up/Down history, attachments (map-layer
references or small UTF-8 text files, sent only with the message)
- Chat and voice panels sit above the dock or dock into the right rail's
column; minimized bars sit centred above the dock, and height animates on
swaps
- A centred panel steps down to its bar for the Location and Visual Presets
trays without moving focus, rests above an open tray, and docks right while
expanded over one; it returns to the centre once the trays close
- Minimized voice previews the last sentence from each side; the empty Voice
Preview suggests things to say
- Settings opened from chat keep chat's frame and show the shared AI provider
and quick commands; voice Settings show Back to voice
- Closing chat ends a typed session; the voice panel's close turns voice off
- The dock widens its voice control for the chat button and restyles its
wings, which show icons only at 900px and below; assistant panels share the
left panels' glass, and Cyber keeps its own surfaces
- Server text-chat handlers with body, time and per-IP limits
(GEV_RATELIMIT_*_CHAT_PER_MIN), documented in SECURITY.md; tests and all
eleven locales
Co-authored-by: Darren Dignam <[email protected]>
Han and kana have no word edges, so a CJK forbidden word now counts anywhere
in the reply. The proxy no longer counts words: the model reads Rue de Rivoli
as one, so only the 80-character word-boundary cap remains.
Also restore src/voice/gevActions.test.mjs, reformatted by mistake in 818d2c82.
- The proxy keeps combining marks, in-word apostrophes and name dots
(ar vowel signs, fr d'Orléans, ja ・), normalizes to NFC, caps every
summary in characters and cuts at a word boundary.
- feedState is nullable instead of a `none` token, and the summary has at
most five words: "exactly five" made the model pad short views with
"no data" or "unavailable".
- The HUD caches only the model's words and adds the current warning on
every paint; an RTL locale gets a leading right-to-left mark.
- The echo check also rejects an all-Latin reply in a non-Latin language,
translated layer ids and, with every feed healthy, the UNAVAILABLE word;
places are matched by word and one-word layer names are skipped in
Latin-script locales.
- The warning tag names the worst layers first; a partial analyst card
shows one partial chip.
- Skip the model when the view has layers but no place, street or nearby
label: with layers alone it invented a place.
- The English-echo check treats only Latin letters as part of a word, so
ids flush against kana or Han and hyphenated names are caught; it skips
nominal, state words a locale keeps in English and names inside place
labels.
- Outside English, per-layer state ids are left out of the request.
- The ja/zh cap steps back to the start of a Latin word, not the last space.
- Fold the CJK cap cases into hudSummaryResponse.test.mjs.
Second follow-up review fixes:
- Japanese and Chinese summaries are capped at 20 characters only, without
the five-token cut first, and the cap steps back to a space instead of
splitting a Latin word.
- The HUD request sends localized layer names in feedProvenance too and
leaves the English narration note out outside English. A non-English reply
that still echoes a feed-state id or a translated English layer name falls
back to the deterministic line.
- A camera move during typing shows the deterministic line, not a fragment.
- A view with no place labels and no enabled layer does not ask the model.
- Behaviour tests on IntelHUD replace the regex checks of hud.js; docs and
the instruction wording updated.
Follow-up review fixes:
- The HUD line is STATE | text | LAYERS for the AI summary and the
fallback line, so the narrow minimal and cyber layouts show both the
warning and the summary. get_hud_caption builds its caption the same
way. The unused "{state} {layers}" key is pruned from the catalogs.
- The 20-character cap applies only when the request locale is Japanese
or Chinese; other languages keep five words around a Han place name.
- enabledLayers names are localized too, and the instructions name layers
only from enabledLayerLabels, never from feedProvenance or its note.
- The keyless test restores GEV_RATELIMIT_OPENAI_PER_MIN again; comments
and docs updated.
Review fixes for the HUD summary language change:
- The translated feed-state tag now leads the one-line summary and the
fallback line, so the readout's ellipsis cannot hide it.
- get_hud_caption checks the reply's feedState, returns it and appends the
state tag; a reply naming another state keeps only the place.
- The model gets the layer names the user sees (localizeLayerName).
- Japanese and Chinese summaries are capped at 20 characters; pt, zh-CN and
zh-Hant get explicit language names.
- A partial view is narrated as incomplete coverage, not as not live.
- provenanceTag and hudSummaryMatchesProvenance move to layerSnapshot.js so
the tools package can use them without browser globals.
- docs/CURRENT-STATE.md, changelog and comments updated; redundant tests
trimmed.
The HUD sends its locale with the summary context; the proxy names that
language in the instructions and the input, English when missing or
unsupported. The model returns {summary, feedState}; the HUD shows the
summary only when feedState matches its own state for a view that is not
nominal, and appends its own translated state tag.
partial joins the shared feed-state table between fallback and nominal,
so one partial layer no longer reads off and partial plus nominal no
longer reads nominal.
Snow is no longer offered anywhere a style can be chosen: the Visual
Presets tray, keyboard shortcut 7, the Cockpit vision carousel, and the
set_visual_style voice action. The shader and its labels go with it;
weather snowfall and the SNOW weather condition are unaffected.
Share links keep working. Snow's enum code (6) stays reserved, so a
compact or long-form link that names it opens on Normal and drops the
Snow shader parameters without reporting them as unknown. A Scene
Director shot saved with Snow plays on Normal too. Snow's frozen share
defaults matched Normal's apart from those parameters, so no other
setting in an old link changes meaning.
The legacy voice-schema pins restore Snow before hashing, so the
unchanged pins still prove nothing else moved; the complete Realtime
payload digest is re-derived.
Display > HUD > Layout and the set_hud voice action now offer Tactical,
Minimal and Cyber. The Operator option, its CSS and its QA coverage are
removed, and the unused "Operator" translations are pruned.
Share-link codes are append-only, so operator keeps its hudVariant code.
A link, scene or saved state that names it restores the default Tactical
layout. The voice schema pins keep their legacy digests by restoring the
retired value before comparing; the full-payload digest is re-derived.
The HUD builds place context for its AI summary whenever the camera
settles, and each build paid for up to four reverse geocodes plus a Nearby
Places search, even when no OpenAI key was configured to use them.
- Ask GET /api/openai/hud-summary/status once; with no summary configured,
or after an unconfigured reply, make no place lookups for the page.
- Reverse-geocode at most two points: the view center and one viewport
point.
- Reuse a lookup until the view center moves a quarter of the visible
width, the camera's distance to it doubles or halves, or the camera
crosses a lookup height. The reused points are asked again, so only a
timeout, failure or empty answer spends.
- Repaint the last AI reply when a small pan leaves the context unchanged.
Cached-only hosts and voice context lookups are unchanged.
add the shared Voice Preview and Settings panel with browser-local preview
and inactivity preferences, keyboard controls and responsive positioning
preserve main’s Rotation and Context behavior, clarify panel routing and
refresh schema pins within the unchanged prompt limit
Knip, configured with the application, package exports, scripts, QA
harnesses, workers, tools and tests as entry points, reports exports
that no module imports. Each was checked by hand against string-based
and namespace imports.
- Symbols that are still used inside their own module lose only the
`export` keyword (176 constants and helpers).
- Unused re-exports are removed from barrel modules (flight, military,
vessel, CCTV, directions, transit history, local ADS-B and street-level
policy modules).
- Declarations that nothing uses at all are deleted, among them
AIS_WATCHDOG_STATUSES, applyFilter, isRadioCountryCode, compactText,
DENSITY_STOPS, the unused CCTV endpoint constants, and the SDR band
constants.
- Default exports that nothing imports are removed. The ones in
src/data/earthquakes.js, src/data/telegeographySubmarineCables.js,
src/data/wind.js and src/data/bhoteKoshiLocator.js each constructed a
layer instance at import time that nothing used; those instances are
no longer created.
Every export of a package.json entry stays, as does TOMTOM_CREDIT,
which the traffic layer reads from the credits namespace it is given.
The login child inherited the whole server environment, including
OPENAI_API_KEY and every other provider credential. Pass only what Codex
needs to find its home, open a browser and reach OpenAI through a proxy.
Keeps the voice-core prompt rules and adds main's world-query and
vessels-vs-visuals rules; the tool catalog wraps the deixis-aware runner.
Also fixes two things the branch had before the merge:
- Live voice could not start: OpenAI rejects track_entity's top-level
anyOf. Model-facing tools now omit top-level combinators; the runner
and benchmark still require a query or a referent.
- The Contacts-centred analyst answer dropped the viewport coverage note.
- turn named places and mapped landmarks into reusable bounded areas
- count loaded records inside resolved, drawn and annotated geometry
- find recent imagery for an area, the view or the pointer snapshot
- keep osm place search behind an operator-configured overpass endpoint
- preserve cancellation, ownership and follow-up memory across every action
- cover geometry, provider lifecycle, cache isolation and voice routing
The voice debug log writes the server's timestamp after the client's record,
so a record can no longer replace loggedAt. The dev and preview servers also
send X-Content-Type-Options: nosniff with their shared browser headers.
From the findings in #710.
Co-authored-by: Sunil <[email protected]>
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
The OpenAI endpoints (/api/realtime/token, /api/openai/hud-summary) now
allow 30 requests per minute per client IP and the Google Places endpoints
120, the caps the Pinokio build already ships. GEV_RATELIMIT_OPENAI_PER_MIN
and GEV_RATELIMIT_GOOGLE_PER_MIN still override them and exactly 0 disables
them; an unreadable value falls back to the default rather than to
unlimited.
Salvaged from #683 onto current main.
Co-authored-by: Sameh Khamis <[email protected]>
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Voice hears audio directly, and "vessels" is easily heard as "visuals",
which the instructions route to visual styles, so asking for vessels turned
on other layers or a CRT style. The instructions now say a vessels request
without a style name means the ships layer, and a style needs its name.
The voice instructions recommended aircraft_in_area, vessels_in_area and
get_earthquakes, which voice no longer offers, and the docs said voice gets
nearly every query. The instructions now name tools voice has and send
aircraft, ship and hazard questions to analyst_query; a test keeps the
instructions to tools the voice session lists.
- Voice layer manifest (src/voice/layerManifest.js) generates the layer enums,
aliases and analyst field hints; a test fails when a catalog layer has no entry.
- analyst_query validates fields, operators, units, values and scopes, counts up
to 250,000 loaded records per layer, reports complete/partial/unanswered, tells
LAYER_OFF / NOT_READY / FEED_UNAVAILABLE apart from zero, refuses mismatched
follow-ups and never lets a superseded query replace follow-up memory. Seven
more layers expose analyst records.
- Tool results carry a code-built say line and a display card; the voice card in
the mic control shows captions, plan steps and the result
(src/voice/resultDisplay.js is the one adapter). Slow tools narrate their
current step out of band; holding Space while the assistant talks cuts it off.
Turn spans go to the debug log; caption transcription is metered; the log
omits what was said unless GEV_VOICE_LOG_CONTENT=1.
- Point and ask: the Space-down pointer snapshot resolves "this", "that one" and
"around here" through 'pointer' targets, and "the second one" through
referent:n from the last numbered list.
- Prompt states each policy once (instructions + tools 48.5k -> 46.7k chars).
- Voice benchmark harness (scripts/qa-voice-bench.mjs, scripts/voice-bench/)
shares its phrase table and grading with scripts/qa-voice-routing.mjs.
Adds a Local ADS-B layer fed by your own receivers: a browser WebUSB
RTL-SDR (with FM listening) and local 1090/978 MHz decoder feeds
(LOCAL_RECEIVER_FEEDS, local and private addresses only). Local aircraft
draw in magenta with class, silhouettes, 3D models, trails and a position
sanity filter; the click card names the band and input that heard them.
Builds on Sameh Khamis's SDR prototype and the initial GEV port by
Mazeyar Moeini Feizabadi (#134); the decoder-feed design follows the
notes on #57. Closes#57.
Co-authored-by: Sameh Khamis <[email protected]>
Co-authored-by: Mazeyar Moeini Feizabadi <[email protected]>
Wind (NOAA GFS and ECMWF IFS: animated particles, colour field, a reading at map centre), observed weather (NOAA MRMS radar, GOES and NESDIS infrared clouds, lightning density) on one shared history timeline, NHC and CPHC cyclone advisories with tracks and cones, and a WEATHER panel in the right rail. On 3D Tiles the observed products render as raised shells with a sharper window around the view.
Co-authored-by: Gustavo Beneduzi <[email protected]>
Co-authored-by: Daniel Slay <[email protected]>
Co-authored-by: Rehaan Delmotra <[email protected]>
Co-authored-by: Cursor Agent <[email protected]>
manjunath22466