The action under way when a reply stops is answered as cancelled, since it
may have taken effect, and only later calls as skipped; a stop during a
round's first action keeps that round too. The closing note no longer
claims the actions succeeded. Tests cover the refusal body read cap and
the release of an unread refusal body.
resolve_area compared the spoken name with the unit's name literally, so
"State of Texas" (admin1) and "USA" or "UK" (country) were refused with
AREA_IDENTITY_MISMATCH although the bundled matcher found exactly those
units. Both identity checks, the bundled one and the outline ladder's, now
also accept the prefix forms and same-unit country aliases the matcher
reads, at the level they imply, so the level check still applies.
"Great Britain", "Britain", "America" and "Korea" still need their own
names, and the natural-region check stays literal.
osm_query said "At least N" when the server capped its list, but handed
the places to the analyst engine with truncated:false, so "which of those
have emergency?" answered as if the list were whole. The handoff now
carries the cap and its caveat, and follow-ups (and their follow-ups)
stay incomplete with an unknown total.
The voice answer waited 25 s for /api/osm/features, while the server let
Overpass run for 25 s and itself waited up to 40 s per endpoint and 45 s
in all. When the browser gave up first it closed the request, the route
cancelled the upstream query, and nothing was cached, so the OSM_TIMEOUT
retry started the same query from scratch and timed out again.
The server now tells Overpass 20 s, waits 22 s per endpoint and 22 s in
all, which ends inside the client's 25 s. A test pins that ordering.
A reply stopped partway through a round now keeps the actions that ran,
answering the calls it never reached as skipped, so the history stays
paired for both providers. A failed typed turn no longer resets the
pointer, referents and follow-up memory, since its conversation continues.
The server now tells a rejected key (401, or Gemini's 400 with reason
API_KEY_INVALID, matched exactly), denied access (403) and an account out
of credit (OpenAI's 429 insufficient_quota) apart; each cools down for a
minute and gets its own message in the chat.
A zoom in during a pivoting tilt stops at the range that keeps the camera 150 m above the pivot at its current pitch. A zoom that moves the range by less than half a metre (at the clearance, the 80 m floor or the 5,000 km cap) now answers not zoomed instead of ok, and a tilt reports tiltRangeAdjusted rather than orbitRadiusAdjusted.
#1308 moved /api/overpass to the keyless cross-site gate: foreign origins
and cross-site fetches are still refused, but forwarding headers are not,
so the map works behind a reverse proxy. Voice place search spends the same
operator Overpass, so it now uses the same gate; with the stricter one,
place search failed behind a proxy while the map kept working. The prompt
lint note records the merged size, 55,370 with an operator Overpass.
Picks up #1308 (Overpass admission), #114 (Sea Marks and Ski Pistes),
#1206 (radio country coverage) and #1435. Resolutions:
- SECURITY.md: main's rewritten cross-site paragraph, with voice place
search listed beside /api/overpass among the keyless map routes.
- scripts/format-scope.json: both sides. A merge on main dropped 97 entries
(being restored there); keeping them, this PR's 12 and main's 2 new ones
keeps every file in format:check. All 1,222 paths exist.
- Translation catalogs: main's two new keys and this PR's 47, no overlap.
- Layer count 36 (33 + OSM Places + Sea Marks + Ski Pistes); both schema
pins re-derived, the first-run length is exactly 30,919 + 292 + 107.
Typed chat sessions no longer end on the voice inactivity timeout, and a
failed or stopped reply keeps the conversation: Stop cancels only the
reply in progress, and map actions that already ran stay in the model's
context. Each upstream request and map action gets its own deadline, old
tool results shrink instead of the history overflowing, and step limits,
a missing or rejected key, the OAuth lock and throttling each get their
own message.
- Server: OpenAI chat is off under GEV_PREFER_CODEX_OAUTH; chat throttles
default to the provider's throttle; replies carry API_KEY_MISSING,
API_KEY_REJECTED, CODEX_OAUTH_REQUIRED and RATE_LIMITED codes; upstream
Retry-After passes through, and a rejected key (Gemini's invalid-key 400
included) cools down for a minute
- Pinokio forwards the chat throttle settings
- UI: the reply reveal publishes only the message list; the border orbit
runs only while busy; layers attach with a button or Enter; the log is
aria-busy while a reply types out; auto-scroll respects a reader who
scrolled up; Escape closes the attach menu; the chat gear says Settings;
a note says chat bills the OpenAI key when voice uses ChatGPT sign-in
- Layout does less work while panel height animates; unused blurs removed
- Gemini voice start ends chat; the voice preview clears on every new
session
- Tests for the route gates, client limits, attachments and the fixes
The two Update-branch merges resolved the locale conflicts by keeping
this branch's side, which dropped nine strings main had added (Utilities
panel, drawing colours, cockpit exit). Rebuild each catalog as main's copy
plus Sea Marks and Ski Pistes, and re-pin the voice tool schema length and
digests on top of main's Utilities target.
- A motion waiting on a voice fly-to follows a later fly-to in the
same reply that cancels it, within one deadline, instead of
reporting that its flight did not arrive. A stale reply stops
following.
- The wait is 15 s, down from 30 s: it holds the tool output, and on
Gemini the later tools in the turn.
- Beyond 1,000 km of range a tilt happens in place instead of
swinging thousands of km about the crosshair point.
- The tilt-nudge limits read the pivot pitch the motion starts from,
including for a voice tilt from a tracked view, whose follow camera
is released before it moves.
- move_camera's argument checks run before the flight wait, so a
malformed call is refused at once.
- Tests: t.after cleanup in the fly-then-motion loop, the pitchDeg
pin asserts the field before removing it, and the handoff-order
pin follows the extracted argument checks.
Tightening the AREAS rule dropped "to outline it (draw=true)", and on Gemini
"Outline the state of Texas" stopped passing draw: none of 5 runs set it, so
the model drew the state again with annotate_map. With the clause back all 4
runs called resolve_area with draw. The prompt is 55,263 characters with an
operator Overpass, ~240 under the 55,500 ceiling.
From review:
- Card lines the actions write in English (area basis, part counts, merged
duplicates, OSM caveats, the imagery window) are shown in the UI
language through the analyst card's note path; the model still gets the
English. One OSM place reads "1 hospital".
- Same-named clarification choices show their region code (WA, DC).
- OSM Places shows as unavailable once the server says no Overpass is
configured, and its points are depth-tested past 2.5 km so they never
show through the globe.
- find_imagery puts the operator's sources and mode back when a box is
refused, and a voice search cancelled mid-flight leaves "Search
cancelled" instead of an empty strip.
- The README lists OSM Places with the other Utilities.
From review:
- The browser client for /api/osm/features gets its own tests (request
shape, abort, network failure, coded refusals), and a 200 whose body is
not JSON is now a failure instead of zero places.
- An end-to-end test runs resolve_area, osm_query and an analyst follow-up
through the real runner and checks the follow-up keeps the area; it fails
with the wiring reverted.
- The Realtime token test pins Overpass off and compares the full tool
schemas again instead of names taken from the environment.
From review:
- An answer the view picked ("Springfield" over Illinois) is reused or
shared only within the same ~2° view cell, so asking again elsewhere no
longer gets the first state's area.
- Admin names compare with accents folded on Latin letters only: "Quebec"
matches "Québec" and "Cordoba" asks between Argentina and Spain, while a
Thai or Devanagari mark still tells names apart.
- A Natural Earth name used by several features is one place with every
part when they touch (the Great Rift Valley, the Lau Group across 180°)
and a question when they are far apart (Flores); natural places can now
be candidates.
- The AREAS rule and the area tool descriptions are tighter, leaving a
~270-character margin under the 55,500 prompt ceiling, and resolve_area
no longer names osm_query when that tool is not offered.
- A zoom during a pivoting tilt now scales the range the tilt holds,
as it already did for orbit, instead of being undone on the next
frame while reporting success. The zoom limits no longer move a view
the wrong way: a zoom in leaves a view closer than 80 m in place, and
a zoom out leaves an orbit beyond 5,000 km where it is.
- A grab, touch or wheel on the map while a voice fly-to is in flight
drops the motion queued behind it. Cesium lands the flight anyway,
so the orbit or tilt would otherwise take the camera back. Chained
fly-tos in one reply keep the earliest snapshot.
- A pivot tilt toward the horizon stops lowering the camera at 150 m
above the pivot and tilts the rest in place, so a close view no
longer ends inside tiles or a hillside.
- pitchDeg is an absolute angle. A direction that contradicts it is
refused, judged from the pitch the swing starts at, and the tool
description says so.
A radius the user gives always draws a buffer, but the enclosing outline
was still looked up first (up to 6 s and a guarded Nominatim request) and
then thrown away. It is no longer requested. The slow-outline test now
checks the default 250 m fallback it was meant to cover.
Three resolver fixes from checking the review focus:
- A natural region named like a country or state no longer hides it.
"India" was the peninsula, "Japan" an island group, "Florida" the
peninsula and "Punjab" a desert, so bare "Punjab" never asked which one.
The region still answers when asked for by level, and when no unit truly
has the name ("Great Britain" finds the United Kingdom only by alias).
- Same-named units get their own candidate: "Washington" offered WA and DC
with one id, so either choice was DC and a DC lookup overwrote a stored
Washington State. Ids now carry the region code, and choices show it.
- A bundled unit that only shares the name ("Azad Kashmir" for "Kashmir")
no longer ends the search; the outline ladder still looks, and the
mismatch is kept if it finds nothing. Retired zones stay refused.
osm_query handed its places to the analyst with no presentation, so "which of
those have emergency?" after "hospitals in Kathmandu" was answered as "anywhere
in the loaded data". The handoff now carries the scope the places were found
in ("inside Kathmandu" with its areaId, or "in view"), as an analyst count over
that area would.
Every name ending in "Zone" was kept without a level and refused as a retired
administrative identity. That is right for Nepal, which replaced its zones in
2015, but zones are current units elsewhere: Ethiopia's "Gamo Zone" was
refused. The selector now keeps the verified level unless display_name places
the zone in Nepal or names no country, and the client no longer reads "Zone"
as a retired level when the source verified one.
The cancelled and failed branches ran before the say-line branch, so a
cancelled panel change no longer said "Cockpit exited; the voice panel was
left as you last set it", and AREA_TIMEOUT / OSM_TIMEOUT got "tell the user it
failed" instead of the tool description's "retry once". A cancelled or failed
result with a say line now has it spoken, and those two timeouts are retried
once. Both providers share this instruction.
"Fly to Paris and orbit it" reaches the app as two sibling tool calls,
which OpenAI runs at once. With a place that needs a lookup, the orbit
started over the old view and the late flight then replaced it; with a
known place, pan and rotate cancelled the flight already in the air.
Orbit, pan and rotate now wait for a pending voice flight the same way
tilt does: up to 30 s, and not run (saying so) if the flight is
cancelled, finds nothing or times out. Stop and rotate_globe do not
wait. Orbit keeps its own arming for flights the voice runner does not
track.
"Fly to London and tilt the camera" reaches the app as two sibling tool
calls, and OpenAI runs them concurrently. The tilt's navigation step
cancelled the flight already in the air, so the camera tilted where it
was; when the place still needed a lookup, the late flight replaced the
tilt and London came out top-down.
The voice runner now tracks its latest flight. A tilt sent while that
flight is still looking up or flying waits for it to land (up to 30 s),
then tilts at the destination. If the flight is cancelled, finds nothing
or times out, the tilt is not run and says so.
A voice fly-to frames a city top-down, at pitch -90. The tilt clamp
rejected any step that left the pitch below -89, so the first frame of
"tilt up" stopped the motion and "tilt down" was refused: from the view
voice lands on, no tilt ever moved the camera. A bare "tilt the camera"
also failed because tilt required a direction.
- Clamp tilt only in the direction of travel, so -90 can tilt up.
- Swing the camera about the ground under the crosshair, so the place
stays centred and the heading is kept. A tracked view still tilts in
place, and a pivot tilt yields when tracking takes the camera.
- Add an optional move_camera pitchDeg for "tilt 45 degrees"; a tilt
with neither angle nor direction settles at -45 (oblique view).
- Re-derive the voice tool schema pins for the new argument and wording,
and keep the Realtime prompt under its 52,000-character budget.
Picks up the Radio Next/Previous fix (#1474) and the Codespaces docs
(#1515). The one conflict is the Radio cycle assertion in
gevActions.test.mjs: main's new fallback flag is kept in this branch's
layout.
- strip control and format characters from OSM tag values, keeping the
zero-width (non-)joiners Persian and Indic names spell with, and cut
them at 120 code points so a name cannot steer or flood the model
- adopt the nine new geometry test files in format-scope.json; five of
them change by formatting only
Both stack parents (#837, #842) are on main, so the voice geometry branch
now builds on main directly. Conflict resolutions keep main's behaviour and
add the geometry work on top: osm_query gating on every provider, restored
admin identity for Nominatim outlines, merged analyst follow-up memory, and
the cancelled Globe View reset instruction ahead of the generic cancellation.
Resolve the CURRENT-STATE.md layer table by keeping main's realigned
table and adding this layer's row. Main added an Arabic locale, so
translate the 12 layer strings and list ar.json in the drone-airspace
package.
Draw and Import used to be reachable by voice through DISPLAY. set_panel_open
now takes 'utilities-panel', and "utilities", "draw", "drawing" and "import"
reach it. This adds one value to the Realtime tool schema, so the pinned
payload digest, enum list and serialized length are re-derived with a note.
The show_data_layers_menu test now finds a row only inside the list that
really holds it, so dropping #utilities-toggles from the lookup fails it.
main moved the panels to Vue views and a one-open-panel rail, and removed
the two templates this branch edited. This merge carries the branch onto
that architecture; it adds no behaviour of its own beyond the port.
- The Utilities panel is a RailPanel in LayerPanelsView with its own rail
item (handyman). LayerPanel renders the Directions rows into its list with
LayerRows, and falls back to Data Layers without it.
- Draw leaves DisplayPanel for DrawControls, DrawMarkList and DrawSwatches,
which render what the draw tool publishes to the display store and call
its actions, as GeoImport does. drawMarks.js words the rows; the DOM-built
drawMarkList.js is gone. A row's fly-to is an immediate navigation like
Import's ('drawing' gets its Cockpit refusal), the name field keeps a
draft through v-field-value, and the colours are one radio group with a
single tab stop.
- UiSegmented draws an option's icon, named by its label, for the shape
buttons.
- The catalogs take main's languages with this branch's strings, Arabic
included, and the tests follow the views to their new homes.
Each overlay is now its own Infrastructure row instead of a chip on one
shared row: sea-marks (token 6) and ski-pistes (token 7), both
enabled-only, built by createOsmOverlayLayers() from one descriptor
list. Drops the option group and its share-codec defaults, gives each
layer its own name, voice entry and CURRENT-STATE row, and shifts the
token-ledger fixtures to the next free digit 8.
Main moved on since this PR opened; the overlay layer is ported to the
current contracts:
- Layer-state token 6 (j belongs to transit; new single-character
tokens are digits) with a reservation-ledger row, and the option owner
renamed to the layer id so the compact share codec can encode it.
- Constructed per catalog by createOsmOverlaysLayer() in
constructCatalog.js; src/standalone/data.js no longer registers layers.
- Drapes through maps/imageryHost.js, so the overlays show on Google 3D
(tileset imagery) as well as the globe stacks, and follow map-stack
switches. Imagery layers are created on enable and destroyed on
disable/destroy.
- Translated chip labels and layer name in every locale, LAYER_NAME_KEYS,
Infrastructure panel group, voice manifest entry.
- Attribution: OpenSeaMap tiles CC BY-SA 2.0, OpenSnowMap credit as
required, OSM ODbL credit for both; bulk-download note in DATA_SOURCES.
- CURRENT-STATE row, changelog fragment, share-codec absent defaults,
updated token-ledger fixtures and voice schema pins.
Chat sits next to the mic in the command dock. Replies are silent and never
request the microphone; the first message connects with the selected provider
and runs the same map actions as voice through each provider's text API
(OpenAI OPENAI_CHAT_MODEL, Gemini GEMINI_CHAT_MODEL via generateContent, which
never starts Live). Only one mode runs at a time.
- Chat panel rendered by the voice control presenter: conversation, recent and
quick-command suggestions, Up/Down history, attachments (map-layer
references or small UTF-8 text files, sent only with the message)
- Chat and voice panels sit above the dock or dock into the right rail's
column; minimized bars sit centred above the dock, and height animates on
swaps
- A centred panel steps down to its bar for the Location and Visual Presets
trays without moving focus, rests above an open tray, and docks right while
expanded over one; it returns to the centre once the trays close
- Minimized voice previews the last sentence from each side; the empty Voice
Preview suggests things to say
- Settings opened from chat keep chat's frame and show the shared AI provider
and quick commands; voice Settings show Back to voice
- Closing chat ends a typed session; the voice panel's close turns voice off
- The dock widens its voice control for the chat button and restyles its
wings, which show icons only at 900px and below; assistant panels share the
left panels' glass, and Cyber keeps its own surfaces
- Server text-chat handlers with body, time and per-IP limits
(GEV_RATELIMIT_*_CHAT_PER_MIN), documented in SECURITY.md; tests and all
eleven locales
Co-authored-by: Darren Dignam <[email protected]>