OpenAI refused every typed chat request with 400 invalid_function_parameters:
resolve_area's area-id pattern uses Unicode property classes (\p{L}), which
Chat Completions does not accept as a regex. OpenAI chat now leaves such
patterns out of the tool schemas it sends; each action still checks its
arguments against its own contract, and realtime voice keeps the patterns.
The trim that fitted the tilt wording under the Realtime prompt ceiling dropped the -45° default from pitchDeg, so the model no longer read why a bare tilt goes oblique. pitchDeg reads 'Absolute; -45 default, -89 top-down.' again, and the direction description becomes 'Pan/rotate/tilt steps; orbit/rotate_globe default right.' to pay for it: the Overpass prompt measures 55,499, 1 under the 55,500 ceiling. The schema pin and the measured-size note follow.
With main merged, the Overpass prompt measured 55,549 against the 55,500 ceiling. The pitchDeg and direction descriptions now read 'Absolute; -89 top-down.' and 'Needed for pan/rotate/tilt steps', bringing it to 55,497. The schema pin and the measured-size note follow.
- A tilt's angle, limits and "unchanged" check use the camera's own
pitch. At regional range the pivot pitch is up to about 8 degrees
shallower, so "tilt to 5 degrees" from 900 km ended at -12.9 degrees.
The swing now solves for the pivot pitch that gives the camera angle.
- A zoom out during a pivoting tilt stops at the 1,000 km pivot limit
instead of the orbit's 5,000 km, and a zoom in that reaches the
clearance is applied before the pivot ends.
- An angle the camera already holds still takes navigation, so it stops
a running motion and releases the follow camera.
- A motion queued behind a fly-to moves to a newer fly-to as soon as it
begins, so a stalled lookup no longer holds it past a flight that
landed. Wait listeners are removed when a wait ends, and a lookup that
found nothing points straight at the flight that moved.
- A fly-to that stops a running motion names it (stoppedMotion).
- A tilt left or right is refused, and the pitchDeg description says
steps use direction.
Cockpit entry no longer needs Contacts or both flight feeds: a civilian
aircraft needs Live Flights, a military one Military Flights. Contacts and
every other layer can be turned on or off before or during a session, and
turning off the ridden layer ends it. A Context mode change in flight still
blocks entry.
When the other air feed is off, Live Signals and the Contact card say
FLIGHTS FEED OFF or MILITARY FEED OFF · PARTIAL PICTURE instead of
presenting a partial picture as complete. Voice refuses entry with
subject-layer-off rather than contacts-inactive, and its tool description
says Contacts is not required.
Double-clicking the followed aircraft enters Cockpit through the same gate.
The Cockpit takes the scene's double-click before Cesium's stock handler,
which would otherwise untrack a followed aircraft whose model is a
primitive. The entry also shows under the Intel HUD readout, sharing the
Contacts button's face. A test keeps the classes the Cockpit adds to the
body from matching another element's rule.
Main now offers osm_query, and names it in the prompt, only when Overpass
is configured. Typed chat follows the same switch on both providers: its
prompt, the tools it offers and the calls it accepts.
Re-pin the Realtime tool payload hash for the merged tree (voice geometry plus pitchDeg). To fit main's 55,500 prompt ceiling with Overpass on, the pitchDeg schema drops its -90..90 range (the runner already clamps to -89..-5, now pinned by a test) and its description is shorter. The Overpass prompt measures 55,492.
Typed chat sessions no longer end on the voice inactivity timeout, and a
failed or stopped reply keeps the conversation: Stop cancels only the
reply in progress, and map actions that already ran stay in the model's
context. Each upstream request and map action gets its own deadline, old
tool results shrink instead of the history overflowing, and step limits,
a missing or rejected key, the OAuth lock and throttling each get their
own message.
- Server: OpenAI chat is off under GEV_PREFER_CODEX_OAUTH; chat throttles
default to the provider's throttle; replies carry API_KEY_MISSING,
API_KEY_REJECTED, CODEX_OAUTH_REQUIRED and RATE_LIMITED codes; upstream
Retry-After passes through, and a rejected key (Gemini's invalid-key 400
included) cools down for a minute
- Pinokio forwards the chat throttle settings
- UI: the reply reveal publishes only the message list; the border orbit
runs only while busy; layers attach with a button or Enter; the log is
aria-busy while a reply types out; auto-scroll respects a reader who
scrolled up; Escape closes the attach menu; the chat gear says Settings;
a note says chat bills the OpenAI key when voice uses ChatGPT sign-in
- Layout does less work while panel height animates; unused blurs removed
- Gemini voice start ends chat; the voice preview clears on every new
session
- Tests for the route gates, client limits, attachments and the fixes
Tightening the AREAS rule dropped "to outline it (draw=true)", and on Gemini
"Outline the state of Texas" stopped passing draw: none of 5 runs set it, so
the model drew the state again with annotate_map. With the clause back all 4
runs called resolve_area with draw. The prompt is 55,263 characters with an
operator Overpass, ~240 under the 55,500 ceiling.
From review:
- An answer the view picked ("Springfield" over Illinois) is reused or
shared only within the same ~2° view cell, so asking again elsewhere no
longer gets the first state's area.
- Admin names compare with accents folded on Latin letters only: "Quebec"
matches "Québec" and "Cordoba" asks between Argentina and Spain, while a
Thai or Devanagari mark still tells names apart.
- A Natural Earth name used by several features is one place with every
part when they touch (the Great Rift Valley, the Lau Group across 180°)
and a question when they are far apart (Flores); natural places can now
be candidates.
- The AREAS rule and the area tool descriptions are tighter, leaving a
~270-character margin under the 55,500 prompt ceiling, and resolve_area
no longer names osm_query when that tool is not offered.
- A zoom during a pivoting tilt now scales the range the tilt holds,
as it already did for orbit, instead of being undone on the next
frame while reporting success. The zoom limits no longer move a view
the wrong way: a zoom in leaves a view closer than 80 m in place, and
a zoom out leaves an orbit beyond 5,000 km where it is.
- A grab, touch or wheel on the map while a voice fly-to is in flight
drops the motion queued behind it. Cesium lands the flight anyway,
so the orbit or tilt would otherwise take the camera back. Chained
fly-tos in one reply keep the earliest snapshot.
- A pivot tilt toward the horizon stops lowering the camera at 150 m
above the pivot and tilts the rest in place, so a close view no
longer ends inside tiles or a hillside.
- pitchDeg is an absolute angle. A direction that contradicts it is
refused, judged from the pitch the swing starts at, and the tool
description says so.
A voice fly-to frames a city top-down, at pitch -90. The tilt clamp
rejected any step that left the pitch below -89, so the first frame of
"tilt up" stopped the motion and "tilt down" was refused: from the view
voice lands on, no tilt ever moved the camera. A bare "tilt the camera"
also failed because tilt required a direction.
- Clamp tilt only in the direction of travel, so -90 can tilt up.
- Swing the camera about the ground under the crosshair, so the place
stays centred and the heading is kept. A tracked view still tilts in
place, and a pivot tilt yields when tracking takes the camera.
- Add an optional move_camera pitchDeg for "tilt 45 degrees"; a tilt
with neither angle nor direction settles at -45 (oblique view).
- Re-derive the voice tool schema pins for the new argument and wording,
and keep the Realtime prompt under its 52,000-character budget.
One provenanceLine() builds STATE | text | LAYERS for the HUD and
get_hud_caption. hasWord is one lookaround regex, capAtWord one grapheme
pass. Feed-state helpers are imported from layerSnapshot.js directly, and the
kept-whole sanitizer cases are one table.
Both stack parents (#837, #842) are on main, so the voice geometry branch
now builds on main directly. Conflict resolutions keep main's behaviour and
add the geometry work on top: osm_query gating on every provider, restored
admin identity for Nominatim outlines, merged analyst follow-up memory, and
the cancelled Globe View reset instruction ahead of the generic cancellation.
Chat sits next to the mic in the command dock. Replies are silent and never
request the microphone; the first message connects with the selected provider
and runs the same map actions as voice through each provider's text API
(OpenAI OPENAI_CHAT_MODEL, Gemini GEMINI_CHAT_MODEL via generateContent, which
never starts Live). Only one mode runs at a time.
- Chat panel rendered by the voice control presenter: conversation, recent and
quick-command suggestions, Up/Down history, attachments (map-layer
references or small UTF-8 text files, sent only with the message)
- Chat and voice panels sit above the dock or dock into the right rail's
column; minimized bars sit centred above the dock, and height animates on
swaps
- A centred panel steps down to its bar for the Location and Visual Presets
trays without moving focus, rests above an open tray, and docks right while
expanded over one; it returns to the centre once the trays close
- Minimized voice previews the last sentence from each side; the empty Voice
Preview suggests things to say
- Settings opened from chat keep chat's frame and show the shared AI provider
and quick commands; voice Settings show Back to voice
- Closing chat ends a typed session; the voice panel's close turns voice off
- The dock widens its voice control for the chat button and restyles its
wings, which show icons only at 900px and below; assistant panels share the
left panels' glass, and Cyber keeps its own surfaces
- Server text-chat handlers with body, time and per-IP limits
(GEV_RATELIMIT_*_CHAT_PER_MIN), documented in SECURITY.md; tests and all
eleven locales
Co-authored-by: Darren Dignam <[email protected]>
Han and kana have no word edges, so a CJK forbidden word now counts anywhere
in the reply. The proxy no longer counts words: the model reads Rue de Rivoli
as one, so only the 80-character word-boundary cap remains.
Also restore src/voice/gevActions.test.mjs, reformatted by mistake in 818d2c82.
- The proxy keeps combining marks, in-word apostrophes and name dots
(ar vowel signs, fr d'Orléans, ja ・), normalizes to NFC, caps every
summary in characters and cuts at a word boundary.
- feedState is nullable instead of a `none` token, and the summary has at
most five words: "exactly five" made the model pad short views with
"no data" or "unavailable".
- The HUD caches only the model's words and adds the current warning on
every paint; an RTL locale gets a leading right-to-left mark.
- The echo check also rejects an all-Latin reply in a non-Latin language,
translated layer ids and, with every feed healthy, the UNAVAILABLE word;
places are matched by word and one-word layer names are skipped in
Latin-script locales.
- The warning tag names the worst layers first; a partial analyst card
shows one partial chip.
- Skip the model when the view has layers but no place, street or nearby
label: with layers alone it invented a place.
- The English-echo check treats only Latin letters as part of a word, so
ids flush against kana or Han and hyphenated names are caught; it skips
nominal, state words a locale keeps in English and names inside place
labels.
- Outside English, per-layer state ids are left out of the request.
- The ja/zh cap steps back to the start of a Latin word, not the last space.
- Fold the CJK cap cases into hudSummaryResponse.test.mjs.
Second follow-up review fixes:
- Japanese and Chinese summaries are capped at 20 characters only, without
the five-token cut first, and the cap steps back to a space instead of
splitting a Latin word.
- The HUD request sends localized layer names in feedProvenance too and
leaves the English narration note out outside English. A non-English reply
that still echoes a feed-state id or a translated English layer name falls
back to the deterministic line.
- A camera move during typing shows the deterministic line, not a fragment.
- A view with no place labels and no enabled layer does not ask the model.
- Behaviour tests on IntelHUD replace the regex checks of hud.js; docs and
the instruction wording updated.
Follow-up review fixes:
- The HUD line is STATE | text | LAYERS for the AI summary and the
fallback line, so the narrow minimal and cyber layouts show both the
warning and the summary. get_hud_caption builds its caption the same
way. The unused "{state} {layers}" key is pruned from the catalogs.
- The 20-character cap applies only when the request locale is Japanese
or Chinese; other languages keep five words around a Han place name.
- enabledLayers names are localized too, and the instructions name layers
only from enabledLayerLabels, never from feedProvenance or its note.
- The keyless test restores GEV_RATELIMIT_OPENAI_PER_MIN again; comments
and docs updated.
Review fixes for the HUD summary language change:
- The translated feed-state tag now leads the one-line summary and the
fallback line, so the readout's ellipsis cannot hide it.
- get_hud_caption checks the reply's feedState, returns it and appends the
state tag; a reply naming another state keeps only the place.
- The model gets the layer names the user sees (localizeLayerName).
- Japanese and Chinese summaries are capped at 20 characters; pt, zh-CN and
zh-Hant get explicit language names.
- A partial view is narrated as incomplete coverage, not as not live.
- provenanceTag and hudSummaryMatchesProvenance move to layerSnapshot.js so
the tools package can use them without browser globals.
- docs/CURRENT-STATE.md, changelog and comments updated; redundant tests
trimmed.
The HUD sends its locale with the summary context; the proxy names that
language in the instructions and the input, English when missing or
unsupported. The model returns {summary, feedState}; the HUD shows the
summary only when feedState matches its own state for a view that is not
nominal, and appends its own translated state tag.
partial joins the shared feed-state table between fallback and nominal,
so one partial layer no longer reads off and partial plus nominal no
longer reads nominal.
Snow is no longer offered anywhere a style can be chosen: the Visual
Presets tray, keyboard shortcut 7, the Cockpit vision carousel, and the
set_visual_style voice action. The shader and its labels go with it;
weather snowfall and the SNOW weather condition are unaffected.
Share links keep working. Snow's enum code (6) stays reserved, so a
compact or long-form link that names it opens on Normal and drops the
Snow shader parameters without reporting them as unknown. A Scene
Director shot saved with Snow plays on Normal too. Snow's frozen share
defaults matched Normal's apart from those parameters, so no other
setting in an old link changes meaning.
The legacy voice-schema pins restore Snow before hashing, so the
unchanged pins still prove nothing else moved; the complete Realtime
payload digest is re-derived.
Display > HUD > Layout and the set_hud voice action now offer Tactical,
Minimal and Cyber. The Operator option, its CSS and its QA coverage are
removed, and the unused "Operator" translations are pruned.
Share-link codes are append-only, so operator keeps its hudVariant code.
A link, scene or saved state that names it restores the default Tactical
layout. The voice schema pins keep their legacy digests by restoring the
retired value before comparing; the full-payload digest is re-derived.
The HUD builds place context for its AI summary whenever the camera
settles, and each build paid for up to four reverse geocodes plus a Nearby
Places search, even when no OpenAI key was configured to use them.
- Ask GET /api/openai/hud-summary/status once; with no summary configured,
or after an unconfigured reply, make no place lookups for the page.
- Reverse-geocode at most two points: the view center and one viewport
point.
- Reuse a lookup until the view center moves a quarter of the visible
width, the camera's distance to it doubles or halves, or the camera
crosses a lookup height. The reused points are asked again, so only a
timeout, failure or empty answer spends.
- Repaint the last AI reply when a small pan leaves the context unchanged.
Cached-only hosts and voice context lookups are unchanged.
add the shared Voice Preview and Settings panel with browser-local preview
and inactivity preferences, keyboard controls and responsive positioning
preserve main’s Rotation and Context behavior, clarify panel routing and
refresh schema pins within the unchanged prompt limit
Knip, configured with the application, package exports, scripts, QA
harnesses, workers, tools and tests as entry points, reports exports
that no module imports. Each was checked by hand against string-based
and namespace imports.
- Symbols that are still used inside their own module lose only the
`export` keyword (176 constants and helpers).
- Unused re-exports are removed from barrel modules (flight, military,
vessel, CCTV, directions, transit history, local ADS-B and street-level
policy modules).
- Declarations that nothing uses at all are deleted, among them
AIS_WATCHDOG_STATUSES, applyFilter, isRadioCountryCode, compactText,
DENSITY_STOPS, the unused CCTV endpoint constants, and the SDR band
constants.
- Default exports that nothing imports are removed. The ones in
src/data/earthquakes.js, src/data/telegeographySubmarineCables.js,
src/data/wind.js and src/data/bhoteKoshiLocator.js each constructed a
layer instance at import time that nothing used; those instances are
no longer created.
Every export of a package.json entry stays, as does TOMTOM_CREDIT,
which the traffic layer reads from the credits namespace it is given.
The login child inherited the whole server environment, including
OPENAI_API_KEY and every other provider credential. Pass only what Codex
needs to find its home, open a browser and reach OpenAI through a proxy.
Keeps the voice-core prompt rules and adds main's world-query and
vessels-vs-visuals rules; the tool catalog wraps the deixis-aware runner.
Also fixes two things the branch had before the merge:
- Live voice could not start: OpenAI rejects track_entity's top-level
anyOf. Model-facing tools now omit top-level combinators; the runner
and benchmark still require a query or a referent.
- The Contacts-centred analyst answer dropped the viewport coverage note.
- turn named places and mapped landmarks into reusable bounded areas
- count loaded records inside resolved, drawn and annotated geometry
- find recent imagery for an area, the view or the pointer snapshot
- keep osm place search behind an operator-configured overpass endpoint
- preserve cancellation, ownership and follow-up memory across every action
- cover geometry, provider lifecycle, cache isolation and voice routing
The voice debug log writes the server's timestamp after the client's record,
so a record can no longer replace loggedAt. The dev and preview servers also
send X-Content-Type-Options: nosniff with their shared browser headers.
From the findings in #710.
Co-authored-by: Sunil <[email protected]>
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
The OpenAI endpoints (/api/realtime/token, /api/openai/hud-summary) now
allow 30 requests per minute per client IP and the Google Places endpoints
120, the caps the Pinokio build already ships. GEV_RATELIMIT_OPENAI_PER_MIN
and GEV_RATELIMIT_GOOGLE_PER_MIN still override them and exactly 0 disables
them; an unreadable value falls back to the default rather than to
unlimited.
Salvaged from #683 onto current main.
Co-authored-by: Sameh Khamis <[email protected]>
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>