The HUD builds place context for its AI summary whenever the camera
settles, and each build paid for up to four reverse geocodes plus a Nearby
Places search, even when no OpenAI key was configured to use them.
- Ask GET /api/openai/hud-summary/status once; with no summary configured,
or after an unconfigured reply, make no place lookups for the page.
- Reverse-geocode at most two points: the view center and one viewport
point.
- Reuse a lookup until the view center moves a quarter of the visible
width, the camera's distance to it doubles or halves, or the camera
crosses a lookup height. The reused points are asked again, so only a
timeout, failure or empty answer spends.
- Repaint the last AI reply when a small pan leaves the context unchanged.
Cached-only hosts and voice context lookups are unchanged.
Keeps the voice-core prompt rules and adds main's world-query and
vessels-vs-visuals rules; the tool catalog wraps the deixis-aware runner.
Also fixes two things the branch had before the merge:
- Live voice could not start: OpenAI rejects track_entity's top-level
anyOf. Model-facing tools now omit top-level combinators; the runner
and benchmark still require a query or a referent.
- The Contacts-centred analyst answer dropped the viewport coverage note.
The voice debug log writes the server's timestamp after the client's record,
so a record can no longer replace loggedAt. The dev and preview servers also
send X-Content-Type-Options: nosniff with their shared browser headers.
From the findings in #710.
Co-authored-by: Sunil <[email protected]>
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
show_on_globe replaces open_in_gods_eye_view and names an MCP Apps view, a
ui:// panel that loads the app in embed mode and posts each later view to it,
so the globe in the conversation changes without reloading. It also takes a
view another answer returned. Tools can declare a UI resource, and the MCP
adapter serves resources/list and resources/read and lists each tool's
resource in _meta.ui.resourceUri.
Voice hears audio directly, and "vessels" is easily heard as "visuals",
which the instructions route to visual styles, so asking for vessels turned
on other layers or a CRT style. The instructions now say a vessels request
without a style name means the ships layer, and a style needs its name.
The voice instructions recommended aircraft_in_area, vessels_in_area and
get_earthquakes, which voice no longer offers, and the docs said voice gets
nearly every query. The instructions now name tools voice has and send
aircraft, ship and hazard questions to analyst_query; a test keeps the
instructions to tools the voice session lists.
src/tools/surfaces.js lists the tools a surface leaves out, and the MCP
server, the voice session's tool list and the browser catalog voice calls
all filter through it. Voice leaves out tools that answer with images, link
to the app, or repeat what its app actions already answer.
- Voice layer manifest (src/voice/layerManifest.js) generates the layer enums,
aliases and analyst field hints; a test fails when a catalog layer has no entry.
- analyst_query validates fields, operators, units, values and scopes, counts up
to 250,000 loaded records per layer, reports complete/partial/unanswered, tells
LAYER_OFF / NOT_READY / FEED_UNAVAILABLE apart from zero, refuses mismatched
follow-ups and never lets a superseded query replace follow-up memory. Seven
more layers expose analyst records.
- Tool results carry a code-built say line and a display card; the voice card in
the mic control shows captions, plan steps and the result
(src/voice/resultDisplay.js is the one adapter). Slow tools narrate their
current step out of band; holding Space while the assistant talks cuts it off.
Turn spans go to the debug log; caption transcription is metered; the log
omits what was said unless GEV_VOICE_LOG_CONTENT=1.
- Point and ask: the Space-down pointer snapshot resolves "this", "that one" and
"around here" through 'pointer' targets, and "the second one" through
referent:n from the last numbered list.
- Prompt states each policy once (instructions + tools 48.5k -> 46.7k chars).
- Voice benchmark harness (scripts/qa-voice-bench.mjs, scripts/voice-bench/)
shares its phrase table and grading with scripts/qa-voice-routing.mjs.
Ported onto the split provider layout: the route now lives in
server/providers/openai/debug-log.js and its state belongs to the handler
the factory returns rather than to module scope.
/api/realtime/debug-log is the one route in its family left unhardened.
Both siblings enforce a limiter through openai/rate-limit.js, and this was
the only handler still writing with fs.appendFileSync on the request path.
- Always-on per-IP limiter (120/min, 400 global). Deliberately not the
opt-in openAiRateLimiter(): that one defaults to unlimited so throttling
cannot silently change what a user spends, and this sink spends disk and
event-loop time rather than provider quota.
- Serialized async appends with a 32 MB rotation ceiling and one retained
generation. REALTIME_DEBUG_LOG_MAX_BYTES caps a single request body; it
never bounded the file those bodies accumulate into, so any local page
could grow .gev-logs/realtime-conversations.jsonl for as long as the dev
server ran. Serializing also closes a read-then-write race on the size
check, where concurrent callers all stat under the ceiling and none
rotates.
- Generic 400/500 answers separating a malformed record or oversized body
from a failed write, instead of echoing error.message — an errno and an
absolute path for an fs failure.
- readRequestBody marks an oversized body with code 'BODY_TOO_LARGE', the
marker its sibling readRequestBodyCapped already sets, so the two cases
are distinguishable without matching on text.
Coverage extends src/tooling/localServices.test.mjs, which already drives
the real handler over a temporary sourceRoot. Both new tests fail on the
previous implementation and pass with this one.
Refs #23. Does not add authentication: these proxies have no auth
mechanism today, so introducing one is a maintainer design decision.
Ported onto the split provider layout: the routes now live in
server/providers/openai/{hud-summary,realtime}.js and the camera health
entry in server/providers/cctv.js.
The routes #263 names — celestrak, terrain, adsbdb — were sanitized by
#289. The same defect had moved:
- /api/openai/hud-summary relayed `data.error.message` verbatim on a
non-ok upstream. That is OpenAI's own error text, carrying request ids,
organization hints and quota wording. Its catch echoed the JS error too.
- /api/realtime/token echoed the JS error when minting failed, which for a
network fault is a resolver message naming the upstream host.
- cctvProxy stored the raw error as a camera's health `message`. This is
the least obvious of the three and the reason the PR is worth its size:
GET /api/cctv/health serializes `message`, and src/layers/cctv/frames.js
renders it as the camera's status label, so the detail reached the
screen through a different door than the sanitized 502 beside it.
Logs name the failure and the upstream status without the text, matching
the Launch Library and terrain paths; these lines get pasted into issues.
Coverage extends the existing harnesses rather than adding files —
localServices.test.mjs drives both OpenAI routes over a mocked upstream,
mediaProviders.test.mjs drives a failing media fetch and then reads
/health. All three assertions fail on the previous implementation.
Refs #263. Does not touch /api/realtime/debug-log's identical echo; that
route is the subject of the #23 PR, so the two do not overlap a line.
Sameh Khamis