Compare commits

..
Author SHA1 Message Date
Sameh Khamis aa16b7c3b0 Merge pull request #902 from bilawalsidhu/release/v0.2.1
chore(release): prepare v0.2.1
2026-10-02 22:36:12 -07:00
Sameh Khamis 80fc8f0e1c chore(release): prepare v0.2.1
Date the changes since v0.2.0 as 0.2.1, move the live CCTV video entry
into the 0.2.0 section where it shipped, and align the package metadata.
2026-10-02 22:19:04 -07:00
Sameh Khamis 05df7083b9 Merge pull request #873 from manjunath22466/fix/cyber-voice-popups
fix(ui): harden Cyber panel layout and preset restoration
2026-10-02 21:51:37 -07:00
eaa51a24fe test(qa): cover the seven-mode Cockpit vision carousel
The Cockpit utility gate still expected five modes and an inherited NOIR
entry. Cycle all seven modes, check each drives only its own shader and that
Normal clears them all, enter on a Noir map preset, and restore it through
Exit Cockpit and Reset.

Co-authored-by: Sameh Khamis <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-02 21:43:12 -07:00
Sameh KhamisandClaude Opus 5.5 b33cae4b74 Merge main into #873
Resolve the CHANGELOG and CURRENT-STATE conflicts: the PR's entries go under
Unreleased, after the 0.2.0 release.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-02 21:18:15 -07:00
Sameh Khamis 2a612b7315 Merge pull request #901 from bilawalsidhu/fix/build-hygiene
build: ship each bundled data pack once
2026-10-02 21:12:40 -07:00
Sameh KhamisandClaude Opus 5.5 9c87fabf88 build: ship each bundled data pack once
loadBundledJson's callers passed a literal JSON import as a Node fallback,
which made the production build emit a JavaScript copy of every pack beside
the JSON the browser fetches. The loader now reads file: URLs itself under
Node (through process.getBuiltinModule, synchronously, as promptly as the
cached import it replaces), and the callers pass only the URL. On a Node too
old to provide getBuiltinModule it says that Node 24.14 or newer is needed.
dist/ drops from 54 MB to 42 MB. A test keeps app code from importing a pack
as a module.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-02 21:08:11 -07:00
Sameh Khamis 992412fc9a Merge pull request #900 from bilawalsidhu/fix/security-consolidation
Harden the local server: cross-site gate, CSP, default throttles, LAN host check
2026-10-02 20:26:01 -07:00
Sameh KhamisandClaude Opus 5.5 0646630dec fix(server): let the CSP load the Nepal event's embedded media
The Content-Security-Policy blocked the YouTube player API, the Facebook
video SDK and X's post widget that src/data/bhoteKoshiEmbeddedMedia.js
loads, along with their players and posts in frames. script-src now allows
exactly those three script origins and a new frame-src allows YouTube,
youtube-nocookie, Facebook and X embeds. A test ties the policy to the
frames the media module builds and pins the allowed script origins.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-02 20:21:23 -07:00
Sameh KhamisandClaude Opus 5.5 913d45bc07 fix(server): keep cost throttles on for positive fractional limits
A limit such as 0.5 floored to 0 and disabled the throttle, although only an
explicit 0 is meant to. Positive values below 1 now count as 1.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-02 20:17:21 -07:00
Sameh KhamisandClaude Opus 5.5 9b4ccbff62 fix(server): gate Google text search against cross-site requests
/api/google/text-search spends the same Google quota as nearby-places but
was not behind the cross-site gate. It now refuses cross-site requests
before calling Google, and a test covers both paid Google routes.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-02 20:16:39 -07:00
Sameh KhamisandClaude Opus 5.5 e6f1dd70a1 fix(server): apply the Host check before the app's own routes
Vite installs its Host check after the middleware plugins add in
configureServer, so the provider and /api routes answered any Host,
including a DNS-rebinding name. hostCheckPlugin runs first on the dev and
preview servers and applies Vite's rule with the same allowed hosts: IP
addresses, localhost and *.localhost, the configured server host, and the
listed names. A test drives a real Vite server with an unlisted Host.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-02 20:15:54 -07:00
Sameh KhamisandClaude Opus 5.5 81513f9aae fix(mcp): refuse panel requests to routes reached through a dot suffix
The server routes a mounted path when a dot follows it, so /mcp.json reached
the MCP endpoint while panel_request refused only /mcp and /mcp/...; the same
held for the provider-settings, credential and model routes. The refusals now
match a dot after the route as well. A regression test drives panel_request
against a real Vite server.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-02 20:10:47 -07:00
Sameh Khamis 9954e9f991 Merge pull request #899 from bilawalsidhu/fix/mcp-embed-hardening
Tighten the local MCP server and embed mode
2026-10-02 20:04:56 -07:00
Sameh KhamisandClaude Opus 5.5 6cdd6c8af9 refactor(server): share one proxy-header check across the local gates
localRequestGate.mjs owns the reverse-proxy header list and hasProxySignals;
Provider Settings, the cross-site gate and /mcp all use it instead of two
copies of the same nine names. The changelog credits the cross-site gate.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
2026-10-02 20:02:48 -07:00
bbf4976f5e fix(server): keep debug-log timestamps server-owned and send nosniff
The voice debug log writes the server's timestamp after the client's record,
so a record can no longer replace loggedAt. The dev and preview servers also
send X-Content-Type-Options: nosniff with their shared browser headers.

From the findings in #710.

Co-authored-by: Sunil <[email protected]>
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
2026-10-02 20:00:11 -07:00
98b8c6c581 fix(server): keep the Host-header check when binding to all interfaces
Binding HOST=0.0.0.0 no longer sets Vite's allowedHosts to true. The dev and
preview servers accept IP addresses, localhost and the LAN hostnames listed
in GEV_ALLOWED_HOSTS; suffix and wildcard entries are ignored. The
configuration factory takes the resolved list as an input, so build/vite.js
still reads no environment.

Ported from #97 to the current build/vite.js layout. Fixes #21.

Co-authored-by: Sameh Khamis <[email protected]>
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
2026-10-02 19:59:02 -07:00
bb74071dc2 fix(server): throttle the cost-bearing proxies by default
The OpenAI endpoints (/api/realtime/token, /api/openai/hud-summary) now
allow 30 requests per minute per client IP and the Google Places endpoints
120, the caps the Pinokio build already ships. GEV_RATELIMIT_OPENAI_PER_MIN
and GEV_RATELIMIT_GOOGLE_PER_MIN still override them and exactly 0 disables
them; an unreadable value falls back to the default rather than to
unlimited.

Salvaged from #683 onto current main.

Co-authored-by: Sameh Khamis <[email protected]>
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
2026-10-02 19:57:29 -07:00
7aa9bbd3be fix(server): gate cost/log endpoints against cross-site requests and ship a full CSP
The cost-bearing endpoints (/api/realtime/token, /api/openai/hud-summary,
/api/google/nearby-places) and the voice debug log refuse cross-site browser
requests: a foreign or opaque Origin, a Sec-Fetch-Site other than
same-origin/none, or proxy forwarding headers. Loopback tools without an
Origin and the LAN opt-in keep working. The dev and preview servers send one
shared Content-Security-Policy, and embed-mode framing rewrites only the
frame-ancestors directive of it.

Salvaged from #242 onto current main.

Co-authored-by: Sameh Khamis <[email protected]>
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
2026-10-02 19:56:42 -07:00
Sameh KhamisandClaude Opus 5.5 42faed3aea docs: describe the MCP server and panel in the security guide
SECURITY.md gains a section on what /mcp accepts, the panel's request key,
the browser keys the panel build carries and embed-mode framing. TOOLS.md
now describes what panel_request may request, and the changelog records
the recent MCP and embed changes.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
2026-10-02 19:47:37 -07:00
Sameh KhamisandClaude Opus 5.5 857767842f fix(mcp): bound panel requests, request bodies and cancelled calls
Panels on one MCP server share six requests in flight; further requests
wait their turn (up to 256 waiting) instead of failing, and a waiting
request leaves the queue when its caller cancels. Held responses belong to
their own MCP server rather than to the process. /mcp request bodies must
arrive within 30 seconds. Over stdio, a client's notifications/cancelled
aborts the named request, which then gets no response.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
2026-10-02 19:47:37 -07:00
Sameh KhamisandClaude Opus 5.5 fec93aa817 fix(mcp): accept only direct local requests at /mcp
The /mcp route now applies the same proxy and sharing checks as Provider
Settings: requests carrying forwarding headers, or arriving while launcher
sharing is on, are refused. The Host must name the port the request reached
and a browser Origin must match that host exactly. The proxy and sharing
checks move into shared helpers in keySetupCore; Provider Settings behaves
as before.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
2026-10-02 19:47:37 -07:00
Sameh KhamisandClaude Opus 5.5 f7d3db5875 fix(embed): make embed-mode framing opt-in
Embed-mode documents now keep the server's framing protection unless
GEV_EMBED_FRAME_ANCESTORS names the pages that may frame them (or * for any
page). The MCP Apps panel loads the app into its own page and needs no
framing. View answers are posted to the origin that sent the view rather
than to any origin.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
2026-10-02 19:47:37 -07:00
Sameh KhamisandClaude Opus 5.5 3318390e3f fix(mcp): require the panel key for panel requests
panel_request now requires a key that each MCP server generates and places
in its panel page, which keeps the tool from clients that list it to the
model without loading the panel. The key is not access control: any MCP
client can read the panel page. Routes the panel never loads (provider
settings, credential and model endpoints, the MCP endpoint and development
server internals) are refused to every caller, regardless of letter case or
encoding.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
2026-10-02 19:47:37 -07:00
Sameh Khamis b1a958242c Merge pull request #897 from bilawalsidhu/release/v0.2.0
chore(release): prepare v0.2.0
2026-10-02 19:00:29 -07:00
Sameh Khamis b86239aa6d chore(release): prepare v0.2.0
Group the changes since v0.1.1 under a dated 0.2.0 section and align the
package metadata.
2026-10-02 18:47:29 -07:00
Sameh Khamis 74213d18da Merge pull request #896 from bilawalsidhu/feat/tool-catalog
Add a shared tool catalog and MCP server, with God's Eye View inside Claude and ChatGPT
2026-10-02 18:45:08 -07:00
Sameh Khamis a55d260551 docs: drop the server check from the MCP setup 2026-10-02 18:29:42 -07:00
Sameh Khamis bcbc350354 docs: add a one-page MCP setup for Claude and Codex
Steps people and coding agents can follow to run the app, add the MCP
server to Claude Code, Claude Desktop, the Codex CLI and the Codex and
ChatGPT desktop apps, and check it works, linked from the README and the
tools guide.
2026-10-02 18:29:42 -07:00
Sameh Khamis b69d34bad2 Frame a view that only follows an aircraft where the aircraft is
Asked for an aircraft's cockpit view, assistants passed only the follow,
which failed for want of an area or camera, so every such request took
a second call. show_in_gods_eye_view now looks the aircraft up and frames
the view on it, and says so when the aircraft is not reported.
2026-10-02 18:29:42 -07:00
Sameh Khamis ae00f009e6 Log why a tool call failed in the stdio server's log
A failed call's reason went only to the client, so client logs showed a
failure without its cause.
2026-10-02 18:29:42 -07:00
Sameh Khamis e82ab7440d Run the worker prelude ahead of Cesium's own workers script
Cesium's script sets CESIUM_WORKERS itself when it loads, so the panel's
combined workers script was never used, and neither was the embedded
terrain-heights file: the worker still requested it from the panel
page's site. The panel build now writes only the prelude, and the panel
puts it ahead of whatever Cesium assigns to CESIUM_WORKERS. The separate
esbuild bundle of Cesium's workers is gone.
2026-10-02 18:29:42 -07:00
Sameh Khamis 9e5ef78f91 Embed the files Cesium's workers load in the panel's workers script
The ground-polyline worker loads approximateTerrainHeights.json itself,
and a worker's requests go to the panel page's own site, where they
failed with 404 and stopped rendering once ground-clamped lines such as
military installations appeared. The panel build now embeds that file in
the workers script, which answers the worker's request from memory.
2026-10-02 18:29:42 -07:00
Sameh Khamis d4311a9d54 Let a view show an aircraft's cockpit view
A view's follow may ask for cockpit view, which the app enters after it
follows the aircraft; links cannot carry it and open the app following.
Applying a view no longer flies the camera when the followed entity is
already there, since the flight ended the follow; it flies only while
waiting for the entity. The panel applies its first view again only
when it asks for cockpit view.
2026-10-02 18:29:42 -07:00
Sameh Khamis 89a448ee83 Do not apply a panel's first view twice
The panel starts the app from the first view's link, which the app
restores, following included. Sending the same view again once the app
was ready flew the camera to it, which ended the follow and left cockpit
view. The panel now sends only views that arrive while the app loads.
2026-10-02 18:29:42 -07:00
Sameh Khamis 9bc031c753 docs: describe God's Eye View in conversations in the changelog and current state 2026-10-02 18:29:42 -07:00
Sameh Khamis abca609708 Keep panel requests on the app's server and bounded
panel_request resolved paths with a check for a leading //, but URL
parsing also reads /\host as another host, so it could request other
servers. It now resolves the path against the app's address, refuses
any other origin, matches refused routes decoded as well, and does not
follow redirects. Responses are capped at 64 MiB while reading, with a
deadline; held responses make room on arrival and expire by time only,
so a response being read keeps its parts.

The panel keeps converted images only for its build files, so live
camera frames no longer accumulate, and failed build loads are retried.
The dev server answers a malformed /panel/ path with 400 instead of
crashing.
2026-10-02 18:29:42 -07:00
Sameh Khamis efdee8d255 Clean up the panel after host testing
Drop the render restarts added while chasing the black overlays; keep
the full render-error report and stop the timer after an error, as
Cesium does. Correct comments the investigation outran, share the
panel_request name, and document the host differences the panel works
around.
2026-10-02 18:29:42 -07:00
Sameh Khamis d0da4b0722 Keep the panel's 2D canvases in memory
In Claude, the app's 2D overlay canvases covered the globe in black
while the host treated the panel as off screen, and the globe showed
once they were hidden. The panel now asks for 2D canvases backed by
memory rather than the GPU.
2026-10-02 18:29:42 -07:00
Sameh Khamis dbbdef0f78 Offer fullscreen in the panel, and hold images in inserted HTML
Where the host offers fullscreen, an Expand button asks for it; the page
fills the room it is given. Images in markup the app inserts as HTML
text now wait for their files instead of requesting the panel host's
site, which ended the mic.svg and Cesium credit 404s.
2026-10-02 18:29:42 -07:00
Sameh Khamis 5ad0dd0004 Report a panel's render errors in full and restart drawing
Cesium stops drawing after a render error and prints worker failures as
[object Object]. An inline panel now logs the error's details and starts
drawing again, up to three times.
2026-10-02 18:29:42 -07:00
Sameh Khamis 47742f335d Keep an inline panel drawing when its host reports it hidden
Claude reports a panel on screen as hidden, which stopped the app's
render loop and the browser's animation frames, so no map was drawn. An
inline panel no longer suspends rendering when hidden, and draws from a
timer while animation frames stop arriving.
2026-10-02 18:29:42 -07:00
Sameh Khamis 62468c547f Fit the panel to Codex's and Claude's panel rules
Images and stylesheet files load as data: URLs, which hosts allow where
some refuse blob: images; the scene-asset source gets an https app
address the panel serves, since Codex pages use their own scheme; and a
panel whose 3D context the browser takes back says so.
2026-10-02 18:29:42 -07:00
Sameh Khamis 996b95eb45 Load the God's Eye View panel through the MCP server
Hosts serve panels from their own sites and may refuse other addresses;
Codex refuses any address on the user's machine. The panel now never
requests the app's server: it loads the app's panel build and data
through panel_request, a tool only the panel may call, which requests
the path from the app's server. One path in every host; only map
providers load directly.

npm run build:panel writes the panel build to dist/panel, served at
/panel/: one app script, one stylesheet, Cesium's script, and Cesium's
workers as one script Cesium runs from memory. Tools may now be marked
app-only. The panel CORS rules are gone.
2026-10-02 18:29:42 -07:00
Sameh Khamis b11d54d8b8 Support panel hosts that serve the page from their own scheme
Codex serves panels from codex-sandbox:// pages, where origins are
opaque: the panel recognizes the app's own addresses by scheme and host,
the vessels feed resolves against the document address, and the dev
server answers Codex's sandbox origins with CORS.
2026-10-02 15:21:37 -07:00
Sameh Khamis 64c1b1951f Answer panel origins only through the CORS plugin 2026-10-02 15:21:37 -07:00
Sameh Khamis 282efac862 Start the panel's app at the first view instead of the default place 2026-10-02 15:21:37 -07:00
Sameh Khamis 5b555d7708 Load images the panel redirects to the app's server with CORS
The app treats those images as its own site's and loads them without
CORS, so once they come from the app's server WebGL refuses them and
Cesium stops rendering.
2026-10-02 15:21:37 -07:00
Sameh Khamis 06460020d0 Send the panel's app requests to the app's server without <base>
Claude refuses <base> in panels, so relative addresses reached the
host's site: styles, Cesium assets and API calls all failed. The panel
now points fetch, XHR, beacons, images, media, links, workers and added
style url()s at the app's server, and declares its policy under OpenAI's
widget key as well.
2026-10-02 15:21:37 -07:00
Sameh Khamis d9ecd126c8 Let the app run under a panel page's document base
A panel page loads the app with <base> pointing at the app's server, so
addresses the app builds resolve against the document base instead of
the page location, and hash updates stay on the page's own address.
The dev server answers panel origins with CORS on the API routes too.
2026-10-02 15:21:37 -07:00
Sameh Khamis 63341b947e Run God's Eye View inside the MCP Apps panel instead of framing it
Hosts such as Claude do not let a panel frame other sites, so the panel
now loads the app's page, styles and scripts into itself in inline embed
mode, and declares the app and map providers as resource and connect
domains. The dev server answers panel origins with CORS.
2026-10-02 15:21:37 -07:00
Sameh Khamis cb04fd5b19 feat: say in the panel what a host's security policy blocked
When a host's content security policy refuses part of the panel, such as
its frame, the panel now names the blocked directive and address instead of
waiting until its load timeout.
2026-10-02 15:21:37 -07:00
Sameh Khamis d368df533d fix: complete the MCP Apps handshake the way hosts validate it
The panel sent clientInfo and capabilities in ui/initialize, where the MCP
Apps schema requires appInfo, so hosts such as Claude rejected the handshake
and never showed the panel. It now sends exactly appCapabilities, appInfo and
protocolVersion, sends initialized without parameters, says so when a host
rejects it, and offers the link when God's Eye View does not load within 20
seconds. Checked against the MCP Apps SDK's own host bridge.
2026-10-02 15:21:37 -07:00
Sameh Khamis ba751d9433 fix: also name a tool's UI resource with the flat ui/resourceUri key
Some MCP Apps hosts read the earlier flat _meta key; the MCP Apps SDK sends
it alongside _meta.ui.resourceUri, and so does the adapter now.
2026-10-02 15:21:37 -07:00
Sameh Khamis 5775eb9053 feat: log local MCP requests to stderr by method
The stdio server writes each request's method, and the tool or resource it
names, to stderr, which clients such as Claude Desktop copy into their logs.
Arguments and results are never logged.
2026-10-02 15:21:36 -07:00
Sameh Khamis 47ef168815 refactor: name the view tool show_in_gods_eye_view
The tool and its panel now say "Show in God's Eye View" rather than
"show on the globe", in tool names, descriptions, server instructions and
docs.
2026-10-02 15:21:36 -07:00
Sameh Khamis 216a75ac71 feat: show views as the live globe in MCP Apps clients
show_on_globe replaces open_in_gods_eye_view and names an MCP Apps view, a
ui:// panel that loads the app in embed mode and posts each later view to it,
so the globe in the conversation changes without reloading. It also takes a
view another answer returned. Tools can declare a UI resource, and the MCP
adapter serves resources/list and resources/read and lists each tool's
resource in _meta.ui.resourceUri.
2026-10-02 15:21:36 -07:00
Sameh Khamis 60c40865e9 feat: add an embed mode that shows only the globe
?embed=1 shows the globe alone: clean view with the HUD, panels, welcome and
setup prompts hidden, and provider attribution kept. The page that frames it
sends views with postMessage; the app applies each through its own actions
and reports every step. Annotations in any link are drawn once it restores.
Only embed-mode documents may be framed; every other page keeps
X-Frame-Options DENY and frame-ancestors 'none', and
GEV_EMBED_FRAME_ANCESTORS can restrict who frames them.
2026-10-02 15:21:36 -07:00
Sameh Khamis f68aa988ec feat: carry annotations in views and aim tilted views at the area
A view can carry the marks the app's annotate_map action draws, and links
carry them in the an parameter, bounded in count and size. A tilted view of
an area now places the camera behind its center so the area stays in the
middle of the frame.
2026-10-02 15:21:36 -07:00
Sameh Khamis af8856cedf feat: lead the MCP tool list with what the globe shows
MCP no longer lists place search, routing, plain weather and wind, the
regional brief, the HUD caption, radio, bike share and transit. Assistants
already cover these or they add little without the globe; voice keeps them,
and tools that combine others still reach them.
2026-10-02 15:21:36 -07:00
Sameh Khamis d6473b6049 feat: return the view that shows each answer
Tool answers with something to show now include data.view: the view with
the matching layers on, the area framed from above or a single aircraft or
satellite followed, and the link that opens it. Area framing moves to
src/tools/views.js so links and answers frame alike.
2026-10-02 15:21:36 -07:00
Sameh Khamis 96a8a9addb feat: describe what the app shows as a view
A view is a camera, data layers, visual style, map imagery and an aircraft,
military aircraft or satellite to follow, written in the share-link format
the app already restores. gods-eye-view/view builds, encodes and decodes one,
and share links take their style names from it. open_in_gods_eye_view now
builds its link from a full view: an area or camera, layers, style, map and
something to follow.
2026-10-02 15:21:36 -07:00
Sameh Khamis de1e0836b3 fix: send "vessels" to the ships layer in voice
Voice hears audio directly, and "vessels" is easily heard as "visuals",
which the instructions route to visual styles, so asking for vessels turned
on other layers or a CRT style. The instructions now say a vessels request
without a style name means the ships layer, and a style needs its name.
2026-10-02 15:21:36 -07:00
Sameh Khamis 759d05db4e fix: name only tools voice offers in its instructions
The voice instructions recommended aircraft_in_area, vessels_in_area and
get_earthquakes, which voice no longer offers, and the docs said voice gets
nearly every query. The instructions now name tools voice has and send
aircraft, ship and hazard questions to analyst_query; a test keeps the
instructions to tools the voice session lists.
2026-10-02 15:21:36 -07:00
Sameh Khamis 33445aec73 fix: keep each aircraft feed's freshness in search answers
find_aircraft merged the records of the feeds that answered and dropped their
freshness, so a stale OpenSky snapshot read as a current "not reported".
Answers now list each feed with its freshness and coverage and say which
feed's data may be stale.
2026-10-02 15:21:36 -07:00
Sameh Khamis 41d1352930 fix: search every vessel the server retains
Vessel tools read the source's default of the newest 12,000 vessels while
the server keeps up to 50,000, so a lookup could miss a vessel still held.
They now ask for more than any server keeps and receive everything it has.
2026-10-02 15:21:36 -07:00
Sameh Khamis ad9a3976a9 fix: say the camera catalog has no cameras rather than none exist
An empty camera answer outside the catalog's regions read as "0 public
cameras", as if the area had none. It now says the catalog has no cameras
there.
2026-10-02 15:21:36 -07:00
Sameh Khamis f610de77dc fix: judge camera catalog trimming by region, not by returned cameras
Coverage was checked only for packs among the cameras found, so an area
whose cameras were all trimmed away answered zero cameras, complete. The
catalog now reports where each trimmed pack's cameras are, and the tool
marks an answer partial whenever its area overlaps that region.
2026-10-02 15:21:36 -07:00
Sameh Khamis 7f56679a46 fix: show the whole weather map when a window cannot cover the area
The map window was moved inside the product's bounds without checking that
it still covered the requested area, so an area across the antimeridian got
an image of only one side. A window must now hold the whole area, clipped to
the product, or the tool returns the product's full extent.
2026-10-02 15:21:36 -07:00
Sameh Khamis 4cfca57cfb fix: bound large boxes by their farthest point on the sphere
A box's radius was the distance to its farthest corner, which falls short for
large boxes: a whole-world box measured about 10,000 km, so bike-share and
transit selection never considered systems beyond it. The radius now checks
whether the box holds the point opposite its center and otherwise samples
every edge.
2026-10-02 15:21:36 -07:00
Sameh Khamis e1ac5ac270 fix: cancel shared place lookups nobody is waiting for
The place cache shared one lookup between callers but started it without a
signal or deadline, so a cancelled request kept running and stayed cached.
A shared lookup now has its own deadline, is cancelled when every caller has
given up, and only a successful answer is kept.
2026-10-02 15:21:36 -07:00
Sameh Khamis 66b07418bd fix: say when the camera catalog serves only part of a region
Camera packs keep their nearest cameras and the catalog cap thins them
further, so London answered 250 cameras while TfL offers many more. The
catalog now records each pack's full count, tags cameras with their pack,
and serves the packs it trimmed; find_cctv_cameras names them.
2026-10-02 15:21:36 -07:00
Sameh Khamis e779432925 fix: report stale and partial data in more tool answers
A sweep for the same pattern found more answers presented as current and
complete: launches and satellite orbits served from a stale proxy cache, a
regional aircraft fallback read as worldwide, license plate cameras trimmed
or partly loaded, and a radio directory of popular stations read as every
station. The launch and satellite sources now report the proxies' stale
markers, and each answer says what it covers.
2026-10-02 15:21:36 -07:00
Sameh Khamis 6a45a8f6a2 fix: check only own properties when validating tool arguments
Inherited names such as constructor, toString and __proto__ passed as
declared properties despite additionalProperties: false, and a schema-valued
additionalProperties was accepted but never enforced. Validation now uses own
properties, and definitions must give additionalProperties as a boolean.
2026-10-02 15:21:36 -07:00
Sameh Khamis c4a4247f70 fix: cancel a local MCP tool call when its client disconnects
The /mcp route built its Request without a signal, so a tool kept working
after the client left. The request's signal now aborts when the connection
closes before the answer is sent.
2026-10-02 15:21:36 -07:00
Sameh Khamis 13f8fa7c8e fix: wait briefly for installation names before answering
Installation tile results can arrive before the name pack, and the tool
answered with generic names such as "Military area" where a moment later
it would say "Camp Mabry". It now waits up to five seconds for the named
result and says when names are still loading.
2026-10-02 15:21:36 -07:00
Sameh Khamis 9e4cc9095b fix: measure traffic inside the requested circle and report missing tiles
Traffic figures for a radius area counted roads anywhere in its bounding
box, and a tile that failed to load silently changed them. Roads are now cut
to the circle, and the flow source reports per request whether any tile
failed so the answer can say its figures are partial.
2026-10-02 15:21:36 -07:00
Sameh Khamis 302a1c1995 fix: say when bike share, transit and place limits leave results out
Bike share and transit read only the three nearest systems, and place
search returns at most five matches within 50 km, while answers read as
complete. Answers now name the systems not searched and say when a place
search limit may have left places out.
2026-10-02 15:21:36 -07:00
Sameh Khamis 56f0d2e203 fix: carry weather and fire freshness into tool answers
get_weather dropped the proxy's stale status and get_active_fires dropped
FIRMS staleness and failed satellite sources, so answers and the HUD caption
read them as current. Both now report them, and the caption context marks
such sections stale or degraded.
2026-10-02 15:21:36 -07:00
Sameh Khamis dd8ba562d2 fix: find fire perimeters that overlap an area
Fire perimeters were placed at their average vertex, so a fire enclosing the
whole requested area was missed. Polygons now count when a vertex is inside
the area, an edge crosses it, or the polygon contains it outside any hole.
2026-10-02 15:21:36 -07:00
Sameh Khamis 4af15aa6a9 fix: keep a named place's own point in situation briefs
Brief sections received a named place as its bounding box, so they centered
on the box's middle: Tokyo's aircraft were sorted from a point in the
Pacific. Sections now receive the place name and resolve it like a direct
query, and place lookups are cached briefly so a brief resolves it once.
2026-10-02 15:21:36 -07:00
Sameh Khamis 1b3aa8de8b fix: stop reading MCP request bodies past the size limit
The HTTP handler read the whole body before checking its size, so a body
without Content-Length could be buffered without bound. It now reads the
stream and stops as soon as the body passes the limit.
2026-10-02 15:21:36 -07:00
Sameh Khamis 6f56c84b0b fix: find cables that cross an area between vertices
Cable searches only tested vertices, so a cable crossing an area with no
vertex inside it was missed. Lines now test each segment against the area's
bounds, across the antimeridian too, and against a radius area's great-circle
distance.
2026-10-02 15:21:36 -07:00
Sameh Khamis 5b73b70d19 fix: bound radius areas by the exact spherical cap
The longitude span divided the angular radius by the center's cosine, which
is too narrow for large radii and near the poles. Areas now use the exact
tangent bound and span every longitude when the circle reaches a pole.
2026-10-02 15:21:36 -07:00
Sameh Khamis cac5ba3b72 fix: derive HUD caption feed states from section results
The caption request declared every answered section nominal, so a stale
aircraft snapshot reached the caption as live. Each section now reports
stale or unavailable from its own result, and the provenance envelope is
built with the app's feedProvenanceEnvelope.
2026-10-02 15:21:36 -07:00
Sameh Khamis 853c60583f fix: report stale transit feeds and drop expired positions
get_transit_vehicles returned positions from a feed the proxy served from
its error cache, and hour-old fixes, as current. It now applies the transit
layer's rules: a feed is stale when the proxy says so or the operator has
not answered recently, and fixes older than the vehicle limit are dropped
and counted.
2026-10-02 15:21:36 -07:00
Sameh Khamis 61974848fe fix: keep aircraft search results when one feed is down
find_aircraft failed whenever either feed failed. It now answers from the
feeds that responded, names the ones that did not, and searches only the
military feed for registrations, the only feed that reports them.
2026-10-02 15:21:36 -07:00
Sameh Khamis 963c791a5c fix: run composite tool sections through the catalog
situation_brief, military_awareness and the HUD caption called other tools
directly, so replaced tools and interceptors did not apply to their sections.
Tools now reach each other through the catalog, and interceptors see these
calls with the calling tool as parent. Surfaces become a view over the
composed catalog, so a composite offered on voice keeps sections that voice
does not list.
2026-10-02 15:21:36 -07:00
Sameh Khamis 7ddee426e7 fix: count every mapped installation in tool answers
The installation source keeps one named site per display cell for the map.
Queries now ask for every site and the exact box, and report when the
source returned only part of the mapped sites.
2026-10-02 15:21:36 -07:00
Sameh Khamis 2e071b62d3 fix: repeat MCP tool data as JSON text for clients without structured content 2026-10-02 15:21:36 -07:00
Sameh Khamis df0e0f32e8 feat: choose per tool whether MCP, voice or both offer it
src/tools/surfaces.js lists the tools a surface leaves out, and the MCP
server, the voice session's tool list and the browser catalog voice calls
all filter through it. Voice leaves out tools that answer with images, link
to the app, or repeat what its app actions already answer.
2026-10-01 14:36:12 -07:00
Sameh Khamis b4067f9859 fix: time-limit OpenSky snapshot requests and retry once
A stalled global snapshot held the request for over a minute. Each attempt
now gets 10 seconds, a timed-out or failed attempt is retried once, and a
second failure is answered from the stale cache or the regional fallback.
The OAuth token request gets the same limit.
2026-10-01 14:26:20 -07:00
Sameh Khamis ec1adbdc19 fix: tighten tool answers found in local testing
Unknown aircraft tracks read as empty instead of a server error, aircraft
info names the aircraft and flight, the HUD caption sends the app's caption
context, and satellite pass, satellites overhead and nearby places take a
place name or point like the other tools.
2026-10-01 14:01:25 -07:00
Sameh Khamis 1a03e7483d feat: share the tool catalog with voice 2026-10-01 13:52:36 -07:00
Sameh Khamis 0953ebb026 feat: add a Bhote Koshi flood event tool 2026-10-01 13:44:41 -07:00
Sameh Khamis a747b8a4e0 feat: add a military awareness tool 2026-10-01 13:41:30 -07:00
Sameh Khamis 28cfd39d1a feat: add a datacenter and dam tool 2026-10-01 13:38:38 -07:00
Sameh Khamis a8e5ff9390 refactor: move infrastructure data parsing into a portable source module
The datacenter and dam layers parsed their bundled GeoJSON Lines and
built analyst records inside the Cesium-backed layer module. Move the
data URLs, the line parser, mapAnalystRecord and layerTitle unchanged to
src/sources/infrastructureData.js; the layer module imports them and
still exports mapAnalystRecord. The consumer-build pin lists the new
module.
2026-10-01 13:34:26 -07:00
Sameh Khamis 0a454815cb feat: add a license plate reader camera tool
Add find_alpr_cameras over the layer's OpenStreetMap camera extracts:
cameras in a US or Canadian area of at most 3° per side, nearest first,
with operator, manufacturer and facing direction, and the source's
attribution. Areas outside coverage or too wide are reported.
2026-10-01 13:31:31 -07:00
Sameh Khamis c1957d884e feat: add a submarine cable tool
Add find_submarine_cables over the bundled TeleGeography data: cables
passing through or landing in an area, or matching a name, with landing
points and the dataset's attribution. The local MCP server's fetch reads
the bundled data files a browser would load by URL, and nothing outside
that directory.
2026-10-01 13:28:02 -07:00
Sameh Khamis 9be90fff44 feat: add a recent satellite imagery tool
Add get_recent_imagery. It searches the recent Harmonized Landsat and
Sentinel-2 catalog for an area, picks the image the layer would show
with the layer's own ranking, and returns it with its date, sensor and
cloud cover, falling back to the VIIRS daily overview.
2026-10-01 13:25:00 -07:00
Sameh Khamis a3aa058569 feat: add weather map and wind tools
Add get_weather_map, which returns the latest NOAA radar, satellite or
lightning map over an area as an image, choosing a window the image
route accepts and reporting areas outside the product's coverage, and
get_wind, which samples GFS or IFS wind at a location with the wind
layer's own grid sampling.
2026-10-01 13:22:16 -07:00
Sameh Khamis c43c94ba5f feat: read weather image frames through the weather source
Add getImage() to the weather source. It reads one frame through the
bounded image route the layer already uses and returns its content type
and bytes, so code outside the layer can show the current map.
2026-10-01 13:18:01 -07:00
Sameh Khamis fe69f22346 feat: turn layers on in links to an area
open_in_gods_eye_view accepts layers and writes them with the app's own
share-link layer codec, so the link opens with those layers on. Layer
names are limited to the registered layer ids.
2026-10-01 13:17:15 -07:00
Sameh Khamis 9de2705eaf refactor: move the layer state coordinator into its own module
layerState.js held both the share-link layer codec and the coordinator
that restores layer choices into the running app and persists them to
browser storage. Move the coordinator, unchanged, to
layerStateCoordinator.js so the codec stays a pure data module that code
outside the browser can use to write share links. The registry stays in
layerState.js, where the token tooling reads it, and the aircraft
barrel re-exports both.
2026-10-01 13:16:19 -07:00
Sameh Khamis a8b5860e9f refactor: share the default tool services between surfaces
Move the composition of the default tool services from the Node MCP
server into gods-eye-view/tools/services, a portable export. Given a
fetch that resolves the sources' /api paths, it builds the same services
in a browser as on the local MCP server, so other surfaces can compose
the same catalog. Queries still read only their services.
2026-10-01 13:12:34 -07:00
Sameh Khamis 0dbd6375fc fix: center named places on their own location
A place's bounds can reach far from the place itself; Tokyo's include
remote Pacific islands, so the middle of its box is open ocean. Keep the
geocoder's point for a named place and use it as the area's center,
while bounds still decide what lies inside the area.
2026-10-01 12:50:15 -07:00
Sameh Khamis 469b1e8d89 feat: serve the MCP server at /mcp on the local servers
Mount the local MCP server at /mcp on the development and preview
servers for clients that connect by URL. The route accepts only
requests from this machine that name a loopback host and, when an
Origin is sent, come from a loopback origin, and drains oversized
bodies so the 413 response is delivered. The provider-order regression
test now pins the route between the providers and the API fallback.
2026-10-01 12:47:44 -07:00
Sameh Khamis b6af8b34e0 feat: add a tool that links to an area in the app
Add open_in_gods_eye_view. It builds a version 2 share link looking
straight down on an area, with altitude chosen from the area's size, on
the app address the application supplies.
2026-10-01 12:42:08 -07:00
Sameh Khamis 9d12da0a79 feat: add a traffic flow tool
Add get_traffic_flow over the traffic source's TomTom flow tiles. It
reports average speed as a share of free-flow speed, congested and
closed road length, and a breakdown by road category, stepping down a
zoom level when an area needs more tiles than the source reads at once.
2026-10-01 12:36:00 -07:00
Sameh Khamis e6cbcc31ad feat: add ship tools
Add vessels_in_area, find_vessel and get_vessel_track over the live
vessel source, and a ships section in situation_brief. Without an AIS
key the source's own message reaches clients as an unavailable feed.
2026-10-01 12:32:09 -07:00
Sameh Khamis 0a74e36ce8 feat: add weather, hazard, terrain and situation tools
Add get_weather, get_regional_brief, get_cyclones, get_fire_perimeters,
get_terrain_height, find_military_installations and get_map_features
over the application request services and the layers' sources.

situation_brief combines weather, earthquakes, fires, aircraft and
cyclones for an area by reusing those tools, marking sections that are
unavailable instead of failing. get_hud_caption returns the app's
heads-up display caption for the same overview.
2026-10-01 12:28:34 -07:00
Sameh Khamis 8c97fed92d feat: add bike-share and transit tools
Add get_bike_share over the public GBFS systems and get_transit_vehicles
over the GTFS-Realtime feeds. Each finds the systems or feeds whose
coverage reaches the area, reads up to three, and reports any that were
unavailable instead of failing the whole answer. Transit results carry
each feed's attribution and license.
2026-10-01 12:22:13 -07:00
Sameh Khamis 2ba751ace8 feat: read the transit feed catalog through the transit source
Add getFeeds() to the transit source. It reads the public catalog route
the server already serves, so code outside the layer can find feeds
near a location without loading the layer.
2026-10-01 12:19:19 -07:00
Sameh Khamis eb12303f46 refactor: move GBFS station parsing into a portable source module
The bike-share layer parsed station documents inside its model factory.
Move the parsers unchanged to src/sources/gbfsStations.js and have the
model import them, so code outside the layer can read GBFS stations
without constructing the layer.
2026-10-01 12:18:48 -07:00
Sameh Khamis e680a59bf5 feat: add place search and routing tools
Add search_places and places_nearby over the app's Google Places routes,
and plan_route over its OpenStreetMap routing. The place search service
reports when no search key is configured, so an unconfigured server is
not mistaken for an empty result, and rate limits become retry_later.
Route endpoints accept a place name or coordinates.
2026-10-01 12:16:01 -07:00
Sameh Khamis 85fee59da1 feat: add public camera and radio tools
Add find_cctv_cameras, get_cctv_snapshot and find_radio_stations. A
snapshot returns the camera's current frame as image content with its
credit. Tool results may now carry base64 images, which the MCP adapter
sends as image content.
2026-10-01 12:11:00 -07:00
Sameh Khamis d65402eae3 feat: read current camera frames through the CCTV source
Add getFrame(camera) to the CCTV source. It reads one frame through the
registered frame endpoint the layer already uses and returns its content
type and bytes, so code outside the layer can show a camera's view.
2026-10-01 12:11:00 -07:00
Sameh Khamis 22c3fe40b0 feat: add aviation and satellite tools
Add aircraft_in_area, find_aircraft, get_aircraft_track and
get_aircraft_info over the live aircraft sources, with an optional
military feed, and next_satellite_pass and satellites_overhead over the
satellite catalog and the existing pass predictor.

Source failures from the live contract now become tool errors with the
matching code, so a rate limit reaches clients as retry_later with a
wait time. Tool schemas may use pattern.
2026-10-01 12:05:50 -07:00
Sameh Khamis 9895942633 refactor: move TLE text parsing into a portable source module
The satellites layer parsed catalog text inside its Cesium-backed orbit
owner. Move the parser unchanged to src/sources/tle.js, add a catalog
number helper, and have the layer delegate to it so code outside the
layer can read TLE text without loading rendering.
2026-10-01 12:05:41 -07:00
Sameh Khamis c3dfa8e853 feat: add a tool catalog and local MCP server
Define tools that answer questions from the app's data once, independent
of the surface that exposes them. A tool declares its arguments and the
services it reads; applications compose tools with their own services and
optional call interceptors. Queries reuse the layers' portable source
factories, so they run wherever an application routes /api requests.

The first queries are get_earthquakes, get_active_fires and
get_recent_launches, with a shared area argument (place name, bounding
box, or point and radius) and capped lists.

gods-eye-view/tools/mcp adapts a catalog to the Model Context Protocol
with a small JSON-RPC handler and a stateless HTTP transport, adding no
dependencies. npm run mcp serves Core's tools over stdio to local clients
such as Claude Code, reading from a running app. Both exports are
portable and checked by check:boundaries; the app does not import them.
2026-10-01 11:51:59 -07:00
Manjunath 1dbc8a4452 fix(ui): preserve Cyber and Cockpit visual state 2026-10-01 17:00:10 +05:30
Manjunath e03bf008f3 fix(ui): harden Cyber panel layout and preset restoration 2026-10-01 01:50:31 +05:30
309 changed files with 20419 additions and 24984 deletions

No files matched your search

+35 -30
View File
@@ -34,16 +34,19 @@ GOOGLE_MAPS_API_KEY=
# single shared key keeps working — splitting is opt-in.
# SERVER-SIDE ONLY.
GOOGLE_MAPS_SERVER_API_KEY=
# Optional: opt-in per-IP rate limit for the Google Places cost endpoint
# (/api/google/nearby-places), in requests per minute.
# DEFAULT IS UNLIMITED — unset (or 0) means no throttling, unchanged behavior.
# Set a positive integer to cap per-IP Places quota spend when self-hosting.
# Recommended whenever HOST is not localhost. NOTE: this is a per-IP,
# process-local guard (in-memory, resets on restart; over-limit requests get a
# sanitized 429) — it is NOT a billing cap. For hard spend protection set
# provider-side budget alerts: Google Cloud Console -> Billing -> Budgets & alerts
# (plus per-API quotas under APIs & Services -> Quotas).
# GEV_RATELIMIT_GOOGLE_PER_MIN=60
# Per-IP rate limit for the Google Places cost endpoints
# (/api/google/nearby-places + /api/google/text-search), in requests per minute.
# DEFAULTS TO 120, the value the Pinokio build already ships. Raise or lower it
# with a positive integer; 60 is a sensible tightening whenever HOST is not
# localhost. Set it to exactly 0 to run unthrottled. An unreadable value (a
# typo, a stray unit) falls back to the default rather than to unlimited, so a
# slip cannot quietly disarm the guard. A positive value below 1 counts as 1.
# NOTE: this is a per-IP, process-local guard (in-memory, resets on restart;
# over-limit requests get a sanitized 429) — it is NOT a billing cap. For hard
# spend protection set provider-side budget alerts: Google Cloud Console ->
# Billing -> Budgets & alerts (plus per-API quotas under APIs & Services ->
# Quotas).
# GEV_RATELIMIT_GOOGLE_PER_MIN=60 # uncomment to tighten from the default 120
# Optional: Google Photorealistic 3D Tiles through Cesium ion, Bing world
# imagery, and Cesium World Terrain. The free Community plan is for eligible
@@ -64,28 +67,20 @@ OPENAI_REALTIME_REASONING_EFFORT=low
# Keep only a short conversational window. Current map state is fetched live.
OPENAI_REALTIME_CONTEXT_TOKENS=3000
OPENAI_REALTIME_CONTEXT_RETENTION=0.5
# Voice-card captions of what you said. Billed separately by OpenAI and
# counted in the session cost meter and cap; off disables them.
OPENAI_REALTIME_TRANSCRIBE_MODEL=gpt-4o-mini-transcribe
# The voice debug log (.gev-logs/realtime-conversations.jsonl, rotated at
# 32 MB, never expired) omits transcripts and typed or spoken text by default.
# Set to 1 to keep them while debugging; delete the log when you are done.
GEV_VOICE_LOG_CONTENT=
OPENAI_HUD_SUMMARY_MODEL=gpt-5-nano
# On-device voice (Cloud / On-device choice in the mic control) runs models
# in the browser. Its natural voice downloads an open-source speech component
# (GPL) to the browser at runtime; set to off to leave natural voice out and
# speak with the browser's on-device voices (or text) instead.
# GEV_NATURAL_VOICE=off
# Optional: opt-in per-IP rate limit for the OpenAI cost endpoints
# Per-IP rate limit for the OpenAI cost endpoints
# (/api/realtime/token + /api/openai/hud-summary), in requests per minute.
# DEFAULT IS UNLIMITED — unset (or 0) means no throttling, unchanged behavior.
# Set a positive integer to cap per-IP spend when self-hosting on a network.
# Recommended whenever HOST is not localhost. NOTE: this is a per-IP,
# process-local guard (in-memory, resets on restart; over-limit requests get a
# sanitized 429) — it is NOT a billing cap. For hard spend protection set a
# provider-side budget: OpenAI platform -> Settings -> Limits (usage limits).
# GEV_RATELIMIT_OPENAI_PER_MIN=30
# DEFAULTS TO 30, the value the Pinokio build also ships. That clears ordinary
# use several times over: an open tab asks for a HUD summary every 15s (4/min)
# and a voice session mints one token. Raise or lower it with a positive
# integer; set it to exactly 0 to run unthrottled. An unreadable value (a typo,
# a stray unit) falls back to the default rather than to unlimited, so a slip
# cannot quietly disarm the guard. A positive value below 1 counts as 1.
# NOTE: this is a per-IP, process-local guard (in-memory, resets on restart;
# over-limit requests get a sanitized 429) — it is NOT a billing cap. For hard
# spend protection set a provider-side budget: OpenAI platform -> Settings ->
# Limits (usage limits).
# GEV_RATELIMIT_OPENAI_PER_MIN=30 # the default; uncomment only to change it
# Optional: OpenSky auth mode (oauth | basic | auto | anon)
# Default: oauth — requires client credentials below
@@ -113,6 +108,11 @@ PORT=4173
# networks you trust, and consider the GEV_RATELIMIT_* throttles above.
# HOST=localhost
# HOST=0.0.0.0
# Optional: comma-separated LAN hostnames to accept, in addition to localhost
# and IP addresses, for example GEV_ALLOWED_HOSTS=globe.lan,globe.internal.
# Binding HOST=0.0.0.0 alone does NOT trust every Host header; suffix and
# wildcard entries are ignored.
# GEV_ALLOWED_HOSTS=
# Optional: NASA FIRMS live active fires. SERVER-SIDE ONLY (never reaches the
# browser — the client polls the /api/firms proxy). Free key:
@@ -198,3 +198,8 @@ AISSTREAM_API_KEY=
# Optional: comma-separated Overpass URLs; use only instances you run or pay for.
OVERPASS_UPSTREAMS=
# Optional: who may frame the app in embed mode (?embed=1), as CSP
# frame-ancestors sources such as "https://example.com", or * for any page.
# Unset, no page may frame it. Other pages of the app can never be framed.
# GEV_EMBED_FRAME_ANCESTORS=
+136 -81
View File
@@ -1,27 +1,106 @@
# Changelog
- The voice card shows analyst counts ("At least 250,000 aircraft …") with
their scope, feed window and caveats, and marks partial answers.
- Voice answers are shorter and fill slow work instead of going silent. Slow
or multi-step requests may open with one short spoken plan; instant ones
just happen. Tool results carry a code-built `say` line and a `display`
card: sources and notes stay on screen, a caveat is spoken only when it
changes the answer, and "at least" and partial answers are always said. A card above the mic shows captions, the plan's steps and the
result. When a tool that reports steps (layers, place lookups, outlines,
nearest-aircraft) is still running after 1.5 s its current step is spoken
once; others say "still working" once after 8 s. A soft
tick plays in push-to-talk. Holding Space while the assistant talks cuts it
off. Each turn writes latency and speech figures to the debug log
(`node scripts/voice-turn-spans.mjs`). Captions of what you say are
counted in the voice cost meter; the debug log leaves out what was said
unless `GEV_VOICE_LOG_CONTENT=1`.
- Point and ask: hold Space with the cursor on an aircraft, ship or spot on
the map and say "what's that", "track that one" or "how many flights
around here". A reticle marks the point and the voice card shows what
"this" meant. In open mic it works when you have just moved the cursor
there. "The second one" picks from the last numbered list. Asking about
bare ground up close gives the assistant a small crop of the map around the
point.
## [Unreleased]
## [0.2.1] — 2026-10-02 — God's Eye View through Your Agent (with security fixes and other features)
- Ship each bundled data pack once. The region, marine, admin-boundary,
county, military-name and neighborhood packs were emitted twice by the
production build, as the JSON the browser fetches and as an unused
JavaScript copy; `dist/` drops from 54 MB to 42 MB. Under Node the loader
reads the JSON file directly (reporting that Node 24.14 or newer is needed
on a runtime too old to do so), and a test keeps app code from importing a
pack as a module.
- Apply the Host check before the app's own routes. Vite installs its Host
check after the middleware that plugins add, so the provider and `/api`
routes used to answer any Host, including a DNS-rebinding name, in every
mode. A first-running middleware now applies the same allowed hosts on the
dev and preview servers.
- Refuse cross-site browser requests to the cost-bearing and log endpoints
(`/api/realtime/token`, `/api/openai/hud-summary`,
`/api/google/nearby-places`, `/api/google/text-search`,
`/api/realtime/debug-log`): a foreign or opaque
Origin, a cross-site `Sec-Fetch-Site`, or proxy forwarding headers get 403,
while loopback tools and LAN use keep working. The dev and preview servers
send one shared Content-Security-Policy (James Sumpter, #242).
- The voice debug log records the server's own time: a record can no longer
supply its `loggedAt`. The dev and preview servers also send
`X-Content-Type-Options: nosniff` (findings by Sunil, #710).
- Keep the Host-header check when binding to all interfaces. `HOST=0.0.0.0`
used to accept any Host; the dev and preview servers now accept IP
addresses, `localhost` and the LAN hostnames listed in `GEV_ALLOWED_HOSTS`
(suffix and wildcard entries are ignored), which also keeps DNS-rebinding
names out in LAN mode. The `.local` suffix is no longer accepted by default;
list such a name explicitly (Puspo Aditya, #97, fixes #21).
- Throttle the cost-bearing proxies by default instead of on request. The
OpenAI endpoints (`/api/realtime/token`, `/api/openai/hud-summary`) now allow
30 requests per minute per client IP and the Google Places endpoints
(`/api/google/nearby-places`, `/api/google/text-search`) 120 — the caps the
Pinokio build already ships, so the packaged app is unaffected and only an
unconfigured server changes, from unlimited to what the product already runs
with. `GEV_RATELIMIT_OPENAI_PER_MIN` and `GEV_RATELIMIT_GOOGLE_PER_MIN` still
override the caps, and exactly `0` disables them; a value that cannot be read
as a number now falls back to the default rather than to unlimited, so a typo
cannot silently disarm the guard (daikaginza, #683).
- Tighten the local MCP server and embed mode. The panel's `panel_request`
needs the key in its MCP server's panel page, and refuses Provider
Settings, credential and model endpoints, `/mcp` and development server
routes in any letter case, encoding or dot suffix; panels on one server share six
requests in flight, and further requests wait their turn. `/mcp` applies Provider Settings' proxy and sharing checks,
requires a Host on the port it reached and an Origin from that host, and
times out request bodies after 30 seconds; stdio honors
`notifications/cancelled`. Embed mode is no longer framable unless
`GEV_EMBED_FRAME_ANCESTORS` allows the framing page, and view answers go
only to the origin that sent the view. SECURITY.md describes the MCP surface
and the panel's browser keys.
- Cockpit now enters on the existing matching map style while keeping one fixed,
duplicate-free carousel over `Normal / CRT / NVG / FLIR / Anime / Noir / Snow`.
Normal remains a real unfiltered option; Cockpit-only choices still restore
the captured map style through both Exit Cockpit and Reset.
- Cyber's compact right-rail and Cockpit utility buttons now center their glyphs
vertically and share the same inset and edge alignment.
- Keep Cyber Voice help/error popups and Location/Visual Presets pins clear of
their decorative frames. Leave space above attribution for its full logo row.
- Add an intentionally future-facing, opt-in panel surface contract for new
panels to inherit compatible Normal, Cyber and Cockpit styling. Its first
production adopter will land separately after this change.
- Restore the user's previous visual preset when they explicitly switch from
Cyber to another HUD layout, without overriding scene or shared-link state.
- Move the aligned Cyber side-panel rails upward on desktop so the left stack
clears the lower coordinate card; keep Cockpit's independent visor layout.
## [0.2.0] — 2026-10-02 — God's Eye View through Your Agent
- Show God's Eye View inside AI conversations. Answers that can be shown
include a view (camera, layers, style, map, marks, something to follow),
and `show_in_gods_eye_view` displays it as the live globe in clients that
support MCP Apps, such as Claude Desktop and the Codex and ChatGPT desktop
apps, and as a link everywhere else. The panel loads the app's panel build
(`npm run build:panel`) and its data through the MCP server, so a local
server needs no HTTPS or tunnel. `?embed=1` shows the globe alone for pages
that frame the app.
- Add tools for language-model clients and a local MCP server, served over
stdio by `npm run mcp` and over HTTP at `/mcp` on the development and preview
servers for local requests only. `npm run mcp`
serves earthquake, active-fire, launch, aircraft, ship, satellite, camera, radio,
place, routing, bike-share, transit, traffic, weather, cyclone, fire-perimeter,
terrain, installation and map-feature queries, plus a situation brief, over
stdio to clients such as Claude Code, reading
from a running app. Tools are defined
once in `gods-eye-view/tools`, reuse the layers' source factories, and are
exposed through the protocol adapter in `gods-eye-view/tools/mcp`. The
catalog also covers weather maps, wind, recent imagery, submarine cables,
license plate reader cameras, datacenters and dams, the Bhote Koshi flood,
military awareness and links that open the app over an area with layers on.
Voice offers the same queries next to its app actions.
- Public Overpass instances are no longer used by default. Street Traffic
roads come from TomTom flow tiles, OpenFreeMap vector tiles, or both, chosen
@@ -172,7 +251,6 @@
geocode and admin-boundary fallback answers `region-timeout` after 3 s
instead of holding the reply.
- Transit and Directions rows repaint as soon as their data lands again:
`refreshLayerStats()` now lives on the layer lifecycle, not only on the
compatibility facade. `scripts/qa-radio.mjs` uses it instead of a private
@@ -206,31 +284,7 @@
(`src/ui/imagerySplit.js`, `src/maps/imageryComparison.js`), and
`MapSourceController.subscribe()` reports every settled map switch.
## Unreleased — voice reaches every layer; honest analyst answers
- Voice can switch on every shipped data layer (the two scene-driven
Bhote Koshi layers stay scene-only), including transit, Recent
Imagery, wind, the weather layers, cyclones and mapped installations, and
understands their spoken names ("buses", "hurricanes", "military bases").
One voice manifest (`src/voice/layerManifest.js`) generates the tool enums
and aliases, and a unit test fails when a new layer has no entry.
- `analyst_query` counts up to 250,000 loaded records per layer instead of the
first 2,000, and says when a layer went past that (`complete: false`). It
also answers over bikeshare stations, transit vehicles, rocket launches,
cyclones, ALPR cameras, mapped installations, local ADS-B aircraft and fire perimeters; fire
records carry their sensor (VIIRS or MODIS).
- Unknown fields, operators, units, value types, scope kinds and invalid
centres are refused with the allowed values instead of answering zero or widening to anywhere. A
layer that is off, still loading or unavailable is reported as such, not
as a count of zero; a partly answerable query is marked `partial`.
Distance sorts honour `sortDir`, a written unit such as `altitudeM(ft)`
converts, and large rankings no longer sort the whole set.
- Results carry `lat`/`lon` for each item and a `display` object with the
scope, the feed window (earthquakes and fires cover the last 24 h) and any
caveat for the screen. The model is asked to answer in one short phrase.
- The weather and Recent Imagery panels can be opened by voice.
## Unreleased — local receiver feeds
### Local receiver feeds
- The Local ADS-B layer also reads local 1090 MHz and 978 MHz UAT decoder
feeds: the `aircraft.json` that dump1090-fa, readsb, tar1090 or skyaware978
@@ -259,7 +313,7 @@
- Records carry `band` (`1090`/`978`) and `source` (`webusb`/`feed`).
- See `docs/LOCAL-RECEIVERS.md`.
## Unreleased — local RTL-SDR and Local ADS-B
### Local RTL-SDR and Local ADS-B
- Add a Local RTL-SDR card to the Radio panel. It connects a USB RTL-SDR in
desktop Chrome or Edge through WebUSB and receives broadcast FM (tune, seek,
@@ -284,7 +338,7 @@
- Add `@jtarrio/webrtlsdr` and `@jtarrio/signals` (Apache-2.0); see
`THIRD_PARTY_NOTICES.md`.
## Unreleased — weather review
### Weather review
- On 3D Tiles, draw a 4096×2048 detail window around the view on each
observed-weather shell except global infrared, sampled by the shell's own
@@ -550,26 +604,26 @@ Analyst records for loaded satellites, datacenters and dams, with explicit bound
- Split application scene, controls, catalog, tools and HTML into reusable components; configure application request services and sources without changing global fetch. Preserve standalone markup and voice behavior. Explicit annotation navigation may resolve a distant named target.
## Satellite pass prediction
### Satellite pass prediction
- Bisect pass rise/set to ~0.2 s and fit peak elevation with a parabola.
- Mark passes visible from Earth-shadow and civil-twilight checks.
- Add `getNextSatellitePass(noradId, options)` for any loaded catalog satellite.
- `next_iss_pass` retains the next geometric pass and adds visibility metadata. `next_satellite_pass` adds bounded loaded-catalog name/NORAD lookup and optional visible-only filtering (Rehaan Delmotra, #451; maintainer adaptation).
## Voice component boundaries
### Voice component boundaries
- Separate voice controls, Realtime connection requests and the action runner.
- Allow compatible endpoints and server-selected models through construction options.
- Cancel pending token/SDP requests on Stop or teardown and reject expired secrets.
## Configurable geospatial services
### Configurable geospatial services
- Compose geocoding, place context and routes through independent providers.
- Allow compatible endpoint configuration without changing voice tools or annotation behavior.
- Isolate configured source caches and reject results after cancellation.
## ALPR camera locations
### ALPR camera locations
- Port Manjunath's (@manjunath22466) cyan camera badges, coral selection brackets,
gradient direction wedges and animated tactical labels into the reusable ALPR
@@ -589,32 +643,32 @@ Analyst records for loaded satellites, datacenters and dams, with explicit bound
- Separate the request adapter, camera model, presentation, and instance lifecycle.
Source cancellation also guards late response bodies and rejects invalid query bounds.
## Release disabled infrastructure rendering
### Release disabled infrastructure rendering
- Remove built Data Center, Dam and Submarine Cable entities when their layers
are disabled, avoiding retained visualizer work and entity memory.
- Keep parsed datasets cached for re-enable; rebuild entities without refetching.
## Camera layer components
### Camera layer components
- Separate camera source requests, placement, frames, projection, cards and calibration.
- Own visibility listeners and pending initialization within each layer lifetime.
- Preserve existing camera catalogs, URL families, geometry and playback behavior.
## Traffic and bikeshare components
### Traffic and bikeshare components
- Separate traffic loading, animation, styling and lifecycle into factory-owned components.
- Give each flow source its own bounded decode cache and cancellation checks.
- Separate bikeshare registry, station requests, rendering, selection and proximity handling.
## Installation and context components
### Installation and context components
- Separate mapped-site requests, records, placement, selection and viewport lifecycle.
- Separate proximity queries, subject tracking, navigation/history, panel and direction rendering.
- Retain source and ground-floor ownership in standalone composition; reject malformed
installation snapshots and ignore failures from cancelled requests.
## Satellite and mission layer components
### Satellite and mission layer components
- Separate catalog loading, orbit calculations, display, tracking and interaction
into instance-owned satellite components.
@@ -622,20 +676,20 @@ Analyst records for loaded satellites, datacenters and dams, with explicit bound
operations, retaining existing layer controls and satellite coordination.
- Cancel late mission source work and reject malformed launch snapshots.
## Fire layer components
### Fire layer components
- Split fire source loading, state, rendering, cards, selection and viewport work
into reusable components with application-owned scene services.
- Cancel late refreshes, retain good data after malformed responses, and preserve
selection identity without repeating a user-selection notification on refresh.
## Earthquake components
### Earthquake components
- Separate earthquake snapshot loading, record validation, and display ownership.
- Cancel pending earthquake refreshes on disable or destruction, retaining the
last good snapshot after malformed or failed refreshes.
## September 8, 2026
### September 8, 2026
Earthquake refreshes validate the complete feed and construct replacement entities before clearing the previous snapshot. Malformed rows and duplicate rendered IDs retain the last good entities, overlays, count and timestamp and report a malformed response; unknown magnitude is excluded from M2.5+ rendering.
@@ -646,7 +700,7 @@ Launch payloads with missing records now say PAYLOAD DATA UNAVAILABLE. Missing n
This changelog records public product changes. For the authoritative description
of current runtime behavior, see [`docs/CURRENT-STATE.md`](docs/CURRENT-STATE.md).
## [Unreleased]
### Other changes
- Add ECMWF IFS model selection to Wind (#464, thanks @beneduzi), with model-scoped forecast-step caches, cancellation of replaced requests, and separate issue/valid timestamps.
@@ -667,7 +721,7 @@ of current runtime behavior, see [`docs/CURRENT-STATE.md`](docs/CURRENT-STATE.md
- Separate submarine cable sources and rendering components, and export bundled geography lookup modules.
### Added
#### Added
- DISPLAY ▸ Draw: draw on the world by hand. Pick Area, Line or Pin, click the
vertices, double-click or press Enter to finish, label and colour it; Backspace
@@ -677,7 +731,7 @@ of current runtime behavior, see [`docs/CURRENT-STATE.md`](docs/CURRENT-STATE.md
together. While you are drawing, the draw tool owns the pointer and no layer
selects what you click through (#235 — thanks @cora-fresh-labs).
### Fixed
#### Fixed
- Keep traffic-road bounds crossing the antimeridian monotonic and inside the
longitude range accepted by the Overpass request path, preserving the small
@@ -744,7 +798,7 @@ of current runtime behavior, see [`docs/CURRENT-STATE.md`](docs/CURRENT-STATE.md
toggle. Camera departure cancels pending work, arrival checks the final view,
and superseded requests cannot keep a newer view loading.
### Added
#### Added
- Add MODIS NRT (Terra+Aqua, ~1 km) active fires to the FIRMS layer, sharing the
existing `FIRMS_MAP_KEY` and 30-minute cache.
@@ -872,9 +926,9 @@ of current runtime behavior, see [`docs/CURRENT-STATE.md`](docs/CURRENT-STATE.md
- Extract shared surface keyboard handling for the welcome launcher and Provider
Settings, preserving Tab/Escape behavior and releasing the listener on teardown.
### Added
#### Added
### Security
#### Security
- The CCTV media route no longer forwards a client `Range` header to the upstream
camera host as it arrived. A single `bytes=` range is canonicalized and
@@ -905,7 +959,7 @@ of current runtime behavior, see [`docs/CURRENT-STATE.md`](docs/CURRENT-STATE.md
- Cancel the active location lookup when its controls are disposed.
### Fixed
#### Fixed
- `DATA_SOURCES.md` states what the project does with camera frame content: a
successful upstream response is relayed as the provider served it, nothing in
@@ -984,7 +1038,7 @@ of current runtime behavior, see [`docs/CURRENT-STATE.md`](docs/CURRENT-STATE.md
fetching, AIS records/tracks and shared request helpers; preserve existing
routes, local setup, fallback behavior and rendering.
### Added
#### Added
- **Directions layer** — keyless A→B directions without a geocoder or a
microphone (thanks @spcpza). The row's chips arm a globe click for A and B
@@ -1012,7 +1066,7 @@ of current runtime behavior, see [`docs/CURRENT-STATE.md`](docs/CURRENT-STATE.md
voice routes since launch, previously uncredited), with the OpenStreetMap
credit and the "fix the map" link the service's usage policy asks for.
### Changed
#### Changed
- The interface asks Google Fonts for only the icon glyphs it draws, instead of
the whole variable icon font, and no longer requests a second icon family that
@@ -1027,7 +1081,7 @@ of current runtime behavior, see [`docs/CURRENT-STATE.md`](docs/CURRENT-STATE.md
- Rename standalone browser startup to `src/standalone/` and add a Node-only
`gods-eye-view/build/vite` export with checked package ownership.
### Development
#### Development
- The CCTV launcher and preview-server tests resolve their temporary fixture
root through `fs.realpath`, so they pass on macOS, where the system temp
@@ -1061,7 +1115,7 @@ of current runtime behavior, see [`docs/CURRENT-STATE.md`](docs/CURRENT-STATE.md
infrastructure modules and their consumer tests. Package boundary checks keep
those exports separate from app startup and local Node services.
### Fixed
#### Fixed
- Reduce terrain-height timeouts when Re:Earth slows down. Batches are
sized against measured response latency on both browser and server to reduce
@@ -1152,7 +1206,7 @@ of current runtime behavior, see [`docs/CURRENT-STATE.md`](docs/CURRENT-STATE.md
and upstream error bodies; response statuses and cache fallback remain intact.
Includes the security fixes contributed by Tom-Neverwinter in PR #171.
### Fixed
#### Fixed
- Map Source keyboard opening retries focus until the selected tile is visible.
Leaving the disclosure, pointer interaction, or closing the tray cancels the
@@ -1194,7 +1248,7 @@ of current runtime behavior, see [`docs/CURRENT-STATE.md`](docs/CURRENT-STATE.md
that had since recovered. A miss is now cached only when every source
consulted actually returned a verdict.
### Added
#### Added
- Keyless place search. The LOCATION search box and the `fly_to_location` voice
tool now resolve place names through Photon (komoot, over OpenStreetMap) when
@@ -1214,6 +1268,13 @@ of current runtime behavior, see [`docs/CURRENT-STATE.md`](docs/CURRENT-STATE.md
to Puppeteer 25.10.0 and Sharp 0.35.4. Cesium remains on 1.138.0.
Browser QA awaits the new asynchronous executable-path lookup.
### Live CCTV video
- Live HLS video shares one decoder between the camera panel and projection,
with a DelDOT HTTPS source pack. Credit: Daniel Slay (@Danielslay86), PR #489.
- Maintainer adjustments bound sessions and downloads, remove disk/subprocess
remuxing, reject redirects, and clean up playback on switching or disabling.
## [0.1.1] — 2026-09-01 — Installation and live-data fixes
### Changed
@@ -1451,9 +1512,3 @@ represent previously published GitHub Releases.
- Initial project version.
### Live CCTV integration candidate
- Live HLS video shares one decoder between the camera panel and projection,
with a DelDOT HTTPS source pack. Credit: Daniel Slay (@Danielslay86), PR #489.
- Maintainer adjustments bound sessions and downloads, remove disk/subprocess
remuxing, reject redirects, and clean up playback on switching or disabling.
-14
View File
@@ -100,20 +100,6 @@ The optional Local ADS-B layer and the broadcast-FM receiver use radio you recei
- **Cockpit regional briefing.** `/api/regional-brief` rounds aircraft coordinates into 0.1° cache cells, caches results for five minutes, and resolves physical regions from bundled public-domain Natural Earth polygons without reverse-geocoding requests. Google News RSS is queried with the resolved locality/region first; GDELT is used only when that RSS query fails or is empty. Google's published Google News terms restrict that source to personal, noncommercial use, so commercial deployments must disable/replace it or obtain separate permission; GDELT permits commercial dataset use with citation. The Data attribution popover identifies the active headline sources; article links retain publisher attribution. Headlines are location-query matches, not verified incidents, risk rankings, or evidence that a location is safe. Empty, partial, stale, and unavailable source states remain distinct. Open-Meteo supplies current conditions independently of the news source. `WX OFF` disables cockpit weather rendering only; the Local Info briefing still fetches its source-backed weather values and displays the required linked Open-Meteo credit.
- **Dynamic weather presentation.** While cockpit mode is active, `/api/weather-effects` requests current Open-Meteo observations for the aircraft/camera location, rounds coordinates into 0.1° cache cells, caches results for five minutes, and may retain a stale observation for up to 30 minutes during a transient outage. WMO condition code selects the visual family; observed cloud cover, precipitation, visibility, wind speed, and wind direction bound its strength and motion. Missing or expired weather renders no synthetic atmospheric effect, and normal globe view never renders the weather overlay.
### On-device voice models (downloaded only when voice runs on this device)
Downloaded by the browser from Hugging Face at pinned revisions and kept in its cache; not stored in this repo.
| Model | Revision | License |
| --- | --- | --- |
| Gemma 4 E2B / E4B / 12B / 26B A4B web builds (`litert-community/gemma-4-*-it-litert-lm`) | per model in `src/voice/local/modelCatalog.js`, SHA-256 checked | Apache-2.0 |
| Moonshine Base (`onnx-community/moonshine-base-ONNX`) | `b1e9b6aa` | MIT |
| Whisper base.en / large-v3-turbo ONNX (`onnx-community/whisper-base.en`, `onnx-community/whisper-large-v3-turbo`) | `51eefc0a` / `360ebcde` | MIT |
| Kokoro-82M and voice vectors (`onnx-community/Kokoro-82M-v1.0-ONNX`) | `1939ad2a` | Apache-2.0 |
Natural voice's phonemizer is fetched the same way, from the public npm CDN (see [THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md)).
---
## Bundled snapshots
+3 -3
View File
@@ -237,8 +237,6 @@ _Why cockpit mode exists: you're riding a real aircraft over real terrain — an
> Voice needs an **OpenAI key**. Without one the entire app still runs — the mic button just reports voice is unavailable. The same key drives the **AI HUD summary**: a terse, five-word intelligence-style readout of the current view that regenerates as you move.
> **No key? Run voice on this device.** Open the tray next to the mic and choose **ON-DEVICE**: Gemma 4, Moonshine and Kokoro run in the browser (WebGPU; current Chrome or Edge) after a one-time download, with no key and no per-use cost. The tray checks your hardware: 16 GB+ Apple Silicon works best with Gemma 4 12B (6 GB download); 8–16 GB with Gemma 4 E4B (3 GB). `?voice=local-web` is a shortcut.
Click **GEV MIC**, grant the microphone, and just talk. This is more than a voice-controlled remote:
- **🧠 It knows what it's looking at.** The agent pulls live scene context before answering — including coordinates, street names, active layers, and view scale. Ask _"what city is this?"_ mid-flight and it knows.
@@ -281,6 +279,8 @@ _Ask for radio near anywhere and the globe starts broadcasting — every station
---
**💬 Or ask from Claude and Codex.** God's Eye View is also an MCP server: ask Claude Desktop, Codex or ChatGPT desktop to show a place, and the live globe opens right in the conversation. Setup takes a minute — see [docs/MCP_SETUP.md](docs/MCP_SETUP.md).
## 🛰️ What's on the Globe
Nineteen layers and map sources. **Seventeen have a keyless path.** Some offer additional capabilities with a provider key. (🟢 no key · 🟡 free key · 🔴 metered.)
@@ -512,7 +512,7 @@ Everything above is the deliberately cheap baseline — enough to get a real tas
### 🔒 Sharing an instance
By default nobody else can reach your server — it binds to localhost. To share on your LAN, opt in explicitly (`npm run dev -- --host 0.0.0.0 --port 4173`, or `HOST=0.0.0.0 ./scripts/dev-fresh.sh` on macOS/Linux) — but know that ⚠️ **a LAN-visible server brokers your configured API keys to anyone who can reach it.** Set the per-IP throttles (`GEV_RATELIMIT_OPENAI_PER_MIN`, `GEV_RATELIMIT_GOOGLE_PER_MIN` — see `.env.example`) and, before anything else, **configure provider quotas, usage limits, and billing alerts**: app-level throttles are not billing caps, and a budget alert alone does not stop spending. Full threat model in [SECURITY.md](SECURITY.md).
By default nobody else can reach your server — it binds to localhost. To share on your LAN, opt in explicitly (`npm run dev -- --host 0.0.0.0 --port 4173`, or `HOST=0.0.0.0 ./scripts/dev-fresh.sh` on macOS/Linux). IP addresses work as they are; if you use a LAN hostname, add it with `GEV_ALLOWED_HOSTS=globe.lan`, since binding to `0.0.0.0` does not trust arbitrary Host headers. But know that ⚠️ **a LAN-visible server brokers your configured API keys to anyone who can reach it.** Per-IP throttles on the cost-bearing endpoints are on by default (`GEV_RATELIMIT_OPENAI_PER_MIN`, `GEV_RATELIMIT_GOOGLE_PER_MIN` — see `.env.example` to tune them), but before anything else, **configure provider quotas, usage limits, and billing alerts**: app-level throttles are not billing caps, and a budget alert alone does not stop spending. Full threat model in [SECURITY.md](SECURITY.md).
Provider Settings is disabled when the server is shared, so remote users cannot
access the key-entry panel.
+13 -2
View File
@@ -71,14 +71,16 @@ The data proxies under `server/providers/` are written so the browser cannot tur
- **Sanitized errors** — internal error details are not echoed back to clients.
- **Coalesced OAuth refresh** and cached successful responses only (OpenSky).
- **Redacted debug logging.** The voice debug log (`.gev-logs/`, gitignored) strips API keys, bearer tokens, client secrets, and image data URLs before writing.
- **Cross-site gate on cost/log endpoints.** The cost-bearing (`/api/realtime/token`, `/api/openai/hud-summary`, `/api/google/nearby-places`, `/api/google/text-search`) and log (`/api/realtime/debug-log`) endpoints refuse cross-site browser requests: a foreign or opaque `Origin`, a `Sec-Fetch-Site` other than `same-origin`/`none` (this is what blocks an `<img>`/navigation that carries no `Origin`), or any reverse-proxy/CDN forwarding header returns `403`. Non-browser loopback tools (the repo QA harnesses POST the token endpoint with no `Origin`) and the explicit `HOST=0.0.0.0` LAN opt-in keep working — the gate deliberately does not require a loopback remote address; that stricter requirement belongs only to the credential panel (`admitKeySetupRequest`). The Host check (`allowedHosts`, applied by `hostCheckPlugin` ahead of every route, since Vite's own check runs only after plugin middleware) rejects a rebinding `Host` before any handler runs, in every mode, and the gate is defense in depth behind it: the LAN opt-in keeps that check and accepts only IP addresses, `localhost` and names listed in `GEV_ALLOWED_HOSTS`. **Limit:** a name you list there is trusted like the app's own address, so list only names you control.
- **Full Content-Security-Policy.** The dev and preview servers send a real CSP (`script-src 'self' 'unsafe-eval' blob:` plus the YouTube, Facebook and X scripts the Nepal event's embedded media loads, with a matching `frame-src` for their players and posts: no inline script and no other script origin; `blob:` covers the object URLs Cesium's bundled workers bootstrap through in the built app), alongside the existing `X-Frame-Options: DENY` and `frame-ancestors 'none'`. `'unsafe-eval'` stays because Knockout, bundled inside Cesium's widgets package, resolves the global object with `eval` at load time and the globe does not initialize without it; styles permit `'unsafe-inline'` and Google Fonts because Vite's dev client injects `<style>` elements.
## Network exposure — the operator threat model
The dev server is a **key broker**: every server-side key above is spendable by anyone who can send HTTP requests to it. That shapes the defaults:
- **Local-only by default.** `./scripts/dev-fresh.sh` (and the Vite config itself) bind to `localhost`, so only your machine can reach the server — and only local names are accepted (`allowedHosts` stays restricted, which also blunts DNS-rebinding tricks).
- **Local-only by default.** `./scripts/dev-fresh.sh` (and the Vite config itself) bind to `localhost`, so only your machine can reach the server — and only local names are accepted (`allowedHosts` stays restricted, which also blunts DNS-rebinding tricks). Binding to all interfaces keeps that restriction: IP addresses and `localhost` work, and LAN hostnames must be listed in `GEV_ALLOWED_HOSTS` (for example `GEV_ALLOWED_HOSTS=globe.lan`); suffix and wildcard entries are ignored.
- **LAN exposure is an explicit opt-in**: `HOST=0.0.0.0 ./scripts/dev-fresh.sh`. The launcher prints a prominent warning plus your LAN URL. Understand what opting in means: **every device on that network can drive the proxies and spend your OpenAI / Google / OpenSky / AISStream / TomTom / FIRMS quota** for as long as the server runs. Do this only on networks you trust.
- **App-level throttles (opt-in):** `GEV_RATELIMIT_OPENAI_PER_MIN` and `GEV_RATELIMIT_GOOGLE_PER_MIN` cap the cost-bearing endpoints per client IP per minute (over-limit requests receive a sanitized `429`). They are **per-IP, process-local, in-memory guards** — they reset on restart and are **not billing caps**.
- **App-level throttles (on by default):** `GEV_RATELIMIT_OPENAI_PER_MIN` (default `30`) and `GEV_RATELIMIT_GOOGLE_PER_MIN` (default `120`) — the values the Pinokio build already ships — cap the cost-bearing endpoints per client IP per minute (over-limit requests receive a sanitized `429`). Both defaults sit well above what the app itself generates; set either to exactly `0` to run unthrottled. They are **per-IP, process-local, in-memory guards** — they reset on restart, a caller with several addresses gets a bucket per address, and they are **not billing caps**.
- **Provider-side budgets are the real backstop.** For hard spend protection, configure limits where the money is: OpenAI platform usage limits, Google Cloud budget alerts + per-API quotas, and equivalent controls for any other keyed provider.
- **Pinokio LAN and Cloudflare sharing are refused.** The current supported
Pinokio release re-reads sharing state when an app registers its Open URL and
@@ -90,6 +92,15 @@ The dev server is a **key broker**: every server-side key above is spendable by
loopback only. Use a separately reviewed authentication proxy for remote
access and keep provider-side quotas as the spend backstop.
## MCP server and panel
God's Eye View's tools are also served to MCP clients: over stdio (`npm run mcp`) and at `/mcp` on the development and preview servers.
- **`/mcp` answers only direct local requests**: a loopback connection naming a loopback host on the port it reached, a browser `Origin` (when sent) from that same host, no proxy forwarding headers, and launcher sharing off. It accepts only JSON, so a web page cannot post to it cross-site. This is transport safety, not authentication: any program on your machine can use the tools, and some of them spend the same provider quotas as the app.
- **What the panel's requests can reach.** In clients that display MCP Apps, the panel loads the app through the `panel_request` tool, which requests paths on the app's server. It is marked for the panel only and requires a key that each MCP server puts in its panel page, which keeps it from clients that list tools to the model without loading the panel. That key is not access control: any MCP client can read the panel page, key included, and then reach the app's files and data routes through `panel_request`, much as its other tools reach the data. Provider Settings, credential and model endpoints, `/mcp` and the development server's internal routes are refused to every caller.
- **The panel carries the two browser keys.** The panel build (`npm run build:panel`) includes the same Google Maps key and Cesium ion token as the app, and they run inside the client's panel page, on the client's site. A key restricted by HTTP referrer must also allow that page, or the panel cannot load tiles. To keep the app's own keys narrower, build the panel with separately restricted keys in the environment of `npm run build:panel`.
- **Embed mode is not framable by default.** `?embed=1` documents keep `X-Frame-Options: DENY` and `frame-ancestors 'none'` unless `GEV_EMBED_FRAME_ANCESTORS` names the pages that may frame them; a framing page can change what the app shows. Other documents can never be framed.
## Scope & expectations
- The Vite server is a **development/preview** server. If you expose it beyond localhost, put it behind your own auth/proxy and review the bindings (see the threat model above).
+2 -23
View File
@@ -1,9 +1,8 @@
# Third-party software notices
The project depends on third-party npm packages whose own licenses apply. This
file records the packages added for the browser-local SDR and on-device voice
features; the complete resolved dependency inventory remains in
`package-lock.json`.
file records the packages added for the browser-local SDR feature; the complete
resolved dependency inventory remains in `package-lock.json`.
## Web RTL-SDR
@@ -20,23 +19,3 @@ features; the complete resolved dependency inventory remains in
- Source: <https://github.com/jtarrio/signals>
- License: Apache License 2.0
- License text: <https://www.apache.org/licenses/LICENSE-2.0>
## On-device voice
Loaded only when voice runs on this device.
| Package | Version | Source | License |
| --- | --- | --- | --- |
| `@litert-lm/core` (runtime and wasm served from the build) | 0.17.1 | <https://github.com/google-ai-edge/LiteRT-LM> | Apache-2.0 |
| `@huggingface/transformers` (speech recognition) | 4.3.0 | <https://github.com/huggingface/transformers.js> | Apache-2.0 |
| `onnxruntime-web` (with transformers.js 4.3.0) | 1.31.0-dev.20260914 | <https://github.com/microsoft/onnxruntime> | MIT |
| `kokoro-js` (natural voice) | 1.2.1 | <https://github.com/hexgrad/kokoro> | Apache-2.0 |
| `@huggingface/transformers` (used by kokoro-js) | 3.8.1 | <https://github.com/huggingface/transformers.js> | Apache-2.0 |
| `onnxruntime-web` (used by kokoro-js) | 1.22.0-dev.20250409 | <https://github.com/microsoft/onnxruntime> | MIT |
Natural voice also uses `phonemizer` 1.2.1 (<https://github.com/xenova/phonemizer.js>),
which embeds eSpeak NG (<https://github.com/espeak-ng/espeak-ng>, GPL-3.0-or-later).
It is not part of this repository or its build: when natural voice first
starts, the user's browser downloads the pinned file from
`cdn.jsdelivr.net/npm/[email protected]` and checks its SHA-256 before use.
`GEV_NATURAL_VOICE=off` turns natural voice off.
+94
View File
@@ -0,0 +1,94 @@
/**
* Host header allowlist for the Vite dev/preview server. The configuration
* factory passes the configured names in; this module reads no environment.
*
* Binding to a wildcard interface only controls which network interfaces accept
* connections. It must not turn into a wildcard Host-header policy: this server
* brokers configured API keys through local proxy routes.
*/
import { isIP } from 'node:net';
export const DEFAULT_ALLOWED_HOSTS = Object.freeze(['localhost', '127.0.0.1']);
/**
* Resolve Vite's allowedHosts setting from an optional comma-separated list.
* Empty entries and suffix/wildcard entries are ignored; repeated names keep
* their first occurrence. Vite treats leading-dot values as suffix wildcards,
* which would defeat an explicit host policy.
*
* @param {string|undefined|null} configuredHosts
* @returns {string[]}
*/
export function resolveAllowedHosts(configuredHosts) {
const hosts = String(configuredHosts ?? '')
.split(',')
.map((host) => host.trim())
.filter((host) => host && !host.startsWith('.') && !host.includes('*'));
return [...new Set([...DEFAULT_ALLOWED_HOSTS, ...hosts])];
}
/**
* Whether a Host header names a host the server answers, by Vite's own rule:
* IP addresses, `localhost` and `*.localhost` always; then `additional` names
* (the configured server host) and `allowedHosts` (a leading dot allows a
* domain and its subdomains). `allowedHosts === true` allows every host.
*
* @param {string|undefined} hostHeader
* @param {string[]|true} allowedHosts
* @param {string[]} [additional]
* @returns {boolean}
*/
export function isAllowedHost(hostHeader, allowedHosts, additional = []) {
if (allowedHosts === true) return true;
const host = String(hostHeader ?? '').trim();
if (!host) return false;
if (host[0] === '[') {
const end = host.indexOf(']');
return end > 0 && isIP(host.slice(1, end)) === 6;
}
const colon = host.indexOf(':');
const hostname = (colon === -1 ? host : host.slice(0, colon)).toLowerCase();
if (isIP(hostname) === 4) return true;
if (hostname === 'localhost' || hostname.endsWith('.localhost')) return true;
if (additional.includes(hostname)) return true;
return (allowedHosts || []).some(
(allowed) =>
allowed === hostname ||
(allowed[0] === '.' &&
(allowed.slice(1) === hostname || hostname.endsWith(allowed))),
);
}
/**
* Vite plugin that applies the Host check before every other middleware.
* Vite installs its own check after the middleware plugins add in
* `configureServer`, so without this the app's `/api` routes would answer any
* Host, including a DNS-rebinding name. It runs first (`enforce: 'pre'`) and
* uses the same allowed hosts as Vite, on the dev and preview servers.
*/
export function hostCheckPlugin() {
const middleware = (config, preview) => {
const allowed = preview
? config.preview.allowedHosts
: config.server.allowedHosts;
const additional = [config.server.host, config.preview.host].filter(
(value) => typeof value === 'string' && value,
);
return (req, res, next) => {
if (isAllowedHost(req.headers.host, allowed, additional)) return next();
res.writeHead(403, { 'Content-Type': 'text/plain' });
res.end('Blocked request. This host is not allowed.');
};
};
return {
name: 'host-check',
enforce: 'pre',
configureServer(server) {
server.middlewares.use(middleware(server.config, false));
},
configurePreviewServer(server) {
server.middlewares.use(middleware(server.config, true));
},
};
}
+73
View File
@@ -0,0 +1,73 @@
/**
* Lets chosen pages frame the app in embed mode (`?embed=1`). Every
* document keeps the server's `X-Frame-Options: DENY` and
* `frame-ancestors 'none'` unless `GEV_EMBED_FRAME_ANCESTORS` names who may
* frame embed-mode documents: CSP frame-ancestors sources, or `*` for any
* page. A framing page can change what the app shows, so no page may frame
* it by default. Other documents, including Provider Settings, are never
* framable. "Any" sends no framing restriction at all: `frame-ancestors *`
* would still refuse sandboxed frames with an opaque origin.
*/
const FRAMING_HEADERS = new Set(['x-frame-options', 'content-security-policy']);
/** The directives of a CSP header value, minus any frame-ancestors rule. */
function withoutFraming(value) {
return String(value ?? '')
.split(';')
.map((directive) => directive.trim())
.filter(Boolean)
.filter((directive) => !/^frame-ancestors\b/i.test(directive));
}
/** Whether a request is for an embed-mode document. */
export function isEmbedDocumentRequest(url) {
let parsed;
try {
parsed = new URL(url || '/', 'http://localhost');
} catch {
return false;
}
return (
parsed.searchParams.get('embed') === '1' &&
(parsed.pathname === '/' || parsed.pathname.endsWith('.html'))
);
}
/** Vite plugin applying the embed framing policy on dev and preview servers. */
export function embedFramingPlugin({
ancestors = process.env.GEV_EMBED_FRAME_ANCESTORS || '',
} = {}) {
const allowed = ancestors.trim();
// Unset: embed documents keep the server's framing protection.
if (!allowed) return { name: 'embed-framing' };
const anywhere = allowed === '*';
const policy = `frame-ancestors ${allowed}`;
const install = (server) => {
server.middlewares.use((req, res, next) => {
if (!isEmbedDocumentRequest(req.url)) return next();
// The server's own headers are written when the response is sent, so
// intercept them rather than setting ours first.
const setHeader = res.setHeader.bind(res);
res.setHeader = (name, value) => {
const key = String(name).toLowerCase();
if (!FRAMING_HEADERS.has(key)) return setHeader(name, value);
if (key === 'content-security-policy') {
// Keep every other directive of the server's policy (script-src,
// connect-src, ...) and only swap the framing rule.
const kept = withoutFraming(value);
const next = anywhere ? kept : [...kept, policy];
return next.length ? setHeader(name, next.join('; ')) : res;
}
return res;
};
if (!anywhere) setHeader('Content-Security-Policy', policy);
next();
});
};
return {
name: 'embed-framing',
configureServer: install,
configurePreviewServer: install,
};
}
-170
View File
@@ -1,170 +0,0 @@
import { createRequire } from 'node:module';
import path from 'node:path';
// Two of the LiteRT-LM package's four builds: relaxed SIMD with JavaScript
// promise integration (current Chrome and Edge) and the most compatible one.
const LITERT_VARIANTS = {
internal: 'litertlm_wasm_internal',
compatAsyncify: 'litertlm_wasm_compat_asyncify_internal',
};
const ASSETS_MODULE = path.join(
'src',
'voice',
'local',
'workers',
'runtimeAssets.js',
);
// kokoro-js imports `phonemizer`, which embeds eSpeak NG. The build never
// includes it: the import resolves to this loader, which downloads the pinned
// file from the public npm CDN at runtime and checks its SHA-256.
const PHONEMIZER_MODULE = path.join(
'src',
'voice',
'local',
'workers',
'phonemizerRuntime.js',
);
/** Strings that only appear in the eSpeak NG phonemizer build. */
export const ESPEAK_MARKERS = Object.freeze([
'eSpeakNGWorker',
'espeak-ng-data',
'espeakEVENT',
]);
/** Absolute path of the runtime phonemizer loader for a checkout. */
export function phonemizerLoaderPath(root = process.cwd()) {
return path.join(root, PHONEMIZER_MODULE);
}
/** Names of emitted chunks or assets that contain eSpeak NG code. */
export function findEspeakChunks(bundle) {
const found = [];
for (const [name, output] of Object.entries(bundle || {})) {
const text =
typeof output.code === 'string'
? output.code
: typeof output.source === 'string'
? output.source
: output.source instanceof Uint8Array
? Buffer.from(output.source).toString('latin1')
: '';
if (ESPEAK_MARKERS.some((marker) => text.includes(marker)))
found.push(name);
}
return found;
}
const posix = (file) => file.split(path.sep).join('/');
/** Directory of the installed package that `from` resolves `name` to. */
function packageDir(from, name) {
const entry = createRequire(from).resolve(name);
const marker = `${path.sep}node_modules${path.sep}${name.split('/').join(path.sep)}${path.sep}`;
const at = entry.lastIndexOf(marker);
if (at < 0) throw new Error(`Cannot locate ${name} from ${from}`);
return entry.slice(0, at + marker.length - 1);
}
/**
* Serves the on-device voice runtimes from the application build instead of
* a CDN. It replaces src/voice/local/workers/runtimeAssets.js with imports
* of the installed LiteRT-LM Emscripten loaders (classic scripts, exported
* here as modules) and wasm files, and the ONNX Runtime Web files that the
* two transformers.js copies (speech recognition, Kokoro) use.
*/
export function onDeviceRuntimePlugin({ root = process.cwd() } = {}) {
let paths = null;
const resolvePaths = () => {
if (paths) return paths;
const rootFile = path.join(root, 'package.json');
const require = createRequire(rootFile);
const litert = path.join(packageDir(rootFile, '@litert-lm/core'), 'wasm');
const ort = path.join(
packageDir(
require.resolve('@huggingface/transformers'),
'onnxruntime-web',
),
'dist',
);
const kokoroTransformers = path.join(
packageDir(require.resolve('kokoro-js'), '@huggingface/transformers'),
'dist',
);
paths = {
assets: path.join(root, ASSETS_MODULE),
loaders: new Set(
Object.values(LITERT_VARIANTS).map((name) =>
path.join(litert, `${name}.js`),
),
),
source() {
const lines = [];
const runtimes = [];
for (const [key, name] of Object.entries(LITERT_VARIANTS)) {
const file = posix(path.join(litert, name));
lines.push(
`import ${key}Factory from ${JSON.stringify(`${file}.js`)};`,
);
lines.push(
`import ${key}Wasm from ${JSON.stringify(`${file}.wasm?url`)};`,
);
runtimes.push(
`${key}: { ModuleFactory: ${key}Factory, wasmUrl: ${key}Wasm }`,
);
}
const onnx = [];
for (const [key, dir, name] of [
['transformers', ort, 'ort-wasm-simd-threaded.asyncify'],
['kokoro', kokoroTransformers, 'ort-wasm-simd-threaded.jsep'],
]) {
const file = posix(path.join(dir, name));
lines.push(
`import ${key}Mjs from ${JSON.stringify(`${file}.mjs?url`)};`,
);
lines.push(
`import ${key}OrtWasm from ${JSON.stringify(`${file}.wasm?url`)};`,
);
onnx.push(`${key}: { mjs: ${key}Mjs, wasm: ${key}OrtWasm }`);
}
lines.push(`export const litertRuntimes = { ${runtimes.join(', ')} };`);
lines.push(`export const onnxWasmPaths = { ${onnx.join(', ')} };`);
lines.push('export function requireRuntime(value) { return value; }');
return lines.join('\n');
},
};
return paths;
};
const phonemizer = phonemizerLoaderPath(root);
return {
name: 'gev-on-device-runtime',
enforce: 'pre',
resolveId(source) {
return source === 'phonemizer' ? phonemizer : null;
},
generateBundle(_options, bundle) {
const found = findEspeakChunks(bundle);
if (found.length)
this.error(
`eSpeak NG phonemizer code must not be bundled (found in ${found.join(', ')})`,
);
},
load(id) {
const file = path.normalize(id.split('?')[0]);
return file === resolvePaths().assets ? resolvePaths().source() : null;
},
transform(code, id) {
const file = path.normalize(id.split('?')[0]);
if (!file.endsWith('.js') || !resolvePaths().loaders.has(file))
return null;
// Modules are strict: the loaders' debug hook declares a function
// inside a block, which only sloppy scripts hoist.
const prelude =
'var custom_dbg = function () { console.warn.apply(console, arguments); };\n';
return {
code: `${prelude}${code}\nexport default ModuleFactory;\n`,
map: null,
};
},
};
}
+153
View File
@@ -0,0 +1,153 @@
/**
* The MCP Apps panel build. A panel loads the app through its MCP server,
* one tool call per file (see src/tools/globePanel.js), so the app must be
* few files: one app script, one stylesheet, and Cesium's own script, which
* carries its workers and starts them from memory. The build is served at
* PANEL_BASE next to the app.
*/
// The panel loads these paths (src/tools/globePanel.js); a test keeps the
// two in step.
export const PANEL_BASE = '/panel/';
/** The script the panel runs ahead of Cesium's workers (workerFilesPrelude). */
export const PANEL_WORKER_PRELUDE_PATH = 'cesium/worker-prelude.js';
export const PANEL_OUT_DIR = 'dist/panel';
/** A browser Vite config changed to produce the panel build. */
export function panelBuildConfig(config) {
return {
...config,
base: PANEL_BASE,
build: {
...config.build,
outDir: PANEL_OUT_DIR,
emptyOutDir: true,
modulePreload: false,
cssCodeSplit: false,
rollupOptions: {
...config.build?.rollupOptions,
output: { inlineDynamicImports: true },
},
},
};
}
const CONTENT_TYPES = {
'.css': 'text/css',
'.geojsonl': 'application/geo+json-seq',
'.glb': 'model/gltf-binary',
'.html': 'text/html',
'.jpg': 'image/jpeg',
'.js': 'text/javascript',
'.json': 'application/json',
'.png': 'image/png',
'.svg': 'image/svg+xml',
'.wasm': 'application/wasm',
'.woff2': 'font/woff2',
'.xml': 'application/xml',
};
/**
* Vite plugin serving the panel build at PANEL_BASE on the dev server, as
* the preview server and production builds serve it from the build output.
*/
export function panelBuildPlugin({ outDir = PANEL_OUT_DIR } = {}) {
return {
name: 'panel-build',
configureServer(server) {
server.middlewares.use(async (req, res, next) => {
const { pathname } = new URL(req.url || '/', 'http://localhost');
if (!pathname.startsWith(PANEL_BASE)) return next();
const { readFile } = await import('node:fs/promises');
const { extname, resolve, sep } = await import('node:path');
const root = resolve(server.config.root, outDir);
let relative;
try {
relative = decodeURIComponent(pathname.slice(PANEL_BASE.length));
} catch {
res.statusCode = 400;
res.end('Bad path');
return;
}
const file = resolve(root, relative || 'index.html');
if (file !== root && !file.startsWith(root + sep)) return next();
try {
const body = await readFile(file);
res.setHeader(
'Content-Type',
CONTENT_TYPES[extname(file)] || 'application/octet-stream',
);
res.end(body);
} catch {
res.statusCode = 404;
res.end(
relative ? 'Not found' : 'No panel build; run npm run build:panel',
);
}
});
},
};
}
/**
* Cesium files its workers load themselves, as paths under Cesium's base.
* Workers request them from the panel page's own site, which has none of
* the app's files, so the panel build embeds them in the workers script.
*/
export const PANEL_WORKER_FILES = Object.freeze([
'Assets/approximateTerrainHeights.json',
]);
/**
* A script, run in each worker before Cesium's, that answers the worker's
* requests for `files` (Cesium-relative path → text) from memory and lets
* every other request through.
*/
export function workerFilesPrelude(files) {
const answer = (FILES) => {
const find = (url) => {
const path = String(url).split(/[?#]/)[0];
for (const name of Object.keys(FILES))
if (path.endsWith(`/${name}`)) return FILES[name];
return null;
};
const xhr = XMLHttpRequest.prototype;
const { open, send, setRequestHeader } = xhr;
xhr.open = function (method, url, ...rest) {
this.__gevFile = find(url);
if (this.__gevFile === null) return open.call(this, method, url, ...rest);
};
xhr.setRequestHeader = function (...args) {
if (this.__gevFile === null) return setRequestHeader.apply(this, args);
};
xhr.send = function (body) {
const text = this.__gevFile;
if (text === null || text === undefined) return send.call(this, body);
const type = this.responseType;
const response =
type === 'json'
? JSON.parse(text)
: type === 'arraybuffer'
? new TextEncoder().encode(text).buffer
: text;
const state = {
readyState: 4,
status: 200,
statusText: 'OK',
response,
responseText: typeof response === 'string' ? response : '',
getAllResponseHeaders: () => 'content-type: application/json\r\n',
getResponseHeader: (name) =>
name.toLowerCase() === 'content-type' ? 'application/json' : null,
};
for (const [name, value] of Object.entries(state))
Object.defineProperty(this, name, { configurable: true, value });
setTimeout(() => {
this.dispatchEvent(new Event('readystatechange'));
this.dispatchEvent(new ProgressEvent('load'));
this.dispatchEvent(new ProgressEvent('loadend'));
});
};
};
return `(${answer})(${JSON.stringify(files)});\n`;
}
+62 -27
View File
@@ -1,9 +1,50 @@
import { applicationHtmlPlugin } from './application-html.js';
import cesium from 'vite-plugin-cesium';
import {
onDeviceRuntimePlugin,
phonemizerLoaderPath,
} from './onDeviceRuntime.js';
import { DEFAULT_ALLOWED_HOSTS, hostCheckPlugin } from './allowedHosts.js';
import { embedFramingPlugin } from './embed-framing.js';
import { panelBuildPlugin } from './panel.js';
/**
* Content-Security-Policy for every document the dev/preview server serves.
* No inline script is permitted, and the only foreign script origins are the
* three that the Bhote Koshi event's embedded media needs (YouTube, Facebook
* and X), which frame-src matches for their players and posts. 'unsafe-eval' is
* required: Knockout (bundled inside @cesium/widgets) resolves the global
* object with `(0, eval)("this")` at module load, and without it the Cesium
* widget never initializes (verified in headless Chrome). It also covers
* Cesium's WASM decoders. `blob:` is required in the built app: Cesium's
* bundled workers bootstrap through `importScripts(blob:...)`, which a worker
* checks against script-src (the dev server loads them by URL instead).
* Widen any other directive only for a real violation.
*/
export const BROWSER_CSP = [
"default-src 'self'",
// The Bhote Koshi event's embedded media: the YouTube player API, the
// Facebook video SDK and X's post widget (src/data/bhoteKoshiEmbeddedMedia.js).
"script-src 'self' 'unsafe-eval' blob: https://www.youtube.com https://connect.facebook.net https://platform.twitter.com",
"style-src 'self' 'unsafe-inline' https://fonts.googleapis.com",
"font-src 'self' data: https://fonts.gstatic.com",
"img-src 'self' data: blob: https:",
"media-src 'self' blob: https:",
"connect-src 'self' blob: data: https: wss: ws:",
"worker-src 'self' blob:",
"child-src 'self' blob:",
// The same media's players and posts, which load in frames.
"frame-src 'self' blob: https://www.youtube-nocookie.com https://www.youtube.com https://www.facebook.com https://platform.twitter.com",
"manifest-src 'self'",
"object-src 'none'",
"base-uri 'self'",
"form-action 'self'",
"frame-ancestors 'none'",
].join('; ');
/** Response headers shared by the dev and preview servers. */
export const BROWSER_HEADERS = Object.freeze({
'X-Frame-Options': 'DENY',
'Content-Security-Policy': BROWSER_CSP,
// Responses are only ever used as the type the server declares.
'X-Content-Type-Options': 'nosniff',
});
/** Build browser assets with explicit inputs; never load environment or providers. */
export function createBrowserViteConfig({
@@ -13,22 +54,19 @@ export function createBrowserViteConfig({
cesiumToken,
host = 'localhost',
port = 4173,
naturalVoice,
allowedHosts = DEFAULT_ALLOWED_HOSTS,
command,
} = {}) {
return {
plugins: [
// First, so no provider route answers a Host the server does not allow.
hostCheckPlugin(),
cesium(),
applicationHtmlPlugin(),
onDeviceRuntimePlugin(),
...plugins,
embedFramingPlugin(),
panelBuildPlugin(),
],
// Workers are bundled separately and need the on-device runtime too.
worker: { plugins: () => [onDeviceRuntimePlugin()] },
// Also applied while pre-bundling kokoro-js for the dev server.
resolve: {
alias: [{ find: /^phonemizer$/, replacement: phonemizerLoaderPath() }],
},
...(publicDir === undefined ? {} : { publicDir }),
// A production build must not clean the dependency cache a running dev
// server is still serving optimized module URLs from.
@@ -42,33 +80,30 @@ export function createBrowserViteConfig({
'@jtarrio/signals/demod/modes.js',
'@jtarrio/webrtlsdr/rtlsdr.js',
'egm96-universal',
'@litert-lm/core',
'@huggingface/transformers',
'kokoro-js',
],
},
server: {
host: host || 'localhost',
port: parseInt(port, 10) || 4173,
allowedHosts:
host === '0.0.0.0' || host === '::'
? true
: ['localhost', '127.0.0.1', '.local'],
// A wildcard bind only chooses interfaces; it never disables Vite's
// Host-header check. LAN hostnames are added explicitly (see
// build/allowedHosts.js); IP addresses are always accepted.
allowedHosts: [...allowedHosts],
fs: {
deny: ['.env', '.env.*', '*.{crt,pem}', '**/.git/**', '**/ENVIRONMENT'],
},
// These headers protect the document containing Provider Settings.
headers: {
'X-Frame-Options': 'DENY',
'Content-Security-Policy': "frame-ancestors 'none'",
},
// These headers protect the document containing Provider Settings and
// give the whole page a real Content-Security-Policy (BROWSER_CSP).
// Embed-mode documents are framable instead: embed-framing.js rewrites
// only the frame-ancestors directive and keeps the rest of the policy.
headers: BROWSER_HEADERS,
},
// The preview server serves the same documents, so it carries the same
// framing + CSP hardening (one constant, no drift).
preview: { headers: BROWSER_HEADERS },
define: {
'import.meta.env.GOOGLE_MAPS_API_KEY': JSON.stringify(googleApiKey),
'import.meta.env.CESIUM_ION_TOKEN': JSON.stringify(cesiumToken),
// GEV_NATURAL_VOICE=off leaves natural (Kokoro) voice out of the
// on-device tier; replies use the browser's on-device voices or text.
'import.meta.env.GEV_NATURAL_VOICE': JSON.stringify(naturalVoice ?? ''),
},
build: { chunkSizeWarningLimit: 1500 },
};
+85 -60
View File
@@ -1,10 +1,51 @@
# God's Eye View Current State
## Cyber HUD — September 23, 2026
## God's Eye View in conversations — October 2, 2026
Tool answers that can be shown in God's Eye View include a view: camera, layers,
style, map, marks and an aircraft or satellite to follow, written in the
share-link format (`gods-eye-view/view`). `show_in_gods_eye_view` shows a view as live
God's Eye View inside clients that display MCP Apps, and as a link everywhere.
The panel runs the app's panel build (`npm run build:panel`, served at
`/panel/`) and loads it, with its data, through the MCP server, so it works
in Claude Desktop and in the Codex and ChatGPT desktop apps with a local
server and no HTTPS. `?embed=1` shows the app as the globe alone and takes
new views from the page that frames it; framing is off unless
`GEV_EMBED_FRAME_ANCESTORS` allows the framing page. MCP leads with the tools that find
what to show. See [tools and the MCP server](TOOLS.md).
## Tools and local MCP server — October 1, 2026
`gods-eye-view/tools` defines queries that answer questions from the app's data,
and `gods-eye-view/tools/mcp` exposes a composed catalog over the Model Context
Protocol. `npm run mcp` serves Core's tools over stdio to a local MCP client,
reading from a running app's `/api` routes (default `http://localhost:4173`).
The development and preview servers also serve the tools over HTTP at `/mcp`,
accepting only direct local requests: a loopback host on the port reached,
an `Origin` (when sent) from that same host, no proxy forwarding headers, and
launcher sharing off.
Queries cover earthquakes, active fires, recent launches, aircraft (in an
area, by identifier, tracks, and type and route lookups), ships (in an area,
by identifier, and tracks), satellites (next
pass over a point, and those overhead now), public cameras (including a
camera's current image), license plate reader cameras, radio stations, place search, routing, bike-share
stations, transit vehicles, road traffic flow, weather, weather map images
(radar, satellite, lightning), wind, the most recent satellite image of an
area, submarine cables, datacenters and dams, the Bhote Koshi flood event pack, regional briefs, tropical cyclones, fire
perimeters, terrain height, military installations and map features, plus a
combined situation brief, military awareness around a point, the app's heads-up display caption, and a link that
opens the app over an area. Tools reuse the layers' portable source factories, take a
shared `area` argument (place name, bounding box, or point and radius) and cap
lists at 25 rows by default. Voice offers the same queries next to its app
actions: the session lists them, and the browser runs them through the same
catalog, loaded on first use. See [tools and the MCP server](TOOLS.md).
## Cyber HUD — September 30, 2026
Display > HUD > Layout includes Cyber, also available through the HUD voice
action and shared visual state. An explicit first transition into Cyber selects
FLIR with Ironbow 0.42; restored links and subsequent visual tuning remain
FLIR with Ironbow 0.42. An explicit switch back to another HUD layout restores
the visual preset used before Cyber; restored links and scenes remain
authoritative. The skin uses shared red/slate panel treatments in map and cockpit,
with compact 200px collapsed controls and wider expanded panels. Other HUD
layouts retain their existing presentation.
@@ -17,6 +58,21 @@ headers and decorative frames. The narrow-screen rail remains scrollable to reac
each panel. Radio retains the shared nested Context player and compact
header disclosure, without relocating playback controls on theme changes.
Voice help/error popups and Location/Visual Presets pins extend beyond their
Cyber frames without being clipped: the shapes decorate non-interactive
pseudo-elements. The lower-left telemetry card leaves room for attribution's
full logo row. New panels can use the [shared surface contract](panel-surfaces.md)
for theme tokens, rail input and a fixed header with a bounded scroll body.
The contract is intentionally future-facing and opt-in; its first production
adopter will land after this change. Existing owners still control disclosure,
persistence and placement, while small-screen rail popup clipping and other
integration limits remain documented in the contract.
On desktop, Cyber raises the left rail so it clears the lower coordinate card.
`layoutRightPanelRail` reads that rendered top as its measured `baseTop`, which
keeps the right rail aligned. Cockpit retains its separate rail placement for
its visor layout.
Sonar has one contact-highlighting method. Native Cesium points, billboards and
labels are treated in GPU draw commands without replacing their positions,
identities or pick commands. Model and canvas/DOM overlays retain their own
@@ -222,7 +278,7 @@ Consult the linked official advisory for safety decisions.
Voice and HUD snapshots reuse the existing feedState classifier. Analyst follow-ups retain their original data provenance; current-view results append provenance without replacing legacy fields. HUD context and deterministic telemetry include non-nominal feed state.
Analyst records come on demand from each layer's `getAnalystRecords`: flights, military, local ADS-B, vessels, satellites, launches, earthquakes, FIRMS (VIIRS and MODIS, with a `sensor` field), fire perimeters, cyclones, transit, bikeshare, ALPR cameras, mapped installations, datacenters and dams. `analyst_query` counts the whole loaded set (up to 250,000 records per layer; past that `coverage.records` reports `returned`, `total` and `truncated: true` and the result carries `complete: false`). Ranking keeps a top-k and scans in 20,000-row slices that yield to the page and stop when superseded. It refuses unknown fields, operators, units that do not convert (`altitudeM(ft)` converts), wrong-typed values, invalid centres, unknown scope kinds and sort fields with `code` and `allowed`, refuses a follow-up with no previous answer (`NO_RESULT_CONTEXT`) or naming other layers than that answer (`FOLLOW_UP_MISMATCH`), and tells `LAYER_OFF`, `NOT_READY` and `FEED_UNAVAILABLE` apart from a real zero; when only some layers can answer the result is `partial` with `unanswered` ids. The Contacts-window path applies the same filters, ranking and item shape to the panel cohort. A superseded query never replaces follow-up memory. Items carry `lat`/`lon`; the feed window (earthquakes and FIRMS: last 24 h), the precise scope and any caveat sit in `display`, which is on-screen detail rather than something the model recites. "In view" is still a radius around the view centre, not the visible footprint. Satellite distance is ground distance.
Satellite and local infrastructure layers expose on-demand analyst records through their current factory owners. Analyst counts and ranks explicitly cover only bounded examined loaded records (default 2,000 per new layer, core satellite rows before dense extras); omitted records can change nearest/count and satellite distance is ground distance. Existing tools and result fields remain available.
Keyless terrain tiles retry when Re:Earth throttles them. The browser fetches
`terrain.reearth.land/cesium-mesh/ellipsoid/{z}/{x}/{y}.terrain` directly; no
@@ -297,45 +353,6 @@ replacement session; audio meters release failed initialization and reject revok
frames. Delayed action results and post-capture continuations cannot resume a
stopped conversation or send output into a replacement. See [voice ownership](VOICE-OWNERSHIP.md).
Voice speech: `src/voice/speech.js` adds `{ say, display, referents?, schedule? }`
to layer toggles, fly_to_location, frame_overhead, select_nearest_aircraft,
get_entity_context and annotate_map (analyst_query plugs in through the
runner's `speechBuilders`). `get_current_view_state` lists enabled layers only.
The voice card nested in the mic control (`voiceCard.js`) shows captions, plan
steps and the result. `resultDisplay.js` is the one adapter from a result to
the card: builder results show as built, and analyst results show a title
with their scope, window and caveats. Caveats are on the card; the model speaks
one only when it changes the answer, and always says lower bounds and partial
answers. `narration.js` speaks one
code-authored line for the current step of a tool still running at 1.5 s
(out of band, never after the result). Tools that report steps: layer
toggles, place lookups, select_nearest_aircraft and annotate_map (finding,
outlining). Other tools get one "still working" line at 8 s. The
earcon plays in push-to-talk only. A claimed Space hold while the assistant
speaks is a barge-in (WebRTC cancel + audio clear), refused while Radio holds
the speaker (`mayVoiceClaimSpeaker()`). Progress lines stay outside the reply
lifecycle and Radio handoff. `turn.span` debug records carry generation and
playback latency, silence, preamble and word-count figures. Caption
transcription (`OPENAI_REALTIME_TRANSCRIBE_MODEL`, default on) is metered into
the session cost and cap. The debug log omits transcripts and text unless
`GEV_VOICE_LOG_CONTENT=1`.
Point and ask: `pointerContext.js` reads what the cursor is on. A Space hold
is the pointing gesture: its keydown snapshot (released with the key) is sent
as an inert `pointer_context` item with the target, with reticle and chip.
Open-mic speech and typed turns count only a pointer that moved on the map in
the last 5 s and send only the kind (`pointing:"aircraft"`/`"ground"`); the chip appears when a tool
resolves it. Losing the pointer sends one `target:"none"` item. `deixis.js` resolves `'pointer'` in
track_entity/fly_to_location queries, get_entity_context scope, analyst_query
`scope.kind` and annotate_map targets, plus `referent:n` from the latest
numbered result (`referents.js`, cleared at session end). A new result set (an analyst count) replaces
the numbered list even when it is empty. A reticle marks the
point during the turn and the voice card shows "THIS"/"HERE"; both hide in
Clean-UI and recording. A ground-only pointer lookup at local/city scale sends a ringed 512 px crop in
the one retained image slot, dropped if the camera moved or a newer capture or
turn started; screenX/screenY against the crop are mapped back to the canvas. Harness: `qa-voice-routing.mjs --layer deixis`,
`--layer deixis-live`, `--layer behavior --deixis-only`.
The application shell composes focused state owners for navigation, destination
lookup/orbit, Cockpit, visual settings, panel layout, aircraft display and layer
bindings. Keyboard/display subscriptions have a separate lifetime; existing
@@ -402,11 +419,9 @@ Reference feed construction is exported through `sources/reference`; the cable s
Source factories have dedicated `layers/<family>/source` exports. ALPR and earthquake record normalization and CCTV source policy no longer pull rendering into source consumers. Catalog construction and voice feed reads use their focused owners; compatibility entries remain available. Settings filesystem hardening lives under `server/standalone/`. Import-direction checks complement export ownership checks; source behavior, settings policy and rendering are unchanged.
Voice controls bind to a protocol-independent session factory. The default WebRTC adapter preserves the existing Realtime connection, push-to-talk, cost controls and radio handoff. Session subscriptions expose state, transcript, action call/progress/result, interruption, completion and turn-metrics events; stopping or replacing a session cancels pending actions. Alternate adapters can use the same controls and action runner.
Voice controls bind to a protocol-independent session factory. The default WebRTC adapter preserves the existing Realtime connection, push-to-talk, cost controls and radio handoff. Session subscriptions expose state, transcript, action call/result, interruption and completion events; stopping or replacing a session cancels pending actions. Alternate adapters can use the same controls and action runner.
Voice action argument schemas have one portable owner under `src/voice/actionSchemas.js`. The Realtime provider builds 30 tools using separate description-only metadata. Description customization cannot replace argument types, enum values or required fields.
`src/voice/layerManifest.js` declares each catalog layer's voice capabilities: spoken aliases, `toggle`, in-view `context`, `track`, and analyst `query` fields with type and unit, or `noQuery` with the reason. The `set_layer_visibility`, `show_data_layers_menu`, `get_entity_context` and `analyst_query` layer enums, the runner's alias table and the analyst field hints are generated from it. `src/voice/layerManifest.test.mjs` fails when a catalog layer is neither listed nor named in `VOICE_OFF_LAYERS` (today only the two scene-driven Bhote Koshi layers). Global Context is listed with `toggle: false`; `set_context_mode` reaches it. `set_panel_open` also opens `weather-panel` and `recent-imagery-panel`.
Voice action argument schemas have one portable owner under `src/voice/actionSchemas.js`. The Realtime provider builds 29 tools, retaining the existing 28 names and argument contracts apart from the three additive analyst-layer enum values using separate description-only metadata. Description customization cannot replace argument types, enum values or required fields.
Portable source exports provide Radio Browser station normalization, CCTV feed types and regional records independently of HTTP middleware. Radio normalization accepts an explicit URL policy; the standalone directory retains its existing HTTPS rules. Tile coordinate validation accepts explicit zoom bounds with unchanged traffic defaults. Cache, request and rendering owners remain unchanged.
@@ -449,10 +464,7 @@ clears caches and removes map-stack listeners. Standalone setup supplies the
existing service clients. HUD summaries, regional brief, cockpit weather, location
framing and annotation boundaries receive their service instances explicitly.
Compatibility entrypoints retain defaults for direct callers; normal startup does
not change global service slots. Analyst follow-up memory belongs to its voice
runner and ends with the voice session: stopping voice invalidates writes still in
flight and forgets the last answer, the pointer and the numbered list; removing
voice disposes the runner.
not change global service slots. Analyst follow-up memory belongs to its voice runner.
Application startup constructs fresh layer instances from explicit source objects.
Standalone composition selects the existing providers. Controls, launch orbit
@@ -919,10 +931,21 @@ errors.
## Places and CCTV request bounds
With a Google key configured, nearby and text search reject missing, blank,
non-numeric and out-of-range coordinates before the opt-in limiter and upstream
non-numeric and out-of-range coordinates before the per-IP limiter and upstream
request. Text search also requires a nonblank query. Keyless requests retain
their `configured: false` response.
The cost-bearing proxies are throttled per client IP without configuration:
`/api/realtime/token` and `/api/openai/hud-summary` share 30 requests per
minute per IP, `/api/google/nearby-places` and `/api/google/text-search` share
120 — the caps the Pinokio build already sets, so the packaged app is
unaffected. `GEV_RATELIMIT_OPENAI_PER_MIN` and `GEV_RATELIMIT_GOOGLE_PER_MIN` override
those; exactly `0` disables the limiter, while a value that cannot be read as a
number falls back to the default rather than to unlimited. Over-limit requests
receive a sanitized `429` with `Retry-After: 5` and never reach the provider.
The client key is the socket peer address only — a forwarded-for header is not
trusted, so a proxied deployment shares one bucket per upstream hop.
CCTV media waits at most 15 seconds for upstream response headers and returns
504 on timeout. Its timer stops when headers arrive, so live bodies can continue
streaming; a separate 30-second idle deadline then bounds the gap between
@@ -1987,11 +2010,12 @@ down | auth-failed` (plus the unchanged `missing-key`/`unsupported`) with
> registered for lifecycle and restoration but is not duplicated in Data Layers.
> Inside Cockpit, the focused summary card is titled Contact in both visible
> copy and its accessible control labels.
> The top-center Cockpit vision cycle shows the inherited map preset name
> (for example, `NOIR`) followed by CRT, NVG, FLIR, and NOIR. That inherited entry
> leaves the selected map preset unchanged inside Cockpit. NONE is not offered
> in the cycle; CRT, NVG, FLIR, and NOIR temporarily override that preset, while returning to it or exiting
> Cockpit restores the captured map style and its exact shader intensities.
> The top-center Cockpit vision cycle is the fixed, duplicate-free sequence
> Normal, CRT, NVG, FLIR, Anime, Noir, and Snow. Entry selects the map preset
> already in use, so a FLIR map opens on the existing FLIR item while
> Normal remains available exactly once. NONE is not offered. Cockpit choices
> temporarily override the map preset, while Exit Cockpit and Reset both restore
> the captured map style and its exact shader intensities.
> Selecting a Cockpit vision treatment with configurable parameters opens
> Cockpit Display and reveals those parameters through the existing right-side
> accordion; an inherited parameterless Normal preset does not force it open.
@@ -3831,8 +3855,9 @@ and unreachable upstream (502/504) separately from road geometry.
first, activates the Contacts sub-view, and returns the settled 250 km window.
Its `aircraft` count is the exact civilian-plus-military total when both feeds
can answer, or `unknown` when either component is unavailable.
- Cockpit's top vision switch cycles five rendered looks: the inherited map
style, CRT, NVG, FLIR, and Noir. There is no empty `NONE` entry.
- Cockpit's top vision switch cycles all seven map looks: Normal, CRT, NVG,
FLIR, Anime, Noir, and Snow. Entry selects the matching map style without adding
a duplicate carousel item. There is no empty `NONE` entry.
### Live AIS Vessels (June 2026)
@@ -3893,7 +3918,7 @@ and unreachable upstream (502/504) separately from road geometry.
- **Reliability**: tool-call dedupe (2.5s window across call/item/args keys); response-create queueing that respects active responses and defers follow-ups when the user starts speaking; per-tool follow-up instructions so the agent confirms only what actually happened (zoom confirms only on `ok=true`).
- **Aircraft identity honesty:** “What is this aircraft?” reads callsign, operator, registration, type, and route only from the selected contact context. Missing operator, route, or type enrichment is named explicitly rather than silently omitted or inferred from the callsign.
- **Diagnostics**: every client/server event is posted to `/api/realtime/debug-log` and appended to `.gev-logs/realtime-conversations.jsonl` (gitignored) with secret/image redaction; last 30 errors persist in `localStorage` (`gev-realtime-errors`); `window.__gevVoiceCommands.getDiagnostics()` in the console.
- **Counting semantics ("near")** — a CONTRACT; new count-bearing tool work inherits it. Three honest numbers exist for one question: the Contacts cohort (250 km around the subject, what the panel shows), `analyst_query`'s count of _currently-loaded_ records for the requested scope, and the layer-wide loaded total in `coverage.layersQueried[].total`. They diverge legitimately — the flights layer loads by viewport, so after a camera dive the loaded set can hold a fraction of the cohort (field case: panel 42, analyst 8). The contract:
- **Counting semantics ("near")** — a CONTRACT; new count-bearing tool work inherits it. Three honest numbers exist for one question: the Contacts cohort (250 km around the subject, what the panel shows), `analyst_query`'s count of _currently-loaded_ records for the requested scope, and the layer-wide loaded total in `coverage.layersQueried[].records`. They diverge legitimately — the flights layer loads by viewport, so after a camera dive the loaded set can hold a fraction of the cohort (field case: panel 42, analyst 8). The contract:
1. **Contacts ACTIVE** → "near / nearby / how many aircraft" means the **Contacts window** — the panel's numbers, spoken verbatim. Mechanism: `contactsWindow` (`{centeredOn, radiusKm, flights, military, vessels}`), carried by both `analyst_query` and `get_current_view_state`, derived by `contactsWindowFromSnapshot()` from the same snapshot the panel renders so the two cannot drift. A cohort whose feed cannot answer reports `'unknown'`, never a confident zero.
2. **Contacts OFF** → "nearby" means **in view**; "near \<place\>" means a radius around that place. A radius query with Contacts active and no explicit centre is centred on the **active contact**, not the camera.
3. **Every count names its scope in words** — "42 in your window", "8 in view", "about 30 within 250 km of Austin" — never a bare number. `analyst_query` returns `scopeLabel` so this is mechanical. Two different numbers with named scopes are not a contradiction.
@@ -4203,7 +4228,7 @@ are omitted rather than framing the wrong part of the globe.
- **Cockpit left-panel clearance:** the Cockpit Contact card and peripheral HUD participate in the adaptive left accordion's live obstacle measurements, including live viewport-height changes. Expanding Layers or Scenes keeps the active panel in the available upper-left corridor with internal scrolling; it does not cover the Contact card, lower Cockpit controls, or Cesium credit line. Outside Cockpit the hidden card does not alter the normal corridor.
- **Cockpit Context scope:** the 250 km radius applies to the air/sea proximity cohorts. Installation counts come only from the currently loaded viewport and are labeled `CURRENT VIEWPORT ONLY` in the cockpit as well as the normal Context panel; neither surface presents them as a complete 250 km installation survey.
- **Cockpit camera anchor:** first-person mode does not write feed-boundary corrections directly into the camera. A cockpit-only inertial anchor advances from the selected aircraft's displayed course and speed, then converges toward the authoritative delayed track with correction capped below forward motion. The displayed kinematics are derived from the same consecutive fix segment as the rendered position, with raw feed speed/course used only as fallback; a transient zero/missing feed speed therefore cannot freeze a visibly moving aircraft after layer enable or a map/cockpit handoff. Rendered altitude continues to come from that interpolated track position. Late ADS-B fixes and short render stalls can remove drift without accelerating or reversing the view. Camera placement runs before scene update/culling at a bounded 20 Hz so a moving cockpit does not force Photoreal 3D Tiles to retraverse on every display frame; textual instruments update at 10 Hz and context/layout work at 4 Hz. Every far Cockpit contact pip shares one stable Cesium texture-atlas entry and skips unused screen-projected course calculations, while only in-range 2D aircraft silhouettes pay the screen-projected rotation cost; ambient glTF collections are hidden/retained rather than synchronously destroyed at cockpit entry, and context rails lay out only on explicit content/state changes and viewport resize. The deliberate 15/30-second layer interpolation delays and per-Cesium-frame position caches remain unchanged.
- **Cockpit route, vision, and view controls:** visible on-screen `COCKPIT`, `RESET`, and `EXIT COCKPIT` controls replace reliance on the `C` shortcut. RESET uses the same canonical globe route as the map and voice actions, exits Cockpit, and releases its camera ownership rather than exposing the hidden map-style top action. When the tracked commercial flight has a plausible ADSBDB route, the top of the right briefing rail shows a compact `FROM → TO` airport strip and the visor shows a centered estimated-destination chevron with its relative bearing; absent or implausible route data hides the strip and cue rather than guessing. The cockpit-local vision control is an interactive `PREV / CURRENT / NEXT` carousel over the inherited map preset, `CRT`, `NVG`, `FLIR`, and `NOIR`; its previous/next actions wrap, and activating the current value advances to the next style. The inherited entry is named directly, such as `NOIR`, and retains that map shader. There is no empty `NONE` entry. CRT, NVG, FLIR, and NOIR temporarily activate the existing Cesium post-process stages, while returning to the inherited entry or exiting Cockpit restores the pre-entry visual style. The regional-news page uses a free Google News RSS locality query first, with the existing GDELT query retained only as a fail-soft fallback; linked headlines remain reporting, not verified incidents or risk intelligence.
- **Cockpit route, vision, and view controls:** visible on-screen `COCKPIT`, `RESET`, and `EXIT COCKPIT` controls replace reliance on the `C` shortcut. RESET uses the same canonical globe route as the map and voice actions, exits Cockpit, and releases its camera ownership rather than exposing the hidden map-style top action. When the tracked commercial flight has a plausible ADSBDB route, the top of the right briefing rail shows a compact `FROM → TO` airport strip and the visor shows a centered estimated-destination chevron with its relative bearing; absent or implausible route data hides the strip and cue rather than guessing. The cockpit-local vision control is an interactive `PREV / CURRENT / NEXT` carousel over `NORMAL`, `CRT`, `NVG`, `FLIR`, `ANIME`, `NOIR`, and `SNOW`; its previous/next actions wrap, and activating the current value advances to the next style. Cockpit enters on the active map preset, so a Cyber FLIR map opens Cockpit on FLIR while Normal remains a distinct option instead of producing a second FLIR entry. Cockpit choices temporarily activate the existing Cesium post-process stages, while exiting Cockpit through either EXIT or RESET restores the pre-entry map style. The regional-news page uses a free Google News RSS locality query first, with the existing GDELT query retained only as a fail-soft fallback; linked headlines remain reporting, not verified incidents or risk intelligence.
- **Cockpit weather status:** the earlier multi-canvas atmospheric compositor remains fail-closed and is not attached to the live viewer. Cockpit clouds are a separate transparent WebGL pass with a capped 520×320 framebuffer, 24 ray steps, three FBM octaves, and a 12 FPS ceiling. It defaults off and starts only when local storage explicitly contains the persisted `WX ON` opt-in (`'1'`). When opted in, observations refresh after five minutes or 25 km of aircraft movement, fail transparent when unavailable or clear, and stop on exit or disable. `WX OFF` governs atmospheric rendering only: the briefing still fetches source-backed Natural Earth region, headline, and Open-Meteo local-information data, aborting and replacing any in-flight request when the selected aircraft changes. No weather effect runs in map mode and no synthetic fallback is shown.
- **Cockpit trail visibility:** entering cockpit hides the selected aircraft's trail body and head so they cannot cross the first-person view; exit restores them. This cockpit-only presentation change does not alter the normal map-mode invariant that aircraft trails render through terrain using their depth-fail material.
- **Aircraft course slew:** civilian and military 3D models retain the 60°/s course limiter, but each rendered frame can consume at most 250 ms of accumulated slew time. A long tile/render stall therefore catches up over multiple visible frames instead of turning one delayed frame into a heading snap.
+71
View File
@@ -0,0 +1,71 @@
# MCP setup
Use God's Eye View from Claude and Codex: tools, links, and the live globe
inside the conversation. Agents: follow these steps in order and check each
one before going on.
## 1. Run the app
```sh
npm install
npm run build:panel # the in-conversation globe; rerun after app changes
npm run dev # http://localhost:5173, keep it running
```
The MCP server reads from this app.
## 2. Add the server to a client
The server is `server/mcp/stdio.js` in this checkout. Use absolute paths
(clients do not load your shell profile), with each argument as its own entry.
| Client | How |
| --- | --- |
| Claude Code | `claude mcp add gods-eye-view -- node /abs/path/gods-eye-view/server/mcp/stdio.js --api-base http://localhost:5173` |
| Codex CLI | `codex mcp add gods-eye-view -- node /abs/path/gods-eye-view/server/mcp/stdio.js --api-base http://localhost:5173` |
| Codex and ChatGPT desktop | Same `~/.codex/config.toml` as the CLI (block below), or Settings → MCP servers → Add server (STDIO). |
| Claude Desktop | Settings → Developer → Edit Config, then add the JSON below. |
```toml
[mcp_servers.gods-eye-view]
command = "node"
args = ["/abs/path/gods-eye-view/server/mcp/stdio.js", "--api-base", "http://localhost:5173"]
```
```json
{
"mcpServers": {
"gods-eye-view": {
"command": "node",
"args": ["/abs/path/gods-eye-view/server/mcp/stdio.js", "--api-base", "http://localhost:5173"]
}
}
}
```
**Windows with the checkout in WSL:** use `"command": "wsl.exe"` and put
`"-e"` and the absolute Linux path to `node` first in `args`, for example
`["-e", "/home/you/.local/share/mise/installs/node/24/bin/node", "/home/you/gods-eye-view/server/mcp/stdio.js", "--api-base", "http://localhost:5173"]`.
## 3. Restart and try it
Fully quit the client (on Windows, from the tray icon) and reopen it, then
start a new chat; clients keep the server they started with. Ask: *"Show San
Diego in God's Eye View with military flights on."*
- Claude Desktop (a chat, not the Code tab), Codex and ChatGPT desktop show
the live globe; the first load takes 10–20 seconds.
- Claude Code and the Codex CLI answer with data and a link.
## Troubleshooting
- **The server does not appear:** run the command from step 2 in a terminal;
it should print `God's Eye View MCP server reading http://localhost:5173`
and wait. A path or `args` mistake fails here.
- **The panel says it could not load:** `npm run build:panel` was not run, or
the dev server is down.
- **Logs:** the server writes each call, and why one failed, to stderr. Claude
Desktop keeps it in its logs folder as `mcp-server-gods-eye-view.log`; in
Codex, right-click the panel → DevTools for the panel's console.
See [tools and the MCP server](TOOLS.md) for what the tools do.
+302
View File
@@ -0,0 +1,302 @@
# Tools and the MCP server
Tools answer questions from God's Eye View data for language-model clients.
They are defined once and exposed through adapters: the Model Context Protocol
(MCP) and function calling, which voice uses.
To use them from Claude or Codex, see [MCP setup](MCP_SETUP.md).
## Layers
| Owner | Responsibility |
| -------------------- | ------------------------------------------------------------------------------------------------- |
| `src/tools/` | Tool definitions, catalog composition, argument validation, shared `area` and result helpers |
| `src/tools/queries/` | Queries, one file per domain, reading only portable source contracts |
| `src/tools/mcp/` | MCP protocol (JSON-RPC) and a stateless HTTP transport; knows the catalog interface, not queries |
| `src/tools/functions.js` | Function-calling adapter: tool records and results for function-calling clients |
| `src/tools/services.js` | The default services: the layers' source factories and request services, given a resolving fetch |
| `server/mcp/` | Node composition: points the sources at a running app's `/api` routes; serves stdio and `/mcp` |
| `server/standalone/voiceTools.js`, `src/standalone/toolCatalog.js` | Standalone voice composition: the session's tool list and the browser catalog |
Dependencies point downward only. `gods-eye-view/tools` and
`gods-eye-view/tools/mcp` are portable exports: they reach no application,
rendering, Node, Cesium or browser-global code, which
`npm run check:boundaries` enforces. In the application, only voice reaches
them: `withToolCatalog` in `src/voice/gevRealtime.js` imports the
function-calling adapter, and the standalone entry supplies the catalog.
## Definitions and composition
`defineTool({ name, kind, title, description, inputSchema, requires, run })`
validates and freezes a tool. `kind` is `query` (answers from data, read-only)
or `action`. `inputSchema` uses a JSON Schema subset that `src/tools/schema.js`
checks completely; unsupported keywords are rejected at definition time.
`run(args, { services, signal })` resolves to `{ summary, data }`: one sentence
for people and a structured object for programs. A tool may also return `images`,
each `{ mimeType, data }` with base64 data; the MCP adapter sends them as image
content. MCP results carry the summary and the data as JSON text, plus the data
as `structuredContent`, for clients that read only one of them.
`composeCatalog({ tools, services, replace, interceptors })` builds a catalog:
- **Tools**: an application adds its own tools to `coreTools`. Reusing a name
fails unless the name is listed in `replace`.
- **Services**: each tool names the services it reads in `requires`. Tools
whose services are not supplied are left out.
- **Interceptors**: `(call, next) => next(call)` functions wrap every call,
outermost first. They can observe, reject or change a call.
- **Composite tools**: `run` receives `tools`, with `has(name)` and
`call(name, args)`, to call other tools through the same catalog, so
replaced tools and interceptors apply. Interceptors see such calls with
`parent`, the calling tool's name.
### Surfaces
`src/tools/surfaces.js` lists which tools MCP and voice offer. A tool is on
both unless `TOOL_SURFACES` turns it off; edit an entry to turn a tool on or
off for one surface. MCP leaves out place search, routing, plain
weather and wind, the regional brief, the HUD caption, radio, bike share and
transit, which assistants already cover or which add little without the
globe. `catalogForSurface(catalog, surface, overrides)` is the
view a surface exposes: it lists and calls only the tools it offers, while
composite tools still reach the whole catalog. `toolsForSurface` gives the
same selection as a list of definitions, such as for the voice session's tool
list. Voice leaves out tools that answer with images, link to the app, or
repeat what its app actions answer.
Expected failures throw `ToolError` with one of `invalid_arguments`,
`unavailable`, `unsupported`, `malformed` or `retry_later`. Other errors are
reported to clients without details.
## Services
`gods-eye-view/tools/services` builds the default set with
`createToolServices({ fetchImpl, appUrl })`. Services are the portable source factories the layers already use, such as
`createUsgsEarthquakeSource`, `createFirmsSource` and `createLaunchSource`,
plus a `places` service with `resolve(name, { signal })`. Sources request
relative `/api/...` paths through an injected `fetchImpl`, so the same tool
code runs wherever an application routes those paths.
The `weather`, `regional`, `terrain`, `summary` and `features` services are the
application request services from `gods-eye-view/application/requests`.
`situation_brief` and `military_awareness` run each section whose services are supplied and mark the
others unavailable. `app` is `{ baseUrl }`, the address links open. The `bikeshare` service is `{ systems, getStations }`: the system registry and
the GBFS source. `createGeocodePlaceService` resolves place names through `/api/geocode`;
`createPlaceSearchService` searches `/api/google/*` and reports when no search key
is configured; `createRouteService` plans routes through `/api/route`.
## Views
`gods-eye-view/view` (`src/view/index.js`) describes what the app shows,
independent of how it is shown: a camera (lat, lon, altitude, heading,
pitch), data layers, visual style, map imagery, and an aircraft, military
aircraft or satellite to follow. `createView` builds and bounds one,
`viewToParams` and `viewFromParams` write and read it in the share-link
format the app restores, and `viewUrl` gives the address that opens it. The
style names are the ones share links use. Ships cannot be followed from a
link yet. An aircraft's `follow` may ask for its cockpit view
(`cockpit: true`); links cannot carry that and open the app following it.
A view can also carry `annotations`, the marks the app's `annotate_map`
action draws (pins, highlights, areas, arrows, routes and labels at a place
name or coordinates). Links carry them in the `an` parameter, and the app
draws them once the link has been restored.
Tools take a view as `VIEW_ARGUMENTS`: an `area` to frame from above, or a
`camera`, plus `layers`, `style`, `map`, `follow` and `annotations`. A
tilted camera over an area looks at its center from behind; camera fields
given with an area override its framing. `resolveViewArguments` turns them
into a view.
### The God's Eye View panel
`show_in_gods_eye_view` names an MCP Apps view (`io.modelcontextprotocol/ui`):
`_meta.ui.resourceUri` is `ui://gods-eye-view/globe`, a `text/html;profile=mcp-app`
resource from `createGlobePanelResource({ runtime })` in
`src/tools/globePanel.js`, with the panel's script from
`src/app/globePanelRuntime.js`. Clients that display apps render it inside
the conversation. The panel completes the MCP Apps handshake
(`ui/initialize`, `ui/notifications/initialized`,
`ui/notifications/size-changed`), and for each
`ui/notifications/tool-result` carrying a view it loads the app in inline
embed mode the first time and posts later views to that same app, so the
globe changes without reloading. Its "Open in God's Eye View" button asks
the host to open the link (`ui/open-link`).
Hosts serve panels from their own sites and may refuse other addresses;
Codex, for one, refuses any address on the user's machine. So the panel
never requests the app's server itself. It loads everything from the app's
own paths through `panel_request`, a tool meant for the panel: it is
marked `_meta.ui.visibility: ["app"]`, and each call must carry the key the
MCP server puts in its panel page. Any client can read that page, so the key
keeps the tool from clients that only list it, and is not access control. The
MCP server requests the path from the app's server and returns the response,
compressed and in parts when large.
The same path works in every host. Only
map imagery, tiles and fonts load directly, from the providers the
resource's `csp` lists. `panel_request` refuses Provider Settings
(`/api/setup`), credential and model endpoints (`/api/realtime`,
`/api/openai`), `/mcp` and the development server's internal routes, in any
letter case, encoding or dot suffix.
The panel loads the app's panel build, which `npm run build:panel` writes to
`dist/panel` and the servers serve at `/panel/`: one app script, one
stylesheet, and Cesium's script, which carries its workers and starts them
from memory (`CESIUM_WORKERS`). Files those workers load themselves are
embedded in a prelude the panel runs ahead of them, since a worker's
requests reach the panel page's own site. Rebuild it after changing
the app.
Hosts differ in ways the panel works around, all inside the panel only:
images and stylesheet files arrive as `data:` URLs, since some hosts refuse
`blob:` images; code that needs an https address for the app gets
`GEV_APP_BASE_URL`, since some hosts serve the page from their own scheme;
the globe keeps drawing from a timer when the host reports the panel hidden
and stops animation frames; and 2D canvases are kept in memory
(`willReadFrequently`), since a host that treats the panel as off screen may
drop their GPU contents and show the overlays as black over the globe.
Tools declare a UI resource with `defineTool({ ui: { resourceUri } })`, and
an app-only tool with `ui: { visibility: ['app'] }`;
`createMcpServer({ resources })` serves `resources/list` and
`resources/read`.
### Embed mode
`?embed=1` shows only the globe: clean view, with the HUD, panels, welcome
and setup prompts hidden; provider attribution stays. A page that frames it
changes the view by posting `{ type: 'gev:view', id, view }` to the frame.
The app applies it through its own actions (style, map, exactly the view's
layers, annotations, then the followed entity, retried until its layer has
it, or the camera when nothing is followed or the entity is not there yet,
since a camera flight would end the follow; then cockpit view when asked) and
answers `{ type: 'gev:view-applied', id, ok, steps }` to the origin that
sent the view. It posts `{ type: 'gev:ready' }` once it can take views, and
only its parent page can send them. See `src/app/embed.js`.
No page may frame the app by default: every document keeps
`X-Frame-Options: DENY` and `frame-ancestors 'none'`. Setting
`GEV_EMBED_FRAME_ANCESTORS` lets the pages it names (CSP frame-ancestors
sources, or `*` for any page) frame embed-mode documents only. The MCP Apps
panel loads the app into its own page and needs no framing.
Answers that have something to show include `data.view`: the view that
shows them, with the matching layers on, an area framed from above, and a
single aircraft or satellite followed, plus `url` to open it (null when the
app's address is not configured). `suggestView` in `src/tools/views.js`
builds one.
## The `area` argument
Location-scoped tools take `area` as exactly one of a `place` name, a `bbox`
(`[west, south, east, north]`, crossing the antimeridian when west exceeds
east), or `lat`, `lon` and `radius_km`. Lists default to 25 rows, at most 200,
and report `total`, `returned` and `truncated`.
## MCP
`createMcpServer({ catalog, name, version, instructions, descriptions, decorate })`
implements `initialize`, `ping`, `tools/list` and `tools/call` for protocol
revisions 2025-11-25, 2025-06-18 and 2025-03-26. `descriptions` overrides a
tool's title or description for this surface; `decorate(definition, tool)`
merges extra fields into each listed definition. `createMcpHttpHandler(server)`
returns a `Request`-to-`Response` handler for stateless Streamable HTTP: one
JSON-RPC message per POST, answered with JSON. The host owns routing and any
access control in front of it.
## Voice
Voice offers the catalog's queries next to its app actions.
`toFunctionTools(tools, { exclude })` turns tools into
`{ type: 'function', name, description, parameters }` records, and
`toFunctionOutput(name, result)` turns a result into
`{ ok, tool, summary, data }`, counting images in `images_omitted` instead of
sending them.
The voice session token endpoint takes its tool list as `realtime.tools`.
`realtimeSessionTools(additional)` appends function tools to the app actions,
skipping names an action already uses, so `next_satellite_pass` stays the
action. The standalone server supplies the core queries that
`src/tools/surfaces.js` offers on voice, which leaves out tools that answer
with images, link to the app, or repeat what voice's app actions answer
(aircraft, ships, earthquakes, fires, datacenters, dams and satellites
overhead, which `analyst_query` covers).
In the browser, `initGevVoiceCommands({ toolCatalog })` takes a function that
resolves a catalog. App action names go to the action runner; other names the
catalog has go to `catalog.call` with the call's abort signal. The standalone
entry composes the catalog with `createToolServices` over the page's fetch and
loads it the first time voice calls a query.
## Running locally
Start the app (`npm run dev` or `npm run preview`), then register the stdio
server with an MCP client, for example Claude Code:
```bash
claude mcp add gods-eye-view -- npm --prefix /path/to/gods-eye-view run --silent mcp
```
`npm run mcp -- --api-base http://localhost:4173` selects another server.
The development and preview servers also serve the same tools over HTTP at
`/mcp`, for clients that connect by URL:
```bash
claude mcp add --transport http gods-eye-view http://localhost:4173/mcp
```
The route accepts only requests from this machine that name a loopback host
on the port they reached and, when a browser sends an `Origin`, come from
that same host. It refuses requests a proxy forwarded and refuses all
requests while launcher sharing is on. This is local transport safety, not
authentication. The
local server's tools request the app's `/api` routes and the public feeds
the sources already use. The panel's `panel_request` also requests the app's
own files and data routes for the panel: it requires the key in the panel's
page, and refuses Provider Settings, credential and model endpoints, `/mcp`
and the development server's internal routes. See SECURITY.md.
## Tools
| Tool | Reads | Returns |
| --------------------- | ------------- | ------------------------------------------------------------- |
| `get_earthquakes` | `earthquakes` | USGS M2.5+ events in the last 24 hours, strongest first |
| `get_active_fires` | `fires` | NASA FIRMS detections in an area, highest radiative power first |
| `get_recent_launches` | `launches` | Launch Library 2 launches in the last 30 days, newest first |
| `aircraft_in_area` | `aircraft` | Aircraft in an area, nearest first; `military: true` reads the `military` feed |
| `find_aircraft` | `aircraft` | Aircraft anywhere by callsign, ICAO address or registration |
| `get_aircraft_track` | `aircraft` | Recent positions of one aircraft, thinned to 200 points |
| `get_aircraft_info` | `aircraft` | Aircraft type and registration, and flight route, from adsbdb |
| `vessels_in_area` | `vessels` | Ships reported by AIS in an area, nearest first, optionally by type |
| `find_vessel` | `vessels` | Ships anywhere by MMSI, IMO number or name |
| `get_vessel_track` | `vessels` | Recent positions of one ship, thinned to 200 points |
| `next_satellite_pass` | `satellites` | Next pass over a place or point (default the ISS), with naked-eye visibility |
| `satellites_overhead` | `satellites` | Satellites in a CelesTrak group above a place or point now, highest first |
| `find_cctv_cameras` | `cctv` | Public cameras in an area, nearest first |
| `get_cctv_snapshot` | `cctv` | The current image from one camera, returned as image content |
| `find_alpr_cameras` | `alpr` | OpenStreetMap-mapped license plate readers in a US/Canadian area up to 3° |
| `find_radio_stations` | `radio` | Radio Browser stations by area and/or search terms, with stream URLs |
| `search_places` | `placeSearch` | Points of interest matching a query within an area (Google Places) |
| `places_nearby` | `placeSearch` | Notable places around a place or point (Google Places) |
| `plan_route` | `routing` | Walking, driving or cycling route over OpenStreetMap, with a simplified path |
| `get_bike_share` | `bikeshare` | Live GBFS stations in an area, with bikes and docks available |
| `get_transit_vehicles` | `transit` | Live GTFS-Realtime vehicle positions in an area, optionally one route |
| `get_traffic_flow` | `traffic` | TomTom flow in a city-sized area: speed vs free flow, congested and closed road |
| `get_weather` | `weather` | Current conditions at a place or point |
| `get_weather_map` | `weatherMaps` | The latest NOAA radar, satellite or lightning map image over an area |
| `get_wind` | `wind` | GFS or IFS model wind 10 m above ground at a location |
| `get_recent_imagery` | `imagery` | The most recent clear Landsat/Sentinel-2 image of an area (VIIRS fallback) |
| `find_submarine_cables` | `cables` | TeleGeography cables and landing points by area or name (CC BY-NC-SA 3.0) |
| `find_infrastructure` | `infrastructure` | OpenStreetMap datacenters or dams in an area, nearest first (ODbL) |
| `get_bhote_koshi_flood` | `events` | The 2026 Bhote Koshi flood: evidence trail in story order, flood path and imagery dates (CC BY-NC 4.0) |
| `get_regional_brief` | `regional` | What and where a location is, its weather and recent headlines |
| `get_cyclones` | `cyclones` | Active NHC/CPHC tropical cyclones, optionally in an area |
| `get_fire_perimeters` | `perimeters` | Mapped WFIGS wildfire perimeters in an area, largest first |
| `get_terrain_height` | `terrain` | Ground, geoid and ellipsoid heights at up to 20 points |
| `find_military_installations` | `installations` | OpenStreetMap military sites in an area of at most 10° per side |
| `get_map_features` | `features` | Administrative areas, named places or monuments at a location (needs Overpass) |
| `situation_brief` | `weather` | Weather, earthquakes, fires, aircraft, ships and cyclones for an area, by section |
| `military_awareness` | `military` | Military and other aircraft, ships and military installations within 250 km of a point, by section |
| `get_hud_caption` | `weather`, `summary` | The app's heads-up display caption for an area |
| `show_in_gods_eye_view` | `app` | A view in God's Eye View: the live panel in clients with MCP Apps, and a link everywhere; takes another answer's view or an area or camera, layers, style, map, marks and something to follow |
| `panel_request` | `app` | Panel only: loads a path from the app's server for the God's Eye View panel |
+1 -7
View File
@@ -14,12 +14,6 @@ The session adapter, action runner and backend interfaces remain unchanged.
| `realtimeCost.js` | Next-session preferences and the current session's model-bound meter |
| `realtimeViewport.js` | One retained image, bounded deletion identities and capture generation |
| `realtimeDiagnostics.js` | Bounded error history and sanitized optional diagnostics |
| `turnMetrics.js` | Per-turn latency and speech spans written to the debug log |
| `narration.js` | Progress lines and the earcon while tools run; spoken through the turns owner |
| `pointerContext.js` | Pointer position, dwell pick and the current turn's pointer snapshot (session-scoped) |
| `referents.js` | Numbered items from the latest result for `referent:n` (session-scoped) |
| `resultDisplay.js` | What the voice card shows for a result and which items it numbers (read by the card and `referents.js`) |
| `gevActions.js` runner | Analyst follow-up memory (session: `resetConversation()` on stop, `dispose()` on removal) |
Pure input policy, preferences and protocol-response policy have separate modules.
Owners receive named readers, operations or focused collaborators. Cross-owner
@@ -35,7 +29,7 @@ Action and post-capture continuations retain their conversation identity before
publishing output, queueing a response or changing status.
Behavioral invariants remain in force: a 500 ms Space hold claims push-to-talk;
short control taps and text entry remain native; a claimed hold while the assistant speaks interrupts it (never during a Radio handoff); a click-started session stays
short control taps and text entry remain native; a click-started session stays
open-mic. Every accepted or refused tool call receives its terminal output once.
New typed input supersedes old intent. Radio waits for the spoken confirmation and
verified muted playback. A live cost meter stays bound to its negotiated model,
+150
View File
@@ -0,0 +1,150 @@
# Panel surface contract
`src/ui/styles/panel-surfaces.css` supplies an opt-in visual surface for new
panels. It uses the application's default color, font, border and radius tokens;
Cyber supplies its existing red/slate palette, monospace typography and shaped
frame. This is CSS shared by the existing panel system, not a panel registry or
a controller. A panel ID does not need a new theme selector.
## Markup
A rail panel uses an outer owner with its existing stable `id`, matching
`data-panel-id`, and `panel-collapsible` class. Its first non-glow child is the
surface. Keep the header and any outward popup outside the scrolling body:
```html
<section
id="example-panel"
data-panel-id="example-panel"
class="panel-collapsible"
>
<div data-panel-surface>
<header data-panel-header>
<h2 class="panel-title panel-surface-title">Example</h2>
<button
class="panel-collapse-btn panel-surface-control"
data-collapse-target="example-panel"
aria-expanded="true"
aria-label="Collapse Example"
>
−
</button>
</header>
<div data-panel-body data-rail-scroller>
<label>Filter <input class="panel-surface-control" type="text" /></label>
<button class="panel-surface-control" type="button">Apply</button>
<p class="panel-surface-error" role="status" hidden></p>
</div>
<aside data-panel-popup hidden aria-label="Example help">
<!-- The panel owns this popup's placement and interactions. -->
</aside>
</div>
</section>
```
Use native controls and accessible names. `.panel-surface-control` is an
optional skin for buttons, text fields, selects and textareas; it provides
hover, focus, disabled and invalid-state styling. Native `disabled` owns actual
disablement; `aria-disabled` alone only describes a state and must be enforced
by the controller. `.panel-surface-error` supplies error color, not announcement
or validation behavior. Keep checkbox/radio semantics and labels intact.
`data-panel-header` stays outside `data-panel-body`, so it remains reachable
when content scrolls. Give the body a bounded height through its containing
surface. The rail's allocation supplies that bound; a dock or dialog owner must
supply its own `max-height` or height. Avoid a flex body whose contents shrink to
fit when those contents should instead scroll.
## Placement and ownership
| Mount | Shared contract | Owner responsibilities |
| ------------------------------------- | --------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Direct child of `#left-panel-stack` | An owner with a direct `[data-panel-surface]` child receives relative positioning and allocated height; inner surface/body skin | Add the wrapper shown above, use the existing layout/disclosure owners, and retain left-rail focus/auto-collapse rules. |
| Direct child of `#right-context-rail` | The same hooks with right-rail allocation and intrinsic measurement | Keep `data-rail-scroller` on the body so measurement preserves its scroll position. Respect Cyber's one-expanded-panel policy. |
| Surface inside `#command-dock` | Surface, controls, popup decoration and inherited dock concealment | Supply geometry, stacking, disclosure, pin behavior if wanted, safe viewport bounds and dismissal. This CSS does not extend the existing Location/Visual Presets tray stack manager to arbitrary panels. |
| `dialog[data-panel-surface]` | The same surface/body skin; a closed dialog remains hidden | Set width/max-height, use `showModal()` or `show()` and `close()`, and own labels, initial/return focus and dismissal. Do not put `panel-collapsible` on a dialog. |
For rails, `--panel-expanded-width` and `--panel-collapsed-width` customize
the desired widths within the rail; otherwise Cyber uses its existing left,
right and collapsed panel widths. `--left-panel-allocated-height` and
`--right-panel-allocated-height` belong
to the existing layout owner, not to individual panels. The surface must be
the wrapper's first non-glow child because left-rail intrinsic measurement
inspects that child. Do not put an absolutely positioned popup before the
surface or inside its body. Position a popup beside or above its surface using
component CSS; the shared rules deliberately do not choose a direction that
could run beyond the viewport.
Rail measurement still considers visible child extents. A popup extending
below the surface can therefore increase its measured height. Prefer a popup
beside or above the surface, or integrate an external overlay host with the
feature's layout owner; this CSS does not change measurement algorithms.
At viewport widths of 720px or less, both rails are intentional scroll
containers. Their overflow can clip outward popups even when the panel's own
surface is unclipped. Use a popup contained within the rail's visible bounds,
or an explicitly owned overlay host outside the rail; the surface contract
does not provide automatic popup escape from these small-screen scroll regions.
For dock disclosure, use a semantic button with `.panel-surface-control` and
the feature's own accessible name, `aria-expanded` state and event binding.
Do not copy the rail example's `.panel-collapse-btn` class into the dock:
existing dock rules intentionally hide that class. The generic surface skin
does not replace the dock's disclosure policy.
The existing `PanelChrome` binds `.panel-collapse-btn[data-collapse-target]`
at initialization and routes state through `setPanelCollapsed`. For additions
present at startup, keep those hooks and the existing chrome lifecycle. A
dynamically mounted panel must bind and dispose its own disclosure using
`bindPanelDisclosure` / `collapsePanelOnEscape` from `panelDisclosure.js`, with
the application owning state changes. Do not rerun the whole application chrome
initializer just to add one panel. `layoutLeftPanelRail` and
`layoutRightPanelRail` accept arbitrary direct `data-panel-id` children; their
host owns layout scheduling, observations and collapse callbacks. Dynamic
content that changes size may require an owner-scheduled layout pass or an
owned `ResizeObserver`; adding CSS attributes does not register new observers.
Persisted/share-link state, pinning, provider loading, keyboard shortcuts and
Cockpit-specific mounting still require deliberate integration with their
current owners. This contract does not add entries to named persistence, dock,
Cockpit or share-state policies. Dispose listeners, observers and timers,
dismiss open popups/dialogs and restore focus before removing a dynamic panel.
## Decoration, popups and visibility
The shared surface reserves `::before` for its non-interactive background. The
Cyber frame clips this pseudo-element only; the surface stays unclipped with
visible overflow. This follows the existing Display and Global Context pattern.
Putting the shape on an interactive ancestor clips all descendant popups even
if their `z-index` is high. A scrolling body also clips descendants by design,
so outward popups must be siblings of the body, or use an explicitly owned
overlay host outside every clipping ancestor.
`data-panel-popup` gives a positioned popup the same decorative and control
tokens. It is not a popover API, focus trap, collision solver or top-layer
manager. The owner must keep it inside the viewport, above relevant siblings,
and reachable by pointer and keyboard. A native modal dialog participates in
the browser top layer; a surface in a rail does not. Transform, containment,
mask, clipping or stacking rules on an additional ancestor can still obstruct
either an ordinary panel or its popup. Arbitrary future markup is not covered.
`hidden`, a closed native dialog and collapsed bodies/popups remain hidden.
Clean UI and recording conceal conforming surfaces, including outward popups;
owners must close modal dialogs when entering these modes so an invisible
modal cannot leave the application inert. Existing Cockpit rail and dock
exclusions remain in force. New left-rail/Cockpit functionality needs an
explicit product integration; the surface attribute grants no mode exception.
Never use theme styling to alter stored preferences or force hidden content
visible.
## Check an addition
Use the real application with the default and Cyber themes. Check open,
collapsed, hidden, disabled/error, theme round trips, clean UI/recording and
the intended Cockpit behavior. Exercise keyboard focus, Escape, pointer input,
the last scrolling item, resize without reload and any pin/unpin controls.
For an outward popup, test visible pixels and pointer hit-testing outside the
surface; a positive bounding box alone does not prove it escaped clipping.
Check each mount the addition actually supports. The CSS contract cannot
establish provider correctness, disclosure lifecycle or responsive placement
without those feature-specific checks.
+33 -1202
View File
File diff suppressed because it is too large. Load diff
+9 -6
View File
@@ -1,7 +1,7 @@
{
"name": "gods-eye-view",
"private": true,
"version": "0.1.1",
"version": "0.2.1",
"description": "A real-time intelligence console for planet Earth — photorealistic 3D globe, live aircraft/ships/satellites/earthquakes/CCTV, and hands-free voice control. Runs in a browser.",
"type": "module",
"license": "MIT",
@@ -41,24 +41,23 @@
"dev:secure": "./scripts/dev-secure.sh",
"opensky:import": "./scripts/opensky-import-client.sh",
"build": "vite build",
"build:panel": "node scripts/build-panel.mjs",
"preview": "vite preview",
"test": "node scripts/run-unit-tests.mjs",
"test:track": "node scripts/track-regression.mjs",
"qa:map-source-tray": "node scripts/qa-map-source-tray.mjs",
"check:boundaries": "node scripts/check-import-directions.mjs && node scripts/check-package-boundaries.mjs",
"qa:transit": "node scripts/qa-transit.mjs"
"qa:transit": "node scripts/qa-transit.mjs",
"mcp": "node server/mcp/stdio.js"
},
"dependencies": {
"@huggingface/transformers": "4.3.0",
"@jtarrio/signals": "^0.10.0",
"@jtarrio/webrtlsdr": "^3.0.6",
"@litert-lm/core": "0.17.1",
"@mapbox/vector-tile": "^3.0.0",
"@meri-imperiumi/eccodes-wasm": "^2.48.2",
"cesium": "^1.124.0",
"egm96-universal": "^1.1.1",
"hls.js": "^1.7.3",
"kokoro-js": "1.2.1",
"mgrs": "^2.1.0",
"pbf": "^5.1.2",
"satellite.js": "^6.0.2"
@@ -253,6 +252,10 @@
"./layers/transit/standalone": "./src/data/transit.js",
"./layers/transit/source": "./src/layers/transit/source.js",
"./sources/transit-service": "./src/sources/transitService.js",
"./sources/map-features": "./src/sources/overpassFeatures.js"
"./sources/map-features": "./src/sources/overpassFeatures.js",
"./tools": "./src/tools/index.js",
"./tools/mcp": "./src/tools/mcp/index.js",
"./tools/services": "./src/tools/services.js",
"./view": "./src/view/index.js"
}
}
+2 -1
View File
@@ -18,7 +18,8 @@ For commercial use, obtain separate permission or exclude both this pack and the
derived coordinate dataset in `src/data/bhoteKoshiFloodPath.js` from the source
and build. Deleting this directory alone does not remove the compiled river
data. Excluding the Nepal scene also requires removing its registrations and
imports before building.
imports before building, including the `get_bhote_koshi_flood` tool in
`src/tools/queries/events.js`.
Sources:
+46
View File
@@ -0,0 +1,46 @@
#!/usr/bin/env node
/**
* Build the MCP Apps panel: the app under dist/panel, served at /panel/.
* See build/panel.js.
*/
import { readFile, rename, rm, writeFile } from 'node:fs/promises';
import { join } from 'node:path';
import { fileURLToPath } from 'node:url';
import { build } from 'vite';
import standaloneConfig from '../server/standalone/vite.config.js';
import {
PANEL_BASE,
PANEL_OUT_DIR,
PANEL_WORKER_FILES,
PANEL_WORKER_PRELUDE_PATH,
panelBuildConfig,
workerFilesPrelude,
} from '../build/panel.js';
const root = fileURLToPath(new URL('../', import.meta.url));
const outDir = join(root, PANEL_OUT_DIR);
const config = standaloneConfig({ command: 'build', mode: 'production' });
await build({
configFile: false,
root,
...panelBuildConfig(config),
});
// Cesium's plugin copies its files under the base path inside outDir.
const nested = join(outDir, PANEL_BASE, 'cesium');
await rm(join(outDir, 'cesium'), { recursive: true, force: true });
await rename(nested, join(outDir, 'cesium'));
await rm(join(outDir, PANEL_BASE.split('/')[1]), { recursive: true });
// Cesium's workers load these files themselves; embed them in a prelude the
// panel runs ahead of Cesium's workers script.
const files = {};
for (const name of PANEL_WORKER_FILES)
files[name] = await readFile(join(outDir, 'cesium', name), 'utf8');
await writeFile(
join(outDir, PANEL_WORKER_PRELUDE_PATH),
workerFilesPrelude(files),
);
console.log(`Panel build written to ${PANEL_OUT_DIR}`);
+5 -7
View File
@@ -24,10 +24,6 @@ const renderer = (file) =>
/^src\/layers\/[^/]+\/(?:index|rendering|snapshotRenderer|overlay|presentation|cards|controls)\.js$/.test(
file,
);
// The runtime phonemizer loader imports a downloaded file only after its
// SHA-256 matches the pinned value; no other module may import a computed URL.
const verifiedRuntimeImport = (file) =>
file === 'src/voice/local/workers/phonemizerRuntime.js';
// Two compatibility composition entries intentionally select standalone defaults.
const compatibilityEdge = (from, to) =>
(from === 'server/providers/local.js' &&
@@ -41,6 +37,10 @@ const portableExport = (key) =>
) ||
[
'./director',
'./tools',
'./tools/mcp',
'./tools/services',
'./view',
'./voice/action-schemas',
'./voice/session',
'./data/lifecycle',
@@ -88,9 +88,7 @@ export function checkImportDirections(root) {
}
let analysis;
try {
analysis = analyzeModule(readFileSync(resolved, 'utf8'), {
allowComputedImport: verifiedRuntimeImport(file),
});
analysis = analyzeModule(readFileSync(resolved, 'utf8'));
} catch (error) {
report(file, error.message);
return null;
+32 -20
View File
@@ -55,6 +55,7 @@
"server/standalone/vite.config.js",
"vite.config.js",
"src/tooling/viteBuild.test.mjs",
"src/tooling/allowedHosts.test.mjs",
"server/providers/common/http.js",
"server/providers/common/query.js",
"server/providers/aircraft/opensky.js",
@@ -76,6 +77,7 @@
"server/providers/places/routes.js",
"src/data/placeProviderPayloads.js",
"src/tooling/placeProviders.test.mjs",
"server/providers/common/same-site.js",
"server/providers/space.js",
"server/providers/space/celestrak.js",
"server/providers/space/launch-library.js",
@@ -692,7 +694,7 @@
"src/sources/protocols.test.mjs",
"src/voice/actionSchemas.js",
"src/voice/actionSchemas.test.mjs",
"src/voice/actionDescriptions.js",
"server/providers/openai/toolDescriptions.js",
"src/voice/session.js",
"src/voice/sessionCommands.js",
"src/voice/realtimeSession.js",
@@ -865,23 +867,33 @@
"scripts/qa-journey-recorder.mjs",
"scripts/qa-alpr-journey.mjs",
"scripts/qa-context-arrivals.mjs",
"build/onDeviceRuntime.js",
"src/ui/styles/local-voice.css",
"src/ui/styles/voice-engine.css",
"src/voice/voiceEngine.js",
"src/voice/voiceEngineControls.js",
"src/voice/voiceEnginePresentation.js",
"src/voice/voiceEngine.test.mjs",
"src/voice/local/naturalVoice.test.mjs",
"src/voice/local/naturalVoiceSetting.js",
"src/voice/local/workers/phonemizerRuntime.js",
"src/voice/local/localVoice.test.mjs",
"src/voice/local/localVoiceUi.test.mjs",
"src/voice/local/localRuntime.test.mjs",
"src/voice/local/localWebProvider.test.mjs",
"src/voice/local/localStreaming.test.mjs",
"src/voice/local/modelCache.test.mjs",
"src/voice/local/localRadio.test.mjs",
"src/voice/local/onDeviceRuntime.test.mjs",
"src/voice/local/localWorkflow.test.mjs"
"src/tools/catalog.test.mjs",
"src/tools/area.test.mjs",
"src/tools/queries/queries.test.mjs",
"src/tools/mcp/mcp.test.mjs",
"src/tools/localServer.test.mjs",
"src/tools/queries/aviation.test.mjs",
"src/tools/queries/space.test.mjs",
"src/tools/queries/media.test.mjs",
"src/tools/queries/places.test.mjs",
"src/layers/transit/source.test.mjs",
"src/tools/queries/mobility.test.mjs",
"src/tools/queries/environment.test.mjs",
"src/tools/queries/maritime.test.mjs",
"src/tools/queries/share.test.mjs",
"src/tools/localRoute.test.mjs",
"src/tools/queries/atmosphere.test.mjs",
"src/tools/queries/imagery.test.mjs",
"src/tools/queries/cables.test.mjs",
"src/tools/queries/surveillance.test.mjs",
"src/tools/queries/infrastructure.test.mjs",
"src/tools/queries/events.test.mjs",
"src/tools/functions.test.mjs",
"src/tools/surfaces.test.mjs",
"src/tools/places.test.mjs",
"src/view/index.test.mjs",
"src/tools/views.test.mjs",
"src/app/embed.test.mjs",
"src/tools/globePanel.test.mjs",
"src/costEndpointThrottle.test.mjs"
]
+4 -9
View File
@@ -15,13 +15,8 @@ const browserGlobals = new Set([
'getComputedStyle',
]);
/**
* Inspect literal module dependencies and browser-platform references without
* executing code. `allowComputedImport` admits dynamic imports of computed
* specifiers (left out of the graph) for the one loader that imports bytes it
* has already verified.
*/
export function analyzeModule(source, { allowComputedImport = false } = {}) {
/** Inspect literal module dependencies and browser-platform references without executing code. */
export function analyzeModule(source) {
const imports = [];
const browser = new Set();
function walk(node, parent, field) {
@@ -35,11 +30,11 @@ export function analyzeModule(source, { allowComputedImport = false } = {}) {
].includes(node.type) &&
node.source
) {
if (node.source.type === 'StringLiteral') imports.push(node.source.value);
else if (!(allowComputedImport && node.type === 'ImportExpression'))
if (node.source.type !== 'StringLiteral')
throw new Error(
'Computed module imports are not allowed in runtime code',
);
imports.push(node.source.value);
}
if (node.type === 'CallExpression' && node.callee?.name === 'require')
throw new Error('Runtime modules must use ES imports');
+174 -88
View File
@@ -10,7 +10,8 @@
"src/data/localGeojsonCore.js",
"src/data/localGeojsonLod.js",
"src/data/inputOwnership.js",
"src/data/infrastructureOverlayEntry.js"
"src/data/infrastructureOverlayEntry.js",
"src/sources/infrastructureData.js"
],
"external": ["cesium"]
},
@@ -30,7 +31,9 @@
"modules": [
"build/vite.js",
"build/application-html.js",
"build/onDeviceRuntime.js"
"build/allowedHosts.js",
"build/embed-framing.js",
"build/panel.js"
],
"external": ["vite-plugin-cesium"]
},
@@ -72,6 +75,8 @@
"server/providers/common/http.js",
"scripts/google-server-key.mjs",
"src/data/placeProviderPayloads.js",
"server/providers/common/same-site.js",
"src/localRequestGate.mjs",
"src/data/routeSteps.js",
"src/sources/httpBody.js",
"src/sources/rateLimit.js"
@@ -285,9 +290,10 @@
"src/loadingFeedback.js",
"src/data/installationFeedback.js",
"src/voice/voiceCost.js",
"server/providers/common/same-site.js",
"src/localRequestGate.mjs",
"src/voice/actionSchemas.js",
"src/voice/layerManifest.js",
"src/voice/actionDescriptions.js",
"server/providers/openai/toolDescriptions.js",
"src/sources/rateLimit.js"
],
"external": []
@@ -300,6 +306,7 @@
"server/providers/common/source-root.js",
"server/standalone/key-setup.js",
"src/keySetupCore.mjs",
"src/localRequestGate.mjs",
"server/standalone/key-setup-hardening.mjs"
],
"external": []
@@ -384,7 +391,8 @@
"src/ui/mapSource.js",
"src/ui/mapSourceControls.js",
"src/mapStackChips.js",
"src/keySetupCore.mjs"
"src/keySetupCore.mjs",
"src/localRequestGate.mjs"
],
"external": []
},
@@ -403,6 +411,7 @@
"src/loadingFeedback.js",
"src/data/installationFeedback.js",
"src/keySetupCore.mjs",
"src/localRequestGate.mjs",
"src/data/feedState.js",
"src/data/layerSnapshot.js"
],
@@ -518,10 +527,12 @@
"src/data/detectionPolicy.js",
"src/data/installationFeedback.js",
"src/data/layerState.js",
"src/data/layerStateCoordinator.js",
"src/data/layerStateTokenReservations.json",
"src/data/militaryAwarenessEngine.js",
"src/data/scenePick.js",
"src/keySetupCore.mjs",
"src/localRequestGate.mjs",
"src/loadingFeedback.js",
"src/locationStatus.js",
"src/mapStackChips.js",
@@ -654,7 +665,6 @@
"src/ui/styles/command-dock-trays.css",
"src/ui/styles/command-dock-sliding.css",
"src/ui/styles/voice-cost.css",
"src/ui/styles/voice-card.css",
"src/ui/styles/first-run.css",
"src/ui/styles/provider-settings.css",
"src/ui/styles/weather.css",
@@ -840,6 +850,7 @@
"src/data/contactMatch.js",
"src/data/iconOrientation.js",
"src/data/layerState.js",
"src/data/layerStateCoordinator.js",
"src/data/layerStateTokenReservations.json",
"src/data/modelVisualAnchor.js",
"src/data/motionModel.js",
@@ -928,6 +939,7 @@
"src/data/issPass.js",
"src/data/satelliteClass.js",
"src/data/satellitePass.js",
"src/sources/tle.js",
"src/layers/satellites/catalog.js",
"src/layers/satellites/controls.js",
"src/layers/satellites/index.js",
@@ -1085,6 +1097,7 @@
"src/layers/bikeshare/ingestion.js",
"src/layers/bikeshare/lifecycle.js",
"src/layers/bikeshare/model.js",
"src/sources/gbfsStations.js",
"src/layers/bikeshare/policy.js",
"src/layers/bikeshare/queries.js",
"src/layers/bikeshare/registry.js",
@@ -1257,6 +1270,7 @@
"src/maps/terrain.js",
"src/maps/terrainRetry.js",
"src/keySetupCore.mjs",
"src/localRequestGate.mjs",
"src/maps/imageryHost.js"
],
"external": ["cesium"]
@@ -1291,24 +1305,7 @@
"src/voice/realtimeFacade.js",
"src/voice/realtimeRadio.js",
"src/voice/realtimeTurns.js",
"src/voice/realtimeConnection.js",
"src/voice/turnMetrics.js",
"src/voice/speechLint.js",
"src/voice/voiceCard.js",
"src/voice/voiceCardPresentation.js",
"src/voice/resultDisplay.js",
"src/voice/referents.js",
"src/voice/layerManifest.js",
"src/voice/speech.js",
"src/voice/narration.js",
"src/voice/earcon.js",
"src/voice/pointerContext.js",
"src/voice/pointerReticle.js",
"src/voice/voiceEngineControls.js",
"src/voice/voiceEnginePresentation.js",
"src/voice/local/modelCatalog.js",
"src/ui/chipGroup.js",
"src/ui/uiLifetime.js"
"src/voice/realtimeConnection.js"
],
"external": []
},
@@ -1408,7 +1405,6 @@
"src/data/aisLiveVessels.js",
"src/data/alprCameras.js",
"src/data/analystEngine.js",
"src/voice/layerManifest.js",
"src/data/bikeshare.js",
"src/data/cctv.js",
"src/data/cctvCards.js",
@@ -1457,6 +1453,7 @@
"src/data/issPass.js",
"src/data/labelArbiter.js",
"src/data/layerState.js",
"src/data/layerStateCoordinator.js",
"src/data/layerStateTokenReservations.json",
"src/data/lifecycle.js",
"src/data/localGeojson.js",
@@ -1519,6 +1516,7 @@
"src/hudSummaryResponse.js",
"src/keySetup.js",
"src/keySetupCore.mjs",
"src/localRequestGate.mjs",
"src/keylessGeocoder.js",
"src/layers/aircraft/classification.js",
"src/layers/alpr/index.js",
@@ -1549,6 +1547,7 @@
"src/layers/bikeshare/ingestion.js",
"src/layers/bikeshare/lifecycle.js",
"src/layers/bikeshare/model.js",
"src/sources/gbfsStations.js",
"src/layers/bikeshare/policy.js",
"src/layers/bikeshare/queries.js",
"src/layers/bikeshare/registry.js",
@@ -1683,6 +1682,7 @@
"src/layers/satellites/labels.js",
"src/layers/satellites/lifecycle.js",
"src/layers/satellites/orbits.js",
"src/sources/tle.js",
"src/layers/satellites/records.js",
"src/layers/satellites/policy.js",
"src/layers/satellites/rendering.js",
@@ -1875,14 +1875,6 @@
"src/voice/commands.js",
"src/voice/control.js",
"src/voice/gevActions.js",
"src/voice/speech.js",
"src/voice/earcon.js",
"src/voice/narration.js",
"src/voice/turnMetrics.js",
"src/voice/speechLint.js",
"src/voice/voiceCard.js",
"src/voice/voiceCardPresentation.js",
"src/voice/resultDisplay.js",
"src/voice/gevRealtime.js",
"src/voice/layerSummary.js",
"src/voice/realtimeBackend.js",
@@ -2046,12 +2038,6 @@
"src/maps/imageryHost.js",
"src/ui/recentImagery.js",
"src/ui/imageryBoxTool.js",
"src/voice/pointerContext.js",
"src/voice/pointerReticle.js",
"src/voice/deixis.js",
"src/voice/referents.js",
"src/voice/pointerPick.js",
"src/data/pickedContext.js",
"src/layers/alpr/tileRecords.js",
"src/sources/capability.js",
"src/sources/openFreeMap.js",
@@ -2066,36 +2052,11 @@
"src/data/local_data/osm_military_names/names.json",
"src/layers/installations/namedMarkers.js",
"src/data/infrastructureOverlayEntry.js",
"src/voice/voiceEngine.js",
"src/voice/voiceEngineControls.js",
"src/voice/voiceEnginePresentation.js",
"src/voice/local/gemmaToolCalls.js",
"src/voice/local/localAudio.js",
"src/voice/local/localEngine.js",
"src/voice/local/localPrompt.js",
"src/voice/local/localVoiceControls.js",
"src/voice/local/localVoicePreferences.js",
"src/voice/local/localVoicePresentation.js",
"src/voice/local/localVoiceStore.js",
"src/voice/local/localWebSession.js",
"src/voice/local/modelCatalog.js",
"src/voice/local/replies.js",
"src/voice/local/speechText.js",
"src/voice/local/toolset.js",
"src/voice/local/workers/modelCache.js",
"src/voice/local/workers/runtimeAssets.js",
"src/voice/local/workers/phonemizerRuntime.js",
"src/voice/local/workerClient.js",
"src/voice/local/localWebProvider.js",
"src/voice/local/localWebFlag.js",
"src/voice/local/localRadio.js",
"src/voice/local/speechOutput.js",
"src/voice/local/streamTee.js",
"src/voice/local/sha256.js",
"src/voice/local/naturalVoiceSetting.js",
"src/ui/styles/local-voice.css",
"src/voice/actionDescriptions.js",
"src/voice/actionSchemas.js"
"src/sources/infrastructureData.js",
"src/tools/functions.js",
"src/voice/actionSchemas.js",
"src/view/index.js",
"src/app/embed.js"
],
"external": [
"hls.js",
@@ -2257,6 +2218,7 @@
"src/layers/bikeshare/ingestion.js",
"src/layers/bikeshare/lifecycle.js",
"src/layers/bikeshare/model.js",
"src/sources/gbfsStations.js",
"src/layers/bikeshare/policy.js",
"src/layers/bikeshare/queries.js",
"src/layers/bikeshare/registry.js",
@@ -2391,6 +2353,7 @@
"src/layers/satellites/labels.js",
"src/layers/satellites/lifecycle.js",
"src/layers/satellites/orbits.js",
"src/sources/tle.js",
"src/layers/satellites/records.js",
"src/layers/satellites/policy.js",
"src/layers/satellites/rendering.js",
@@ -2579,7 +2542,8 @@
"src/sources/requestDeadline.js",
"src/data/local_data/osm_military_names/names.json",
"src/layers/installations/namedMarkers.js",
"src/data/infrastructureOverlayEntry.js"
"src/data/infrastructureOverlayEntry.js",
"src/sources/infrastructureData.js"
],
"external": [
"hls.js",
@@ -2673,6 +2637,7 @@
"modules": [
"src/data/installationFeedback.js",
"src/data/layerState.js",
"src/data/layerStateCoordinator.js",
"src/data/layerStateTokenReservations.json",
"src/loadingFeedback.js",
"src/navigationPolicy.js",
@@ -2804,11 +2769,7 @@
},
"voice-action-schemas": {
"exports": ["./voice/action-schemas"],
"modules": [
"src/voice/actionSchemas.js",
"src/voice/layerManifest.js",
"src/hudLayoutPolicy.js"
],
"modules": ["src/voice/actionSchemas.js", "src/hudLayoutPolicy.js"],
"external": []
},
"voice-session": {
@@ -2821,19 +2782,7 @@
"modules": [
"src/voice/sessionCommands.js",
"src/voice/session.js",
"src/voice/control.js",
"src/voice/voiceCard.js",
"src/voice/voiceCardPresentation.js",
"src/voice/resultDisplay.js",
"src/voice/referents.js",
"src/voice/layerManifest.js",
"src/voice/speech.js",
"src/voice/pointerReticle.js",
"src/voice/voiceEngineControls.js",
"src/voice/voiceEnginePresentation.js",
"src/voice/local/modelCatalog.js",
"src/ui/chipGroup.js",
"src/ui/uiLifetime.js"
"src/voice/control.js"
],
"external": []
},
@@ -3156,5 +3105,142 @@
"src/layers/perimeters/source.js"
],
"external": []
},
"tools": {
"exports": ["./tools"],
"modules": [
"src/tools/index.js",
"src/tools/catalog.js",
"src/tools/schema.js",
"src/tools/area.js",
"src/tools/results.js",
"src/tools/places.js",
"src/tools/queries/hazards.js",
"src/tools/queries/space.js",
"src/tools/queries/aviation.js",
"src/tools/queries/media.js",
"src/tools/queries/places.js",
"src/tools/queries/mobility.js",
"src/tools/queries/environment.js",
"src/tools/queries/brief.js",
"src/tools/queries/maritime.js",
"src/tools/queries/share.js",
"src/data/layerState.js",
"src/data/layerStateTokenReservations.json",
"src/sources/gbfsStations.js",
"src/sources/tle.js",
"src/data/satellitePass.js",
"src/tools/queries/atmosphere.js",
"src/layers/wind/inspection.js",
"src/layers/wind/model.js",
"src/layers/wind/fields.js",
"src/tools/queries/imagery.js",
"src/tools/queries/cables.js",
"src/tools/queries/surveillance.js",
"src/tools/queries/infrastructure.js",
"src/tools/queries/events.js",
"src/data/bhoteKoshiFloodPath.js",
"src/tools/functions.js",
"src/tools/surfaces.js",
"src/layers/transit/policy.js",
"src/data/transitFeeds.js",
"src/layers/transit/movement.js",
"src/data/contactPlayback.js",
"src/data/layerSnapshot.js",
"src/data/feedState.js",
"src/loadingFeedback.js",
"src/data/installationFeedback.js",
"src/view/index.js",
"src/tools/views.js",
"src/tools/globePanel.js",
"src/tools/queries/panelRequest.js"
],
"external": ["satellite.js"]
},
"tools-mcp": {
"exports": ["./tools/mcp"],
"modules": [
"src/tools/mcp/index.js",
"src/tools/mcp/protocol.js",
"src/tools/mcp/http.js",
"src/tools/catalog.js",
"src/tools/schema.js"
],
"external": []
},
"tools-services": {
"exports": ["./tools/services"],
"modules": [
"src/tools/services.js",
"src/layers/bikeshare/registry.js",
"src/layers/bikeshare/source.js",
"src/layers/cctv/source.js",
"src/layers/cctv/sourcePolicy.js",
"src/layers/cyclones/source.js",
"src/sources/httpBody.js",
"src/layers/earthquakes/source.js",
"src/layers/earthquakes/records.js",
"src/layers/firms/source.js",
"src/layers/installations/source.js",
"src/sources/requestDeadline.js",
"src/data/militaryNames.js",
"src/sources/militaryTileGeometry.js",
"src/data/bundledJson.js",
"src/data/retryableLoad.js",
"src/data/local_data/osm_military_names/names.json",
"src/sources/openFreeMap.js",
"src/sources/vectorTiles.js",
"src/sources/phaseTiming.js",
"src/data/tomtomTiles.js",
"src/sources/capability.js",
"src/data/militaryInstallationData.js",
"src/layers/launches/source.js",
"src/layers/perimeters/source.js",
"src/layers/radio/source.js",
"src/layers/radio/policy.js",
"src/layers/satellites/source.js",
"src/layers/traffic/source.js",
"src/layers/traffic/roadModes.js",
"src/data/flowMatch.js",
"src/layers/traffic/flowSource.js",
"src/layers/traffic/flowDecode.js",
"src/sources/overpassRoads.js",
"src/layers/transit/source.js",
"src/sources/transitHistory.js",
"src/services/requests.js",
"src/sources/overpassFeatures.js",
"src/sources/featureSource.js",
"src/sources/overpassFeaturesRecords.js",
"src/sources/featureGeometry.js",
"src/sources/live/standalone.js",
"src/sources/live/contract.js",
"src/sources/live/aircraft.js",
"src/sources/live/vessels.js",
"src/tools/places.js",
"src/tools/catalog.js",
"src/tools/schema.js",
"src/layers/weather/source.js",
"src/layers/wind/source.js",
"src/sources/windBody.js",
"src/layers/recentImagery/catalog.js",
"src/layers/recentImagery/model.js",
"src/layers/submarineCables/bundledSource.js",
"src/layers/alpr/source.js",
"src/layers/alpr/policy.js",
"src/layers/alpr/records.js",
"src/layers/alpr/tileRecords.js",
"src/sources/infrastructureData.js",
"src/sources/eventPacks.js"
],
"external": ["@mapbox/vector-tile", "pbf"]
},
"view": {
"exports": ["./view"],
"modules": [
"src/view/index.js",
"src/data/layerState.js",
"src/data/layerStateTokenReservations.json"
],
"external": []
}
}
+98 -34
View File
@@ -1572,6 +1572,10 @@ try {
const cockpit = manager.cockpitView;
const current = document.getElementById('cockpit-vision-current');
const label = document.getElementById('cockpit-vision-current-label');
const settle = (ms = 900) => new Promise((resolve) => setTimeout(resolve, ms));
const intensities = () => Object.fromEntries(
Object.entries(manager.stages || {}).map(([name, stage]) => [name, stage.uniforms?.intensity]),
);
const read = () => ({
mode: current?.dataset.cockpitVision,
label: label?.textContent?.trim(),
@@ -1582,62 +1586,106 @@ try {
parametersCollapsed: document.getElementById('param-slider-panel')?.classList.contains('collapsed'),
signalCollapsed: cockpit.signalCollapsed,
focusRetained: document.activeElement === current,
activeStyle: manager.activeStyle,
intensities: intensities(),
});
const reenter = async () => {
cockpit.exit({ restoreTracking: true });
await settle();
for (let attempt = 0; attempt < 40; attempt += 1) {
if (cockpit.enter() === true) return true;
await settle(200);
}
return false;
};
manager._setCockpitDisclosure('display', false);
manager._setCockpitDisclosure('radio', false);
cockpit.setVisionMode('optical');
current.focus({ preventScroll: true });
const states = [read()];
for (let index = 0; index < 5; index += 1) {
for (let index = 0; index < 7; index += 1) {
current.click();
states.push(read());
}
const originalStyle = manager.activeStyle;
// A Noir map preset: Cockpit enters on Noir, Normal clears every shader
// without changing the map preset, and Exit Cockpit restores Noir.
cockpit.exit({ restoreTracking: true });
manager.setStyle('noir');
await new Promise((resolve) => setTimeout(resolve, 900));
await settle();
const enteredOnNoir = await reenter();
const entryNoir = read();
cockpit.setVisionMode('optical');
const inheritedNoir = read();
cockpit.setVisionMode('noir');
const explicitNoir = read();
cockpit.setVisionMode('optical');
const restoredNoir = {
...read(),
activeStyle: manager.activeStyle,
intensity: manager.stages.noir?.uniforms?.intensity,
};
const normalInsideNoir = read();
cockpit.setVisionMode('crt');
const cockpitOnlyCrt = read();
const exitedNoir = cockpit.exit({ restoreTracking: true }) === true;
await settle();
const restoredNoir = { ...read(), cockpitActive: cockpit.active };
manager.setStyle(originalStyle);
await new Promise((resolve) => setTimeout(resolve, 900));
await settle();
const reentered = await reenter();
cockpit.setVisionMode('optical');
return { states, inheritedNoir, explicitNoir, restoredNoir };
return {
states,
enteredOnNoir,
entryNoir,
normalInsideNoir,
cockpitOnlyCrt,
exitedNoir,
restoredNoir,
reentered,
};
});
const cycleModes = ['optical', 'crt', 'nvg', 'thermal', 'anime', 'noir', 'snow', 'optical'];
const cycleLabels = ['NORMAL', 'CRT', 'NVG', 'FLIR', 'ANIME', 'NOIR', 'SNOW', 'NORMAL'];
check(
'Cockpit vision control exposes inherited, CRT, NVG, FLIR, and NOIR before wrapping',
JSON.stringify(visionCycle.states.map((state) => state.mode))
=== JSON.stringify(['optical', 'crt', 'nvg', 'thermal', 'noir', 'optical'])
&& visionCycle.states[4]?.label === 'NOIR'
&& visionCycle.states[4]?.aria === 'Current cockpit vision style: Noir. Activate for next style.',
JSON.stringify(visionCycle),
'Cockpit vision control exposes Normal, CRT, NVG, FLIR, Anime, Noir and Snow before wrapping',
JSON.stringify(visionCycle.states.map((state) => state.mode)) === JSON.stringify(cycleModes)
&& JSON.stringify(visionCycle.states.map((state) => state.label)) === JSON.stringify(cycleLabels)
&& visionCycle.states[5]?.aria === 'Current cockpit vision style: Noir. Activate for next style.'
&& visionCycle.states[7]?.aria === 'Current cockpit vision style: Normal. Activate for next style.',
JSON.stringify(visionCycle.states.map(({ mode, label, aria }) => ({ mode, label, aria }))),
);
check(
'user vision changes open Display for every temporary treatment and retain selector focus',
visionCycle.states[1]?.mode === 'crt'
&& visionCycle.states.slice(1, 5).every((state) => state.displayOpen === 'true')
&& visionCycle.states.slice(1, 5).every((state) => state.radioOpen === 'false')
&& visionCycle.states.slice(1, 5).every((state) => state.parametersActive)
&& visionCycle.states.slice(1, 5).every((state) => !state.parametersCollapsed)
&& visionCycle.states.slice(1, 5).every((state) => state.signalCollapsed)
visionCycle.states.slice(1, 7).every((state) => state.displayOpen === 'true')
&& visionCycle.states.slice(1, 7).every((state) => state.radioOpen === 'false')
&& visionCycle.states.slice(1, 7).every((state) => state.parametersActive)
&& visionCycle.states.slice(1, 7).every((state) => !state.parametersCollapsed)
&& visionCycle.states.slice(1, 7).every((state) => state.signalCollapsed)
&& visionCycle.states.every((state) => state.focusRetained),
JSON.stringify(visionCycle.states),
);
check(
'inherited Noir and explicit Noir keep distinct ownership and restore the Noir map preset',
visionCycle.inheritedNoir?.mode === 'optical'
&& visionCycle.inheritedNoir?.label === 'NOIR'
&& visionCycle.explicitNoir?.mode === 'noir'
&& visionCycle.explicitNoir?.label === 'NOIR'
&& visionCycle.restoredNoir?.mode === 'optical'
'each Cockpit vision mode drives only its own shader and Normal clears them all',
visionCycle.states.slice(1, 7).every((state) => {
const target = {
crt: 'retro', nvg: 'surveillance', thermal: 'thermal', anime: 'anime', noir: 'noir', snow: 'snow',
}[state.mode];
return Object.entries(state.intensities).every(([name, value]) =>
name === target ? value > 0 : value === 0);
})
&& Object.values(visionCycle.states[7].intensities).every((value) => value === 0),
JSON.stringify(visionCycle.states.map(({ mode, intensities }) => ({ mode, intensities }))),
);
check(
'Cockpit enters on the Noir map preset, Normal clears it, and Exit Cockpit restores it',
visionCycle.enteredOnNoir
&& visionCycle.entryNoir?.mode === 'noir'
&& visionCycle.entryNoir?.label === 'NOIR'
&& visionCycle.normalInsideNoir?.mode === 'optical'
&& visionCycle.normalInsideNoir?.label === 'NORMAL'
&& visionCycle.normalInsideNoir?.activeStyle === 'noir'
&& Object.values(visionCycle.normalInsideNoir?.intensities || {}).every((value) => value === 0)
&& visionCycle.cockpitOnlyCrt?.mode === 'crt'
&& visionCycle.cockpitOnlyCrt?.activeStyle === 'noir'
&& visionCycle.exitedNoir
&& !visionCycle.restoredNoir?.cockpitActive
&& visionCycle.restoredNoir?.activeStyle === 'noir'
&& visionCycle.restoredNoir?.intensity === 1,
&& visionCycle.restoredNoir?.intensities?.noir === 1
&& visionCycle.restoredNoir?.intensities?.retro === 0
&& visionCycle.reentered,
JSON.stringify(visionCycle),
);
await page.evaluate(() => window.__godsEyeView.styleManager.cockpitView.setVisionMode('noir'));
@@ -2266,10 +2314,16 @@ try {
// Both the UI and compatibility facade delegate to this navigation owner.
const manager = window.__godsEyeView.styleManager._locationNavigation;
const awareness = window.__godsEyeView.dataManager.layers.get('military-awareness')?.module;
const styleManager = window.__godsEyeView.styleManager;
const styleBefore = styleManager.activeStyle;
// A Cockpit-only choice that Reset must undo.
styleManager.cockpitView.setVisionMode('snow');
window.__qaCockpitReset = {
calls: 0,
original: manager.resetToGlobeView,
subjectId: awareness?.getContextSnapshot?.()?.subject?.id || null,
styleBefore,
snowDuringCockpit: styleManager.stages?.snow?.uniforms?.intensity,
};
manager.resetToGlobeView = function qaCountedCockpitReset(...args) {
window.__qaCockpitReset.calls += 1;
@@ -2305,16 +2359,26 @@ try {
resetHidden: document.getElementById('cockpit-reset-globe')?.hidden,
height,
subjectPreserved: Boolean(qa.subjectId && subjectId === qa.subjectId),
styleRestored: gev.styleManager.activeStyle === qa.styleBefore,
snowDuringCockpit: qa.snowDuringCockpit,
// Cockpit-time intensities are Cockpit's own, so check against the map preset.
visionRestored: Object.entries(gev.styleManager.stages || {}).every(([name, stage]) =>
name === gev.styleManager.activeStyle
? stage.uniforms?.intensity > 0
: stage.uniforms?.intensity === 0),
};
});
check(
'keyboard Cockpit Reset uses one canonical route and preserves Contact selection',
'keyboard Cockpit Reset uses one canonical route, preserves Contact selection and restores the map style',
resetState.calls === 1
&& !resetState.cockpitActive
&& !resetState.trackedEntity
&& resetState.resetHidden
&& Math.abs(resetState.height - 18_000_000) < 150_000
&& resetState.subjectPreserved,
&& resetState.subjectPreserved
&& resetState.snowDuringCockpit > 0
&& resetState.styleRestored
&& resetState.visionRestored,
JSON.stringify(resetState),
);
check('runtime console remains clean', consoleErrors.length === 0 && localHttpErrors.length === 0,
-411
View File
@@ -1,411 +0,0 @@
#!/usr/bin/env node
/**
* qa-voice-bench.mjs — provider-neutral TEXT-mode voice-routing benchmark.
*
* Every provider gets the same prompt: production's realtimeInstructions()
* and GEV_REALTIME_TOOLS (the Realtime adapter uses the exact session body the
* token handler mints). Two suites, reported separately:
* core the qa-voice-routing phrase table, graded with its semantics
* (plus benchmark-only typed-argument checks)
* coverage the newer-layers coverage probe, graded correct /
* honest-refusal / hallucinated-capability / wrong-tool
* dialogue two-turn dialogues in one context; turn 2 depends on turn 1
* capability area handles / imagery / OSM / point-and-ask phrases (and two
* dialogues) for the tools added after 4b56d0e9
* Coverage rows are graded twice: v1 (original rubric, 4b56d0e9 tool schema)
* and v2 (rubric for the current tool surface).
*
* Methodology: a fresh context per phrase (new Realtime/Live session, or a
* stateless chat request); stub tool results; continuations until the model
* stops calling tools (max 4). No app page is involved.
*
* Usage:
* node scripts/qa-voice-bench.mjs --provider openai-realtime --model gpt-realtime-2 --reps 2
* node scripts/qa-voice-bench.mjs --provider gemini-live --model gemini-3.8-live
* node scripts/qa-voice-bench.mjs --provider ollama --model gemma4:e4b
* node scripts/qa-voice-bench.mjs --provider openai-chat --base http://localhost:1234/v1 --model <id>
* node scripts/qa-voice-bench.mjs --summarize --out qa-shots/voice-bench/<run>
* Common flags: --out <dir> (reuse one dir to collect several models),
* --suite core,coverage --only <substr> --budget <usd, cloud total>
* --variant <label> --temperature <t> --num-ctx <n> --think --note <text>
* --reps <n> --start-rep <k> (add repetitions k..k+n-1 to an existing run)
* --replace (supersede earlier runs of this model for the suites run)
* --compare <run dir> (add a before → after table against another run)
*
* Output (gitignored): <out>/results.jsonl, summary.json, summary.md.
* Keys come from the environment or the macOS Keychain at runtime only.
*/
import fs from 'node:fs';
import path from 'node:path';
import os from 'node:os';
import { execFileSync } from 'node:child_process';
import { fileURLToPath } from 'node:url';
import {
CORE_PHRASES, COVERAGE_PROBE, DIALOGUES, CAPABILITY_EXTRAS,
dialogueResult, coverageRubric,
} from './voice-bench/phrases.mjs';
import { gradeCore, gradeCoverage, gradeDialogue, expectLabel } from './voice-bench/grade.mjs';
import { stubResult } from './voice-bench/stub.mjs';
import { summarize, renderMarkdown, renderComparison } from './voice-bench/report.mjs';
import { GEV_REALTIME_TOOLS, realtimeInstructions } from './voice-bench/productionConfig.mjs';
import { chatNormalizationDiff } from './voice-bench/toolFormats.mjs';
import { createOpenAiRealtimeAdapter } from './voice-bench/adapters/openaiRealtime.mjs';
import { createGeminiLiveAdapter } from './voice-bench/adapters/geminiLive.mjs';
import { createOpenAiChatAdapter, createOllamaAdapter } from './voice-bench/adapters/localChat.mjs';
const ROOT = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '..');
const argv = process.argv.slice(2);
const opt = (flag, fallback) => {
const i = argv.indexOf(flag);
return i >= 0 && argv[i + 1] !== undefined ? argv[i + 1] : fallback;
};
const has = (flag) => argv.includes(flag);
const PROVIDER = opt('--provider', null);
const MODEL = opt('--model', null);
const CLOUD = PROVIDER === 'openai-realtime' || PROVIDER === 'gemini-live';
const REPS = Number(opt('--reps', CLOUD ? '2' : '1'));
const START_REP = Number(opt('--start-rep', '1'));
const SUITES = opt('--suite', 'core,coverage,dialogue,capability').split(',');
const ONLY = opt('--only', null);
const BUDGET = Number(opt('--budget', '30'));
const VARIANT = opt('--variant', null);
const OUT = path.resolve(ROOT, opt('--out', path.join('qa-shots', 'voice-bench', new Date().toISOString().replace(/[:.]/g, '-'))));
const LEDGER = path.join(ROOT, 'qa-shots', 'voice-bench', 'cloud-spend.json');
fs.mkdirSync(OUT, { recursive: true });
const RESULTS = path.join(OUT, 'results.jsonl');
const NOTES = path.join(OUT, 'notes.json');
function readRows() {
if (!fs.existsSync(RESULTS)) return [];
// Several runs may append to one results file (a cloud run beside a local
// one); skip a line another process is still writing.
return fs.readFileSync(RESULTS, 'utf8').split('\n').filter(Boolean).flatMap((l) => {
try { return [JSON.parse(l)]; } catch { return []; }
});
}
function appendRow(row) {
fs.appendFileSync(RESULTS, `${JSON.stringify(row)}\n`);
}
function readNotes() {
try { return JSON.parse(fs.readFileSync(NOTES, 'utf8')); } catch { return []; }
}
/**
* Re-grade stored turns with the CURRENT phrase tables and grader, so a
* grader fix applies to every model already in the results file. The raw
* calls and text in results.jsonl are the source of truth; `gradeAtRun`
* keeps the verdict recorded at run time.
*/
function regrade(rows) {
const core = new Map(CORE_PHRASES.map((p) => [p.phrase, p]));
const cov = new Map(COVERAGE_PROBE.map((c) => [c.id, c]));
const dia = new Map(DIALOGUES.map((d) => [d.id, d]));
const cap = new Map(CAPABILITY_ITEMS.map((c) => [c.id, c]));
// A run started with --replace supersedes earlier runs of the same
// provider/model/variant for the suites it covers.
const key = (r) => `${r.provider}:${r.model}#${r.variant || ''}`;
const runs = rows.filter((r) => r.type === 'run');
const superseded = new Set();
for (const r of runs.filter((x) => x.replace)) {
for (const older of runs) {
if (older.at < r.at && key(older) === key(r)) superseded.add(`${older.runId}|${(r.suites || []).join(',')}`);
}
}
const isSuperseded = (row) => [...superseded].some((s) => {
const [runId, suites] = s.split('|');
return runId === row.runId && suites.split(',').includes(row.suite);
});
return rows.filter((row) => row.type !== 'turn' || !isSuperseded(row)).map((row) => {
if (row.type !== 'turn') return row;
if (row.suite === 'dialogue' && dia.has(row.phraseId)) {
return { ...row, gradeAtRun: row.grade, grade: gradeDialogue(dia.get(row.phraseId), [row.turn1?.calls || [], row.calls || []]) };
}
if (row.suite === 'core' && core.has(row.phraseId)) {
return { ...row, gradeAtRun: row.grade, grade: gradeCore(core.get(row.phraseId), row.calls || []) };
}
if (row.suite === 'coverage' && cov.has(row.phraseId)) {
const item = cov.get(row.phraseId);
return {
...row,
gradeAtRun: row.grade,
grade: gradeCoverage(item, row.calls || [], row.text, BASELINE_TOOLS),
gradeV2: gradeCoverage(coverageRubric(item, 'v2'), row.calls || [], row.text, GEV_REALTIME_TOOLS),
};
}
if (row.suite === 'capability' && cap.has(row.phraseId)) {
const c = cap.get(row.phraseId);
const grade = c.dialogue ? gradeDialogue(c.dialogue, [row.turn1?.calls || [], row.calls || []]) : gradeCore(c.phrase, row.calls || []);
return { ...row, gradeAtRun: row.grade, grade };
}
return row;
});
}
function readRowsFrom(dir) {
const file = path.join(dir, 'results.jsonl');
if (!fs.existsSync(file)) return [];
return fs.readFileSync(file, 'utf8').split('\n').filter(Boolean).flatMap((l) => {
try { return [JSON.parse(l)]; } catch { return []; }
});
}
function writeSummary() {
const summary = summarize(regrade(readRows()));
// --compare <dir> (remembered in compare-with.txt): a before → after table
// against another run folder, both re-graded with the current grader.
const compareFile = path.join(OUT, 'compare-with.txt');
if (has('--compare')) fs.writeFileSync(compareFile, `${path.resolve(ROOT, opt('--compare', ''))}\n`);
let comparison = '';
if (fs.existsSync(compareFile)) {
const beforeDir = fs.readFileSync(compareFile, 'utf8').trim();
const before = summarize(regrade(readRowsFrom(beforeDir)));
const beforeHead = readRowsFrom(beforeDir).find((r) => r.type === 'run' && r.prompt?.gitHead)?.prompt?.gitHead || '4b56d0e9';
const afterHead = readRows().find((r) => r.type === 'run' && r.prompt?.gitHead)?.prompt?.gitHead || '?';
comparison = renderComparison(before, summary, {
beforeLabel: `${path.relative(ROOT, beforeDir)} (prompt/tools @ ${beforeHead})`,
afterLabel: `${path.relative(ROOT, OUT)} (prompt/tools @ ${afterHead})`,
});
summary.comparison = { beforeDir: path.relative(ROOT, beforeDir), before: before.models.map((m) => ({ key: m.key, coreFull: m.core.fullCallAccuracyPct, coverageV1Pct: m.coverage.acceptablePct, hcV1: m.coverage.counts['hallucinated-capability'], dialogueT2: m.dialogue.turn2FullPct, toolCallP50: m.latencyMs.toolCallP50, promptTokens: m.warmup?.promptTokens ?? m.tokens.inputPerRequestMean })) };
fs.writeFileSync(path.join(OUT, 'comparison.md'), `${comparison}\n`);
}
fs.writeFileSync(path.join(OUT, 'summary.json'), JSON.stringify(summary, null, 2));
const md = renderMarkdown(summary, { notes: readNotes(), payload: payloadChanges() });
fs.writeFileSync(path.join(OUT, 'summary.md'), comparison ? md.replace('## Routing accuracy', `${comparison}\n## Routing accuracy`) : md);
return summary;
}
/** Exactly what each provider receives differently from production's payload. */
function payloadChanges() {
const chat = chatNormalizationDiff(GEV_REALTIME_TOOLS)
.map((c) => `\`${c.path}\`: ${JSON.stringify(c.from)} → ${JSON.stringify(c.to)}`);
return [
`openai-realtime: none — the session body is captured from production's token handler (${GEV_REALTIME_TOOLS.length} tools, instructions, tool_choice auto, reasoning low, truncation). Only difference from a mic session: response.create asks for output_modalities ["text"] (as qa-voice-routing does).`,
'gemini-live: same instructions text as systemInstruction; the same tool schemas passed verbatim as functionDeclarations[].parametersJsonSchema (no keyword changes); tool_choice has no Live equivalent (AUTO is the default); responseModalities AUDIO + outputAudioTranscription (TEXT is rejected by the model).',
`ollama / openai-chat: same instructions as the system message; tools wrapped as {type:"function",function:{name,description,parameters}} with ${chat.length} schema change(s): ${chat.join('; ') || 'none'}. Descriptions, enums and required lists are unchanged; nothing is stringified.`,
];
}
function ledgerSpent() {
try { return JSON.parse(fs.readFileSync(LEDGER, 'utf8')).spentUsd || 0; } catch { return 0; }
}
function ledgerAdd(usd) {
const spent = ledgerSpent() + usd;
fs.mkdirSync(path.dirname(LEDGER), { recursive: true });
fs.writeFileSync(LEDGER, JSON.stringify({ spentUsd: spent, updatedAt: new Date().toISOString() }));
return spent;
}
const sh = (cmd, args) => {
try { return execFileSync(cmd, args, { encoding: 'utf8', stdio: ['ignore', 'pipe', 'ignore'], timeout: 10_000 }).trim(); } catch { return null; }
};
/** Snapshot of what else is using the machine (GPU-heavy servers, load). */
function machineSnapshot() {
const load = os.loadavg().map((x) => x.toFixed(2)).join(' ');
const ollamaPs = sh('ollama', ['ps']);
const procs = sh('ps', ['-Ao', 'pcpu=,rss=,comm=']) || '';
const heavy = procs.split('\n').map((l) => l.trim().split(/\s+/)).filter((p) => Number(p[0]) > 20)
.map((p) => `${path.basename(p.slice(2).join(' '))} ${p[0]}%cpu`).slice(0, 6);
const gpuServers = procs.split('\n').filter((l) => /ollama runner|llama-server|LM Studio|lms|mlx|whisper/i.test(l))
.map((l) => path.basename(l.trim().split(/\s+/).slice(2).join(' '))).slice(0, 6);
return {
at: new Date().toISOString(),
loadavg: load,
ollamaPs,
busyProcesses: heavy,
modelServers: gpuServers,
summary: `load ${load}; busy: ${heavy.join(', ') || 'none'}`,
};
}
// ── Ollama lifecycle ────────────────────────────────────────────────────────
const OLLAMA = opt('--base', 'http://localhost:11434').replace(/\/v1\/?$/, '');
async function ollamaUnloadAll() {
const ps = await fetch(`${OLLAMA}/api/ps`).then((r) => r.json()).catch(() => ({ models: [] }));
for (const m of ps.models || []) {
await fetch(`${OLLAMA}/api/generate`, { method: 'POST', body: JSON.stringify({ model: m.name, keep_alive: 0 }) }).then((r) => r.text()).catch(() => null);
}
return (ps.models || []).map((m) => m.name);
}
async function ollamaLoad(model, numCtx) {
const t = Date.now();
const res = await fetch(`${OLLAMA}/api/generate`, {
method: 'POST',
body: JSON.stringify({ model, keep_alive: '30m', options: { num_ctx: numCtx } }),
signal: AbortSignal.timeout(600_000),
});
const body = await res.json().catch(() => ({}));
if (!res.ok || body.error) throw new Error(`load failed: ${body.error || res.status}`);
return { wallMs: Date.now() - t, loadMs: body.load_duration ? body.load_duration / 1e6 : null };
}
// ── Main ────────────────────────────────────────────────────────────────────
function buildAdapter() {
const temperature = Number(opt('--temperature', '0.8'));
switch (PROVIDER) {
case 'openai-realtime':
return createOpenAiRealtimeAdapter({ model: MODEL });
case 'gemini-live':
return createGeminiLiveAdapter({ model: MODEL, schemaMode: opt('--gemini-schema', 'jsonSchema') });
case 'ollama':
return createOllamaAdapter({ model: MODEL, baseUrl: OLLAMA, temperature, numCtx: Number(opt('--num-ctx', '32768')), think: has('--think') });
case 'openai-chat':
return createOpenAiChatAdapter({ model: MODEL, baseUrl: opt('--base', 'http://localhost:1234/v1'), temperature });
default:
throw new Error(`unknown --provider ${PROVIDER}`);
}
}
const COLD_PHRASE = 'Take me to Tokyo';
/** Capability suite: single phrases (optional pointer context) and dialogues. */
const CAPABILITY_ITEMS = CAPABILITY_EXTRAS.map((c) => (c.turns
? { suite: 'capability', id: c.id, text: c.turns.join(' → '), dialogue: c }
: { suite: 'capability', id: c.context ? `${c.phrase} [pointer]` : c.phrase, text: c.phrase, phrase: c, context: c.context || null }));
/** The 4b56d0e9 tool schema, for grading coverage by the original yardstick. */
const BASELINE_TOOLS = JSON.parse(fs.readFileSync(path.join(ROOT, 'scripts', 'voice-bench', 'baseline-tools-4b56d0e9.json'), 'utf8'));
/** What prompt this run actually sent (so before/after runs are identifiable). */
function promptVersion() {
let head = null;
try { head = execFileSync('git', ['-C', ROOT, 'rev-parse', '--short', 'HEAD'], { encoding: 'utf8' }).trim(); } catch { /* not a checkout */ }
return {
gitHead: head,
tools: GEV_REALTIME_TOOLS.length,
toolJsonChars: JSON.stringify(GEV_REALTIME_TOOLS).length,
instructionsChars: realtimeInstructions().length,
};
}
function workList() {
const items = [];
if (SUITES.includes('core')) {
for (const p of CORE_PHRASES) items.push({ suite: 'core', id: p.phrase, text: p.phrase, phrase: p });
}
if (SUITES.includes('coverage')) {
for (const c of COVERAGE_PROBE) items.push({ suite: 'coverage', id: c.id, text: c.utterance, item: c });
}
if (SUITES.includes('dialogue')) {
for (const d of DIALOGUES) items.push({ suite: 'dialogue', id: d.id, text: d.turns.join(' → '), dialogue: d });
}
if (SUITES.includes('capability')) items.push(...CAPABILITY_ITEMS);
return items.filter((w) => !ONLY || w.text.toLowerCase().includes(ONLY.toLowerCase()));
}
async function main() {
if (has('--note')) {
const notes = readNotes();
notes.push(opt('--note', ''));
fs.writeFileSync(NOTES, JSON.stringify(notes, null, 2));
}
if (has('--summarize') || !PROVIDER) {
writeSummary();
console.log(`summary written to ${path.join(OUT, 'summary.md')}`);
return;
}
if (!MODEL) throw new Error('--model is required');
const adapter = buildAdapter();
await adapter.open();
let warmup = null;
if (PROVIDER === 'ollama') {
const unloaded = await ollamaUnloadAll();
const numCtx = Number(opt('--num-ctx', '32768'));
const load = await ollamaLoad(MODEL, numCtx);
// COLD turn: the first request after load pays the full-prompt prefill.
// It is recorded separately (warmup.cold*) and primes the prefix cache
// the way a live session's first turn would; every benchmark turn after
// it is steady state.
const w = await adapter.runTurn({ text: COLD_PHRASE, onCall: () => ({ ok: true }) });
warmup = { unloaded, loadWallMs: load.wallMs, loadMs: load.loadMs, coldPhrase: COLD_PHRASE, prefillTokS: w.local.prefillTokS, promptTokens: w.local.promptEvalCount, coldFirstTokenMs: w.tFirstTokenMs, coldToolCallMs: w.tFirstCallMs, coldCalls: w.calls.map((c) => c.name), warmupTurnMs: w.tDoneMs, errors: w.errors };
console.log(`warm-up: load ${Math.round(load.wallMs)}ms, prefill ${Math.round(w.local.prefillTokS || 0)} tok/s over ${w.local.promptEvalCount} tokens`);
if (w.errors.length) console.log(` warm-up errors: ${w.errors.join('; ')}`);
} else if (PROVIDER === 'openai-chat') {
const w = await adapter.runTurn({ text: COLD_PHRASE, onCall: () => ({ ok: true }) });
warmup = { loadMs: Number(opt('--load-ms', 'NaN')) || null, coldPhrase: COLD_PHRASE, prefillTokS: w.local.prefillTokS, promptTokens: w.local.promptEvalCount, coldFirstTokenMs: w.tFirstTokenMs, coldToolCallMs: w.tFirstCallMs, coldCalls: w.calls.map((c) => c.name), warmupTurnMs: w.tDoneMs, errors: w.errors };
console.log(`warm-up: ${w.tDoneMs}ms prefill≈${Math.round(w.local.prefillTokS || 0)} tok/s`);
if (w.errors.length) throw new Error(`warm-up failed, not benchmarking: ${w.errors[0]}`);
}
const runId = `${Date.now().toString(36)}`;
const machine = CLOUD ? null : machineSnapshot();
appendRow({ type: 'run', runId, at: new Date().toISOString(), provider: PROVIDER, model: MODEL, variant: VARIANT, reps: REPS, suites: SUITES, settings: adapter.settings, machine, warmup, replace: has('--replace'), prompt: promptVersion(), methodology: 'fresh context per phrase/dialogue; stub tool results (canned results for dialogue turn 1); ≤4 continuations' });
const items = workList();
let stopReason = null;
let turnCosts = [];
outer: for (let rep = START_REP; rep < START_REP + REPS; rep += 1) {
console.log(`\n== ${PROVIDER} ${MODEL}${VARIANT ? ` (${VARIANT})` : ''} — rep ${rep}/${REPS}, ${items.length} phrases`);
for (const w of items) {
if (CLOUD) {
const avg = turnCosts.length ? turnCosts.reduce((a, b) => a + b, 0) / turnCosts.length : 0.1;
if (ledgerSpent() + avg > BUDGET) { stopReason = `cloud budget $${BUDGET} reached (ledger $${ledgerSpent().toFixed(2)})`; break outer; }
}
let r;
let turn1 = null;
let grade;
if (w.dialogue) {
const rs = await adapter.runDialogue({
turns: w.dialogue.turns,
onCall: (c, turn) => dialogueResult(w.dialogue, c.name, turn) ?? stubResult('core', c, GEV_REALTIME_TOOLS),
});
const [a, b] = rs;
turn1 = { text: a.text.slice(0, 400), calls: a.calls, tFirstCallMs: a.tFirstCallMs, tDoneMs: a.tDoneMs, errors: a.errors };
r = {
...b,
errors: [...a.errors, ...b.errors],
timeout: a.timeout || b.timeout,
costUsd: a.costUsd + b.costUsd,
usage: Object.fromEntries(Object.keys(b.usage).map((k) => [k, (a.usage[k] || 0) + (b.usage[k] || 0)])),
setupMs: a.setupMs,
};
grade = gradeDialogue(w.dialogue, [a.calls, b.calls]);
} else {
r = await adapter.runTurn({ text: w.text, context: w.context || undefined, onCall: (c) => stubResult(w.suite, c, GEV_REALTIME_TOOLS) });
grade = w.suite === 'coverage' ? gradeCoverage(w.item, r.calls, r.text, GEV_REALTIME_TOOLS) : gradeCore(w.phrase, r.calls);
}
const row = {
type: 'turn', runId, provider: PROVIDER, model: MODEL, variant: VARIANT, cloud: CLOUD, rep, suite: w.suite,
phraseId: w.id, phrase: w.text,
expect: w.dialogue ? `t1 ${expectLabel(w.dialogue.first)} → t2 ${expectLabel(w.dialogue.second)}` : w.suite === 'coverage' ? w.item.expect : expectLabel(w.phrase),
context: w.context || undefined,
expectNone: Boolean(w.phrase?.expectNone), supported: w.item ? w.item.supported : undefined,
calls: r.calls, text: r.text.slice(0, 800), grade, turn1: turn1 || undefined,
tFirstTokenMs: r.tFirstTokenMs, tFirstCallMs: r.tFirstCallMs, tDoneMs: r.tDoneMs, setupMs: r.setupMs,
responses: r.responses, usage: r.usage, costUsd: r.costUsd, errors: r.errors, timeout: r.timeout,
local: r.local || undefined,
at: new Date().toISOString(),
};
appendRow(row);
if (CLOUD) { ledgerAdd(r.costUsd); turnCosts.push(r.costUsd); }
const verdict = w.suite === 'coverage' ? grade.verdict : (grade.ok ? 'PASS' : 'FAIL');
console.log(` [${verdict}] ${w.text} → ${turn1 ? `${turn1.calls.map((c) => c.name).join(',') || '(none)'} ⇒ ` : ''}${r.calls.map((c) => c.name).join(',') || '(none)'} tool@${r.tFirstCallMs ?? '—'}ms done@${r.tDoneMs}ms $${r.costUsd.toFixed(4)}${r.errors.length ? ` ERR ${r.errors[0]}` : ''}${r.timeout ? ' TIMEOUT' : ''}`);
if (r.errors.some((e) => /HTTP 4\d\d|close 100[78]|invalid|not found|model_not_found|does not exist/i.test(e)) && r.calls.length === 0 && !r.text) {
const same = readRows().filter((x) => x.type === 'turn' && x.runId === runId).slice(-3);
if (same.length === 3 && same.every((x) => x.errors?.length && !x.calls.length)) { stopReason = `3 consecutive hard errors: ${r.errors[0]}`; break outer; }
}
}
writeSummary();
}
await adapter.close();
if (PROVIDER === 'ollama' && !has('--keep-loaded')) {
await fetch(`${OLLAMA}/api/generate`, { method: 'POST', body: JSON.stringify({ model: MODEL, keep_alive: 0 }) }).catch(() => null);
}
if (stopReason) {
console.log(`STOPPED: ${stopReason}`);
appendRow({ type: 'stop', runId, provider: PROVIDER, model: MODEL, reason: stopReason, at: new Date().toISOString() });
}
const summary = writeSummary();
const mine = summary.models.find((m) => m.provider === PROVIDER && m.model === MODEL && (m.variant || null) === (VARIANT || null));
if (mine) console.log(`\n${mine.key}: core ${mine.core.accuracyPct}% | coverage acceptable ${mine.coverage.acceptablePct}% | p50 tool ${mine.latencyMs.toolCallP50}ms | $${mine.costUsd}`);
console.log(`results: ${RESULTS}\nsummary: ${path.join(OUT, 'summary.md')}\ncloud ledger: $${ledgerSpent().toFixed(3)}`);
}
main().catch((e) => {
console.error(e);
process.exitCode = 1;
});
+151 -561
View File
@@ -23,9 +23,6 @@
* node scripts/qa-voice-routing.mjs # both layers, :4415
* node scripts/qa-voice-routing.mjs --layer routing --budget 120
* node scripts/qa-voice-routing.mjs --layer behavior --url http://localhost:4173
* node scripts/qa-voice-routing.mjs --layer behavior --deixis-only --headful --shots qa-shots/point-and-ask
* node scripts/qa-voice-routing.mjs --layer deixis --budget 12 # pointer_context routing (model turns)
* node scripts/qa-voice-routing.mjs --layer deixis-live --url ... # in-app text turns (model turns)
*
* House gotchas honored: camera.cancelFlight() before every teleport; puppeteer
* suites must run sequentially with other harnesses (SwiftShader saturation);
@@ -37,10 +34,6 @@ import path from 'node:path';
import { fileURLToPath } from 'node:url';
import WebSocket from 'ws';
import puppeteer from 'puppeteer';
import { ROUTING_PHRASES } from './voice-bench/phrases.mjs';
import { gradeCore, expectLabel } from './voice-bench/grade.mjs';
import { countWords, findHedges } from '../src/voice/speechLint.js';
import { buildPointerSnapshot, pointerContextItem } from '../src/voice/pointerContext.js';
const __dirname = path.dirname(fileURLToPath(import.meta.url));
const ROOT = path.resolve(__dirname, '..');
@@ -60,13 +53,10 @@ function getOpt(flag, fallback) {
return i >= 0 && process.argv[i + 1] ? process.argv[i + 1] : fallback;
}
const APP_URL = getOpt('--url', 'http://localhost:4415');
const LAYER = getOpt('--layer', 'all'); // routing | behavior | all | deixis | deixis-live
const LAYER = getOpt('--layer', 'all'); // routing | behavior | all
const TURN_BUDGET = Number(getOpt('--budget', '120'));
const PHRASES_PER_SESSION = Number(getOpt('--batch', '6'));
const ONLY = getOpt('--only', null); // substring filter on phrase text
const DEIXIS_ONLY = process.argv.includes('--deixis-only'); // behavior: point-and-ask block only
const HEADFUL = process.argv.includes('--headful'); // real GPU, visible window
const SHOT_DIR = getOpt('--shots', null); // point-and-ask evidence screenshots
// ── Reporting ───────────────────────────────────────────────
let pass = 0, fail = 0, skip = 0;
@@ -85,10 +75,114 @@ function skipped(label, detail) {
// LAYER 1 — ROUTING
// ════════════════════════════════════════════════════════════
// Phrase table + grading live in scripts/voice-bench/ (shared with the
// provider-neutral benchmark, qa-voice-bench.mjs). See phrases.mjs for the
// expect / oneOf / expectNone / args / argsByTool semantics.
const PHRASES = ROUTING_PHRASES;
/**
* Phrase table. `expect` = tool name(s) the model must call (string, or array
* meaning "all of these", or {oneOf:[...]} for documented acceptable variance).
* `expectNone` pins conversational turns that must NOT call tools.
* `args` = spot-check subset matched against the first matching call's args
* (substring match for strings, exact for booleans/numbers).
*/
const PHRASES = [
// — navigation & framing —
{ phrase: 'Take me to Tokyo', expect: 'fly_to_location' },
{ phrase: 'Fly to the Golden Gate Bridge', expect: 'fly_to_location' },
{ phrase: 'Go to Sixth Street in Austin', expect: 'fly_to_location' },
{ phrase: 'Show me the Alps from above', expect: { oneOf: ['fly_to_location', 'frame_overhead'] } },
{ phrase: 'Zoom in a bit', expect: 'adjust_camera_zoom' },
{ phrase: 'Zoom out a little', expect: 'adjust_camera_zoom' },
{ phrase: 'Zoom out to a globe view', expect: 'zoom_to_globe' },
{ phrase: 'Show me the whole earth', expect: 'zoom_to_globe' },
{ phrase: 'Frame the aircraft near us from overhead', expect: 'frame_overhead' },
// — the satellites trap: data layer, never basemap —
{ phrase: 'Show me the satellites', expect: { oneOf: ['set_layer_visibility', 'frame_overhead'] } },
{ phrase: 'Turn off the satellites', expect: 'set_layer_visibility', args: { layerId: 'satellites' } },
{ phrase: 'Switch to Bing aerial', expect: 'set_map_stack' },
{ phrase: 'Switch the basemap to OSM', expect: 'set_map_stack' },
// — layers —
{ phrase: 'Turn on the flights layer', expect: 'set_layer_visibility', args: { layerId: 'flights' } },
{ phrase: 'Show me live vessels', expect: 'set_layer_visibility' },
{ phrase: 'Turn on the fires layer', expect: 'set_layer_visibility' },
{ phrase: 'Turn on street traffic', expect: 'set_layer_visibility', args: { layerId: 'traffic' } },
{ phrase: 'Open the data layers menu', expect: 'show_data_layers_menu' },
{ phrase: 'Show me the datacenter layers', expect: 'show_data_layers_menu' },
{ phrase: 'Turn on the datacenters layer', expect: 'set_layer_visibility' },
// — visual styles & post-fx —
{ phrase: 'Give me night vision', expect: 'set_visual_style' },
{ phrase: 'Switch to thermal view', expect: 'set_visual_style' },
{ phrase: 'Back to the normal look', expect: 'set_visual_style' },
{ phrase: 'Turn on bloom', expect: 'set_post_processing' },
{ phrase: 'Sharpen the image a touch', expect: 'set_post_processing' },
// — HUD / detection / panels —
{ phrase: 'Turn the HUD off', expect: 'set_hud' },
{ phrase: 'Switch to the tactical layout', expect: 'set_hud' },
{ phrase: 'Turn on detection', expect: 'set_detection' },
{ phrase: 'Set detection density to fifty percent', expect: 'set_detection' },
// — context questions —
{ phrase: 'What am I looking at right now?', expect: 'get_entity_context' },
{ phrase: 'What city is this below us?', expect: 'get_entity_context' },
{ phrase: 'Is there anything interesting in view?', expect: 'get_entity_context' },
// — tracking —
// Context-free text turns may reasonably look before tracking; either
// routing is correct (production sessions always carry screen context).
{ phrase: 'Track that plane', expect: { oneOf: ['track_entity', 'get_entity_context'] } },
{ phrase: 'Follow the nearest aircraft', expect: { oneOf: ['track_entity', 'get_entity_context'] } },
{ phrase: 'Stop tracking', expect: 'stop_tracking' },
// — CCTV / scenes / ISS —
{ phrase: 'Show me the nearest traffic camera', expect: 'control_cctv' },
{ phrase: 'Turn on the camera viewsheds', expect: 'control_cctv' },
{ phrase: 'Play a news radio station near Austin', expect: 'control_radio', args: { action: 'select', category: 'news', locationId: 'austin' } },
{ phrase: 'Turn on the radio', expect: 'control_radio', args: { action: 'play' } },
{ phrase: 'Set the radio volume to thirty percent', expect: 'control_radio', args: { action: 'volume', volumePct: 30 } },
{ phrase: 'Pause the radio', expect: 'control_radio', args: { action: 'pause' } },
{ phrase: 'Stop the radio', expect: 'control_radio', args: { action: 'stop' } },
{ phrase: 'When does the ISS pass over next?', expect: 'next_iss_pass' },
// — annotations —
{ phrase: 'Annotate the Texas State Capitol and its grounds', expect: 'annotate_map' },
{ phrase: 'Outline the state of Texas', expect: 'annotate_map' },
{ phrase: 'Outline Lady Bird Lake', expect: 'annotate_map' },
{ phrase: 'Draw the walking route from the Capitol to Zilker Park', expect: 'annotate_map' },
{ phrase: 'How far is the Eiffel Tower from the Louvre?', expect: 'annotate_map' },
{ phrase: 'Clear the map', expect: 'clear_annotations' },
// — multi-intent (assert ALL tools fire before speech) —
{
phrase: 'Switch to night vision and turn on the flights layer',
expect: ['set_visual_style', 'set_layer_visibility'],
},
{
phrase: 'Turn off the HUD and take me to Paris',
expect: ['set_hud', 'fly_to_location'],
},
{
phrase: 'Go to full planet view and then turn on the radio',
expect: ['zoom_to_globe', 'control_radio'],
argsByTool: { control_radio: { action: 'play' } },
},
// — camera verbs (tools #23/#24) —
{ phrase: 'Orbit around this area slowly', expect: 'move_camera' },
{ phrase: 'Pan left a bit', expect: 'move_camera' },
{ phrase: 'Stop moving the camera', expect: 'move_camera' },
{ phrase: 'Fly the route we just drew', expect: 'fly_route' },
// — analyst queries (tool #22) —
{ phrase: 'How many flights are over Texas right now?', expect: 'analyst_query' },
{ phrase: 'Which ships are headed to Oakland?', expect: 'analyst_query' },
{ phrase: 'What is the biggest fire near Los Angeles?', expect: 'analyst_query' },
{ phrase: 'Is anything flying above forty thousand feet?', expect: 'analyst_query' },
// — negative controls: conversation must NOT tool-call —
{ phrase: 'How is your evening going?', expectNone: true },
{ phrase: 'Tell me a fun fact about maps', expectNone: true },
];
/** Mint a session via the app's own endpoint. */
async function mintSession() {
@@ -127,14 +221,7 @@ class RoutingSession {
try { msg = JSON.parse(raw.toString()); } catch { return; }
this.evidence.write(`${JSON.stringify({ at: new Date().toISOString(), dir: 'recv', type: msg.type, item: msg.item?.type, name: msg.item?.name ?? msg.name, error: msg.error?.message })}\n`);
if (!this.pending) return;
if (msg.type === 'response.output_item.done' && msg.item?.type === 'message') {
// Speech stats only: text said before a call in the same response is a
// preamble; text after the last call is the spoken answer.
const text = (msg.item.content || []).map((part) => part?.text ?? part?.transcript ?? '').join(' ').trim();
if (text) this.pending.messages.push({ text, response: this.pending.continuations, callsBefore: this.pending.calls.length });
}
if (msg.type === 'response.output_item.done' && msg.item?.type === 'function_call') {
this.pending.callResponses.push(this.pending.continuations);
let parsed = {};
try { parsed = JSON.parse(msg.item.arguments || '{}'); } catch { /* keep {} */ }
this.pending.calls.push({ name: msg.item.name, args: parsed });
@@ -147,7 +234,7 @@ class RoutingSession {
item: {
type: 'function_call_output',
call_id: msg.item.call_id,
output: JSON.stringify(this.stubResult || { ok: true, note: 'qa-harness stub result' }),
output: JSON.stringify({ ok: true, note: 'qa-harness stub result' }),
},
});
this.pending.needContinuation = true;
@@ -162,7 +249,6 @@ class RoutingSession {
const p = this.pending;
this.pending = null;
p.resolve(p.calls);
this.lastTurn = p;
}
if (msg.type === 'error') {
// Log-and-continue: a per-turn API hiccup should surface as that turn's
@@ -176,18 +262,10 @@ class RoutingSession {
this.ws.send(JSON.stringify(obj));
}
/** An inert system item (the app's notifyMapEvent shape), e.g. pointer_context. */
sendSystemItem(payload) {
this.send({
type: 'conversation.item.create',
item: { type: 'message', role: 'system', content: [{ type: 'input_text', text: JSON.stringify(payload) }] },
});
}
/** Send one user text turn; resolve with the list of tool calls it produced. */
runTurn(text) {
return new Promise((resolve) => {
this.pending = { calls: [], messages: [], callResponses: [], continuations: 0, needContinuation: false, errors: [], resolve };
this.pending = { calls: [], continuations: 0, needContinuation: false, errors: [], resolve };
const timer = setTimeout(() => {
if (this.pending) { const p = this.pending; this.pending = null; p.resolve(p.calls); }
}, 45000);
@@ -204,6 +282,14 @@ class RoutingSession {
close() { try { this.ws?.close(); } catch { /* noop */ } }
}
function matchArgs(expected, actual) {
return Object.entries(expected).every(([k, v]) => {
const got = actual?.[k];
if (typeof v === 'string') return String(got ?? '').toLowerCase().includes(v.toLowerCase());
return got === v;
});
}
async function runRoutingLayer() {
console.log(`\nLAYER 1 — routing assertions (budget ${TURN_BUDGET} model turns)`);
const logDir = path.join(ROOT, '.gev-logs', 'qa-voice-routing');
@@ -238,505 +324,44 @@ async function runRoutingLayer() {
report(false, `route: "${p.phrase}"`, `turn error: ${e.message}`);
continue;
}
recordSpeech(p.phrase, calls.map((c) => c.name), session.lastTurn);
const grade = gradeCore(p, calls);
report(grade.ok, `route: "${p.phrase}" → ${expectLabel(p)}`, grade.detail);
const names = calls.map((c) => c.name);
if (p.expectNone) {
report(names.length === 0, `route: "${p.phrase}" → (no tool)`, names.length ? `unexpected calls: ${names.join(',')}` : 'clean conversational turn');
continue;
}
let ok; let detail = `called: ${names.join(',') || '(none)'}`;
if (Array.isArray(p.expect)) {
ok = p.expect.every((n) => names.includes(n));
} else if (p.expect && typeof p.expect === 'object' && p.expect.oneOf) {
ok = names.some((n) => p.expect.oneOf.includes(n));
} else {
ok = names.includes(p.expect);
}
if (ok && p.args) {
const call = calls.find((c) => (Array.isArray(p.expect) ? true : c.name === p.expect));
if (!matchArgs(p.args, call?.args)) { ok = false; detail += ` args mismatch: ${JSON.stringify(call?.args)}`; }
}
if (ok && p.argsByTool) {
for (const [toolName, expectedArgs] of Object.entries(p.argsByTool)) {
const call = calls.find((candidate) => candidate.name === toolName);
if (!call || !matchArgs(expectedArgs, call.args)) {
ok = false;
detail += ` ${toolName} args mismatch: ${JSON.stringify(call?.args)}`;
}
}
}
report(ok, `route: "${p.phrase}" → ${Array.isArray(p.expect) ? p.expect.join('+') : (p.expect.oneOf ? p.expect.oneOf.join('|') : p.expect)}`, detail);
}
session.close();
}
evidence.end();
console.log(` routing turns used: ${turnsUsed}/${TURN_BUDGET}`);
summarizeSpeech(logDir);
}
// ════════════════════════════════════════════════════════════
// LAYER 1b — DEIXIS ROUTING (costs model turns)
// ════════════════════════════════════════════════════════════
//
// Point-and-ask phrases against the production instructions and tools. Each
// scenario runs in its own fresh session: a pointer_context item built by the
// app's own formatter lands first (exactly what the client sends at speech
// start), then the user's words. Asserts the model passes the 'pointer'
// sentinel (or referent:n) instead of guessing coordinates or names.
const AIRCRAFT_POINTER = pointerContextItem(buildPointerSnapshot({
at: 'speech_start', x: 912, y: 388, width: 1500, height: 950,
pick: {
entity: { layerId: 'flights', id: 'a4f2c1', label: 'UPS793', kind: 'aircraft', lat: 30.3121, lon: -97.6013 },
ground: { lat: 30.3, lon: -97.6 },
},
}));
const GROUND_POINTER = pointerContextItem(buildPointerSnapshot({
at: 'speech_start', x: 640, y: 520, width: 1500, height: 950,
pick: { entity: null, ground: { lat: 30.2849, lon: -97.7341, heightM: 160 } },
}));
const FRAMED_STUB = {
ok: true,
action: 'frame_overhead',
layerId: 'flights',
count: 3,
say: 'Framed 3 aircraft within 150 km.',
display: { title: '3 aircraft in frame' },
referents: [
{ n: 1, id: 'a11111', label: 'SWA1445', layerId: 'flights' },
{ n: 2, id: 'a22222', label: 'UPS793', layerId: 'flights' },
{ n: 3, id: 'a33333', label: 'AAL2231', layerId: 'flights' },
],
};
const pointerArg = (value) => String(value || '').toLowerCase() === 'pointer';
const DEIXIS_SCENARIOS = [
{ pointer: AIRCRAFT_POINTER, phrase: 'Tell me about this', check: (c) => c.name === 'get_entity_context' && pointerArg(c.args.scope), want: 'get_entity_context{scope:pointer}' },
{ pointer: AIRCRAFT_POINTER, phrase: "What's that plane?", check: (c) => c.name === 'get_entity_context' && pointerArg(c.args.scope), want: 'get_entity_context{scope:pointer}' },
{ pointer: AIRCRAFT_POINTER, phrase: 'Track that one', check: (c) => c.name === 'track_entity' && (pointerArg(c.args.query) || /a4f2c1|ups793/i.test(c.args.query || '')), want: 'track_entity{query:pointer}' },
{ pointer: GROUND_POINTER, phrase: 'How many flights are around here?', check: (c) => c.name === 'analyst_query' && pointerArg(c.args.scope?.kind), want: 'analyst_query{scope.kind:pointer}' },
{ pointer: GROUND_POINTER, phrase: 'Take me there', check: (c) => c.name === 'fly_to_location' && pointerArg(c.args.query), want: 'fly_to_location{query:pointer}' },
{ pointer: GROUND_POINTER, phrase: 'Drop a pin right here', check: (c) => c.name === 'annotate_map' && (c.args.annotations || []).some((a) => pointerArg(a.target)), want: 'annotate_map{target:pointer}' },
{ pointer: GROUND_POINTER, phrase: 'What is this place?', check: (c) => c.name === 'get_entity_context' && pointerArg(c.args.scope), want: 'get_entity_context{scope:pointer}' },
// Open mic / typed turns: the model only learns that something is pointed at.
{ pointer: { type: 'pointer_context', at: 'speech_start', pointing: 'aircraft' }, phrase: "What's that?", check: (c) => c.name === 'get_entity_context' && pointerArg(c.args.scope), want: 'get_entity_context{scope:pointer} (pointing:aircraft)' },
{ pointer: { type: 'pointer_context', at: 'speech_start', pointing: 'ground' }, phrase: 'How many flights are around here?', check: (c) => c.name === 'analyst_query' && pointerArg(c.args.scope?.kind), want: 'analyst_query{scope.kind:pointer} (pointing:ground)' },
{ pointer: { type: 'pointer_context', at: 'speech_start', pointing: 'aircraft' }, phrase: 'Follow that one', check: (c) => c.name === 'track_entity' && pointerArg(c.args.query), want: 'track_entity{query:pointer} (pointing:aircraft)' },
// A named place wins over the pointer.
{ pointer: GROUND_POINTER, phrase: 'How many flights are over Texas right now?', check: (c) => c.name === 'analyst_query' && !pointerArg(c.args.scope?.kind), want: 'analyst_query not pointer (named place)' },
// No pointer_context: the model may still pass the sentinel (observed); the
// runner then falls back to scope auto, so only the tool choice is pinned.
{ pointer: null, phrase: 'Tell me about this', check: (c) => c.name === 'get_entity_context', want: 'get_entity_context (runner falls back to auto)' },
// Referents: a numbered list, then "the second one".
{ setup: { phrase: 'Frame the aircraft near us', stub: FRAMED_STUB }, pointer: null, phrase: 'Track the second one', check: (c) => c.name === 'track_entity' && (c.args.referent === 2 || /ups793|a22222/i.test(c.args.query || '')), want: 'track_entity{referent:2}' },
];
async function runDeixisRoutingLayer() {
console.log(`\nLAYER 1b — deixis routing (budget ${TURN_BUDGET} model turns)`);
const logDir = path.join(ROOT, '.gev-logs', 'qa-voice-routing');
fs.mkdirSync(logDir, { recursive: true });
const evidence = fs.createWriteStream(path.join(logDir, `deixis-${Date.now()}.jsonl`));
let turnsUsed = 0;
let referentUse = 0;
for (const scenario of DEIXIS_SCENARIOS) {
const where = !scenario.pointer ? 'no pointer' : scenario.pointer.pointing ? `pointing ${scenario.pointer.pointing}` : scenario.pointer.target ? scenario.pointer.target.label : 'ground';
const label = `deixis route: [${where}] "${scenario.phrase}" → ${scenario.want}`;
const cost = scenario.setup ? 2 : 1;
if (turnsUsed + cost > TURN_BUDGET) { skipped(label, 'turn budget exhausted'); continue; }
let session;
try {
const { value, model } = await mintSession();
session = new RoutingSession(value, model, evidence);
await session.connect();
} catch (e) {
report(false, label, `session setup failed: ${e.message}`);
continue;
}
try {
if (scenario.setup) {
turnsUsed += 1;
session.stubResult = scenario.setup.stub;
await session.runTurn(scenario.setup.phrase);
session.stubResult = null;
}
if (scenario.pointer) session.sendSystemItem(scenario.pointer);
turnsUsed += 1;
const calls = await session.runTurn(scenario.phrase);
const hit = calls.find(scenario.check);
if (hit?.args?.referent !== undefined) referentUse += 1;
report(Boolean(hit), label, `called: ${calls.map((c) => `${c.name}${JSON.stringify(c.args)}`).join(' ').slice(0, 220) || '(none)'}`);
} catch (e) {
report(false, label, `turn error: ${e.message}`);
} finally {
session.close();
}
}
evidence.end();
console.log(` deixis routing turns used: ${turnsUsed}/${TURN_BUDGET}; referent:n used ${referentUse}x`);
}
// ════════════════════════════════════════════════════════════
// LAYER 1c — DEIXIS LIVE (in-app text turns; costs model turns)
// ════════════════════════════════════════════════════════════
//
// The whole loop in the real app: a synthetic pointer on the live canvas, a
// real Realtime session (push-to-talk mode, so the fake microphone stays
// muted), a typed turn, the model's tool call, the runner's resolution and
// the tool result — observed through the session's own events.
async function runDeixisLiveLayer() {
console.log('\nLAYER 1c — deixis live (in-app text turns)');
const browser = await puppeteer.launch({
headless: HEADFUL ? false : 'new',
...(CHROME_EXECUTABLE ? { executablePath: CHROME_EXECUTABLE } : {}),
args: [
'--no-sandbox',
'--disable-setuid-sandbox',
...(HEADFUL ? [] : ['--use-gl=angle', '--use-angle=swiftshader']),
'--disable-dev-shm-usage',
'--disable-background-timer-throttling',
'--disable-renderer-backgrounding',
'--use-fake-device-for-media-stream',
'--use-fake-ui-for-media-stream',
'--autoplay-policy=no-user-gesture-required',
'--window-size=1500,950',
],
protocolTimeout: 180000,
});
const appUrl = new URL(APP_URL);
appUrl.searchParams.set('welcome', '0');
await browser.defaultBrowserContext().overridePermissions(appUrl.origin, ['microphone']);
const page = await browser.newPage();
await page.setViewport({ width: 1500, height: 950 });
const settle = (ms) => new Promise((r) => setTimeout(r, ms));
const run = (name, args) => page.evaluate(
(n, a) => Promise.resolve(window.__gevVoiceCommands.runner(n, a)).catch((e) => ({ ok: false, error: String(e?.message || e) })),
name, args,
);
let turnsUsed = 0;
try {
await page.goto(appUrl.href, { waitUntil: 'domcontentloaded', timeout: 60000 });
await page.waitForFunction(() => window.__godsEyeView?.viewer && window.__gevVoiceCommands?.session, { timeout: 120000, polling: 250 });
await page.evaluate(() => window.__godsEyeView.viewer.camera.cancelFlight());
await run('fly_to_location', { query: 'Austin, Texas' });
await settle(9000);
await run('set_layer_visibility', { layerId: 'flights', enabled: true });
await settle(40000);
await run('frame_overhead', { target: 'flights' });
await settle(7000);
await page.evaluate(() => {
window.__qaEvents = [];
window.__gevVoiceCommands.session.subscribe((event) => {
window.__qaEvents.push({
type: event.type,
name: event.name || null,
state: event.state || null,
args: event.arguments || null,
result: event.result
? { ok: event.result.ok, error: event.result.error || null, say: event.result.say ?? null, resolvedFrom: event.result.resolvedFrom || null, selectedId: event.result.selected?.id || null, scopeLabel: event.result.scopeLabel || null, label: event.result.label || null }
: null,
text: event.type === 'transcript' && event.final ? event.text : null,
role: event.role || null,
});
});
return window.__gevVoiceCommands.session.start({ pushToTalk: true });
});
await page.waitForFunction(() => window.__gevVoiceCommands.status === 'listening', { timeout: 45000, polling: 250 });
// The pointer moves and the text is sent in one step, as when someone
// points and types: typed turns count only a pointer moved in the last
// few seconds.
const turn = async (text, point = null) => {
turnsUsed += 1;
const from = await page.evaluate(() => window.__qaEvents.length);
await page.evaluate((t, p) => {
let at = p;
if (p?.aircraftId) {
// Aircraft move: project the contact's live position in this step.
const { viewer, dataManager } = window.__godsEyeView;
// The billboard position, which is what the pick hits.
const found = (dataManager.layers.get('flights')?.module?.getAllPositions?.(3000) || []).find((e) => e.id === p.aircraftId);
viewer.scene.render();
at = found?.position ? viewer.scene.cartesianToCanvasCoordinates(found.position) : null;
}
if (at) window.__gevVoiceCommands.pointer.moveTo(at.x, at.y);
window.__gevVoiceCommands.sendTextCommand(t);
}, text, point);
// Done when a reply completes after the last tool result (or no tool).
const deadline = Date.now() + 90000;
let events = [];
while (Date.now() < deadline) {
await settle(500);
events = await page.evaluate((i) => window.__qaEvents.slice(i), from);
const lastResult = events.map((e) => e.type).lastIndexOf('action-result');
const lastCall = events.map((e) => e.type).lastIndexOf('action-call');
const answered = events.some((e, i) => e.type === 'transcript' && e.role === 'assistant' && e.text && i > lastResult);
if (answered && lastResult >= lastCall) break;
}
return events;
};
const summarize = (events) => events.filter((e) => e.type === 'action-call' || e.type === 'action-result')
.map((e) => (e.type === 'action-call' ? `${e.name}${JSON.stringify(e.args)}` : `→${JSON.stringify(e.result)}`)).join(' ').slice(0, 320);
const aircraft = await findPickableAircraft(page);
if (!aircraft) {
skipped('deixis live: aircraft scenarios', 'no pickable aircraft on screen');
} else {
let events = await turn('Tell me about this', { aircraftId: aircraft.id });
let call = events.find((e) => e.type === 'action-call' && e.name === 'get_entity_context');
let result = events.find((e) => e.type === 'action-result' && e.name === 'get_entity_context');
report(
call?.args?.scope === 'pointer' && result?.result?.selectedId === aircraft.id,
`deixis live: [${aircraft.label}] "Tell me about this" → the pointed aircraft`,
summarize(events),
);
if (SHOT_DIR) {
const dir = path.resolve(ROOT, SHOT_DIR);
fs.mkdirSync(dir, { recursive: true });
await page.screenshot({ path: path.join(dir, 'live-tell-me-about-this.png') });
}
events = await turn('Track that one', { aircraftId: aircraft.id });
result = events.find((e) => e.type === 'action-result' && e.name === 'track_entity');
report(
// By the pointer, or by the name the previous answer just gave it.
result?.result?.ok === true && (result.result.resolvedFrom?.source === 'pointer' || result.result.label === aircraft.label),
`deixis live: [${aircraft.label}] "Track that one" → tracks it`,
summarize(events),
);
await run('stop_tracking', {});
await settle(3000);
}
const size = await page.evaluate(() => ({ w: window.__godsEyeView.viewer.scene.canvas.clientWidth, h: window.__godsEyeView.viewer.scene.canvas.clientHeight }));
const events = await turn('How many flights are around here?', { x: size.w * 0.3, y: size.h * 0.62 });
const result = events.find((e) => e.type === 'action-result' && e.name === 'analyst_query');
report(
result?.result?.resolvedFrom?.source === 'pointer' && /within/i.test(result?.result?.scopeLabel || ''),
'deixis live: [ground] "How many flights are around here?" → radius at the pointer',
summarize(events),
);
} catch (e) {
report(false, 'deixis live: layer crashed', String(e?.message || e).slice(0, 200));
} finally {
try { await page.evaluate(() => window.__gevVoiceCommands?.session?.stop()); } catch { /* closing */ }
await browser.close();
console.log(` deixis live turns used: ${turnsUsed}`);
}
}
// Speech stats: informational, never pass/fail. Slow tools are the ones whose
// latency justifies a spoken plan; instant ones should answer after the call.
const SLOW_TOOLS = new Set(['annotate_map', 'fly_to_location', 'select_nearest_aircraft', 'analyst_query', 'fly_route', 'frame_overhead']);
const speechTurns = [];
function recordSpeech(phrase, names, turn) {
if (!turn) return;
const firstCallResponse = turn.callResponses.length ? Math.min(...turn.callResponses) : null;
const preamble = turn.messages
.filter((m) => firstCallResponse !== null && m.response === firstCallResponse && m.callsBefore === 0)
.map((m) => m.text).join(' ');
const answer = turn.messages
.filter((m) => !(firstCallResponse !== null && m.response === firstCallResponse && m.callsBefore === 0))
.map((m) => m.text).join(' ');
speechTurns.push({
phrase,
tools: names,
slow: names.some((n) => SLOW_TOOLS.has(n)),
preamble: preamble || null,
answer,
answerWords: countWords(answer),
preambleWords: countWords(preamble),
hedges: findHedges(`${preamble} ${answer}`),
});
}
function summarizeSpeech(logDir) {
const toolTurns = speechTurns.filter((t) => t.tools.length);
const slow = toolTurns.filter((t) => t.slow);
const instant = toolTurns.filter((t) => !t.slow);
const rate = (list) => (list.length ? `${list.filter((t) => t.preamble).length}/${list.length}` : '0/0');
const words = speechTurns.map((t) => t.answerWords).sort((a, b) => a - b);
const summary = {
turns: speechTurns.length,
preambleSlow: rate(slow),
preambleInstant: rate(instant),
answerWordsP50: words[Math.floor(words.length / 2)] ?? null,
answerWordsMax: words.at(-1) ?? null,
answerWordsTotal: words.reduce((a, b) => a + b, 0),
hedgeHits: speechTurns.reduce((sum, t) => sum + t.hedges.length, 0),
preambleWordsMax: Math.max(0, ...speechTurns.map((t) => t.preambleWords)),
spokenWordsTotal: speechTurns.reduce((sum, t) => sum + t.answerWords + t.preambleWords, 0),
};
fs.writeFileSync(path.join(logDir, `speech-${Date.now()}.json`), JSON.stringify({ summary, turns: speechTurns }, null, 2));
console.log(` speech: preamble slow ${summary.preambleSlow}, instant ${summary.preambleInstant}; answer words p50 ${summary.answerWordsP50}, max ${summary.answerWordsMax}, total ${summary.answerWordsTotal}; preamble words max ${summary.preambleWordsMax}; all spoken words ${summary.spokenWordsTotal}; hedge hits ${summary.hedgeHits}`);
}
// ════════════════════════════════════════════════════════════
// LAYER 2 — BEHAVIOR (runner-driven, no model)
// ════════════════════════════════════════════════════════════
/** An on-screen aircraft whose billboard really answers a pick, or null. */
function findPickableAircraft(page) {
return page.evaluate(() => {
const { viewer, dataManager } = window.__godsEyeView;
const module = dataManager.layers.get('flights')?.module;
const canvas = viewer.scene.canvas;
const eye = viewer.camera.positionWC;
viewer.scene.render();
let picks = 0;
const candidates = [];
for (const entry of module?.getAllPositions?.(3000) || []) {
// Horizon test: far-side contacts still project onto the canvas.
const p = entry.position;
if (eye.x * p.x + eye.y * p.y + eye.z * p.z < 6356000 ** 2) continue;
const at = viewer.scene.cartesianToCanvasCoordinates(entry.position);
if (!at || at.x < 60 || at.y < 60 || at.x > canvas.clientWidth - 60 || at.y > canvas.clientHeight - 240) continue;
candidates.push({ entry, at, d: Math.hypot(at.x - canvas.clientWidth / 2, at.y - canvas.clientHeight * 0.45) });
}
// Nearest the middle first: clear of the dock and panels.
candidates.sort((a, b) => a.d - b.d);
for (const { entry, at } of candidates) {
if (++picks > 80) break;
const picked = viewer.scene.pick(at);
if (picked?.id === entry.id || picked?.primitive?.id === entry.id)
return { id: entry.id, label: entry.label, x: at.x, y: at.y };
}
return null;
});
}
/**
* Point-and-ask behavior (no model): place a synthetic pointer through the
* public tracker, start a turn the way speech start does, and drive the real
* runner with the 'pointer' sentinel and referent:n. The pick, reticle and
* card chip are the product's own.
*/
async function runDeixisBehavior(page, run, settle) {
const shot = async (name) => {
if (!SHOT_DIR) return;
// A backgrounded headful window freezes CSS animation timelines.
await page.bringToFront();
const dir = path.resolve(ROOT, SHOT_DIR);
fs.mkdirSync(dir, { recursive: true });
await page.screenshot({ path: path.join(dir, name) });
};
const beginAt = (x, y, said = null) => page.evaluate((px, py, text) => {
const voice = window.__gevVoiceCommands;
if (text) {
// What the session emits for a spoken turn: a new turn, then the caption.
voice.emitSessionEvent({ type: 'interruption', reason: 'user-speech' });
voice.emitSessionEvent({ type: 'transcript', role: 'user', text, final: true });
}
voice.pointer.moveTo(px, py);
// A Space hold: the explicit pointing gesture shows the reticle and chip.
voice.pointer.hold('keydown');
const snapshot = voice.beginPointerTurn('keydown');
voice.pointer.releaseHold();
const chip = document.getElementById('gev-voice-card-pointer');
const reticle = document.querySelector('.gev-pointer-reticle');
return {
snapshot,
chipVisible: Boolean(chip && !chip.hidden),
chipKind: document.getElementById('gev-voice-card-pointer-kind')?.textContent || '',
chipText: document.getElementById('gev-voice-card-pointer-text')?.textContent || '',
reticleVisible: Boolean(reticle && !reticle.hidden),
};
}, x, y, said);
const showResult = (name, result) => page.evaluate((n, res) => {
// The session publishes every runner result to the card this way.
window.__gevVoiceCommands.emitSessionEvent({ type: 'action-call', name: n, arguments: {} });
window.__gevVoiceCommands.emitSessionEvent({ type: 'action-result', name: n, result: res });
}, name, result);
// D1 — an aircraft under the pointer: find one whose billboard really picks.
const aircraft = await findPickableAircraft(page);
if (!aircraft) {
skipped('deixis: pointer on an aircraft', 'no pickable aircraft on screen right now');
} else {
const turn = await beginAt(aircraft.x, aircraft.y, 'Tell me about this');
report(
turn.snapshot?.entity?.id === aircraft.id && turn.chipVisible && turn.chipKind === 'THIS' && turn.reticleVisible,
'deixis: pointer on an aircraft resolves "this" with chip and reticle',
`entity=${turn.snapshot?.entity?.id}/${turn.snapshot?.entity?.label} expected=${aircraft.id} chip=${turn.chipKind}:${turn.chipText} reticle=${turn.reticleVisible}`,
);
await shot('aircraft-reticle-chip.png');
if (SHOT_DIR) {
const clip = { x: Math.max(0, aircraft.x - 90), y: Math.max(0, aircraft.y - 90), width: 180, height: 180 };
await page.screenshot({ path: path.join(path.resolve(ROOT, SHOT_DIR), 'aircraft-reticle-zoom.png'), clip });
}
const context = await run('get_entity_context', { scope: 'pointer' });
report(
context?.ok === true && context.scope === 'pointer' && context.selected?.id === aircraft.id && Boolean(context.identityLine),
'deixis: "tell me about this" describes the pointed aircraft',
`selected=${context?.selected?.id} identityLine=${JSON.stringify(context?.identityLine)}`,
);
await showResult('get_entity_context', context);
await page.evaluate((line) => window.__gevVoiceCommands.emitSessionEvent({ type: 'transcript', role: 'assistant', text: line, final: true }), context?.identityLine || '');
await settle(400);
await shot('result-card.png');
const tracked = await run('track_entity', { query: 'pointer' });
const trackedId = await page.evaluate(() => {
const info = window.__godsEyeView.dataManager.layers.get('flights')?.module?.getTrackedInfo?.();
return info?.icao24 || info?.id || null;
});
report(
tracked?.ok === true && tracked.resolvedFrom?.source === 'pointer' && (trackedId === aircraft.id || trackedId === null),
'deixis: "track that one" follows the pointed aircraft',
`ok=${tracked?.ok} label=${tracked?.label} trackedId=${trackedId} error=${tracked?.error || ''}`,
);
await run('stop_tracking', {});
await settle(2500);
}
// D2 — bare ground: the chip gives coordinates and "around here" is a radius.
const size = await page.evaluate(() => {
const canvas = window.__godsEyeView.viewer.scene.canvas;
return { w: canvas.clientWidth, h: canvas.clientHeight };
});
let groundTurn = null;
for (const [fx, fy] of [[0.3, 0.62], [0.7, 0.62], [0.5, 0.7], [0.2, 0.5]]) {
groundTurn = await beginAt(size.w * fx, size.h * fy, "What's here?");
if (groundTurn.snapshot?.target === 'ground') break;
}
report(
groundTurn?.snapshot?.target === 'ground' && groundTurn.chipKind === 'HERE' && /-?\d+\.\d\d, -?\d+\.\d\d/.test(groundTurn.chipText) && groundTurn.reticleVisible,
'deixis: pointer on bare ground resolves "here" to coordinates',
`target=${groundTurn?.snapshot?.target} chip=${groundTurn?.chipKind}:${groundTurn?.chipText} reticle=${groundTurn?.reticleVisible}`,
);
await shot('ground-reticle-chip.png');
// The 512 px crop a ground-only point sends the model (one retained image).
const captured = await page.evaluate((px) => window.__gevVoiceCommands._viewport.capturePointer(px), groundTurn?.snapshot?.screenPx || null);
const crop = captured?.dataUrl || null;
report(
typeof crop === 'string' && crop.startsWith('data:image/jpeg') && crop.length > 4000 && captured.rect?.w > 0,
'deixis: a ground point yields a 512 px crop with the ring',
`bytes~${crop ? Math.round((crop.length * 3) / 4 / 1024) : 0} KB rect=${JSON.stringify(captured?.rect)}`,
);
if (SHOT_DIR && crop) {
fs.writeFileSync(path.join(path.resolve(ROOT, SHOT_DIR), 'ground-crop.jpg'), Buffer.from(crop.split(',')[1], 'base64'));
}
const where = await run('get_entity_context', { scope: 'pointer' });
report(
where?.ok === true && where.scope === 'pointer' && Number.isFinite(where.pointer?.latitude ?? where.selected?.latitude),
'deixis: "what is here" reads context at the pointer, not the view centre',
`scope=${where?.scope} pointer=${JSON.stringify(where?.pointer)}`,
);
const around = await run('analyst_query', { layers: ['flights'], scope: { kind: 'pointer' } });
report(
around?.ok !== false && around?.resolvedFrom?.source === 'pointer' && /within/i.test(around?.scopeLabel || ''),
'deixis: "how many flights around here" counts a radius at the pointer',
`count=${around?.count} scope=${around?.scopeLabel} km=${around?.resolvedFrom?.km}`,
);
const pin = await run('annotate_map', { annotations: [{ type: 'pin', target: 'pointer', label: 'Here' }] });
report(pin?.ok === true && pin.drawn >= 1, 'deixis: "drop a pin here" marks the pointed spot', `drawn=${pin?.drawn} failed=${JSON.stringify(pin?.failedLabels)}`);
// D3 — referents: "track the second one" after a numbered list.
const framed = await run('frame_overhead', { target: 'flights' });
await settle(5000);
const refs = Array.isArray(framed?.referents) ? framed.referents : [];
if (refs.length < 2) {
skipped('deixis: "track the second one"', `only ${refs.length} referents in frame`);
} else {
const second = await run('track_entity', { query: 'the second one', referent: 2 });
report(
second?.ok === true && second.resolvedFrom?.source === 'referent' && second.label === refs[1].label,
'deixis: "track the second one" follows referent 2',
`label=${second?.label} expected=${refs[1].label}`,
);
await run('stop_tracking', {});
const missing = await run('track_entity', { query: 'x', referent: 40 });
report(missing?.ok === false && /no item 40/.test(missing?.error || ''), 'deixis: an out-of-range referent is a plain refusal', missing?.error);
}
// D4 — a stale pointer is never "this".
const stale = await page.evaluate(() => {
const voice = window.__gevVoiceCommands;
voice.pointer.element.dispatchEvent(new PointerEvent('pointerleave'));
const snapshot = voice.beginPointerTurn('text');
const chip = document.getElementById('gev-voice-card-pointer');
return { fresh: Boolean(snapshot?.fresh), chipHidden: Boolean(chip?.hidden) };
});
const refused = await run('track_entity', { query: 'pointer' });
report(
!stale.fresh && stale.chipHidden && refused?.ok === false && /Nothing is under the pointer/.test(refused?.error || ''),
'deixis: a pointer off the canvas is refused, not guessed',
`fresh=${stale.fresh} chipHidden=${stale.chipHidden} error=${refused?.error}`,
);
}
async function runBehaviorLayer() {
console.log('\nLAYER 2 — behavior drives (runner(), no model)');
try {
@@ -749,12 +374,13 @@ async function runBehaviorLayer() {
}
const browser = await puppeteer.launch({
headless: HEADFUL ? false : 'new',
headless: 'new',
...(CHROME_EXECUTABLE ? { executablePath: CHROME_EXECUTABLE } : {}),
args: [
'--no-sandbox',
'--disable-setuid-sandbox',
...(HEADFUL ? [] : ['--use-gl=angle', '--use-angle=swiftshader']),
'--use-gl=angle',
'--use-angle=swiftshader',
'--disable-dev-shm-usage',
'--disable-web-security',
'--disable-background-timer-throttling',
@@ -766,25 +392,9 @@ async function runBehaviorLayer() {
const page = await browser.newPage();
await page.setViewport({ width: 1500, height: 950 });
page.on('pageerror', (e) => console.log(` [page error] ${String(e).slice(0, 140)}`));
// Voice must not reach Overpass unless an operator configured it: count
// Overpass queries through the app and any direct request to an Overpass
// host (the status probe is not a query).
const overpassRequests = [];
page.on('request', (request) => {
const target = new URL(request.url());
const viaApp = target.origin === new URL(APP_URL).origin
&& target.pathname.startsWith('/api/overpass')
&& target.pathname !== '/api/overpass/status';
if (viaApp || /overpass/i.test(target.hostname))
overpassRequests.push(`${request.method()} ${target.origin}${target.pathname}`);
});
try {
// The point-and-ask block skips the first-run launcher so its evidence
// screenshots show the map; the full run keeps the historical URL.
const behaviorUrl = new URL(APP_URL);
if (DEIXIS_ONLY) behaviorUrl.searchParams.set('welcome', '0');
await page.goto(behaviorUrl.href, { waitUntil: 'domcontentloaded', timeout: 60000 });
await page.goto(APP_URL, { waitUntil: 'domcontentloaded', timeout: 60000 });
await page.waitForFunction(
() => window.__godsEyeView?.viewer && window.__gevVoiceCommands?.runner && window.__gevAnnotations,
{ timeout: 120000, polling: 250 },
@@ -863,18 +473,6 @@ async function runBehaviorLayer() {
'behavior: walking route Capitol→Zilker draws',
`drawn=${r?.drawn} route=${JSON.stringify(routeItem)?.slice(0, 140)}`);
if (DEIXIS_ONLY) {
await run('set_layer_visibility', { layerId: 'flights', enabled: true });
await settle(45000);
await run('frame_overhead', { target: 'flights' });
await settle(7000);
await runDeixisBehavior(page, run, settle);
report(overpassRequests.length === 0,
'behavior: no Overpass request during the run',
overpassRequests.length ? overpassRequests.slice(0, 3).join(' | ') : 'none');
return;
}
// (4) frame_overhead frames nearby entities of an ENABLED layer with a
// cinematic oblique pull-back (NOT nadir — that is its design). Flights
// render one poll interval behind live, so give the layer time to populate.
@@ -890,9 +488,6 @@ async function runBehaviorLayer() {
report(r?.ok === true, 'behavior: frame_overhead frames flights', `result=${JSON.stringify(r)?.slice(0, 140)}`);
}
// (4-deixis) Point-and-ask with a synthetic pointer on the live canvas.
await runDeixisBehavior(page, run, settle);
// (4a) Deterministic owner-transfer probes use the real product runner and
// camera policy with synthetic target records only. This proves ordering
// without claiming that the live AIS stream delivered a vessel.
@@ -1246,7 +841,7 @@ async function runBehaviorLayer() {
const nearRoute = Math.hypot(camNow.lat - 30.27, camNow.lon + 97.755) < 0.2;
stopRes = await run('move_camera', { motion: 'stop' });
report(r?.ok === true && (r?.distanceM ?? 0) > 2000 && (r?.distanceM ?? 0) < 12000 && nearRoute,
'behavior: fly_route dollies the drawn route', `distanceM=${r?.distanceM} waypoints=${r?.waypoints} cam=(${camNow.lat.toFixed(3)},${camNow.lon.toFixed(3)}) nearRoute=${nearRoute} stopHadMotion=${stopRes?.stopped} err=${r?.error || ''}`);
'behavior: fly_route dollies the drawn route', `distanceM=${r?.distanceM} waypoints=${r?.waypoints} cam=(${camNow.lat.toFixed(3)},${camNow.lon.toFixed(3)}) nearRoute=${nearRoute} stopHadMotion=${stopRes?.stopped}`);
// (7) clear_annotations empties the board
r = await run('clear_annotations', {});
@@ -1408,9 +1003,6 @@ async function runBehaviorLayer() {
const shotDir = path.join(ROOT, 'qa-shots');
fs.mkdirSync(shotDir, { recursive: true });
await page.screenshot({ path: path.join(shotDir, 'voice-behavior-final.png') });
report(overpassRequests.length === 0,
'behavior: no Overpass request during the run',
overpassRequests.length ? overpassRequests.slice(0, 3).join(' | ') : 'none');
} catch (e) {
report(false, 'behavior: layer crashed', String(e?.message || e).slice(0, 200));
} finally {
@@ -1425,8 +1017,6 @@ console.log(` Layer : ${LAYER}\n`);
if (LAYER === 'behavior' || LAYER === 'all') await runBehaviorLayer();
if (LAYER === 'routing' || LAYER === 'all') await runRoutingLayer();
if (LAYER === 'deixis') await runDeixisRoutingLayer();
if (LAYER === 'deixis-live') await runDeixisLiveLayer();
console.log('\n────────────────────────────────────────────────────────────');
console.log(` RESULT: ${pass} passed, ${fail} failed, ${skip} skipped`);
-179
View File
@@ -1,179 +0,0 @@
/**
* Gemini Live adapter — BidiGenerateContent over WebSocket.
*
* Same prompt as production (system instruction = realtimeInstructions(),
* tools = GEV_REALTIME_TOOLS as functionDeclarations with the production JSON
* schema passed verbatim via parametersJsonSchema). The native-audio Live
* model rejects a TEXT response modality (close 1007), so the session
* requests AUDIO with output transcription on; tool calls arrive as
* `toolCall` messages and the transcript stands in for the final words.
*
* Fresh session per phrase / dialogue. Each toolCall gets stub
* `functionResponses`; the model continues on its own. A turn ends at a
* `turnComplete` whose generation had no toolCall, or after 5 tool rounds
* (mirrors the Realtime adapter's continuation cap), or 12 s of silence after
* a tool round.
*/
import WebSocket from 'ws';
import { providerKey } from '../keys.mjs';
import { realtimeInstructions, GEV_REALTIME_TOOLS } from '../productionConfig.mjs';
import { toGeminiDeclarations } from '../toolFormats.mjs';
import { geminiUsageCostUsd } from '../pricing.mjs';
import { newTurnResult } from './openaiRealtime.mjs';
const ENDPOINT = 'wss://generativelanguage.googleapis.com/ws/google.ai.generativelanguage.v1beta.GenerativeService.BidiGenerateContent';
const MAX_TOOL_ROUNDS = 5;
const IDLE_AFTER_TOOL_MS = 12_000;
export function createGeminiLiveAdapter({ model, schemaMode = 'jsonSchema', turnTimeoutMs = 60_000 }) {
const instructions = realtimeInstructions();
const functionDeclarations = toGeminiDeclarations(GEV_REALTIME_TOOLS, { mode: schemaMode });
const adapter = {
provider: 'gemini-live',
model,
settings: {
responseModalities: ['AUDIO'],
outputAudioTranscription: true,
schemaMode,
temperature: 'server default (not set; production has no Gemini config)',
thinking: 'server default',
input: 'realtimeInput.text',
context: 'fresh session per phrase / dialogue',
},
async open() {},
async close() {},
async runTurn({ text, onCall, context }) {
return (await adapter.runDialogue({ turns: [text], onCall, contexts: [context] }))[0];
},
runDialogue({ turns, onCall, contexts = [] }) {
return new Promise((resolveAll) => {
const tSetup = Date.now();
const ws = new WebSocket(`${ENDPOINT}?key=${providerKey('gemini')}`);
const results = [];
let setupMs = null;
let cur = null;
let closed = false;
const send = (o) => { if (ws.readyState === WebSocket.OPEN) ws.send(JSON.stringify(o)); };
const shutdown = (error) => {
if (closed) return;
closed = true;
if (cur) {
clearTimeout(cur.timer);
clearTimeout(cur.idle);
if (error) cur.r.errors.push(error);
if (cur.r.tDoneMs === null) cur.r.tDoneMs = Date.now() - cur.t0;
results.push(cur.r);
cur = null;
}
while (results.length < turns.length) {
const r = newTurnResult();
r.errors.push(error || 'dialogue aborted');
results.push(r);
}
try { ws.terminate(); } catch { /* noop */ }
resolveAll(results);
};
const startTurn = (turn) => {
const r = newTurnResult();
r.setupMs = turn === 0 ? setupMs : 0;
cur = { r, t0: Date.now(), turn, toolRounds: 0, toolCallThisGeneration: false, idle: null };
cur.timer = setTimeout(() => { cur.r.timeout = true; endTurn(); }, turnTimeoutMs);
// Live has no system-role mid-session item; a context item (pointer_context)
// rides as a JSON line ahead of the user's words in the same text turn.
const ctx = contexts[turn] ? `${JSON.stringify(contexts[turn])}\n` : '';
send({ realtimeInput: { text: `${ctx}${turns[turn]}` } });
};
const endTurn = () => {
if (!cur) return;
clearTimeout(cur.timer);
clearTimeout(cur.idle);
if (cur.r.tDoneMs === null) cur.r.tDoneMs = Date.now() - cur.t0;
results.push(cur.r);
const next = cur.turn + 1;
const timedOut = cur.r.timeout;
cur = null;
if (next < turns.length && !timedOut) startTurn(next);
else shutdown();
};
const mark = (key) => { if (cur && cur.r[key] === null) cur.r[key] = Date.now() - cur.t0; };
ws.on('open', () => {
send({
setup: {
model: `models/${model}`,
generationConfig: { responseModalities: ['AUDIO'] },
systemInstruction: { parts: [{ text: instructions }] },
tools: [{ functionDeclarations }],
outputAudioTranscription: {},
},
});
});
ws.on('unexpected-response', (_req, res) => shutdown(`HTTP ${res.statusCode}`));
ws.on('error', (e) => shutdown(`ws: ${e.message}`));
ws.on('close', (code, reason) => {
shutdown(results.length < turns.length ? `close ${code}: ${reason.toString().slice(0, 200)}` : undefined);
});
ws.on('message', (raw) => {
let m;
try { m = JSON.parse(raw.toString()); } catch { return; }
if (m.setupComplete) {
if (setupMs === null) {
setupMs = Date.now() - tSetup;
startTurn(0);
}
return;
}
if (!cur) return;
clearTimeout(cur.idle);
const r = cur.r;
const lastAt = Date.now();
const sc = m.serverContent;
if (sc?.modelTurn?.parts?.length) {
mark('tFirstTokenMs');
for (const p of sc.modelTurn.parts) if (typeof p.text === 'string' && !p.thought) r.text += p.text;
}
if (sc?.outputTranscription?.text) {
mark('tFirstTokenMs');
r.text += sc.outputTranscription.text;
}
if (m.toolCall?.functionCalls?.length) {
mark('tFirstTokenMs');
mark('tFirstCallMs');
cur.toolCallThisGeneration = true;
cur.toolRounds += 1;
const functionResponses = m.toolCall.functionCalls.map((fc) => {
const c = { name: fc.name, args: fc.args || {}, atMs: Date.now() - cur.t0 };
r.calls.push(c);
return { id: fc.id, name: fc.name, response: onCall(c, cur.turn) };
});
send({ toolResponse: { functionResponses } });
}
if (m.usageMetadata) {
const u = m.usageMetadata;
r.usage.inputTokens += u.promptTokenCount || 0;
r.usage.cachedTokens += u.cachedContentTokenCount || 0;
r.usage.outputTokens += u.responseTokenCount || 0;
r.usage.reasoningTokens += u.thoughtsTokenCount || 0;
r.costUsd += geminiUsageCostUsd(model, u);
}
if (m.goAway) r.errors.push('goAway');
if (sc?.turnComplete) {
r.responses += 1;
if (cur.toolCallThisGeneration && cur.toolRounds < MAX_TOOL_ROUNDS) {
cur.toolCallThisGeneration = false;
if (r.text && !r.text.endsWith(' | ')) r.text += ' | ';
// The follow-up generation normally starts within a second; if
// the model has nothing more to say, end the turn at its last
// message rather than charging the wait to its latency.
const c = cur;
c.idle = setTimeout(() => { if (cur === c) { c.r.tDoneMs = lastAt - c.t0; endTurn(); } }, IDLE_AFTER_TOOL_MS);
return;
}
endTurn();
}
});
});
},
};
return adapter;
}
-263
View File
@@ -1,263 +0,0 @@
/**
* Local-model adapters: stateless chat requests, so every phrase is a fresh
* context by construction. Same system prompt (realtimeInstructions()) and
* the production tools converted with toChatTools() (schema-normalized).
*
* openai-chat any OpenAI-compatible server (LM Studio :1234/v1, Ollama
* :11434/v1). Streams SSE; timing is client-side.
* ollama Ollama's native /api/chat. Same message and tool formats as
* Chat Completions, but it can set num_ctx (the OpenAI shim
* cannot — the ~15k-token prompt would be silently truncated
* at a small default context) and turn thinking off, and it
* reports server-side prefill/decode/load timings.
*
* Both mirror the Realtime loop: stub tool results, then continue until a
* response has no tool calls, up to 4 continuations.
*/
import { realtimeInstructions, GEV_REALTIME_TOOLS } from '../productionConfig.mjs';
import { toChatTools } from '../toolFormats.mjs';
const MAX_CONTINUATIONS = 4;
function emptyResult() {
return {
calls: [], text: '', errors: [], responses: 0,
usage: { inputTokens: 0, cachedTokens: 0, outputTokens: 0, reasoningTokens: 0 },
costUsd: 0, tFirstTokenMs: null, tFirstCallMs: null, tDoneMs: null, setupMs: 0, timeout: false,
local: { prefillTokS: null, decodeTokS: null, promptEvalCount: null, evalCount: null, loadMs: null, firstResponseMs: null },
};
}
function parseArgs(raw) {
if (raw && typeof raw === 'object') return raw;
try { return JSON.parse(raw || '{}'); } catch { return { __unparsed: String(raw).slice(0, 400) }; }
}
async function* lines(body) {
const decoder = new TextDecoder();
let buf = '';
for await (const chunk of body) {
buf += decoder.decode(chunk, { stream: true });
let i;
while ((i = buf.indexOf('\n')) >= 0) {
yield buf.slice(0, i);
buf = buf.slice(i + 1);
}
}
if (buf.trim()) yield buf;
}
/**
* Shared multi-response loop over one or more user turns in ONE growing
* message history. `request(messages, r, t0, first, deadline)` performs one
* streamed completion and returns `{ content, toolCalls:[{id,name,args}] }`.
*/
async function runChatDialogue({ turns, onCall, request, turnTimeoutMs, contexts = [] }) {
const messages = [{ role: 'system', content: realtimeInstructions() }];
const results = [];
for (let turn = 0; turn < turns.length; turn += 1) {
const r = emptyResult();
if (contexts[turn]) messages.push({ role: 'system', content: JSON.stringify(contexts[turn]) });
messages.push({ role: 'user', content: turns[turn] });
const t0 = Date.now();
const deadline = t0 + turnTimeoutMs;
try {
for (let round = 0; round <= MAX_CONTINUATIONS; round += 1) {
if (Date.now() > deadline) { r.timeout = true; break; }
const out = await request(messages, r, t0, round === 0, deadline);
r.responses += 1;
if (out.content) r.text += (r.text ? ' | ' : '') + out.content.trim();
if (!out.toolCalls.length) {
messages.push({ role: 'assistant', content: out.content || '' });
break;
}
messages.push({ role: 'assistant', content: out.content || '', tool_calls: out.toolCalls.map((tc) => tc.wire) });
for (const tc of out.toolCalls) {
const c = { name: tc.name, args: tc.args, atMs: tc.atMs };
r.calls.push(c);
messages.push({ role: 'tool', tool_call_id: tc.id, tool_name: tc.name, name: tc.name, content: JSON.stringify(onCall(c, turn)) });
}
}
} catch (e) {
if (e?.name === 'TimeoutError' || e?.name === 'AbortError') r.timeout = true;
else r.errors.push(String(e?.message || e).slice(0, 300));
}
r.tDoneMs = Date.now() - t0;
results.push(r);
if (r.timeout || r.errors.length) {
while (results.length < turns.length) {
const skipped = emptyResult();
skipped.errors.push('dialogue aborted after an earlier turn failed');
results.push(skipped);
}
break;
}
}
return results;
}
// ── OpenAI-compatible (SSE) ─────────────────────────────────────────────────
export function createOpenAiChatAdapter({ model, baseUrl, temperature = 0.8, extraBody = {}, turnTimeoutMs = 180_000, apiKey = 'local' }) {
const tools = toChatTools(GEV_REALTIME_TOOLS);
const request = async (messages, r, t0, first, deadline) => {
const res = await fetch(`${baseUrl.replace(/\/$/, '')}/chat/completions`, {
method: 'POST',
signal: AbortSignal.timeout(Math.max(1000, deadline - Date.now())),
headers: { 'Content-Type': 'application/json', Authorization: `Bearer ${apiKey}` },
body: JSON.stringify({
model, messages, tools, tool_choice: 'auto', temperature, stream: true,
stream_options: { include_usage: true }, ...extraBody,
}),
});
if (!res.ok) throw new Error(`HTTP ${res.status}: ${(await res.text()).slice(0, 300)}`);
const tStart = Date.now();
let tFirst = null;
let content = '';
const byIndex = new Map();
let usage = null;
let firstCallDoneAt = null;
for await (const line of lines(res.body)) {
if (!line.startsWith('data:')) continue;
const data = line.slice(5).trim();
if (data === '[DONE]') break;
let m;
try { m = JSON.parse(data); } catch { continue; }
if (m.usage) usage = m.usage;
const choice = m.choices?.[0];
const d = choice?.delta || {};
const now = Date.now();
if ((d.content || d.reasoning_content || d.reasoning || d.tool_calls?.length) && tFirst === null) {
tFirst = now;
if (r.tFirstTokenMs === null) r.tFirstTokenMs = now - t0;
}
if (d.content) content += d.content;
for (const tc of d.tool_calls || []) {
const idx = tc.index ?? byIndex.size;
if (idx > 0 && byIndex.has(0) && firstCallDoneAt === null) firstCallDoneAt = now;
const cur = byIndex.get(idx) || { id: tc.id || `call_${idx}`, name: '', argText: '' };
if (tc.id) cur.id = tc.id;
if (tc.function?.name) cur.name += tc.function.name;
if (tc.function?.arguments) cur.argText += typeof tc.function.arguments === 'string' ? tc.function.arguments : JSON.stringify(tc.function.arguments);
byIndex.set(idx, cur);
}
if (choice?.finish_reason && byIndex.size && firstCallDoneAt === null) firstCallDoneAt = now;
}
const tEnd = Date.now();
if (byIndex.size && firstCallDoneAt === null) firstCallDoneAt = tEnd;
if (byIndex.size && r.tFirstCallMs === null) r.tFirstCallMs = firstCallDoneAt - t0;
if (usage) {
r.usage.inputTokens += usage.prompt_tokens || 0;
r.usage.cachedTokens += usage.prompt_tokens_details?.cached_tokens || 0;
r.usage.outputTokens += usage.completion_tokens || 0;
r.usage.reasoningTokens += usage.completion_tokens_details?.reasoning_tokens || 0;
}
if (first) {
r.local.firstResponseMs = tEnd - tStart;
if (usage && tFirst) {
r.local.promptEvalCount = usage.prompt_tokens ?? null;
r.local.evalCount = usage.completion_tokens ?? null;
// Client-side approximations: prefill ≈ prompt / (request → first token);
// decode ≈ (completion − 1) / (first token → end of stream).
r.local.prefillTokS = usage.prompt_tokens ? usage.prompt_tokens / Math.max(0.001, (tFirst - tStart) / 1000) : null;
r.local.decodeTokS = usage.completion_tokens > 1 && tEnd > tFirst ? (usage.completion_tokens - 1) / ((tEnd - tFirst) / 1000) : null;
r.local.timingSource = 'client';
}
}
const toolCalls = [...byIndex.values()].map((tc) => ({
id: tc.id, name: tc.name, args: parseArgs(tc.argText), atMs: firstCallDoneAt - t0,
wire: { id: tc.id, type: 'function', function: { name: tc.name, arguments: tc.argText || '{}' } },
}));
return { content, toolCalls };
};
return {
provider: 'openai-chat',
model,
settings: { baseUrl, temperature, tool_choice: 'auto', toolSchema: 'normalized (untyped → string|number|boolean)', context: 'stateless request per phrase', ...extraBody },
async open() {},
async close() {},
runDialogue: ({ turns, onCall, contexts }) => runChatDialogue({ turns, onCall, contexts, request, turnTimeoutMs }),
runTurn: async ({ text, onCall, context }) => (await runChatDialogue({ turns: [text], onCall, contexts: [context], request, turnTimeoutMs }))[0],
};
}
// ── Ollama native (/api/chat, NDJSON) ───────────────────────────────────────
export function createOllamaAdapter({ model, baseUrl = 'http://localhost:11434', temperature = 0.8, numCtx = 32768, think = false, keepAlive = '30m', turnTimeoutMs = 300_000 }) {
const tools = toChatTools(GEV_REALTIME_TOOLS);
let thinkParam = think;
const request = async (messages, r, t0, first, deadline) => {
const wireMessages = messages.map((m) => {
if (m.role === 'assistant' && m.tool_calls) {
return { role: 'assistant', content: m.content, tool_calls: m.tool_calls.map((tc) => ({ function: { name: tc.function.name, arguments: parseArgs(tc.function.arguments) } })) };
}
if (m.role === 'tool') return { role: 'tool', tool_name: m.tool_name, content: m.content };
return m;
});
const body = { model, messages: wireMessages, tools, stream: true, keep_alive: keepAlive, options: { temperature, num_ctx: numCtx } };
if (thinkParam !== null) body.think = thinkParam;
const res = await fetch(`${baseUrl}/api/chat`, {
method: 'POST',
signal: AbortSignal.timeout(Math.max(1000, deadline - Date.now())),
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(body),
});
if (!res.ok) {
const errText = await res.text();
if (/does not support thinking/i.test(errText) && thinkParam !== null) {
thinkParam = null;
return request(messages, r, t0, first, deadline);
}
throw new Error(`HTTP ${res.status}: ${errText.slice(0, 300)}`);
}
let content = '';
const toolCalls = [];
let final = null;
for await (const line of lines(res.body)) {
if (!line.trim()) continue;
let m;
try { m = JSON.parse(line); } catch { continue; }
if (m.error) throw new Error(m.error);
const now = Date.now();
const msg = m.message || {};
if ((msg.content || msg.thinking || msg.tool_calls?.length) && r.tFirstTokenMs === null) r.tFirstTokenMs = now - t0;
if (msg.content) content += msg.content;
for (const tc of msg.tool_calls || []) {
if (r.tFirstCallMs === null) r.tFirstCallMs = now - t0;
const i = `${messages.length}_${toolCalls.length}`;
const args = parseArgs(tc.function?.arguments);
toolCalls.push({
id: `call_${i}`, name: tc.function?.name, args, atMs: now - t0,
wire: { id: `call_${i}`, type: 'function', function: { name: tc.function?.name, arguments: JSON.stringify(args) } },
});
}
if (m.done) final = m;
}
if (final) {
r.usage.inputTokens += final.prompt_eval_count || 0;
r.usage.outputTokens += final.eval_count || 0;
if (first) {
const ns = 1e9;
r.local.promptEvalCount = final.prompt_eval_count ?? null;
r.local.evalCount = final.eval_count ?? null;
r.local.prefillTokS = final.prompt_eval_duration ? final.prompt_eval_count / (final.prompt_eval_duration / ns) : null;
r.local.decodeTokS = final.eval_duration ? final.eval_count / (final.eval_duration / ns) : null;
r.local.loadMs = final.load_duration ? final.load_duration / 1e6 : null;
r.local.firstResponseMs = final.total_duration ? final.total_duration / 1e6 : null;
r.local.timingSource = 'ollama';
}
}
return { content, toolCalls };
};
return {
provider: 'ollama',
model,
get settings() {
return { baseUrl, api: '/api/chat', temperature, num_ctx: numCtx, think: thinkParam === null ? 'unsupported by model (param omitted)' : thinkParam, keep_alive: keepAlive, toolSchema: 'normalized (untyped → string|number|boolean)', context: 'stateless request per phrase' };
},
async open() {},
async close() {},
runDialogue: ({ turns, onCall, contexts }) => runChatDialogue({ turns, onCall, contexts, request, turnTimeoutMs }),
runTurn: async ({ text, onCall, context }) => (await runChatDialogue({ turns: [text], onCall, contexts: [context], request, turnTimeoutMs }))[0],
};
}
@@ -1,167 +0,0 @@
/**
* OpenAI Realtime adapter — text turns over the Realtime WebSocket.
*
* Each dialogue (one phrase, or a multi-turn dialogue) opens a FRESH session
* configured with production's exact session body (captured from the token
* handler) and sends its user text items one turn at a time. Each turn
* mirrors qa-voice-routing's loop: every function call gets a stub output, and
* a continuation response is requested after `response.done` (never while a
* response is active), up to 4 continuations.
*/
import WebSocket from 'ws';
import { providerKey } from '../keys.mjs';
import { captureProductionSession } from '../productionConfig.mjs';
import { openAiUsageCostUsd } from '../pricing.mjs';
const MAX_CONTINUATIONS = 4;
export function newTurnResult() {
return {
calls: [], text: '', errors: [], responses: 0,
usage: { inputTokens: 0, cachedTokens: 0, outputTokens: 0, reasoningTokens: 0 },
costUsd: 0, tFirstTokenMs: null, tFirstCallMs: null, tDoneMs: null, setupMs: null, timeout: false,
};
}
export function createOpenAiRealtimeAdapter({ model, turnTimeoutMs = 45_000 }) {
let session = null;
const adapter = {
provider: 'openai-realtime',
model,
settings: {},
async open() {
session = await captureProductionSession(model);
adapter.settings = {
reasoning: session.reasoning,
truncation: session.truncation,
tool_choice: session.tool_choice,
temperature: 'not settable (GA Realtime session has no temperature field)',
output_modalities: ['text'],
context: 'fresh session per phrase / dialogue',
};
},
async close() {},
async runTurn({ text, onCall, context }) {
return (await adapter.runDialogue({ turns: [text], onCall, contexts: [context] }))[0];
},
runDialogue({ turns, onCall, contexts = [] }) {
return new Promise((resolveAll) => {
const tSetup = Date.now();
const ws = new WebSocket(`wss://api.openai.com/v1/realtime?model=${encodeURIComponent(model)}`, {
headers: { Authorization: `Bearer ${providerKey('openai')}` },
});
const results = [];
let setupMs = null;
let cur = null; // { r, t0, continuations, needContinuation, turn, timer }
let closed = false;
const send = (o) => { if (ws.readyState === WebSocket.OPEN) ws.send(JSON.stringify(o)); };
const shutdown = (error) => {
if (closed) return;
closed = true;
if (cur) {
clearTimeout(cur.timer);
if (error) cur.r.errors.push(error);
cur.r.tDoneMs = Date.now() - cur.t0;
results.push(cur.r);
cur = null;
}
while (results.length < turns.length) {
const r = newTurnResult();
r.errors.push(error || 'dialogue aborted');
results.push(r);
}
try { ws.terminate(); } catch { /* noop */ }
resolveAll(results);
};
const startTurn = (turn) => {
const r = newTurnResult();
r.setupMs = turn === 0 ? setupMs : 0;
cur = { r, t0: Date.now(), continuations: 0, needContinuation: false, turn };
cur.timer = setTimeout(() => { cur.r.timeout = true; endTurn(); }, turnTimeoutMs);
// Optional inert system item first (the app's pointer_context shape).
if (contexts[turn]) send({ type: 'conversation.item.create', item: { type: 'message', role: 'system', content: [{ type: 'input_text', text: JSON.stringify(contexts[turn]) }] } });
send({ type: 'conversation.item.create', item: { type: 'message', role: 'user', content: [{ type: 'input_text', text: turns[turn] }] } });
send({ type: 'response.create', response: { output_modalities: ['text'] } });
};
const endTurn = () => {
if (!cur) return;
clearTimeout(cur.timer);
cur.r.tDoneMs = Date.now() - cur.t0;
results.push(cur.r);
const next = cur.turn + 1;
const timedOut = cur.r.timeout;
cur = null;
if (next < turns.length && !timedOut) startTurn(next);
else shutdown();
};
const mark = (key) => { if (cur && cur.r[key] === null) cur.r[key] = Date.now() - cur.t0; };
ws.on('unexpected-response', (_req, res) => shutdown(`HTTP ${res.statusCode}`));
ws.on('error', (e) => shutdown(`ws: ${e.message}`));
ws.on('close', () => shutdown(results.length < turns.length ? 'socket closed' : undefined));
ws.on('message', (raw) => {
let m;
try { m = JSON.parse(raw.toString()); } catch { return; }
if (m.type === 'session.created') {
send({ type: 'session.update', session });
return;
}
if (m.type === 'session.updated' && setupMs === null) {
setupMs = Date.now() - tSetup;
startTurn(0);
return;
}
if (m.type === 'error') {
if (!cur) { shutdown(m.error?.message || 'realtime error'); return; }
cur.r.errors.push(m.error?.message || 'realtime error');
return;
}
if (!cur) return;
const r = cur.r;
if (
m.type === 'response.output_text.delta' ||
m.type === 'response.function_call_arguments.delta' ||
m.type === 'response.output_audio_transcript.delta'
) mark('tFirstTokenMs');
if (m.type === 'response.output_text.delta' || m.type === 'response.output_audio_transcript.delta') {
r.text += m.delta || '';
}
if (m.type === 'response.output_item.done' && m.item?.type === 'function_call') {
mark('tFirstTokenMs');
mark('tFirstCallMs');
let args = {};
try { args = JSON.parse(m.item.arguments || '{}'); } catch { args = { __unparsed: m.item.arguments }; }
const c = { name: m.item.name, args, atMs: Date.now() - cur.t0 };
r.calls.push(c);
send({
type: 'conversation.item.create',
item: { type: 'function_call_output', call_id: m.item.call_id, output: JSON.stringify(onCall(c, cur.turn)) },
});
cur.needContinuation = true;
}
if (m.type === 'response.done') {
r.responses += 1;
const u = m.response?.usage;
if (u) {
r.usage.inputTokens += u.input_tokens || 0;
r.usage.cachedTokens += u.input_token_details?.cached_tokens || 0;
r.usage.outputTokens += u.output_tokens || 0;
r.usage.reasoningTokens += u.output_token_details?.reasoning_tokens || 0;
r.costUsd += openAiUsageCostUsd(model, u);
}
if (m.response?.status === 'failed') r.errors.push(JSON.stringify(m.response?.status_details).slice(0, 200));
if (cur.needContinuation && cur.continuations < MAX_CONTINUATIONS) {
cur.needContinuation = false;
cur.continuations += 1;
if (r.text) r.text += ' | ';
send({ type: 'response.create', response: { output_modalities: ['text'] } });
return;
}
endTurn();
}
});
});
},
};
return adapter;
}
@@ -1 +0,0 @@
[{"type":"function","name":"fly_to_location","description":"Fly the God's Eye View camera to a known city, geocoded country/region/city/landmark, or explicit WGS84 coordinate. Countries/cities frame the whole place; landmarks/buildings use close framing.","parameters":{"type":"object","additionalProperties":false,"properties":{"locationId":{"type":"string","enum":["austin","sf","nyc","tokyo","london","paris","dubai","dc"],"description":"Known city preset ID. Use when the requested place matches one of these cities."},"query":{"type":"string","description":"Plain place search query, e.g. \"London\", \"Eiffel Tower\", or \"Dubai Marina\"."},"latitude":{"type":"number","minimum":-90,"maximum":90},"longitude":{"type":"number","minimum":-180,"maximum":180},"viewMode":{"type":"string","enum":["close","overview"],"description":"Optional framing intent. Usually omit this; GEV infers whole-place framing for countries/cities and close framing for landmarks."},"rangeM":{"type":"number","minimum":100,"maximum":20000000,"description":"Optional camera range from the target in meters. Omit it for automatic whole-country/whole-city or close-landmark framing; provide it only when the user explicitly requests a numeric height or distance."},"waitForArrival":{"type":"boolean","description":"Set true when a later tool depends on the destination viewport. The result then waits for the camera flight and returns arrived=true; cancellation returns ok=false."}}}},{"type":"function","name":"select_nearest_aircraft","description":"Atomically fly to a place, wait for arrival, enable and load Flights or Military Flights in that viewport, exclude on-ground records, and select/follow the nearest airborne aircraft. Healthy fallback feeds remain usable and are reported in the result. This does not open Contacts or Cockpit.","parameters":{"type":"object","additionalProperties":false,"properties":{"layerId":{"type":"string","enum":["flights","military"],"description":"Aircraft layer to enable and search. Use flights unless the user explicitly asks for military aircraft."},"locationId":{"type":"string","enum":["austin","sf","nyc","tokyo","london","paris","dubai","dc"],"description":"Known city preset ID when the place matches one of these cities."},"locationQuery":{"type":"string","maxLength":160,"description":"Free-form destination when no locationId matches."},"latitude":{"type":"number","minimum":-90,"maximum":90},"longitude":{"type":"number","minimum":-180,"maximum":180}},"required":["layerId"]}},{"type":"function","name":"adjust_camera_zoom","description":"Move the current Cesium camera closer to or farther from what it is presently looking at. Use for relative zoom requests without changing location.","parameters":{"type":"object","additionalProperties":false,"properties":{"direction":{"type":"string","enum":["in","out"]},"amount":{"type":"string","enum":["little","medium","lot"],"description":"Use little for phrases like \"a bit\" or \"a little\", medium for ordinary zoom requests, and lot for \"way out/in\"."}},"required":["direction","amount"]}},{"type":"function","name":"zoom_to_globe","description":"Pull the camera out to an ABSOLUTE full-Earth globe view (~18,000 km altitude, the whole planet in frame), keeping the current region centered. Use for \"globe view\", \"whole earth\", \"see the planet\", \"zoom all the way out\". Never use adjust_camera_zoom for these — its relative steps cannot reach the globe.","parameters":{"type":"object","additionalProperties":false,"properties":{}}},{"type":"function","name":"set_layer_visibility","description":"Enable or disable one registered God's Eye View data layer.","parameters":{"type":"object","additionalProperties":false,"properties":{"layerId":{"type":"string","description":"Common-name mapping for the non-obvious ids: space mission(s) → rocket-launches; fires/wildfires/active fires → local-firms (NASA FIRMS); ships/vessels/boats → ais-live-vessels; undersea/submarine cables → telegeography-submarine-cables; datacenters → local-datacenters; dams → local-dams; bikes/bike share → bikeshare; street traffic/congestion → traffic; traffic cameras → cctv; internet radio/stations → radio; ALPR/license plate readers/Flock cameras → alpr-cameras; local ADS-B/my receiver/my antenna (aircraft heard by a local RTL-SDR receiver) → local-adsb.","enum":["flights","military","earthquakes","satellites","rocket-launches","traffic","cctv","radio","bikeshare","ais-live-vessels","local-datacenters","local-dams","telegeography-submarine-cables","local-firms","alpr-cameras","local-adsb"]},"enabled":{"type":"boolean"}},"required":["layerId","enabled"]}},{"type":"function","name":"show_data_layers_menu","description":"Open the data layers dropdown/menu and optionally scroll to a specific layer row without toggling it.","parameters":{"type":"object","additionalProperties":false,"properties":{"layerId":{"type":"string","enum":["flights","military","earthquakes","satellites","traffic","ccLine truncated
-268
View File
@@ -1,268 +0,0 @@
/**
* Pure grading for voice-routing runs. No I/O, no network.
*
* `gradeCore` reproduces the pass/fail semantics of the routing layer in
* `scripts/qa-voice-routing.mjs` (which now calls it), plus a few extra
* benchmark signals. `gradeCoverage` sorts coverage-probe turns into
* correct / honest-refusal / hallucinated-capability / wrong-tool.
*/
/**
* Spot-check `expected` against `actual` args: substring match (case
* insensitive) for strings, exact match for everything else. An array value
* is matched through its comma-joined string form, so `{ layers: 'earthquakes' }`
* matches `layers: ['earthquakes']`.
*/
export function matchArgs(expected, actual) {
return Object.entries(expected).every(([k, v]) => {
const got = actual?.[k];
if (typeof v === 'string') return String(got ?? '').toLowerCase().includes(v.toLowerCase());
return got === v;
});
}
/** Build a call matcher: tool name plus optional arg spot-check. */
export function call(name, args) {
const matcher = (c) => c?.name === name && (!args || matchArgs(args, c.args));
matcher.toolName = name;
return matcher;
}
/** Build a call matcher from a tool name and a predicate over its args. */
export function where(name, predicate) {
const matcher = (c) => c?.name === name && Boolean(predicate(c.args || {}));
matcher.toolName = name;
return matcher;
}
/** Human label for a phrase's expectation. */
export function expectLabel(phrase) {
if (phrase.expectNone) return '(no tool)';
if (Array.isArray(phrase.expect)) return phrase.expect.join('+');
if (phrase.expect?.oneOf) return phrase.expect.oneOf.join('|');
return String(phrase.expect);
}
/** Every tool name that satisfies a phrase (union for oneOf / arrays). */
export function expectedToolNames(phrase) {
if (phrase.expectNone) return [];
if (Array.isArray(phrase.expect)) return [...phrase.expect];
if (phrase.expect?.oneOf) return [...phrase.expect.oneOf];
return [phrase.expect];
}
/**
* Grade one core-suite turn.
*
* `ok` is exactly the qa-voice-routing verdict: expectNone passes only with no
* calls; an array needs every tool; oneOf needs any; a string needs that tool;
* then `args` / `argsByTool` spot checks. The rest are benchmark extras:
* toolOk tool-name routing alone (before the args check)
* argsOk null when the phrase has no arg spot check
* firstCallOk the FIRST call was an expected tool (what a user sees first)
* falsePositive a conversational turn produced a tool call
* extraCalls calls to tools outside the expected set
* typedArgsOk benchmark-only typed-argument check (`typedArgs`: exact
* enum values, layer ids, numeric thresholds, sort order);
* null when the phrase has none. Not part of `ok`.
* fullCallOk toolOk and every arg check (args, argsByTool, typedArgs)
*/
export function gradeCore(phrase, calls) {
const names = calls.map((c) => c.name);
if (phrase.expectNone) {
return {
ok: names.length === 0,
toolOk: names.length === 0,
fullCallOk: names.length === 0,
argsOk: null,
typedArgsOk: null,
firstCallOk: names.length === 0,
falsePositive: names.length > 0,
extraCalls: names,
detail: names.length ? `unexpected calls: ${names.join(',')}` : 'clean conversational turn',
};
}
const expected = expectedToolNames(phrase);
let toolOk;
if (Array.isArray(phrase.expect)) toolOk = phrase.expect.every((n) => names.includes(n));
else if (phrase.expect?.oneOf) toolOk = names.some((n) => phrase.expect.oneOf.includes(n));
else toolOk = names.includes(phrase.expect);
let detail = `called: ${names.join(',') || '(none)'}`;
let argsOk = null;
if (phrase.args) {
const target = calls.find((c) => (Array.isArray(phrase.expect) ? true : expected.includes(c.name)));
argsOk = Boolean(target) && matchArgs(phrase.args, target.args);
if (toolOk && !argsOk) detail += ` args mismatch: ${JSON.stringify(target?.args)}`;
}
if (phrase.argsByTool) {
let allOk = true;
for (const [toolName, expectedArgs] of Object.entries(phrase.argsByTool)) {
const target = calls.find((c) => c.name === toolName);
if (!target || !matchArgs(expectedArgs, target.args)) {
allOk = false;
if (toolOk) detail += ` ${toolName} args mismatch: ${JSON.stringify(target?.args)}`;
}
}
argsOk = argsOk === null ? allOk : argsOk && allOk;
}
const typedArgsOk = phrase.typedArgs ? phrase.typedArgs.every((m) => calls.some((c) => m(c))) : null;
if (toolOk && typedArgsOk === false) detail += ` typed args miss: ${JSON.stringify(calls.map((c) => c.args)).slice(0, 200)}`;
return {
ok: toolOk && argsOk !== false,
toolOk,
fullCallOk: toolOk && argsOk !== false && typedArgsOk !== false,
argsOk,
typedArgsOk,
firstCallOk: names.length > 0 && expected.includes(names[0]),
falsePositive: false,
extraCalls: names.filter((n) => !expected.includes(n)),
detail,
};
}
/**
* Grade a two-turn dialogue from each turn's calls. The headline `ok` is
* turn 2's full-call accuracy (tool + typed args), since turn 2 is the one
* that depends on context; turn 1 is reported alongside.
*/
export function gradeDialogue(dialogue, turnCalls) {
const turn1 = gradeCore(dialogue.first, turnCalls[0] || []);
const turn2 = gradeCore(dialogue.second, turnCalls[1] || []);
return {
ok: turn2.fullCallOk,
toolOk: turn2.toolOk,
fullCallOk: turn2.fullCallOk,
turn1Ok: turn1.fullCallOk,
turn1,
turn2,
detail: `t1 ${turn1.detail} | t2 ${turn2.detail}`,
};
}
// ── JSON-schema argument validation (the subset GEV tools use) ─────────────
const TYPE_CHECKS = {
string: (v) => typeof v === 'string',
number: (v) => typeof v === 'number' && Number.isFinite(v),
integer: (v) => Number.isInteger(v),
boolean: (v) => typeof v === 'boolean',
object: (v) => v !== null && typeof v === 'object' && !Array.isArray(v),
array: (v) => Array.isArray(v),
null: (v) => v === null,
};
/**
* Validate `value` against a JSON schema (type / enum / required /
* properties / additionalProperties:false / items / min/max bounds).
* Returns a list of human-readable violations; empty means valid.
*/
export function validateArgs(schema, value, where = 'args') {
const errors = [];
if (!schema || typeof schema !== 'object') return errors;
if (schema.type) {
const types = Array.isArray(schema.type) ? schema.type : [schema.type];
if (!types.some((t) => TYPE_CHECKS[t]?.(value))) {
errors.push(`${where} should be ${types.join('|')}`);
return errors;
}
}
if (schema.enum && !schema.enum.includes(value)) {
errors.push(`${where}=${JSON.stringify(value)} is not one of ${schema.enum.join(', ')}`);
}
if (typeof value === 'number') {
if (typeof schema.minimum === 'number' && value < schema.minimum) errors.push(`${where} below ${schema.minimum}`);
if (typeof schema.maximum === 'number' && value > schema.maximum) errors.push(`${where} above ${schema.maximum}`);
}
if (Array.isArray(value)) {
if (typeof schema.minItems === 'number' && value.length < schema.minItems) errors.push(`${where} needs ≥${schema.minItems} items`);
if (schema.items) value.forEach((item, i) => errors.push(...validateArgs(schema.items, item, `${where}[${i}]`)));
}
if (TYPE_CHECKS.object(value) && (schema.properties || schema.required)) {
for (const key of schema.required || []) {
if (!(key in value)) errors.push(`${where}.${key} is required`);
}
for (const [key, v] of Object.entries(value)) {
const sub = schema.properties?.[key];
if (!sub) {
if (schema.additionalProperties === false) errors.push(`${where}.${key} is not a known argument`);
continue;
}
errors.push(...validateArgs(sub, v, `${where}.${key}`));
}
}
return errors;
}
/** Validate a tool call against the Realtime tool list. */
export function validateCall(tools, c) {
const tool = tools.find((t) => t.name === c.name);
if (!tool) return [`unknown tool ${c.name}`];
return validateArgs(tool.parameters, c.args ?? {});
}
// ── Coverage probe grading ─────────────────────────────────────────────────
/** Tools that only read state; they never count as acting on the request. */
export const LOOKUP_TOOLS = Object.freeze([
'get_entity_context',
'get_current_view_state',
'show_data_layers_menu',
'set_panel_open',
]);
export const REFUSAL_RE =
/\b(can(?:no|')t|cannot|unable|not able|don'?t have|do not have|doesn'?t have|does not have|isn'?t (?:available|supported|something)|aren'?t (?:available|supported)|not (?:available|supported|possible|yet)|no (?:access|way|tool|data|layer|feed|live|transit|weather|wind|cyclone|radar|lightning|receiver)|n'?t (?:connected|see any|find)|(?:isn'?t|is no|are no) (?:an? )?(?:\w+ ){0,2}(?:layer|feed|tool|data source|integration)s?|outside what i can|beyond what i can|only (?:have|support|show|cover)|unfortunately|sorry)\b/i;
/**
* Grade one coverage-probe turn.
*
* Order matters:
* 1. a call pattern in `correct` → correct
* 2. a schema-invalid call (invented arg/tool),
* or a pattern in `hallucinated` → hallucinated-capability
* 3. any other acting call (not lookup/neutral) → wrong-tool
* 4. no acting call: text matches `claimText`
* and no refusal wording → hallucinated-capability
* 5. no acting call: refusal wording or a question → honest-refusal
* 6. no acting call, no refusal → hallucinated-capability
* (it answered as if it had the data)
*
* `acceptable` = correct, or honest-refusal on an unsupported item.
*/
export function gradeCoverage(item, calls, text, tools = []) {
// Models write typographic apostrophes ("can’t"); fold them before matching.
const finalText = String(text || '').replace(/[\u2018\u2019\u02bc]/g, "'");
const verdict = (v, reason) => ({
verdict: v,
reason,
acceptable: v === 'correct' || (v === 'honest-refusal' && !item.supported),
});
for (const alternative of item.correct || []) {
if (alternative.every((m) => calls.some((c) => m(c)))) {
return verdict('correct', `matched ${alternative.map((m) => m.toolName || 'pattern').join('+')}`);
}
}
if (tools.length) {
for (const c of calls) {
const errors = validateCall(tools, c);
if (errors.length) return verdict('hallucinated-capability', `invalid ${c.name}: ${errors[0]}`);
}
}
for (const m of item.hallucinated || []) {
const hit = calls.find((c) => m(c));
if (hit) return verdict('hallucinated-capability', `faked capability via ${hit.name}(${JSON.stringify(hit.args).slice(0, 80)})`);
}
const neutral = new Set([...LOOKUP_TOOLS, ...(item.neutral || [])]);
const acting = calls.filter((c) => !neutral.has(c.name));
if (acting.length) return verdict('wrong-tool', `called ${acting.map((c) => c.name).join(',')}`);
const refused = REFUSAL_RE.test(finalText);
if (item.claimText && item.claimText.test(finalText) && !refused) {
return verdict('hallucinated-capability', 'asserted data it cannot have');
}
if (refused || /\?\s*$/.test(finalText.trim())) return verdict('honest-refusal', 'refused or asked');
if (!finalText.trim().replace(/\|/g, '').trim()) {
return verdict('wrong-tool', calls.length ? `only lookups/neutral calls (${calls.map((c) => c.name).join(',')}) and no answer` : 'no tool call and no answer');
}
return verdict('hallucinated-capability', 'answered without a tool or a refusal');
}
-35
View File
@@ -1,35 +0,0 @@
/**
* Runtime key lookup for the voice benchmark. Keys are read from the
* environment, then the macOS Keychain, at call time. They are never written
* to results, logs, or files.
*/
import { execFileSync } from 'node:child_process';
const KEYCHAIN = {
openai: { env: 'OPENAI_API_KEY', service: 'openai-api', account: 'api-key' },
gemini: { env: 'GEMINI_API_KEY', service: 'gemini-api', account: 'api-key' },
};
const cache = new Map();
/** Resolve a provider key from the environment, then the Keychain. */
export function providerKey(provider) {
if (cache.has(provider)) return cache.get(provider);
const spec = KEYCHAIN[provider];
if (!spec) throw new Error(`unknown key provider: ${provider}`);
let value = process.env[spec.env] || '';
if (!value && process.platform === 'darwin') {
try {
value = execFileSync(
'security',
['find-generic-password', '-s', spec.service, '-a', spec.account, '-w'],
{ encoding: 'utf8', stdio: ['ignore', 'pipe', 'ignore'] },
).trim();
} catch {
value = '';
}
}
if (!value) throw new Error(`${spec.env} is not available (env or Keychain)`);
cache.set(provider, value);
return value;
}
-600
View File
@@ -1,600 +0,0 @@
/**
* Shared voice-routing phrase tables.
*
* CORE_PHRASES is the routing table used by `scripts/qa-voice-routing.mjs`
* (layer 1) and the core suite of `scripts/qa-voice-bench.mjs`.
*
* `expect` tool name the model must call (string), every tool in an
* array, or `{ oneOf: [...] }` for documented acceptable variance.
* `expectNone` conversational turns that must NOT call tools.
* `args` spot-check subset matched against the first matching call's
* args (substring match for strings, exact for booleans/numbers).
* `argsByTool` the same spot check, keyed by tool, for multi-intent phrases.
* `typedArgs` benchmark-only matchers for typed arguments (exact enum
* values, layer ids, thresholds, sort direction). They feed the
* benchmark's full-call accuracy and never change the
* qa-voice-routing verdict.
*
* COVERAGE_PROBE is the newer-layers coverage probe: utterances about layers
* and questions that today's tool surface only partly supports. Each item is
* graded into correct / honest-refusal / hallucinated-capability / wrong-tool
* by `gradeCoverage` in ./grade.mjs.
*/
import { call, where } from './grade.mjs';
export const CORE_PHRASES = Object.freeze([
// — navigation & framing —
{ phrase: 'Take me to Tokyo', expect: 'fly_to_location', typedArgs: [where('fly_to_location', (a) => a.locationId === 'tokyo' || /tokyo/i.test(a.query || ''))] },
{ phrase: 'Fly to the Golden Gate Bridge', expect: 'fly_to_location' },
{ phrase: 'Go to Sixth Street in Austin', expect: 'fly_to_location' },
{ phrase: 'Show me the Alps from above', expect: { oneOf: ['fly_to_location', 'frame_overhead'] } },
{ phrase: 'Zoom in a bit', expect: 'adjust_camera_zoom', typedArgs: [where('adjust_camera_zoom', (a) => a.direction === 'in' && a.amount === 'little')] },
{ phrase: 'Zoom out a little', expect: 'adjust_camera_zoom', typedArgs: [where('adjust_camera_zoom', (a) => a.direction === 'out' && a.amount === 'little')] },
{ phrase: 'Zoom out to a globe view', expect: 'zoom_to_globe' },
{ phrase: 'Show me the whole earth', expect: 'zoom_to_globe' },
{ phrase: 'Frame the aircraft near us from overhead', expect: 'frame_overhead', typedArgs: [where('frame_overhead', (a) => a.target === 'flights')] },
// — the satellites trap: data layer, never basemap —
{ phrase: 'Show me the satellites', expect: { oneOf: ['set_layer_visibility', 'frame_overhead'] } },
{ phrase: 'Turn off the satellites', expect: 'set_layer_visibility', args: { layerId: 'satellites' }, typedArgs: [where('set_layer_visibility', (a) => a.layerId === 'satellites' && a.enabled === false)] },
{ phrase: 'Switch to Bing aerial', expect: 'set_map_stack', typedArgs: [where('set_map_stack', (a) => a.stack === 'bing-aerial')] },
{ phrase: 'Switch the basemap to OSM', expect: 'set_map_stack', typedArgs: [where('set_map_stack', (a) => a.stack === 'osm')] },
// — layers —
{ phrase: 'Turn on the flights layer', expect: 'set_layer_visibility', args: { layerId: 'flights' }, typedArgs: [where('set_layer_visibility', (a) => a.layerId === 'flights' && a.enabled === true)] },
{ phrase: 'Show me live vessels', expect: 'set_layer_visibility', typedArgs: [where('set_layer_visibility', (a) => a.layerId === 'ais-live-vessels' && a.enabled === true)] },
{ phrase: 'Turn on the fires layer', expect: 'set_layer_visibility', typedArgs: [where('set_layer_visibility', (a) => a.layerId === 'local-firms' && a.enabled === true)] },
{ phrase: 'Turn on street traffic', expect: 'set_layer_visibility', args: { layerId: 'traffic' }, typedArgs: [where('set_layer_visibility', (a) => a.layerId === 'traffic' && a.enabled === true)] },
{ phrase: 'Open the data layers menu', expect: 'show_data_layers_menu' },
{ phrase: 'Show me the datacenter layers', expect: 'show_data_layers_menu', typedArgs: [where('show_data_layers_menu', (a) => a.layerId === 'local-datacenters')] },
{ phrase: 'Turn on the datacenters layer', expect: 'set_layer_visibility', typedArgs: [where('set_layer_visibility', (a) => a.layerId === 'local-datacenters' && a.enabled === true)] },
// — visual styles & post-fx —
{ phrase: 'Give me night vision', expect: 'set_visual_style', typedArgs: [where('set_visual_style', (a) => a.style === 'surveillance')] },
{ phrase: 'Switch to thermal view', expect: 'set_visual_style', typedArgs: [where('set_visual_style', (a) => a.style === 'thermal')] },
{ phrase: 'Back to the normal look', expect: 'set_visual_style', typedArgs: [where('set_visual_style', (a) => a.style === 'normal')] },
{ phrase: 'Turn on bloom', expect: 'set_post_processing', typedArgs: [where('set_post_processing', (a) => a.bloom?.enabled === true)] },
{ phrase: 'Sharpen the image a touch', expect: 'set_post_processing', typedArgs: [where('set_post_processing', (a) => a.sharpen && a.sharpen.enabled !== false)] },
// — HUD / detection / panels —
{ phrase: 'Turn the HUD off', expect: 'set_hud', typedArgs: [where('set_hud', (a) => a.visible === 'off')] },
{ phrase: 'Switch to the tactical layout', expect: 'set_hud', typedArgs: [where('set_hud', (a) => a.layout === 'tactical')] },
{ phrase: 'Turn on detection', expect: 'set_detection', typedArgs: [where('set_detection', (a) => a.enabled !== false)] },
{ phrase: 'Set detection density to fifty percent', expect: 'set_detection', typedArgs: [where('set_detection', (a) => a.densityPct === 50 || a.mode === 'balanced')] },
// — context questions —
{ phrase: 'What am I looking at right now?', expect: 'get_entity_context' },
{ phrase: 'What city is this below us?', expect: 'get_entity_context' },
{ phrase: 'Is there anything interesting in view?', expect: 'get_entity_context' },
// — tracking —
// Context-free text turns may reasonably look before tracking; either
// routing is correct (production sessions always carry screen context).
{ phrase: 'Track that plane', expect: { oneOf: ['track_entity', 'get_entity_context'] } },
// The session instructions route "nearest" through analyst_query first
// (sortBy distance, limit 1); stubbed results make that first hop the whole
// observable turn.
{ phrase: 'Follow the nearest aircraft', expect: { oneOf: ['track_entity', 'get_entity_context', 'analyst_query'] } },
{ phrase: 'Stop tracking', expect: 'stop_tracking' },
// — CCTV / scenes / ISS —
{ phrase: 'Show me the nearest traffic camera', expect: 'control_cctv', typedArgs: [where('control_cctv', (a) => a.action === 'nearest')] },
{ phrase: 'Turn on the camera viewsheds', expect: 'control_cctv', typedArgs: [where('control_cctv', (a) => a.action === 'viewshed' && a.enabled !== false)] },
{ phrase: 'Play a news radio station near Austin', expect: 'control_radio', args: { action: 'select', category: 'news', locationId: 'austin' }, typedArgs: [where('control_radio', (a) => a.action === 'select' && a.category === 'news' && a.locationId === 'austin')] },
{ phrase: 'Turn on the radio', expect: 'control_radio', args: { action: 'play' }, typedArgs: [where('control_radio', (a) => a.action === 'play')] },
{ phrase: 'Set the radio volume to thirty percent', expect: 'control_radio', args: { action: 'volume', volumePct: 30 } },
{ phrase: 'Pause the radio', expect: 'control_radio', args: { action: 'pause' }, typedArgs: [where('control_radio', (a) => a.action === 'pause')] },
{ phrase: 'Stop the radio', expect: 'control_radio', args: { action: 'stop' }, typedArgs: [where('control_radio', (a) => a.action === 'stop')] },
{ phrase: 'When does the ISS pass over next?', expect: 'next_iss_pass' },
// — annotations —
{ phrase: 'Annotate the Texas State Capitol and its grounds', expect: 'annotate_map', typedArgs: [where('annotate_map', (a) => (a.annotations || []).some((x) => x.type === 'area'))] },
{ phrase: 'Outline the state of Texas', expect: 'annotate_map', typedArgs: [where('annotate_map', (a) => (a.annotations || []).some((x) => x.type === 'area'))] },
{ phrase: 'Outline Lady Bird Lake', expect: 'annotate_map', typedArgs: [where('annotate_map', (a) => (a.annotations || []).some((x) => x.type === 'area'))] },
{ phrase: 'Draw the walking route from the Capitol to Zilker Park', expect: 'annotate_map', typedArgs: [where('annotate_map', (a) => (a.annotations || []).some((x) => x.type === 'route' && (x.mode ?? 'walking') === 'walking'))] },
{ phrase: 'How far is the Eiffel Tower from the Louvre?', expect: 'annotate_map', typedArgs: [where('annotate_map', (a) => (a.annotations || []).some((x) => x.type === 'arrow'))] },
{ phrase: 'Clear the map', expect: 'clear_annotations' },
// — multi-intent (assert ALL tools fire before speech) —
{
phrase: 'Switch to night vision and turn on the flights layer',
expect: ['set_visual_style', 'set_layer_visibility'],
typedArgs: [where('set_visual_style', (a) => a.style === 'surveillance'), where('set_layer_visibility', (a) => a.layerId === 'flights' && a.enabled === true)],
},
{
phrase: 'Turn off the HUD and take me to Paris',
expect: ['set_hud', 'fly_to_location'],
typedArgs: [where('set_hud', (a) => a.visible === 'off'), where('fly_to_location', (a) => a.locationId === 'paris' || /paris/i.test(a.query || ''))],
},
{
phrase: 'Go to full planet view and then turn on the radio',
expect: ['zoom_to_globe', 'control_radio'],
argsByTool: { control_radio: { action: 'play' } },
},
// — camera verbs (tools #23/#24) —
{ phrase: 'Orbit around this area slowly', expect: 'move_camera', typedArgs: [where('move_camera', (a) => a.motion === 'orbit' && a.speed === 'slow')] },
{ phrase: 'Pan left a bit', expect: 'move_camera', typedArgs: [where('move_camera', (a) => a.motion === 'pan' && a.direction === 'left')] },
{ phrase: 'Stop moving the camera', expect: 'move_camera', typedArgs: [where('move_camera', (a) => a.motion === 'stop')] },
{ phrase: 'Fly the route we just drew', expect: 'fly_route' },
// — analyst queries (tool #22) —
{ phrase: 'How many flights are over Texas right now?', expect: 'analyst_query', typedArgs: [where('analyst_query', (a) => (a.layers || []).includes('flights') && a.scope?.kind === 'region' && /texas/i.test(a.scope?.name || ''))] },
{ phrase: 'Which ships are headed to Oakland?', expect: 'analyst_query', typedArgs: [where('analyst_query', (a) => (a.layers || []).includes('ais-live-vessels') && (a.filters || []).some((f) => f.field === 'destination' && /oakland/i.test(String(f.value))))] },
{ phrase: 'What is the biggest fire near Los Angeles?', expect: 'analyst_query', typedArgs: [where('analyst_query', (a) => (a.layers || []).includes('local-firms') && ['frp', 'acres'].includes(a.sortBy) && a.sortDir !== 'asc')] },
{ phrase: 'Is anything flying above forty thousand feet?', expect: 'analyst_query', typedArgs: [where('analyst_query', (a) => (a.layers || []).some((l) => l === 'flights' || l === 'military') && (a.filters || []).some((f) => f.field === 'altitudeM' && (f.op === 'gt' || f.op === 'gte') && Math.abs(Number(f.value) - 12192) <= 50))] },
// — negative controls: conversation must NOT tool-call —
{ phrase: 'How is your evening going?', expectNone: true },
{ phrase: 'Tell me a fun fact about maps', expectNone: true },
]);
// ── Coverage probe ──────────────────────────────────────────────────────────
//
// Text-mode port of the voice-coverage routing probe. The live probe ran each
// utterance against a real app page (setup steps, real tool results); this
// benchmark has no app, so every item states the scene it assumes in `scene`
// and the model sees only stub tool results. Grading therefore judges the
// ROUTING DECISION and the honesty of the final words, not app behaviour.
//
// supported true when today's tools can satisfy the ask (a refusal is a
// miss); false when the best possible outcome is an honest
// refusal, optionally after a neutral helper call.
// correct call patterns that satisfy the ask (any alternative).
// neutral extra tools that neither help nor hurt (navigation to the
// named place, enabling a related layer that exists).
// hallucinated call patterns that fake the capability (e.g. toggling the
// satellites DATA layer for "recent satellite imagery").
// claimText final-text pattern that asserts data the model cannot have.
//
// Any call whose arguments violate the tool schema (an invented layer id) is
// graded hallucinated-capability regardless of the item.
//
// Two rubrics, so a before/after comparison stays honest:
// v1 the original rubric, graded against the tool schema of 4b56d0e9
// (baseline-tools-4b56d0e9.json) — "by the old yardstick".
// v2 `item.v2` overrides (supported / correct / hallucinated / neutral),
// graded against the CURRENT production tools after the voice layer
// manifest and point-and-ask landed. Use
// coverageRubric(item, 'v2') to get the merged item.
const hasRoute = (c) =>
c.name === 'annotate_map' &&
Array.isArray(c.args?.annotations) &&
c.args.annotations.some((a) => a?.type === 'route');
/** The coverage item as graded under rubric 'v1' (original) or 'v2' (current tools). */
export function coverageRubric(item, version = 'v1') {
if (version !== 'v2' || !item.v2) return item;
return { ...item, ...item.v2 };
}
export const COVERAGE_PROBE = Object.freeze([
{
id: 'quakes-m5-week',
v2: { supported: true },
utterance: 'Show me earthquakes above magnitude 5 this week',
scene: 'globe, no layers on',
expect: 'enable earthquakes + analyst_query magnitude>=5; must say feed covers last 24h only (no week)',
supported: true,
correct: [
[call('analyst_query', { layers: 'earthquakes' })],
[call('set_layer_visibility', { layerId: 'earthquakes', enabled: true })],
],
neutral: ['zoom_to_globe'],
},
{
id: 'trains-london',
v2: { supported: true, correct: [[call('analyst_query', { layers: 'transit' })], [call('set_layer_visibility', { layerId: 'transit', enabled: true })]], hallucinated: [] },
utterance: 'How many trains are moving in London right now?',
scene: 'London',
expect: 'honest "no transit data for London / transit not queryable" — no invented count',
supported: false,
neutral: ['fly_to_location'],
hallucinated: [call('analyst_query')],
claimText: /\b\d+\s+(trains?|services)\b/i,
},
{
id: 'transit-on',
v2: { supported: true, correct: [[call('set_layer_visibility', { layerId: 'transit', enabled: true })]] },
utterance: 'Turn on transit',
scene: 'Austin',
expect: 'set_layer_visibility transit (no transit id in the voice enum today) — honest otherwise',
supported: false,
hallucinated: [call('set_layer_visibility', { layerId: 'traffic', enabled: true })],
},
{
id: 'ship-fastest-singapore',
v2: { supported: true },
utterance: 'Which ship near Singapore is fastest?',
scene: 'Singapore Strait, vessels layer on',
expect: 'analyst_query ais-live-vessels near Singapore sortBy speedKts desc',
supported: true,
correct: [[call('analyst_query', { layers: 'ais-live-vessels' })]],
neutral: ['fly_to_location', 'set_layer_visibility'],
},
{
id: 'recent-imagery',
v2: { supported: true, correct: [[call('set_layer_visibility', { layerId: 'recent-imagery', enabled: true })]] },
utterance: 'Show recent satellite imagery of this area',
scene: 'Austin',
expect: 'enable recent-imagery for current view (no tool today) — must NOT toggle satellites data layer or silently switch basemap',
supported: false,
hallucinated: [
call('set_layer_visibility', { layerId: 'satellites' }),
call('set_map_stack'),
call('frame_overhead', { target: 'satellites' }),
],
},
{
id: 'wind-here',
v2: { supported: true, correct: [[call('set_layer_visibility', { layerId: 'wind', enabled: true })]] },
utterance: "What's the wind doing here?",
scene: 'Austin',
expect: 'enable/inspect wind layer at view centre (no tool today) — honest otherwise',
supported: false,
claimText: /\b\d+\s*(mph|km\/?h|knots?|kts|m\/s|meters per second|miles per hour)\b/i,
},
{
id: 'cyclones-active',
v2: { supported: true, correct: [[call('analyst_query', { layers: 'weather-cyclones' })], [call('set_layer_visibility', { layerId: 'weather-cyclones', enabled: true })]] },
utterance: 'Are there any cyclones active right now?',
scene: 'globe',
expect: 'enable cyclones + list active storms (no tool today) — honest otherwise',
supported: false,
claimText: /\b(there (are|is)|currently)\b[^.?!]*\b(cyclones?|storms?|hurricanes?|typhoons?)\b/i,
},
{
id: 'fires-in-drawn-area',
// There is no drawn-area scope yet; the prompt says to say so. Any count
// presented as inside the drawn area is a faked capability.
v2: {
supported: false,
correct: [],
hallucinated: [where('analyst_query', (a) => (a.layers || []).includes('local-firms'))],
},
utterance: 'Count the fires inside the area I drew',
scene: 'Los Angeles, fires layer on, an area drawn around Angeles National Forest',
expect: 'analyst_query local-firms scoped to the drawn polygon (no polygon scope today)',
supported: true,
correct: [[call('analyst_query', { layers: 'local-firms' })]],
},
{
id: 'local-adsb-on',
v2: { supported: true },
utterance: 'Turn on the local ADS-B layer',
scene: 'globe',
expect: 'set_layer_visibility local-adsb; narrate receiver state honestly',
supported: true,
correct: [[call('set_layer_visibility', { layerId: 'local-adsb', enabled: true })]],
},
{
id: 'bikes-near-me',
v2: { supported: true, correct: [[call('analyst_query', { layers: 'bikeshare' })]] },
utterance: 'How many bikes are available near me?',
scene: 'New York, bikeshare layer on',
expect: 'count available bikes at nearby stations (bikeshare not in analyst today) — honest otherwise',
supported: false,
neutral: ['set_layer_visibility'],
claimText: /\b\d+\s+(available\s+)?(bikes?|bicycles?)\b/i,
},
{
id: 'rocket-launch-track',
v2: { supported: true, correct: [[call('set_layer_visibility', { layerId: 'rocket-launches', enabled: true })], [call('set_context_mode', { mode: 'space-missions' })], [call('set_context_mode', { mode: 'missions' })], [call('analyst_query', { layers: 'rocket-launches' })]] },
utterance: 'Track the next rocket launch',
scene: 'globe',
expect: 'enable rocket-launches / space-missions and select next launch (track_entity cannot) — honest otherwise',
supported: true,
correct: [
[call('set_layer_visibility', { layerId: 'rocket-launches', enabled: true })],
[call('set_context_mode', { mode: 'space-missions' })],
[call('set_context_mode', { mode: 'missions' })],
],
hallucinated: [call('track_entity')],
},
{
id: 'weather-radar-on',
v2: { supported: true, correct: [[call('set_layer_visibility', { layerId: 'weather-radar', enabled: true })]] },
utterance: 'Turn on the weather radar',
scene: 'Austin',
expect: 'set_layer_visibility weather-radar (not in the voice enum today) — honest otherwise',
supported: false,
hallucinated: [call('set_map_stack'), call('set_visual_style')],
},
{
id: 'lightning',
v2: { supported: true, correct: [[call('set_layer_visibility', { layerId: 'weather-lightning', enabled: true })]] },
utterance: 'Show me lightning strikes',
scene: 'globe',
expect: 'set_layer_visibility weather-lightning (not in the voice enum today) — honest otherwise',
supported: false,
},
{
id: 'military-bases',
v2: { supported: true, correct: [[call('set_layer_visibility', { layerId: 'military-installations', enabled: true })], [call('analyst_query', { layers: 'military-installations' })]], hallucinated: [] },
utterance: 'Show military bases near here',
scene: 'San Diego',
expect: 'set_layer_visibility military-installations (not in the voice enum today) — honest otherwise',
supported: false,
hallucinated: [call('annotate_map')],
},
{
id: 'driving-directions',
v2: { supported: true, correct: [[hasRoute], [call('set_layer_visibility', { layerId: 'directions', enabled: true })]] },
utterance: 'Give me driving directions from the Texas Capitol to the Austin airport',
scene: 'Austin',
expect: 'annotate_map route driving (or directions layer)',
supported: true,
correct: [[hasRoute]],
neutral: ['fly_to_location'],
},
{
id: 'satellites-overhead-count',
v2: { supported: true },
utterance: 'How many satellites are overhead right now?',
scene: 'globe, satellites layer on',
expect: 'analyst_query satellites (control: supported)',
supported: true,
correct: [
[call('analyst_query', { layers: 'satellites' })],
[call('frame_overhead', { target: 'satellites' })],
],
neutral: ['set_layer_visibility'],
},
{
id: 'biggest-quake-today',
v2: { supported: true },
utterance: "What's the biggest earthquake today?",
scene: 'globe, earthquakes layer on',
expect: 'analyst_query earthquakes sortBy magnitude desc (control: supported)',
supported: true,
correct: [[call('analyst_query', { layers: 'earthquakes' })]],
neutral: ['set_layer_visibility'],
},
{
id: 'flock-on',
v2: { supported: true },
utterance: 'Turn on the Flock cameras',
scene: 'Austin',
expect: 'set_layer_visibility alpr-cameras (control: supported)',
supported: true,
correct: [[call('set_layer_visibility', { layerId: 'alpr-cameras', enabled: true })]],
hallucinated: [call('control_cctv')],
},
{
id: 'alpr-count',
v2: { supported: true, correct: [[call('analyst_query', { layers: 'alpr-cameras' })]] },
utterance: 'How many license plate cameras are around here?',
scene: 'Austin, ALPR layer on',
expect: 'count alpr-cameras in view (not in analyst today) — honest otherwise',
supported: false,
neutral: ['set_layer_visibility'],
claimText: /\b\d+\s+(license|plate|alpr|flock|cameras?)\b/i,
},
{
id: 'nearest-bus',
v2: { supported: true, correct: [[call('analyst_query', { layers: 'transit' })]] },
utterance: 'Which bus is closest to the center of the screen?',
scene: 'Austin, transit layer on',
expect: 'nearest transit vehicle (no transit query/context today) — honest otherwise',
supported: false,
claimText: /\b(route|bus|line)\s+#?\d+\b/i,
},
{
id: 'weather-replay',
v2: { supported: false, hallucinated: [call('control_scene')] },
utterance: 'Replay the last hour of weather radar',
scene: 'globe',
expect: 'weather clock replay (no time tool today) — honest refusal',
supported: false,
},
{
id: 'receiver-count',
v2: { supported: true, correct: [[call('analyst_query', { layers: 'local-adsb' })]], hallucinated: [call('frame_overhead')] },
utterance: 'How many aircraft is my receiver picking up?',
scene: 'globe, no receiver connected',
expect: 'count local-adsb (not in analyst) — honest: no receiver connected',
supported: false,
neutral: ['set_layer_visibility'],
hallucinated: [call('analyst_query'), call('frame_overhead')],
claimText: /\b\d+\s+(aircraft|planes?|contacts?)\b/i,
},
{
id: 'nearest-traffic-camera',
v2: { supported: true },
utterance: 'Show me the nearest traffic camera',
scene: 'Austin',
expect: 'control_cctv nearest (control: supported)',
supported: true,
correct: [[call('control_cctv', { action: 'nearest' })]],
neutral: ['set_layer_visibility'],
},
{
id: 'hurricane-strongest',
v2: { supported: true, correct: [[call('set_layer_visibility', { layerId: 'weather-cyclones', enabled: true }), call('analyst_query', { layers: 'weather-cyclones' })], [call('analyst_query', { layers: 'weather-cyclones' })]], hallucinated: [] },
utterance: 'Turn on the hurricane tracks and fly me to the strongest storm',
scene: 'globe',
expect: 'enable cyclones + fly to strongest (no tool today) — honest otherwise',
supported: false,
hallucinated: [call('fly_to_location')],
},
]);
// ── Multi-turn dialogues ───────────────────────────────────────────────────
//
// Two-turn dialogues where turn 2 only makes sense given turn 1. Both turns
// run in ONE session/context. Turn 1's tools get the plausible canned results
// in `results` (anything not listed gets the neutral stub), so turn 2 has
// something real to refer back to. `first` and `second` use the CORE_PHRASES
// grading shape (expect / oneOf / expectNone / typedArgs); the headline score
// is turn 2's full-call accuracy.
const flightsTexas = {
ok: true,
count: 212,
scopeLabel: '212 flights over Texas',
feedProvenance: { overall: 'nominal' },
examples: [
{ callsign: 'AAL1402', operator: 'American Airlines', altitudeM: 10972, military: false },
{ callsign: 'SWA2210', operator: 'Southwest Airlines', altitudeM: 11278, military: false },
{ callsign: 'RCH451', operator: 'US Air Force', altitudeM: 9144, military: true },
],
coverage: 'Counts cover data loaded by enabled layers.',
};
const shipsOakland = {
ok: true,
count: 4,
scopeLabel: '4 ships headed to Oakland',
feedProvenance: { overall: 'nominal' },
examples: [
{ name: 'MAERSK KENSINGTON', destination: 'OAKLAND', speedKts: 14.2, distanceKm: 180 },
{ name: 'EVER LOVELY', destination: 'OAKLAND', speedKts: 16.8, distanceKm: 410 },
{ name: 'APL SINGAPURA', destination: 'OAKLAND', speedKts: 12.1, distanceKm: 62 },
],
};
export const DIALOGUES = Object.freeze([
{
id: 'flights-then-military',
turns: ['How many flights are over Texas right now?', 'Now just the military ones'],
results: { analyst_query: [flightsTexas, { ...flightsTexas, count: 9, scopeLabel: '9 military aircraft over Texas', examples: [flightsTexas.examples[2]] }] },
first: { expect: 'analyst_query' },
second: {
expect: 'analyst_query',
typedArgs: [where('analyst_query', (a) => a.followUp === true || (a.layers || []).includes('military') || (a.filters || []).some((f) => f.field === 'military'))],
},
},
{
id: 'quakes-then-biggest',
turns: ['Turn on the earthquakes layer', 'Take me to the biggest one'],
results: {
set_layer_visibility: [{ ok: true, layerId: 'earthquakes', enabled: true, feedState: 'nominal', count: 143 }],
analyst_query: [null, { ok: true, count: 143, results: [{ id: 'us7000abcd', place: 'Kermadec Islands', magnitude: 6.4, latitude: -29.9, longitude: -177.8 }] }],
},
first: { expect: 'set_layer_visibility', typedArgs: [where('set_layer_visibility', (a) => a.layerId === 'earthquakes' && a.enabled === true)] },
second: {
expect: { oneOf: ['track_entity', 'analyst_query'] },
typedArgs: [(c) =>
(c.name === 'track_entity' && /quake|biggest|largest|strongest|kermadec/i.test(c.args?.query || '')) ||
(c.name === 'analyst_query' && (c.args?.layers || []).includes('earthquakes') && c.args?.sortBy === 'magnitude' && c.args?.sortDir !== 'asc')],
},
},
{
id: 'track-then-stop',
turns: ['Track UAL428', 'Stop'],
results: {
track_entity: [{ ok: true, tracking: 'UAL428', layerId: 'flights' }],
move_camera: [null, { ok: true, stopped: false, tracking: true, note: 'No camera motion was active; an entity is being tracked.' }],
},
first: { expect: 'track_entity', typedArgs: [where('track_entity', (a) => /UAL\s*428/i.test(a.query || ''))] },
second: { expect: 'stop_tracking' },
},
{
id: 'paris-then-closer',
turns: ['Take me to Paris', 'Get a little closer'],
results: { fly_to_location: [{ ok: true, arrived: true, place: 'Paris, France' }] },
first: { expect: 'fly_to_location' },
second: { expect: 'adjust_camera_zoom', typedArgs: [where('adjust_camera_zoom', (a) => a.direction === 'in')] },
},
{
id: 'thermal-then-back',
turns: ['Switch to thermal view', 'Actually, put it back the way it was'],
results: { set_visual_style: [{ ok: true, style: 'thermal', previousStyle: 'normal' }] },
first: { expect: 'set_visual_style', typedArgs: [where('set_visual_style', (a) => a.style === 'thermal')] },
second: { expect: 'set_visual_style', typedArgs: [where('set_visual_style', (a) => a.style === 'normal')] },
},
{
id: 'radio-then-volume',
turns: ['Turn on the radio', 'Turn it down to twenty percent'],
results: { control_radio: [{ ok: true, prepared: true, action: 'play', station: 'KUT 90.5 Austin' }] },
first: { expect: 'control_radio', typedArgs: [where('control_radio', (a) => a.action === 'play')] },
second: { expect: 'control_radio', typedArgs: [where('control_radio', (a) => a.action === 'volume' && a.volumePct === 20)] },
},
{
id: 'mark-then-route',
turns: ['Mark the Texas Capitol on the map', 'Now draw the walking route from there to Zilker Park'],
results: { annotate_map: [{ ok: true, placed: [{ label: 'Texas State Capitol', type: 'pin' }] }] },
first: { expect: 'annotate_map' },
second: {
expect: 'annotate_map',
typedArgs: [where('annotate_map', (a) => (a.annotations || []).some((x) =>
x.type === 'route' && (x.mode ?? 'walking') === 'walking' && /capitol/i.test(x.points?.[0]?.target || '')))],
},
},
{
id: 'ships-then-closest',
turns: ['Which ships are headed to Oakland?', 'Which of those is closest?'],
results: { analyst_query: [shipsOakland, { ...shipsOakland, count: 1, examples: [shipsOakland.examples[2]] }] },
first: { expect: 'analyst_query' },
second: { expect: 'analyst_query', typedArgs: [where('analyst_query', (a) => a.followUp === true && a.sortBy === 'distance')] },
},
{
id: 'flights-then-thanks',
turns: ['Turn on the flights layer', "Thanks, that's perfect"],
results: { set_layer_visibility: [{ ok: true, layerId: 'flights', enabled: true, feedState: 'nominal', count: 1840 }] },
first: { expect: 'set_layer_visibility' },
second: { expectNone: true },
},
]);
/** Canned result for a dialogue tool call, or null for the neutral stub. */
export function dialogueResult(dialogue, toolName, turn) {
const byTurn = dialogue.results?.[toolName];
if (!byTurn) return null;
return byTurn[turn] ?? null;
}
// ── New capabilities (point-and-ask, layer reach) ─────────────────────────
//
// ROUTING_PHRASES is the qa-voice-routing table, so that harness runs exactly
// its own table. The benchmark runs CAPABILITY_EXTRAS as the separate
// `capability` suite so the core suite stays comparable across tool versions.
/** The full qa-voice-routing table. */
export const ROUTING_PHRASES = CORE_PHRASES;
// Pointer context items in the app's own format (src/voice/pointerContext.js
// output), sent as an inert system item just before the user's words.
const AIRCRAFT_POINTER = {
type: 'pointer_context', at: 'speech_start',
target: { kind: 'aircraft', layer: 'flights', label: 'UPS793' },
ground: { lat: 30.3121, lon: -97.6013 }, screen: { x: 0.608, y: 0.408 },
};
const GROUND_POINTER = {
type: 'pointer_context', at: 'speech_start',
ground: { lat: 30.2849, lon: -97.7341 }, screen: { x: 0.427, y: 0.547 },
};
const isPointer = (v) => String(v || '').toLowerCase() === 'pointer';
const flightsInView = {
ok: true,
count: 3,
scopeLabel: 'in view',
items: [
{ n: 1, id: 'a1b2c3', callsign: 'UAL428', layerKey: 'flights', lat: 30.2, lon: -97.7 },
{ n: 2, id: 'a4b5c6', callsign: 'SWA1201', layerKey: 'flights', lat: 30.3, lon: -97.6 },
{ n: 3, id: 'a7b8c9', callsign: 'AAL88', layerKey: 'flights', lat: 30.1, lon: -97.8 },
],
};
/**
* Extra single-turn phrases (with an optional pointer `context`) and two-turn
* dialogues for the new capabilities. Items with `turns` are graded like
* DIALOGUES (turn 2 full-call); the rest like CORE_PHRASES.
*/
export const CAPABILITY_EXTRAS = Object.freeze([
{ phrase: 'Tell me about this', context: AIRCRAFT_POINTER, expect: 'get_entity_context', typedArgs: [where('get_entity_context', (a) => isPointer(a.scope))] },
{ phrase: 'How many flights are around here?', context: GROUND_POINTER, expect: 'analyst_query', typedArgs: [where('analyst_query', (a) => isPointer(a.scope?.kind))] },
{ phrase: 'Take me there', context: GROUND_POINTER, expect: 'fly_to_location', typedArgs: [where('fly_to_location', (a) => isPointer(a.query))] },
{ phrase: 'Track that one', context: AIRCRAFT_POINTER, expect: 'track_entity', typedArgs: [where('track_entity', (a) => isPointer(a.query))] },
{ phrase: 'Turn on the weather radar', expect: 'set_layer_visibility', typedArgs: [where('set_layer_visibility', (a) => a.layerId === 'weather-radar' && a.enabled === true)] },
{ phrase: 'Are there any hurricanes right now?', expect: { oneOf: ['analyst_query', 'set_layer_visibility'] }, typedArgs: [(c) => (c.args?.layers || []).includes('weather-cyclones') || c.args?.layerId === 'weather-cyclones'] },
{ phrase: 'Which bus is closest to the middle of the screen?', expect: 'analyst_query', typedArgs: [where('analyst_query', (a) => (a.layers || []).includes('transit'))] },
{
id: 'flights-then-second',
turns: ['How many flights are in view?', 'Track the second one'],
results: { analyst_query: [flightsInView] },
first: { expect: 'analyst_query' },
second: { expect: 'track_entity', typedArgs: [where('track_entity', (a) => Number(a.referent) === 2 || /SWA\s*1201/i.test(a.query || ''))] },
},
]);
-58
View File
@@ -1,58 +0,0 @@
/**
* Cost estimation for benchmark turns (USD). Pure.
*
* OpenAI Realtime: the app's own rate table (`src/voice/voiceCost.js`).
* `gpt-realtime-2.1` has no registry entry, so it resolves to the priciest
* known table (gpt-realtime-2) — which matches OpenAI's published 2.1 rates
* as read on 2026-09-23 ($4 / $0.40 cached / $24 per 1M text tokens).
*
* Gemini Live: https://ai.google.dev/gemini-api/docs/pricing, read 2026-09-23,
* gemini-3.8-live paid tier, USD per 1M tokens. Thinking tokens bill as text
* output. The native-audio Live model rejects TEXT output, so the benchmark
* requests AUDIO and the spoken confirmation bills at the audio-output rate.
*/
import { estimateUsageCostUsd, resolveVoiceModelById } from '../../src/voice/voiceCost.js';
export const GEMINI_RATES = Object.freeze({
'gemini-3.8-live': Object.freeze({
textInput: 0.75,
audioInput: 3.0,
textOutput: 4.5,
audioOutput: 12.0,
verifiedOn: '2026-09-23',
}),
});
/** Cost of one Realtime `response.usage` for `model`. */
export function openAiUsageCostUsd(model, usage) {
return estimateUsageCostUsd(usage, resolveVoiceModelById(model).rates);
}
const modalityCount = (details, modality) =>
(Array.isArray(details) ? details : [])
.filter((d) => String(d?.modality || '').toUpperCase() === modality)
.reduce((sum, d) => sum + (Number(d?.tokenCount) || 0), 0);
/**
* Cost of one Gemini Live `usageMetadata`. Unattributed prompt tokens bill
* as audio input and unattributed response tokens as audio output — the
* conservative direction for a spend cap.
*/
export function geminiUsageCostUsd(model, usage) {
const rates = GEMINI_RATES[model] || GEMINI_RATES['gemini-3.8-live'];
if (!usage) return 0;
const prompt = Number(usage.promptTokenCount) || 0;
const response = Number(usage.responseTokenCount) || 0;
const thoughts = Number(usage.thoughtsTokenCount) || 0;
const textIn = modalityCount(usage.promptTokensDetails, 'TEXT');
const audioIn = Math.max(0, prompt - textIn);
const textOut = modalityCount(usage.responseTokensDetails, 'TEXT');
const audioOut = Math.max(0, response - textOut);
const usd =
(textIn * rates.textInput +
audioIn * rates.audioInput +
(textOut + thoughts) * rates.textOutput +
audioOut * rates.audioOutput) /
1_000_000;
return Number.isFinite(usd) && usd > 0 ? usd : 0;
}
-36
View File
@@ -1,36 +0,0 @@
/**
* Capture the exact Realtime session config production mints.
*
* Rather than re-typing the session fields, this drives the production token
* handler (`server/providers/openai/realtime.js`) with a fake upstream fetch
* and returns the body it would have POSTed to OpenAI. Instructions, tools,
* tool_choice, reasoning, and truncation therefore cannot drift from what a
* real GEV mic session receives.
*/
import { createRealtimeTokenHandler } from '../../server/providers/openai/realtime.js';
import { realtimeInstructions } from '../../server/providers/openai/instructions.js';
import { GEV_REALTIME_TOOLS } from '../../server/providers/openai/tools.js';
export { realtimeInstructions, GEV_REALTIME_TOOLS };
/** Return production's `{ session: {...} }` client-secret body for `model`. */
export async function captureProductionSession(model) {
let captured = null;
const handler = createRealtimeTokenHandler({
resolveApiKey: () => 'bench-placeholder',
models: { standard: model, mini: model },
fetchImpl: async (_url, init) => {
captured = JSON.parse(init.body);
return new Response('{}', { status: 200, headers: { 'content-type': 'application/json' } });
},
});
const res = {
statusCode: 0,
headers: {},
setHeader(k, v) { this.headers[k] = v; },
end() {},
};
await handler({ method: 'POST', url: '/api/realtime/token', headers: {}, socket: {} }, res);
if (!captured?.session) throw new Error('production token handler did not produce a session config');
return captured.session;
}
-272
View File
@@ -1,272 +0,0 @@
/**
* Pure aggregation of benchmark rows into summary.json / summary.md.
* Rows: `{ type:'run', ... }` run metadata and `{ type:'turn', ... }` turns.
*/
export function percentile(values, p) {
const xs = values.filter((v) => typeof v === 'number' && Number.isFinite(v)).sort((a, b) => a - b);
if (!xs.length) return null;
const idx = Math.min(xs.length - 1, Math.max(0, Math.ceil((p / 100) * xs.length) - 1));
return xs[idx];
}
const pct = (n, d) => (d ? Math.round((1000 * n) / d) / 10 : null);
const sum = (xs) => xs.reduce((a, b) => a + (Number(b) || 0), 0);
const mean = (xs) => {
const v = xs.filter((x) => typeof x === 'number' && Number.isFinite(x));
return v.length ? sum(v) / v.length : null;
};
const round = (x, d = 0) => (x === null || x === undefined ? null : Math.round(x * 10 ** d) / 10 ** d);
export const COVERAGE_VERDICTS = Object.freeze(['correct', 'honest-refusal', 'hallucinated-capability', 'wrong-tool']);
export function modelKey(row) {
return `${row.provider}:${row.model}${row.variant ? `#${row.variant}` : ''}`;
}
/** Aggregate turn rows into one summary entry per provider×model(×variant). */
export function summarize(rows) {
const runs = rows.filter((r) => r.type === 'run');
const turns = rows.filter((r) => r.type === 'turn');
const keys = [...new Set(turns.map(modelKey))];
const models = keys.map((key) => {
const mine = turns.filter((t) => modelKey(t) === key);
const core = mine.filter((t) => t.suite === 'core');
const cov = mine.filter((t) => t.suite === 'coverage');
const dia = mine.filter((t) => t.suite === 'dialogue');
const capAll = mine.filter((t) => t.suite === 'capability');
// Latency percentiles stay on the core + coverage phrase sets so runs of
// different builds remain comparable.
const single = mine.filter((t) => t.suite === 'core' || t.suite === 'coverage');
const covV2 = cov.filter((t) => t.gradeV2);
const reps = [...new Set(core.map((t) => t.rep))].sort((a, b) => a - b);
const withCall = single.filter((t) => t.tFirstCallMs !== null && t.tFirstCallMs !== undefined);
const typed = core.filter((t) => t.grade?.typedArgsOk !== null && t.grade?.typedArgsOk !== undefined);
const diaFailures = dia.filter((t) => !t.grade?.ok).map((t) => ({
id: t.phraseId, rep: t.rep, expect: t.expect,
turn1: (t.turn1?.calls || []).map((c) => `${c.name}(${JSON.stringify(c.args).slice(0, 80)})`).join(', ') || '(none)',
turn2: (t.calls || []).map((c) => `${c.name}(${JSON.stringify(c.args).slice(0, 80)})`).join(', ') || '(none)',
}));
const coverageV2Counts = Object.fromEntries(COVERAGE_VERDICTS.map((v) => [v, covV2.filter((t) => t.gradeV2?.verdict === v).length]));
const capFailures = {};
for (const t of capAll.filter((x) => !x.grade?.fullCallOk)) {
const k = t.phraseId;
capFailures[k] = capFailures[k] || { id: k, expect: t.expect, fails: 0, of: capAll.filter((x) => x.phraseId === k).length, called: [] };
capFailures[k].fails += 1;
capFailures[k].called.push((t.calls || []).map((c) => `${c.name}(${JSON.stringify(c.args).slice(0, 70)})`).join(', ') || '(none)');
}
const coverageCounts = Object.fromEntries(COVERAGE_VERDICTS.map((v) => [v, cov.filter((t) => t.grade?.verdict === v).length]));
const coreFailures = {};
for (const t of core.filter((x) => !x.grade?.ok)) {
const k = t.phrase;
coreFailures[k] = coreFailures[k] || { phrase: k, expect: t.expect, fails: 0, of: core.filter((x) => x.phrase === k).length, called: [] };
coreFailures[k].fails += 1;
coreFailures[k].called.push((t.calls || []).map((c) => c.name).join(',') || '(none)');
}
const covByItem = {};
for (const t of cov) {
covByItem[t.phraseId] = covByItem[t.phraseId] || { id: t.phraseId, utterance: t.phrase, supported: t.supported, verdicts: [] };
covByItem[t.phraseId].verdicts.push(t.grade?.verdict);
}
const run = runs.filter((r) => modelKey(r) === key).at(-1) || {};
const local = mine.filter((t) => t.local);
return {
key,
provider: mine[0].provider,
model: mine[0].model,
variant: mine[0].variant || null,
settings: run.settings || null,
machine: run.machine || null,
warmup: run.warmup || null,
turns: mine.length,
core: {
n: core.length,
reps: reps.length,
accuracyPct: pct(core.filter((t) => t.grade?.ok).length, core.length),
toolAccuracyPct: pct(core.filter((t) => t.grade?.toolOk).length, core.length),
fullCallAccuracyPct: pct(core.filter((t) => t.grade?.fullCallOk).length, core.length),
typedArgsAccuracyPct: pct(typed.filter((t) => t.grade?.typedArgsOk).length, typed.length),
typedArgsN: typed.length,
argsAccuracyPct: pct(core.filter((t) => t.grade?.argsOk === true).length, core.filter((t) => t.grade?.argsOk !== null && t.grade?.argsOk !== undefined).length),
firstCallAccuracyPct: pct(core.filter((t) => t.grade?.firstCallOk).length, core.length),
falsePositives: core.filter((t) => t.grade?.falsePositive).length,
negativeTurns: core.filter((t) => t.expectNone).length,
perRepAccuracyPct: reps.map((rep) => {
const r = core.filter((t) => t.rep === rep);
return pct(r.filter((t) => t.grade?.ok).length, r.length);
}),
failures: Object.values(coreFailures).sort((a, b) => b.fails - a.fails),
},
coverage: {
n: cov.length,
counts: coverageCounts,
acceptablePct: pct(cov.filter((t) => t.grade?.acceptable).length, cov.length),
supportedCorrectPct: pct(cov.filter((t) => t.supported && t.grade?.verdict === 'correct').length, cov.filter((t) => t.supported).length),
unsupportedHonestPct: pct(cov.filter((t) => !t.supported && t.grade?.acceptable).length, cov.filter((t) => !t.supported).length),
items: Object.values(covByItem),
},
coverageV2: {
n: covV2.length,
counts: coverageV2Counts,
acceptablePct: pct(covV2.filter((t) => t.gradeV2?.acceptable).length, covV2.length),
},
capability: {
n: capAll.length,
toolPct: pct(capAll.filter((t) => t.grade?.toolOk).length, capAll.length),
fullPct: pct(capAll.filter((t) => t.grade?.fullCallOk).length, capAll.length),
failures: Object.values(capFailures).sort((a, b) => b.fails - a.fails),
},
prompt: run.prompt || null,
dialogue: {
n: dia.length,
turn2FullPct: pct(dia.filter((t) => t.grade?.fullCallOk).length, dia.length),
turn2ToolPct: pct(dia.filter((t) => t.grade?.toolOk).length, dia.length),
turn1FullPct: pct(dia.filter((t) => t.grade?.turn1Ok).length, dia.length),
turn2ToolCallP50: percentile(dia.map((t) => t.tFirstCallMs), 50),
failures: diaFailures,
},
latencyMs: {
scope: 'single-turn phrases (core + coverage); send → event, text mode',
firstTokenP50: percentile(single.map((t) => t.tFirstTokenMs), 50),
firstTokenP90: percentile(single.map((t) => t.tFirstTokenMs), 90),
toolCallP50: percentile(withCall.map((t) => t.tFirstCallMs), 50),
toolCallP90: percentile(withCall.map((t) => t.tFirstCallMs), 90),
turnDoneP50: percentile(single.map((t) => t.tDoneMs), 50),
turnDoneP90: percentile(single.map((t) => t.tDoneMs), 90),
setupP50: percentile(single.map((t) => t.setupMs), 50),
},
tokens: {
input: sum(mine.map((t) => t.usage?.inputTokens)),
cached: sum(mine.map((t) => t.usage?.cachedTokens)),
output: sum(mine.map((t) => t.usage?.outputTokens)),
reasoning: sum(mine.map((t) => t.usage?.reasoningTokens)),
inputPerTurnMean: round(mean(mine.map((t) => t.usage?.inputTokens))),
inputPerRequestMean: round(mean(single.filter((t) => t.responses > 0 && t.usage?.inputTokens).map((t) => t.usage.inputTokens / t.responses))),
},
costUsd: round(sum(mine.map((t) => t.costUsd)), 4),
costPerTurnUsd: mine.length ? round(sum(mine.map((t) => t.costUsd)) / mine.length, 5) : null,
errors: mine.filter((t) => t.errors?.length).length,
timeouts: mine.filter((t) => t.timeout).length,
local: local.length
? {
prefillTokSMean: round(mean(local.map((t) => t.local.prefillTokS)), 1),
prefillTokSP50: round(percentile(local.map((t) => t.local.prefillTokS), 50), 0),
decodeTokSMean: round(mean(local.map((t) => t.local.decodeTokS)), 1),
decodeTokSP50: round(percentile(local.map((t) => t.local.decodeTokS), 50), 1),
decodeTokSP90: round(percentile(local.map((t) => t.local.decodeTokS), 90), 1),
}
: null,
};
});
return {
generatedAt: new Date().toISOString(),
cloudSpendUsd: round(sum(turns.filter((t) => t.cloud).map((t) => t.costUsd)), 4),
models,
};
}
const machineBrief = (machine) => {
if (!machine) return '—';
const load1 = String(machine.loadavg || '').split(' ')[0] || '?';
const busy = (machine.busyProcesses || []).filter((p) => Number(p.match(/([\d.]+)%cpu/)?.[1]) >= 100)
.map((p) => p.replace(/ Helper \(GPU\)/, ' GPU').replace(/Google Chrome for Testing/, 'Chrome-for-Testing'));
return `load ${load1}${busy.length ? `; ${busy.join(', ')}` : ''}`;
};
const fmtCounts = (block) => (block?.n
? `${block.counts.correct} / ${block.counts['honest-refusal']} / ${block.counts['hallucinated-capability']} / ${block.counts['wrong-tool']} (n=${block.n})`
: '—');
const fmtMs = (v) => (v === null || v === undefined ? '—' : v >= 10_000 ? `${(v / 1000).toFixed(1)}s` : `${Math.round(v)}ms`);
const fmtPct = (v) => (v === null || v === undefined ? '—' : `${v}%`);
/** Render summary.md from a summarize() result plus free-form notes. */
export function renderMarkdown(summary, { title = 'GEV voice-routing benchmark', notes = [], payload = [] } = {}) {
const out = [`# ${title}`, '', `Generated ${summary.generatedAt}. Cloud spend so far: **$${summary.cloudSpendUsd?.toFixed(2)}**.`, ''];
out.push('> **What this measures.** Tool CHOICE for text turns sent over each provider\'s API with production\'s instructions and tools, with stubbed (always-successful) tool results — the same method as `qa-voice-routing.mjs` layer 1. It is **not** end-to-end spoken accuracy (no speech recognition, no audio, no screenshot/scene context) and **not** spoken latency: every latency below is "send text → event", before any audio would be synthesized or played.', '');
out.push('## Routing accuracy', '');
out.push('| model | core tool-name | core full-call (tool + all arg checks) | typed args (n) | harness verdict (legacy) | first call right | false +ve | per-rep harness verdict | dialogue turn-2 full-call | dialogue turn-1 | coverage v1 C / HR / HC / WT | coverage v1 acceptable | coverage v2 C / HR / HC / WT | coverage v2 acceptable | capability tool / full-call |');
out.push('|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|');
for (const m of summary.models) {
const c = m.coverage.counts;
out.push(`| ${m.key} | ${fmtPct(m.core.toolAccuracyPct)} | ${fmtPct(m.core.fullCallAccuracyPct)} | ${fmtPct(m.core.typedArgsAccuracyPct)} (${m.core.typedArgsN}) | ${fmtPct(m.core.accuracyPct)} (n=${m.core.n}) | ${fmtPct(m.core.firstCallAccuracyPct)} | ${m.core.falsePositives}/${m.core.negativeTurns} | ${m.core.perRepAccuracyPct.map(fmtPct).join(', ')} | ${fmtPct(m.dialogue.turn2FullPct)} (n=${m.dialogue.n}) | ${fmtPct(m.dialogue.turn1FullPct)} | ${c.correct} / ${c['honest-refusal']} / ${c['hallucinated-capability']} / ${c['wrong-tool']} (n=${m.coverage.n}) | ${fmtPct(m.coverage.acceptablePct)} | ${fmtCounts(m.coverageV2)} | ${fmtPct(m.coverageV2.acceptablePct)} | ${m.capability.n ? `${fmtPct(m.capability.toolPct)} / ${fmtPct(m.capability.fullPct)} (n=${m.capability.n})` : '—'} |`);
}
out.push('', 'Core tool-name = right tool(s) called. Full-call = right tool(s) AND every argument check (the legacy `args` spot checks plus the benchmark\'s typed checks: exact layer ids, enum values, numeric thresholds, sort direction). Legacy harness verdict = exactly what `qa-voice-routing.mjs` would report. Coverage: C = correct, HR = honest refusal, HC = hallucinated capability, WT = wrong tool; "acceptable" = correct, or an honest refusal where today\'s tools cannot do it.', '');
out.push('## Latency and cost', '');
out.push('| model | p50 send → first token (text) | p50 send → tool call (text) | p90 send → tool call (text) | p50 send → turn done (text) | dialogue turn-2 p50 → tool call | session setup p50 | input tok/turn | $ total | $/turn |');
out.push('|---|---|---|---|---|---|---|---|---|---|');
for (const m of summary.models) {
out.push(`| ${m.key} | ${fmtMs(m.latencyMs.firstTokenP50)} | ${fmtMs(m.latencyMs.toolCallP50)} | ${fmtMs(m.latencyMs.toolCallP90)} | ${fmtMs(m.latencyMs.turnDoneP50)} | ${fmtMs(m.dialogue.turn2ToolCallP50)} | ${fmtMs(m.latencyMs.setupP50)} | ${m.tokens.inputPerTurnMean ?? '—'} | ${m.costUsd?.toFixed(3)} | ${m.costPerTurnUsd ?? '—'} |`);
}
out.push('', 'Latencies are over single-turn phrases (core + coverage). "Turn done" includes the continuation response after the stubbed tool result. Cloud session setup (WebSocket + session config) is excluded from every latency and shown separately.', '');
const locals = summary.models.filter((m) => m.local);
if (locals.length) {
out.push('## Local runtime (cold vs warm)', '');
out.push('| model | load (wall) | COLD: first request after load → tool call | cold prefill tok/s (full prompt) | prompt tokens | WARM p50 → tool call | WARM p90 → tool call | warm per-turn prefill tok/s p50 | decode tok/s p50 / p90 | machine at start |');
out.push('|---|---|---|---|---|---|---|---|---|---|');
for (const m of locals) {
const w = m.warmup || {};
out.push(`| ${m.key} | ${fmtMs(w.loadWallMs ?? w.loadMs)} | ${fmtMs(w.coldToolCallMs ?? w.warmupTurnMs)}${w.coldToolCallMs == null ? ' (turn, no tool)' : ''} | ${w.prefillTokS ? Math.round(w.prefillTokS) : '—'} | ${w.promptTokens ?? '—'} | ${fmtMs(m.latencyMs.toolCallP50)} | ${fmtMs(m.latencyMs.toolCallP90)} | ${m.local.prefillTokSP50 ?? '—'} | ${m.local.decodeTokSP50 ?? '—'} / ${m.local.decodeTokSP90 ?? '—'} | ${machineBrief(m.machine)} |`);
}
out.push('', 'COLD = the first request after the model is loaded (pays the full ~11k-token prompt prefill). WARM = every later turn. "Warm per-turn prefill tok/s" is prompt tokens ÷ prompt-eval time as the runtime reports it: a value in the tens of thousands means the cached prompt prefix was reused and only the new user text was prefilled; a value near the cold rate means the WHOLE prompt was re-prefilled on every turn. A real voice session pays the cold cost once per model load. "Machine at start" = 1-minute load average and processes above 100% CPU when the run began (other GPU/CPU users on the machine inflate local timings).', '');
}
if (payload.length) {
out.push('## Payload changes per provider', '');
for (const p of payload) out.push(`- ${p}`);
out.push('');
}
out.push('## Failures by model', '');
for (const m of summary.models) {
out.push(`### ${m.key}`, '');
out.push(`Settings: \`${JSON.stringify(m.settings)}\``, '');
if (m.errors || m.timeouts) out.push(`Errors: ${m.errors} turns, timeouts: ${m.timeouts}.`, '');
if (m.core.failures.length) {
out.push('Core misses (legacy verdict):', '');
for (const f of m.core.failures) out.push(`- "${f.phrase}" (expect ${f.expect}) — failed ${f.fails}/${f.of}; called: ${f.called.join(' ; ')}`);
out.push('');
}
if (m.capability.failures.length) {
out.push('Capability misses (full call):', '');
for (const f of m.capability.failures) out.push(`- "${f.id}" (expect ${f.expect}) — failed ${f.fails}/${f.of}; called: ${f.called.join(' ; ').slice(0, 400)}`);
out.push('');
}
if (m.dialogue.failures.length) {
out.push('Dialogue misses (turn 2):', '');
for (const f of m.dialogue.failures) out.push(`- ${f.id} rep ${f.rep} (expect ${f.expect}) — t1: ${f.turn1} ⇒ t2: ${f.turn2}`);
out.push('');
}
const badCov = m.coverage.items.filter((i) => i.verdicts.some((v) => v === 'hallucinated-capability' || v === 'wrong-tool' || (i.supported && v !== 'correct')));
if (badCov.length) {
out.push('Coverage misses:', '');
for (const i of badCov) out.push(`- "${i.utterance}" (${i.supported ? 'supported' : 'unsupported'}) — ${i.verdicts.join(', ')}`);
out.push('');
}
}
if (notes.length) {
out.push('## Notes', '');
for (const n of notes) out.push(`- ${n}`);
out.push('');
}
return out.join('\n');
}
/**
* Before → after table for two summarize() results (matched by model key).
* Coverage "before" is the v1 rubric (what the old tools could do); "after"
* shows both v1 (same yardstick) and v2 (the current tool surface).
*/
export function renderComparison(before, after, { beforeLabel = 'before', afterLabel = 'after' } = {}) {
const out = ['## Before → after', '', `Before = ${beforeLabel}; after = ${afterLabel}. Core accuracy = tool + all argument checks (full call). Coverage OK % = correct, or an honest refusal where the tools of that build cannot do it. HC = hallucinated-capability count. p50 = send → tool call (text). Prompt tokens = mean input tokens per model request (cloud: as billed, before cache discounts; local: the cold full-prompt count).`, ''];
out.push('| model | core full-call | coverage OK % (v1 → v1 / v2) | HC (v1 → v1 / v2) | dialogue turn-2 | capability full-call (after) | p50 → tool call | prompt tokens |');
out.push('|---|---|---|---|---|---|---|---|');
const hc = (block) => (block?.n ? block.counts['hallucinated-capability'] : null);
const promptTok = (m) => (m?.warmup?.promptTokens ?? m?.tokens?.inputPerRequestMean ?? null);
for (const a of after.models) {
const b = before.models.find((x) => x.key === a.key);
const arrow = (x, y, f) => `${f(x)} → ${f(y)}`;
const pctF = (v) => (v === null || v === undefined ? '—' : `${v}%`);
const msF = (v) => (v === null || v === undefined ? '—' : `${Math.round(v)}ms`);
const num = (v) => (v === null || v === undefined ? '—' : String(v));
out.push(`| ${a.key} | ${arrow(b?.core.fullCallAccuracyPct, a.core.fullCallAccuracyPct, pctF)} | ${pctF(b?.coverage.acceptablePct)} → ${pctF(a.coverage.acceptablePct)} / ${pctF(a.coverageV2.acceptablePct)} | ${num(hc(b?.coverage))} → ${num(hc(a.coverage))} / ${num(hc(a.coverageV2))} | ${arrow(b?.dialogue.turn2FullPct, a.dialogue.turn2FullPct, pctF)} | ${pctF(a.capability.fullPct)} | ${arrow(b?.latencyMs.toolCallP50, a.latencyMs.toolCallP50, msF)} | ${arrow(promptTok(b), promptTok(a), num)} |`);
}
out.push('');
return out.join('\n');
}
-21
View File
@@ -1,21 +0,0 @@
/**
* Stub tool results fed back to the model after each call. Pure.
*
* core suite: the neutral `{ ok:true }` stub qa-voice-routing has always
* used, so continuations behave exactly as in that harness.
* coverage suite: the same stub, except a call whose arguments violate the
* tool schema gets `{ ok:false, error }` — what the app's runner returns
* for an unknown layer id — so the model's follow-up words can be judged
* for honesty.
*/
import { validateCall } from './grade.mjs';
export const STUB_OK = Object.freeze({ ok: true, note: 'qa-harness stub result' });
export function stubResult(suite, c, tools) {
if (suite === 'coverage') {
const errors = validateCall(tools, c);
if (errors.length) return { ok: false, error: `Invalid arguments: ${errors.join('; ')}` };
}
return { ...STUB_OK };
}
-119
View File
@@ -1,119 +0,0 @@
/**
* Pure converters from the production Realtime tool list
* (`{ type:'function', name, description, parameters }`) to other providers'
* tool formats. Descriptions and schemas are carried over unchanged except
* where a target cannot represent them; every such change is listed below.
*/
const PERMISSIVE_SCALAR = Object.freeze(['string', 'number', 'boolean']);
const clone = (value) => JSON.parse(JSON.stringify(value));
/**
* Normalize a JSON schema for OpenAI-compatible local servers.
*
* Only one change: a property schema with no `type` (and no enum / anyOf /
* oneOf / $ref / const that would imply one) gets a permissive scalar union.
* `analyst_query.filters[].value` is `{}` in production — valid JSON Schema,
* but chat templates that print `type` for every property (Gemma's among
* them) crash on it.
*/
export function normalizeSchemaForChat(schema) {
const out = clone(schema);
const visit = (node, isRoot) => {
if (!node || typeof node !== 'object' || Array.isArray(node)) return;
const implied = ['type', 'enum', 'anyOf', 'oneOf', 'allOf', '$ref', 'const'].some((k) => k in node);
if (!isRoot && !implied) node.type = [...PERMISSIVE_SCALAR];
if (node.properties) for (const sub of Object.values(node.properties)) visit(sub, false);
if (node.items) visit(node.items, false);
for (const key of ['anyOf', 'oneOf', 'allOf']) if (Array.isArray(node[key])) node[key].forEach((s) => visit(s, false));
};
visit(out, true);
return out;
}
/**
* Every schema location normalizeSchemaForChat changes, as
* `{ tool, path, from, to }` — logged in summary.md so the per-provider
* payload difference is explicit rather than implied.
*/
export function chatNormalizationDiff(realtimeTools) {
const changes = [];
const walk = (before, after, tool, where) => {
if (!before || typeof before !== 'object' || Array.isArray(before)) return;
if (JSON.stringify(before.type) !== JSON.stringify(after.type)) {
changes.push({ tool, path: where, from: before.type === undefined ? '(untyped)' : before.type, to: after.type });
}
for (const [k, v] of Object.entries(before.properties || {})) walk(v, after.properties[k], tool, `${where}.${k}`);
if (before.items) walk(before.items, after.items, tool, `${where}[]`);
};
for (const tool of realtimeTools) walk(tool.parameters, normalizeSchemaForChat(tool.parameters), tool.name, tool.name);
return changes;
}
/** Realtime tools → Chat Completions tools (`{type:'function', function:{...}}`). */
export function toChatTools(realtimeTools, { normalize = true } = {}) {
return realtimeTools.map((tool) => ({
type: 'function',
function: {
name: tool.name,
description: tool.description,
parameters: normalize ? normalizeSchemaForChat(tool.parameters) : clone(tool.parameters),
},
}));
}
// Keywords the Gemini OpenAPI-subset `Schema` does not accept.
const GEMINI_DROP = new Set(['additionalProperties', '$schema', 'default', 'examples', '$id', 'const']);
/**
* Convert a JSON schema to the Gemini OpenAPI-subset `Schema`.
* Drops unsupported keywords (additionalProperties, $schema, default, ...);
* an untyped property becomes `anyOf` string|number|boolean; a type array
* becomes `anyOf` of single types.
*/
export function toGeminiSchema(schema) {
if (!schema || typeof schema !== 'object' || Array.isArray(schema)) return schema;
const out = {};
for (const [key, value] of Object.entries(schema)) {
if (GEMINI_DROP.has(key)) continue;
if (key === 'properties') {
out.properties = Object.fromEntries(Object.entries(value).map(([k, v]) => [k, toGeminiSchema(v)]));
} else if (key === 'items') {
out.items = toGeminiSchema(value);
} else if (key === 'anyOf' || key === 'oneOf') {
out.anyOf = value.map((s) => toGeminiSchema(s));
} else {
out[key] = clone(value);
}
}
if (Array.isArray(out.type)) {
const types = out.type;
delete out.type;
out.anyOf = types.map((t) => ({ type: t }));
}
const implied = ['type', 'enum', 'anyOf'].some((k) => k in out);
if (!implied) out.anyOf = PERMISSIVE_SCALAR.map((t) => ({ type: t }));
return out;
}
/**
* Realtime tools → Gemini `functionDeclarations`.
* mode 'jsonSchema' (default): pass the production schema verbatim through
* `parametersJsonSchema` — full JSON Schema, closest to prompt parity.
* mode 'openapi': convert to the OpenAPI-subset `parameters` field.
* A tool with no properties gets no parameters field in openapi mode (Gemini
* rejects an empty OBJECT schema there).
*/
export function toGeminiDeclarations(realtimeTools, { mode = 'jsonSchema' } = {}) {
return realtimeTools.map((tool) => {
const decl = { name: tool.name, description: tool.description };
const params = tool.parameters;
if (mode === 'jsonSchema') {
decl.parametersJsonSchema = clone(params);
} else if (params && Object.keys(params.properties || {}).length) {
decl.parameters = toGeminiSchema(params);
}
return decl;
});
}
-111
View File
@@ -1,111 +0,0 @@
#!/usr/bin/env node
/**
* Summarize voice turn spans from the Realtime debug log.
*
* Every closed voice turn writes one `turn.span` record (see
* src/voice/turnMetrics.js). This prints each span and p50/p90 figures for
* end-of-speech → first audio, the longest silence, tool time, preamble rate,
* spoken words and hedge hits.
*
* Usage:
* node scripts/voice-turn-spans.mjs [log.jsonl] [--since <ISO time>] [--session <id>] [--json]
*/
import fs from 'node:fs';
import path from 'node:path';
import { fileURLToPath } from 'node:url';
const root = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '..');
const args = process.argv.slice(2);
const option = (flag) => {
const index = args.indexOf(flag);
return index >= 0 ? args[index + 1] : null;
};
const positional = args.filter(
(arg, index) => !arg.startsWith('--') && !args[index - 1]?.startsWith('--'),
);
const logPath =
positional[0] || path.join(root, '.gev-logs', 'realtime-conversations.jsonl');
const since = option('--since') ? Date.parse(option('--since')) : null;
const session = option('--session');
const spans = [];
for (const line of fs.readFileSync(logPath, 'utf8').split('\n')) {
if (!line.includes('"turn.span"')) continue;
let record;
try {
record = JSON.parse(line);
} catch {
continue;
}
if (record.event !== 'turn.span') continue;
if (since && Date.parse(record.timestamp) < since) continue;
if (session && record.sessionId !== session) continue;
spans.push({
at: record.timestamp,
session: record.sessionId,
...record.payload,
});
}
const quantile = (values, q) => {
const sorted = values.filter(Number.isFinite).sort((a, b) => a - b);
if (!sorted.length) return null;
return sorted[Math.min(sorted.length - 1, Math.floor(q * sorted.length))];
};
const withTools = spans.filter((span) => span.tools?.length);
const slowToolTurns = withTools.filter((span) =>
span.tools.some((tool) => tool.ms >= 700),
);
const summary = {
turns: spans.length,
toolTurns: withTools.length,
firstAudioMs: {
p50: quantile(spans.map((s) => s.first_audio_ms), 0.5),
p90: quantile(spans.map((s) => s.first_audio_ms), 0.9),
},
maxSilenceMs: {
p50: quantile(spans.map((s) => s.max_silence_ms), 0.5),
p90: quantile(spans.map((s) => s.max_silence_ms), 0.9),
},
toolMs: {
p50: quantile(withTools.flatMap((s) => s.tools.map((t) => t.ms)), 0.5),
p90: quantile(withTools.flatMap((s) => s.tools.map((t) => t.ms)), 0.9),
},
preambleRate: withTools.length
? withTools.filter((s) => s.preamble).length / withTools.length
: null,
slowToolPreambleOrProgressRate: slowToolTurns.length
? slowToolTurns.filter((s) => s.preamble || s.progress_lines > 0).length /
slowToolTurns.length
: null,
answerWords: {
p50: quantile(spans.map((s) => s.answer_words), 0.5),
p90: quantile(spans.map((s) => s.answer_words), 0.9),
},
spokenWordsTotal: spans.reduce((sum, s) => sum + (s.spoken_words || 0), 0),
hedgeHits: spans.reduce((sum, s) => sum + (s.hedge_hits?.length || 0), 0),
};
if (args.includes('--json')) {
console.log(JSON.stringify({ summary, spans }, null, 2));
} else {
for (const span of spans) {
const tools = (span.tools || [])
.map((tool) => `${tool.name}:${tool.ms}ms`)
.join(',');
console.log(
[
span.at,
`first=${span.first_audio_ms ?? '-'}ms`,
`maxSilence=${span.max_silence_ms ?? '-'}ms`,
`preamble=${span.preamble ? 'y' : 'n'}`,
`progress=${span.progress_lines}`,
`words=${span.answer_words}/${span.spoken_words}`,
`hedges=${span.hedge_hits?.length || 0}`,
tools || '(no tools)',
span.answer_text ? `"${span.answer_text.slice(0, 80)}"` : '',
].join(' '),
);
}
console.log(JSON.stringify(summary, null, 2));
}
+179
View File
@@ -0,0 +1,179 @@
/**
* Serves the local MCP server at /mcp on the development and preview servers.
*
* This is local transport safety, not authentication: requests must come from
* this machine, carry no proxy forwarding headers, arrive while sharing is
* off, name a loopback host on the port they arrived at (blocking DNS
* rebinding) and, when a browser sends an Origin, come from that same host.
*/
import { isSharingEnabled } from '../../src/keySetupCore.mjs';
import { hasProxySignals } from '../../src/localRequestGate.mjs';
import { createMcpHttpHandler } from '../../src/tools/mcp/index.js';
import { createLocalMcpServer } from './server.js';
const LOOPBACK_HOSTS = new Set(['localhost', '127.0.0.1', '[::1]']);
const LOOPBACK_ADDRESSES = new Set(['127.0.0.1', '::1', '::ffff:127.0.0.1']);
const MAX_BODY_BYTES = 1024 * 1024;
/** How long a client may take to send its request body. */
const BODY_TIMEOUT_MS = 30 * 1000;
/**
* Whether a request may reach the local MCP server. `localPort` is the port
* the connection arrived at; `headers` are the request's, keyed by lower-case
* name; `env` is checked for launcher sharing.
*/
export function isLocalMcpRequest({
remoteAddress,
localPort,
host,
origin,
headers = {},
env = {},
}) {
if (!LOOPBACK_ADDRESSES.has(remoteAddress)) return false;
// A proxy or tunnel in front of the server can make a remote request look
// local, and sharing exposes the server beyond this machine.
if (hasProxySignals(headers) || isSharingEnabled(env)) return false;
let hostUrl;
try {
hostUrl = new URL(`http://${host}`);
} catch {
return false;
}
if (!LOOPBACK_HOSTS.has(hostUrl.hostname)) return false;
// The Host names the port the request reached, so tools are never pointed
// at another local service.
if (localPort != null && (hostUrl.port || '80') !== String(localPort))
return false;
if (origin == null) return true;
try {
const originUrl = new URL(origin);
return (
['http:', 'https:'].includes(originUrl.protocol) &&
originUrl.host === hostUrl.host
);
} catch {
return false;
}
}
/** Vite plugin that mounts the local MCP server at /mcp. */
export function localMcpPlugin({
createServer = createLocalMcpServer,
bodyTimeoutMs = BODY_TIMEOUT_MS,
} = {}) {
const handlers = new Map();
const handlerFor = (apiBase) => {
if (!handlers.has(apiBase))
handlers.set(apiBase, createMcpHttpHandler(createServer({ apiBase })));
return handlers.get(apiBase);
};
const install = (server) => {
server.middlewares.use('/mcp', async (req, res) => {
const host = req.headers.host || '';
if (
!isLocalMcpRequest({
remoteAddress: req.socket?.remoteAddress,
localPort: req.socket?.localPort,
host,
origin: req.headers.origin,
headers: req.headers,
env: process.env,
})
) {
res.writeHead(403, {
'Content-Type': 'application/json',
'Cache-Control': 'no-store',
});
res.end(
JSON.stringify({
error: 'The MCP server only accepts local requests',
}),
);
return;
}
// A client that disconnects before its answer cancels the tool call.
const disconnect = new AbortController();
const onClose = () => {
if (!res.writableFinished) disconnect.abort();
};
res.on('close', onClose);
try {
const body =
req.method === 'POST'
? await readBody(req, bodyTimeoutMs)
: undefined;
const request = new Request(`http://${host}/mcp`, {
method: req.method,
headers: Object.entries(req.headers).flatMap(([name, value]) =>
value === undefined ? [] : [[name, String(value)]],
),
body,
signal: disconnect.signal,
});
const response = await handlerFor(`http://${host}`)(request);
res.writeHead(response.status, Object.fromEntries(response.headers));
res.end(Buffer.from(await response.arrayBuffer()));
} catch (error) {
if (disconnect.signal.aborted) return;
if (res.headersSent) return res.destroy();
res.writeHead([408, 413].includes(error?.status) ? error.status : 400, {
'Content-Type': 'application/json',
'Cache-Control': 'no-store',
});
res.end(
JSON.stringify({
error:
error?.status === 413
? 'Request too large'
: error?.status === 408
? 'Request timed out'
: 'Bad request',
}),
);
} finally {
res.off('close', onClose);
}
});
};
return {
name: 'local-mcp',
configureServer: install,
configurePreviewServer: install,
};
}
function readBody(req, timeoutMs = BODY_TIMEOUT_MS) {
return new Promise((resolve, reject) => {
const chunks = [];
let size = 0;
let tooLarge = false;
// A client that stops sending must not hold the request open.
const timer = setTimeout(
() =>
reject(Object.assign(new Error('Request timed out'), { status: 408 })),
timeoutMs,
);
const fail = (error) => {
clearTimeout(timer);
reject(error);
};
req.on('data', (chunk) => {
size += chunk.length;
// Keep draining without storing, so the 413 response can still be sent.
if (tooLarge || size <= MAX_BODY_BYTES) {
if (!tooLarge) chunks.push(chunk);
return;
}
tooLarge = true;
chunks.length = 0;
fail(Object.assign(new Error('Request too large'), { status: 413 }));
});
req.on('end', () => {
clearTimeout(timer);
resolve(Buffer.concat(chunks));
});
req.on('error', fail);
});
}
+49
View File
@@ -0,0 +1,49 @@
/** The local MCP server: Core's tools over services backed by a running app. */
import { randomBytes } from 'node:crypto';
import { readFileSync } from 'node:fs';
import {
composeCatalog,
coreTools,
catalogForSurface,
} from '../../src/tools/index.js';
import { panelRuntime } from '../../src/app/globePanelRuntime.js';
import { createGlobePanelResource } from '../../src/tools/globePanel.js';
import { createMcpServer } from '../../src/tools/mcp/index.js';
import { DEFAULT_API_BASE, createLocalToolServices } from './services.js';
const INSTRUCTIONS =
"Tools answer questions from God's Eye View's live public data. Location " +
'tools take an area: a place name, a bbox, or lat/lon with radius_km. ' +
'Results are capped; check truncated and total before concluding there is nothing more. ' +
"Answers that can be shown in God's Eye View include data.view; to show one, call " +
'show_in_gods_eye_view with that view, adding layers, style, a camera or marks as ' +
"needed. It shows live God's Eye View where the client displays apps and " +
'returns a link everywhere.';
/** Construct the local MCP server for Core's tools. */
export function createLocalMcpServer({
apiBase = DEFAULT_API_BASE,
fetchImpl,
} = {}) {
const { version } = JSON.parse(
readFileSync(new URL('../../package.json', import.meta.url), 'utf8'),
);
// Each server makes a new key for its panel. The panel's page carries it
// and panel_request requires it; any client may read the page, so it keeps
// the tool from clients that only list it, and is not access control.
const panelKey = randomBytes(32).toString('base64url');
return createMcpServer({
catalog: catalogForSurface(
composeCatalog({
tools: coreTools,
services: createLocalToolServices({ apiBase, fetchImpl, panelKey }),
}),
'mcp',
),
name: 'gods-eye-view',
version,
instructions: INSTRUCTIONS,
resources: [createGlobePanelResource({ runtime: panelRuntime, panelKey })],
});
}
+64
View File
@@ -0,0 +1,64 @@
/**
* Local composition of tool services: Core's default services, pointed at a
* running God's Eye View server's `/api` routes.
*/
import { readFile } from 'node:fs/promises';
import { fileURLToPath } from 'node:url';
import { createToolServices } from '../../src/tools/services.js';
// Bundled data some sources read by module-relative URL.
const BUNDLED_DATA = new URL('../../src/data/local_data/', import.meta.url);
export const DEFAULT_API_BASE = 'http://localhost:4173';
/**
* Resolve the sources' relative `/api/...` requests against `apiBase`, and
* serve `file:` URLs inside the bundled data directory from disk, as a
* browser would load those module-relative assets.
*/
export function createApiFetch({
apiBase = DEFAULT_API_BASE,
fetchImpl = (...args) => globalThis.fetch(...args),
} = {}) {
const base = new URL(apiBase);
if (!['http:', 'https:'].includes(base.protocol))
throw new TypeError(`apiBase must be an http(s) URL: ${apiBase}`);
return (input, init) => {
const url = String(input instanceof Request ? input.url : input);
if (url.startsWith('file:')) return readBundled(url);
return fetchImpl(
typeof input === 'string' && input.startsWith('/')
? new URL(input, base)
: input,
init,
);
};
}
async function readBundled(url) {
const href = new URL(url).href;
if (!href.startsWith(BUNDLED_DATA.href) || href.includes('/../'))
return new Response(null, { status: 404 });
try {
const body = await readFile(fileURLToPath(href));
return new Response(body, {
headers: {
'Content-Type': href.endsWith('.json')
? 'application/json'
: 'application/octet-stream',
},
});
} catch {
return new Response(null, { status: 404 });
}
}
/** Construct every service Core's tools read, backed by the local server. */
export function createLocalToolServices(options = {}) {
return createToolServices({
fetchImpl: createApiFetch(options),
appUrl: options.apiBase ?? DEFAULT_API_BASE,
panelKey: options.panelKey,
});
}
+109
View File
@@ -0,0 +1,109 @@
/**
* Local MCP server over stdio for agents such as Claude Code and Codex.
*
* npm run mcp -- [--api-base http://localhost:4173]
*
* Reads newline-delimited JSON-RPC from stdin and writes responses to stdout.
* Tools read data from a running God's Eye View server (`npm run dev` or
* `npm run preview`) at the API base. Diagnostics go to stderr.
*/
import { createInterface } from 'node:readline';
import { fileURLToPath } from 'node:url';
import { parseErrorResponse } from '../../src/tools/mcp/index.js';
import { createLocalMcpServer } from './server.js';
import { DEFAULT_API_BASE } from './services.js';
/**
* A one-line description of a request for the diagnostic log: its method,
* and the tool or resource it names. Arguments and data are never logged.
*/
export function describeRequest(message) {
const method = typeof message?.method === 'string' ? message.method : null;
if (!method) return null;
const subject =
method === 'tools/call'
? message.params?.name
: method === 'resources/read'
? message.params?.uri
: null;
return typeof subject === 'string' ? `${method} ${subject}` : method;
}
/**
* Serve newline-delimited JSON-RPC until `input` ends. A client's
* `notifications/cancelled` aborts the named request, which then gets no
* response, as the protocol asks.
*/
export async function serveStdio(server, { input, output, log = () => {} }) {
const lines = createInterface({ input, crlfDelay: Infinity });
const pending = new Set();
const running = new Map();
for await (const line of lines) {
if (!line.trim()) continue;
const work = respond(server, line, log, running).then((response) => {
if (response) output.write(`${JSON.stringify(response)}\n`);
});
pending.add(work);
work.finally(() => pending.delete(work));
}
await Promise.all(pending);
}
async function respond(server, line, log, running) {
let message;
try {
message = JSON.parse(line);
} catch {
return parseErrorResponse();
}
if (message?.method === 'notifications/cancelled') {
running.get(message.params?.requestId)?.abort();
return null;
}
const described = describeRequest(message);
if (described) log(described);
const id = message?.id;
const cancellable = id !== undefined && id !== null && !running.has(id);
const controller = new AbortController();
if (cancellable) running.set(id, controller);
let response;
try {
response = await server.handle(message, { signal: controller.signal });
} finally {
if (cancellable) running.delete(id);
}
if (controller.signal.aborted) return null;
// A failed tool call answers the client, not the log; say why here too.
if (response?.result?.isError)
log(` ${described} failed: ${response.result.content?.[0]?.text}`);
return response;
}
/** Read `--api-base <url>` from command-line arguments. */
export function parseArgs(argv) {
const options = {};
for (let index = 0; index < argv.length; index += 1) {
if (argv[index] === '--api-base' && argv[index + 1])
options.apiBase = argv[(index += 1)];
else throw new Error(`Unknown argument: ${argv[index]}`);
}
return options;
}
if (process.argv[1] && fileURLToPath(import.meta.url) === process.argv[1]) {
try {
const options = parseArgs(process.argv.slice(2));
console.error(
`God's Eye View MCP server reading ${options.apiBase || DEFAULT_API_BASE}`,
);
await serveStdio(createLocalMcpServer(options), {
input: process.stdin,
output: process.stdout,
log: (line) => console.error(`<- ${line}`),
});
} catch (error) {
console.error(error.message);
process.exitCode = 1;
}
}
+33 -9
View File
@@ -25,6 +25,36 @@ let _openskyCacheMeta = null;
let _openskyCacheSourceEpochMs = null;
/** TTL for the OpenSky response cache (ms). */
const OPENSKY_CACHE_MS = 9000;
/** Per-attempt limit for the global snapshot; it normally arrives in ~1.5 s. */
const OPENSKY_ATTEMPT_TIMEOUT_MS = 10_000;
const OPENSKY_STATES_URL =
'https://opensky-network.org/api/states/all?extended=1';
/**
* Fetch and read the global snapshot, giving each attempt
* OPENSKY_ATTEMPT_TIMEOUT_MS and retrying once when an attempt times out or
* the connection fails. A second failure throws, which the proxy answers
* from its stale cache or the regional fallback.
* @param {Record<string, string>} headers Request headers.
* @returns {Promise<{upstream: Response, body: string}>}
*/
async function fetchOpenSkyStates(headers) {
for (let attempt = 1; ; attempt++) {
try {
const upstream = await fetch(OPENSKY_STATES_URL, {
headers,
signal: AbortSignal.timeout(OPENSKY_ATTEMPT_TIMEOUT_MS),
});
return { upstream, body: await upstream.text() };
} catch (error) {
if (attempt >= 2) throw error;
console.warn(
'[OpenSky] global snapshot attempt failed, retrying:',
error?.message || error,
);
}
}
}
// --- OpenSky credit governor (field-test fix 2026-07-06) -------------------
// The global /states/all this proxy fetches costs 4 CREDITS per call against
// OpenSky's ~4000/day authenticated budget — a day with the app open burned
@@ -107,6 +137,7 @@ export async function getOpenSkyToken() {
{
method: 'POST',
headers: { 'Content-Type': 'application/x-www-form-urlencoded' },
signal: AbortSignal.timeout(OPENSKY_ATTEMPT_TIMEOUT_MS),
body: `grant_type=client_credentials&client_id=${encodeURIComponent(clientId)}&client_secret=${encodeURIComponent(clientSecret)}`,
},
);
@@ -470,10 +501,7 @@ export function openSkyProxy() {
}
}
let upstream = await fetch(
'https://opensky-network.org/api/states/all?extended=1',
{ headers },
);
let { upstream, body } = await fetchOpenSkyStates(headers);
// Auto-mode fallback: if OAuth was rejected, retry with Basic credentials
if (
(upstream.status === 401 || upstream.status === 403) &&
@@ -485,15 +513,11 @@ export function openSkyProxy() {
Accept: 'application/json',
Authorization: `Basic ${Buffer.from(`${basicUser}:${basicPass}`).toString('base64')}`,
};
upstream = await fetch(
'https://opensky-network.org/api/states/all?extended=1',
{ headers: retryHeaders },
);
({ upstream, body } = await fetchOpenSkyStates(retryHeaders));
usedMode = 'basic';
reason = 'oauth_rejected_fallback_basic';
}
let body = await upstream.text();
const sourceEpochMs = upstream.ok ? openSkySourceEpochMs(body) : null;
if (
upstream.ok &&
+2
View File
@@ -143,8 +143,10 @@ export function cctvProxy({ sourceRoot = process.cwd() } = {}) {
if (url.pathname === '/sources') {
const body = {
trimmedPacks: getCctvSources.trimmedPacks?.() ?? [],
sources: sources.map((source) => ({
id: source.id,
pack: source.pack,
name: source.name,
city: source.city,
cityId: source.cityId,
+22 -2
View File
@@ -3,7 +3,7 @@ import path from 'node:path';
import { DEFAULT_CCTV_SOURCE_FILE, CCTV_SOURCE_CACHE_MS } from './constants.js';
import { allocateSourceCap, resolveCatalogCap } from './cap.js';
import { loadGroundHeights, joinGroundHeights } from './groundHeights.js';
import { normalizeSourceItem } from './normalize.js';
import { cameraRegion, normalizeSourceItem } from './normalize.js';
import {
loadAustinSourcesFromOpenData,
loadCaltransSourcesFromOpenData,
@@ -143,6 +143,9 @@ export function createCctvCatalog({ sourceRoot = process.cwd() } = {}) {
/** @type {Promise<Array<object>>|null} In-flight refresh, shared by concurrent
* callers so a post-TTL burst launches ONE refetch, not one per request. */
let _cctvSourceInflight = null;
/** @type {Array<{pack: string, available: number, served: number}>} Packs
* that serve fewer cameras than they offer. */
let _cctvTrimmedPacks = [];
/**
* Assemble and cache the merged CCTV source list.
@@ -209,8 +212,12 @@ export function createCctvCatalog({ sourceRoot = process.cwd() } = {}) {
name,
sources: items
.filter((item) => item && typeof item === 'object')
.map((item) => normalizeSourceItem(item))
.map((item) => ({ ...normalizeSourceItem(item), pack: name }))
.filter((item) => item.id),
available: Number.isFinite(items.available)
? items.available
: items.length,
region: items.region ?? null,
});
const packs = [
...LIVE_PACKS.map((pack, index) =>
@@ -234,6 +241,16 @@ export function createCctvCatalog({ sourceRoot = process.cwd() } = {}) {
loadGroundHeights(sourceRoot),
);
const trimmed = allocation.packs.filter((pack) => pack.kept < pack.offered);
// Each pack's own nearest-first cap and the shared catalog cap both trim.
const trimmedPacks = packs
.map((pack, index) => ({
pack: pack.name,
available: Math.max(pack.available, allocation.packs[index].offered),
served: allocation.packs[index].kept,
// Where the pack's cameras are, including those not served.
region: pack.region ?? cameraRegion(pack.sources),
}))
.filter((pack) => pack.served < pack.available);
if (trimmed.length) {
const detail = trimmed
.map((pack) => `${pack.name} ${pack.kept}/${pack.offered}`)
@@ -244,6 +261,7 @@ export function createCctvCatalog({ sourceRoot = process.cwd() } = {}) {
}
if (capped.length > 0 || _cctvSourceCache.length === 0) {
_cctvSourceCache = capped;
_cctvTrimmedPacks = trimmedPacks;
} else {
// Every source came back empty (all live packs timed out / upstream outage)
// but a good catalog is already cached — serve it stale rather than blanking
@@ -257,5 +275,7 @@ export function createCctvCatalog({ sourceRoot = process.cwd() } = {}) {
return _cctvSourceCache;
}
/** Packs serving fewer cameras than they offer, as of the last refresh. */
getCctvSources.trimmedPacks = () => _cctvTrimmedPacks;
return getCctvSources;
}
+25 -1
View File
@@ -470,7 +470,31 @@ export function prioritizeSources(cameras, maxCount, anchors) {
return a.idx - b.idx;
});
return scored.slice(0, cap).map((entry) => entry.camera);
const kept = scored.slice(0, cap).map((entry) => entry.camera);
// How many cameras the pack offered before trimming, and where, for
// coverage reports.
Object.defineProperty(kept, 'available', { value: list.length });
Object.defineProperty(kept, 'region', { value: cameraRegion(list) });
return kept;
}
/** The bounding box of cameras with coordinates, or null. */
export function cameraRegion(cameras) {
let region = null;
for (const camera of cameras || []) {
const lat = Number(camera?.lat);
const lon = Number(camera?.lon);
if (!Number.isFinite(lat) || !Number.isFinite(lon)) continue;
region = region
? {
west: Math.min(region.west, lon),
south: Math.min(region.south, lat),
east: Math.max(region.east, lon),
north: Math.max(region.north, lat),
}
: { west: lon, south: lat, east: lon, north: lat };
}
return region;
}
/**
+41 -13
View File
@@ -2,24 +2,52 @@ import { makeRateLimiter } from '../../../src/sources/rateLimit.js';
export { makeRateLimiter } from '../../../src/sources/rateLimit.js';
/**
* Opt-in per-IP rate limiter for the cost-bearing API proxies (OpenAI / Google).
* DEFAULT IS UNLIMITED: when the env var is unset, `0`, or non-numeric, this
* returns `null` and the caller skips the check entirely — a runtime no-op that
* preserves the original behavior. Only a positive integer N enables a fixed
* 60s window of N requests/IP (built lazily once, then reused so its per-IP
* window state persists across requests). The global backstop is set to a
* generous multiple of the per-IP cap so a single host can't starve the rest.
* Resolve the per-IP, per-minute cap for a cost-bearing proxy from its env var.
*
* @param {string|undefined} envValue - Raw env value (requests/min/IP).
* The default is ON. An unset or blank value takes `defaultPerMin`; a positive
* integer overrides it; an explicit `0` disables the limiter for deployments
* that genuinely want no throttle.
*
* A value that cannot be read as a number — a typo, a stray unit, a negative —
* also takes the default rather than disabling the guard. That asymmetry is the
* point: `GEV_RATELIMIT_OPENAI_PER_MIN=3O` should not quietly hand an exposed
* server's API key to whoever asks. Removing the throttle stays possible, but
* only by saying `0`, which cannot be typed by accident.
*
* @param {string|number|undefined|null} envValue - Raw env value (requests/min/IP).
* @param {number} defaultPerMin - Cap to apply when the value is absent or unreadable.
* @returns {number} Requests/min/IP; `0` means unlimited.
*/
export function resolvePerMinuteCap(envValue, defaultPerMin) {
const raw = typeof envValue === 'string' ? envValue.trim() : envValue;
if (raw === undefined || raw === null || raw === '') return defaultPerMin;
const parsed = Number(raw);
if (!Number.isFinite(parsed) || parsed < 0) return defaultPerMin;
// Only an explicit 0 disables the guard: a positive fraction such as 0.5
// would floor to 0, so it takes the strictest real cap instead.
if (parsed > 0 && parsed < 1) return 1;
return Math.floor(parsed); // includes the explicit 0 = unlimited opt-out
}
/**
* Per-IP rate limiter for the cost-bearing API proxies (OpenAI / Google).
* Returns `null` only when the cap resolves to `0` (the explicit opt-out), in
* which case the caller skips the check entirely. Otherwise a fixed 60s window
* of N requests/IP, built lazily once and reused so its per-IP window state
* persists across requests. The global backstop is a generous multiple of the
* per-IP cap so a single host can't starve the rest.
*
* @param {string|number|undefined|null} envValue - Raw env value (requests/min/IP).
* @param {number} defaultPerMin - Cap to apply when the value is absent or unreadable.
* @returns {((key:string)=>boolean)|null} An `allow(key)` fn, or null when unlimited.
*/
export function makeOptInRateLimiter(envValue) {
const max = Number(envValue);
if (!Number.isFinite(max) || max <= 0) return null; // unset/0/garbage -> unlimited
export function makeCostRateLimiter(envValue, defaultPerMin) {
const max = resolvePerMinuteCap(envValue, defaultPerMin);
if (max <= 0) return null; // explicit opt-out
return makeRateLimiter({
windowMs: 60_000,
max: Math.floor(max),
globalMax: Math.floor(max) * 20,
max,
globalMax: max * 20,
});
}
+43
View File
@@ -0,0 +1,43 @@
import { admitSameSiteRequest } from '../../../src/localRequestGate.mjs';
/**
* Cross-site request gate for cost-bearing and log endpoints. Feeds the
* request to the pure, unit-tested `admitSameSiteRequest` and answers 403
* itself when the request is refused. Returns true when it has already
* responded (the caller returns); false when the handler should continue.
* Refuses cross-site browser requests (foreign or opaque Origin, a
* Sec-Fetch-Site other than same-origin/none, or reverse-proxy headers) while
* keeping loopback non-browser tools and the HOST=0.0.0.0 LAN opt-in working.
* @param {import('node:http').IncomingMessage} req
* @param {import('node:http').ServerResponse} res
* @returns {boolean} True when a 403 was sent.
*/
export function admitSameSite(req, res) {
const verdict = admitSameSiteRequest({
hostHeader: req.headers?.host,
protocol: req.socket?.encrypted ? 'https:' : 'http:',
origin: req.headers?.origin,
secFetchSite: req.headers?.['sec-fetch-site'],
proxyHeaders: req.headers || {},
});
if (verdict.ok) return false;
res.statusCode = verdict.status;
res.setHeader('Content-Type', 'application/json');
res.setHeader('Cache-Control', 'no-store');
res.end(JSON.stringify({ error: verdict.error }));
return true;
}
/**
* Wrap a middleware handler so cross-site requests are refused with 403
* before the handler runs. Loopback tools and same-origin browser requests
* pass straight through.
* @param {(req: import('node:http').IncomingMessage, res: import('node:http').ServerResponse, next?: Function) => unknown} handler
* @returns {(req: import('node:http').IncomingMessage, res: import('node:http').ServerResponse, next?: Function) => unknown}
*/
export function sameSiteGated(handler) {
return (req, res, next) => {
if (admitSameSite(req, res)) return undefined;
return handler(req, res, next);
};
}
+6 -3
View File
@@ -26,8 +26,11 @@ import { firePerimetersProxy } from './firePerimeters.js';
import { cycloneProxy } from './cyclones.js';
import { windProxy } from './wind.js';
/** Construct the local provider plugins in their established order. */
function localProviderPlugins() {
/**
* Construct the local provider plugins in their established order.
* `realtime` configures the voice session token endpoint.
*/
function localProviderPlugins({ realtime } = {}) {
return [
openSkyProxy(),
celestrakProxy(),
@@ -49,7 +52,7 @@ function localProviderPlugins() {
adsbLolProxy(),
aisLiveProxy(),
trackBackfillProxies(),
openAiRealtimeProxy(),
openAiRealtimeProxy({ realtime }),
googlePlacesContextProxy(),
windProxy(),
weatherProxy(),
+8 -3
View File
@@ -2,6 +2,7 @@ import { defaultSourceRoot } from './common/source-root.js';
import { handleHudSummary } from './openai/hud-summary.js';
import { createDebugLogHandler } from './openai/debug-log.js';
import { createRealtimeTokenHandler } from './openai/realtime.js';
import { sameSiteGated } from './common/same-site.js';
/**
* Vite plugin: OpenAI Realtime ephemeral client secret.
@@ -15,16 +16,20 @@ function openAiRealtimeProxy({
realtime = {},
} = {}) {
function install(middlewares) {
middlewares.use('/api/openai/hud-summary', handleHudSummary);
// Cost-bearing and log endpoints refuse cross-site browser requests
// (see server/providers/common/same-site.js and SECURITY.md).
middlewares.use('/api/openai/hud-summary', sameSiteGated(handleHudSummary));
middlewares.use(
'/api/realtime/debug-log',
createDebugLogHandler({ sourceRoot }),
sameSiteGated(createDebugLogHandler({ sourceRoot })),
);
middlewares.use(
'/api/realtime/token',
createRealtimeTokenHandler({ ...realtime, annotationGuidance }),
sameSiteGated(
createRealtimeTokenHandler({ ...realtime, annotationGuidance }),
),
);
}
-4
View File
@@ -16,9 +16,6 @@ const OPENAI_REALTIME_CONTEXT_RETENTION_DEFAULT = 0.5;
const OPENAI_HUD_SUMMARY_MODEL_DEFAULT = 'gpt-5-nano';
/** Input transcription for voice captions; OPENAI_REALTIME_TRANSCRIBE_MODEL=off disables it. */
const OPENAI_REALTIME_TRANSCRIBE_MODEL_DEFAULT = 'gpt-4o-mini-transcribe';
export {
OPENAI_HUD_SUMMARY_MODEL_DEFAULT,
OPENAI_REALTIME_MODEL_MINI_DEFAULT,
@@ -27,5 +24,4 @@ export {
OPENAI_REALTIME_REASONING_DEFAULT,
OPENAI_REALTIME_CONTEXT_TOKENS_DEFAULT,
OPENAI_REALTIME_CONTEXT_RETENTION_DEFAULT,
OPENAI_REALTIME_TRANSCRIBE_MODEL_DEFAULT,
};
+5 -38
View File
@@ -23,40 +23,7 @@ const REALTIME_DEBUG_LOG_MAX_FILE_BYTES = 32 * 1024 * 1024;
*/
const REALTIME_DEBUG_LOG_MAX_PER_MIN = 120;
/**
* Fields that carry what people said or heard: transcripts, typed and spoken
* text, streamed deltas and the span's quoted replies. They are dropped from
* the persisted log unless content logging is explicitly enabled
* (GEV_VOICE_LOG_CONTENT=1), so timing and tool diagnostics never retain
* speech by default.
*/
const VOICE_CONTENT_FIELDS = new Set([
'transcript',
'text',
'delta',
'answer_text',
'preamble_text',
]);
/** Replace spoken or typed content with its length. */
function omitVoiceContent(value, depth = 0) {
if (depth > 12 || value === null || typeof value !== 'object') return value;
if (Array.isArray(value))
return value.map((item) => omitVoiceContent(item, depth + 1));
const output = {};
for (const [key, item] of Object.entries(value)) {
output[key] =
VOICE_CONTENT_FIELDS.has(key) && typeof item === 'string'
? `[omitted ${item.length} chars]`
: omitVoiceContent(item, depth + 1);
}
return output;
}
function createDebugLogHandler({
sourceRoot = defaultSourceRoot,
includeContent = process.env.GEV_VOICE_LOG_CONTENT === '1',
} = {}) {
function createDebugLogHandler({ sourceRoot = defaultSourceRoot } = {}) {
const logDir = path.join(sourceRoot, '.gev-logs');
const logFile = path.join(logDir, 'realtime-conversations.jsonl');
const allow = makeRateLimiter({
@@ -112,12 +79,12 @@ function createDebugLogHandler({
try {
const body = await readRequestBody(req, REALTIME_DEBUG_LOG_MAX_BYTES);
const parsed = JSON.parse(body || '{}');
const record = includeContent ? parsed : omitVoiceContent(parsed);
const record = JSON.parse(body || '{}');
// The server's own timestamp comes last, so a record cannot supply one.
await append(
`${JSON.stringify({
loggedAt: new Date().toISOString(),
...record,
loggedAt: new Date().toISOString(),
})}\n`,
);
res.statusCode = 204;
@@ -137,4 +104,4 @@ function createDebugLogHandler({
};
}
export { createDebugLogHandler, omitVoiceContent };
export { createDebugLogHandler };
+5 -5
View File
@@ -2,7 +2,7 @@ import {
HUD_SUMMARY_INSTRUCTIONS,
keylessHudSummaryResponse,
} from '../../../src/hudSummaryResponse.js';
import { enforceOptInRateLimit, openAiRateLimiter } from './rate-limit.js';
import { enforceRateLimit, openAiRateLimiter } from './rate-limit.js';
import { readRequestBody } from '../common/request.js';
import { OPENAI_HUD_SUMMARY_MODEL_DEFAULT } from './constants.js';
@@ -46,10 +46,10 @@ async function handleHudSummary(req, res) {
return;
}
// Opt-in per-IP throttle (GEV_RATELIMIT_OPENAI_PER_MIN). Keyless HUD
// fallback has no provider cost and resolves above without consuming a
// paid-endpoint quota slot.
if (!enforceOptInRateLimit(openAiRateLimiter(), req, res)) return;
// Per-IP throttle (GEV_RATELIMIT_OPENAI_PER_MIN). On by default; 0 disables.
// Keyless HUD fallback has no provider cost and resolves above without
// consuming a paid-endpoint quota slot.
if (!enforceRateLimit(openAiRateLimiter(), req, res)) return;
try {
const body = await readRequestBody(req, 64 * 1024);
+26 -17
View File
@@ -1,5 +1,5 @@
const DEFAULT_ANNOTATION_GUIDANCE =
'Put related places in one annotate_map call. Set flyTo true only when the user is not already looking at the place (off-screen marks auto-frame, so when unsure leave it false). Do not say you are drawing or annotating — talk about the places while the marks appear. Marks ACCUMULATE AND PERSIST across flights and topics so the user can build up the map: never pass clearPrevious, and call clear_annotations ONLY when the user EXPLICITLY asks to clear or reset the map.';
'Use a single annotate_map call with several annotations when you are describing multiple related places at once. Set flyTo true only when the user is not already looking at the place; if every mark in a call lands off-screen the app auto-frames them, so when unsure leave flyTo false. Do NOT say out loud that you are drawing, highlighting, or annotating — just speak naturally about the places while the marks appear. ANNOTATIONS ACCUMULATE AND PERSIST — keep adding marks as you explore; you can fly around, change topic, and jump between far-apart places and the marks STAY, so the user can build up the map and show people things. Do NOT clear on your own initiative: never pass clearPrevious, and call clear_annotations ONLY when the user EXPLICITLY asks to clear or reset the map.';
function realtimeInstructions(
annotationGuidance = DEFAULT_ANNOTATION_GUIDANCE,
@@ -13,23 +13,27 @@ function realtimeInstructions(
'For requests to open, show, reveal, or focus a menu/panel, call set_panel_open or show_data_layers_menu. "Open Context" means only set_panel_open{panelId:"global-context-panel",open:true}; it does not activate a Context sub-mode. "Open Contacts" means set_context_mode{mode:"contacts"}; that action expands the parent Context panel before activating Contacts.',
'For requests like "show me the datacenter layers", open the data layers menu and focus the matching layer row; do not enable the layer unless the user asks to turn it on.',
'For questions like "what am I looking at?", "what is in view?", "what is this?", "that selected thing", nearby datacenter, dam, cable, ship, or current view contents, call get_entity_context first, then answer from the returned scene/entity context.',
'For "what is this aircraft?", answer with get_entity_context\'s identityLine: callsign, operator, type and route from returned fields only, airport codes verbatim. Other questions about the selection answer from its properties. Never infer them from the callsign.',
'While a camera motion or route flight is active, a bare "stop" means move_camera{motion:stop} — NOT control_scene and NOT stop_tracking (those need explicit words like "stop the scene" / "stop tracking"). If move_camera stop returns stopped:false and an entity is being tracked, call stop_tracking next — the user means "stop whatever is moving". Flying somewhere while tracking stops the tracking.',
'For "what is this aircraft?" answers, read the callsign, operator, registration, type, and route only from get_entity_context selected.properties. Treat route, routeOrigin, and routeDestination as the only authoritative route fields. Every aircraft identity answer MUST explicitly cover operator, type, and route. When a route is present, repeat its endpoint codes exactly; do not expand airport codes into city names. For a missing field say exactly "Operator details are unavailable", "Aircraft type is unavailable", or "Route details are unavailable" as applicable. Never silently omit missing enrichment or infer it from the callsign.',
'While a camera motion or route flight is active, a bare "stop" means move_camera{motion:stop} — NOT control_scene and NOT stop_tracking (those need explicit words like "stop the scene" / "stop tracking"). If move_camera stop returns stopped:false and an entity is being tracked, call stop_tracking next — the user means "stop whatever is moving". Flying somewhere while tracking automatically stops the tracking (the result says so): mention it briefly.',
'For camera-motion requests — "orbit around this", "pan left", "tilt up", "stop moving" — call move_camera. For "fly the route" over a drawn route, call fly_route. Confirm with the RESULTING state ("Orbiting slowly", "Flying the route").',
'analyst_query ANSWERS questions; it never moves the camera or starts tracking. For requests to FOLLOW or TRACK a specific aircraft/ship, call track_entity (get_entity_context first when the target is ambiguous), never analyst_query as the final or only action. For "follow/track the nearest aircraft", first call analyst_query with the aircraft layer(s), sortBy=distance, and limit=1, then call track_entity with the returned aircraft identity in the same turn. The lookup alone does not fulfill a follow/track command.',
'For a request to enable an aircraft layer and SELECT or FIND the nearest/closest aircraft near a named place — for example, "Turn on flights and select the closest aircraft to Austin" — call select_nearest_aircraft once. It atomically turns on the requested aircraft layer first, waits for location arrival, refreshes that layer for the destination viewport, filters out landed/on-ground records, and selects the nearest airborne result. A healthy fallback feed is valid data, never an enable failure. Do not also call fly_to_location, set_layer_visibility, analyst_query, track_entity, set_context_mode, or control_cockpit for the same request. SELECT/FIND never implies Contacts or Cockpit unless the user explicitly asks for either mode.',
'For ANALYTICAL questions about loaded layer data — how many / which / fastest / highest / biggest / nearest / next flights, ships, fires, earthquakes, satellites, launches, storms, transit, bikes, bases, datacenters or dams ("how many flights over Texas", "anything above 40,000 feet") — call analyst_query, not get_entity_context. Always set scope by WHERE: "around here" or "near this" while pointing is kind pointer, never view. Filter on the attribute the user names ("MODIS" → sensor eq MODIS). Say the count with its scope in a few words ("37 loaded flights over Texas") plus one or two examples by name. If the needed layer is off, say so and offer to turn it on. "Which of THOSE…" about the last answer → followUp=true with only the new filter/sort.',
'WHERE, for every tool that takes a place or scope, in this order: (1) a place the user names; (2) explicit pointing — "this/that one/here/around here" while pointer_context shows pointing or a target → "pointer"; (3) the Contacts subject while Contacts is active, or the selection when the user refers to it; (4) the current view. A selected datacenter, dam, fire or cable never silently becomes the centre.',
'FEED STATE: when a count\'s feed is stale or degraded, add that one word ("12 flights in view, stale"); never call it live. If unavailable, give no number: say the feed is unavailable. Fallback feeds are valid data; name them only if asked.',
'COUNTING CONTRACT — what "near" means. (1) While Contacts is ACTIVE, "near / nearby / how many aircraft" means the Contacts window: answer from contactsWindow in the tool result — the exact numbers on the user\'s panel (set_context_mode, analyst_query and get_current_view_state carry it once Contacts settles). For "Open Contacts and tell me how many aircraft are within 250 km", call set_context_mode{mode:"contacts"} first and answer from contactsWindow.aircraft, not from a pre-Contacts analyst query; analyst_query\'s own count measures loaded records and is usually lower — never give it as the window count. The centre follows WHERE; with Contacts active but no subject yet, use the view, never an empty panel. (2) With Contacts OFF, "nearby" means in view; "near <place>" means a radius around that place. (3) EVERY count names its scope in words — "42 in your window", "8 in view", "30 within 250 km of Austin" — never a bare number; analyst_query returns scopeLabel for exactly this. Two numbers with named scopes are not a contradiction; say both if asked. (4) State counts VERBATIM — never estimate, round, or hedge: if a tool returns 46, say 46.',
'For a request to enable an aircraft layer and SELECT or FIND the nearest/closest aircraft near a named place — for example, "Turn on flights and select the closest aircraft to Austin" — call select_nearest_aircraft once. It atomically turns on the requested aircraft layer first, waits for location arrival, refreshes that layer for the destination viewport, filters out landed/on-ground records, and selects the nearest airborne result. A healthy fallback feed is valid data: report the returned feed source briefly, never call it an enable failure. Do not also call fly_to_location, set_layer_visibility, analyst_query, track_entity, set_context_mode, or control_cockpit for the same request. SELECT/FIND never implies Contacts or Cockpit unless the user explicitly asks for either mode.',
'For ANALYTICAL questions about layer data — how many / which / fastest / highest / biggest / nearest flights, ships, fires, earthquakes, satellites, datacenters, or dams ("how many flights over Texas", "biggest fire near LA", "which ships are headed to Oakland", "anything above 40,000 feet") — call analyst_query, not get_entity_context. Narrate the count plus two or three notable examples by name, and reflect the result\'s coverage note honestly: the answer covers data loaded by enabled layers, not the whole world. If the needed layer is disabled, say so and offer to enable it. For follow-ups about the same set ("which of THOSE is closest?"), call analyst_query with followUp=true and only the new filter/sort.',
'WORLD QUERIES — the query tools (get_weather, get_wind, search_places, places_nearby, plan_route, get_cyclones, get_fire_perimeters, get_bike_share, get_transit_vehicles, get_traffic_flow, situation_brief, military_awareness and the other get_* and find_* tools you have) look up data anywhere in the world without changing the map and without any layer being on. Use them when the user asks about a place they are not looking at or data whose layer is off ("what is the weather in Tokyo", "how do I drive from Austin to Dallas", "any wildfires in California"). For aircraft, ships, earthquakes, active fires, datacenters, dams and satellites overhead, use analyst_query over the layers instead, turning the layer on first when the user wants it. Use analyst_query and get_entity_context for what the app has loaded or what is on screen. Answer from the result summary and data, mention its source when it has one, and never claim the map changed. If the user also wants to SEE it, use the matching app action (fly_to_location, set_layer_visibility) as well.',
'FEED STATE CONTRACT — analyst_query and get_current_view_state carry feedProvenance from the same layerFeedState the Data Layers chips use (nominal/loading/degraded/stale/fallback/unavailable, or off). NEVER narrate a count as live or current when feedProvenance.overall is not nominal. Say the feedState in the same breath as the count ("12 flights in view, STALE"). If unavailable, do not invent a count; say the feed is unavailable. Fallback is valid data — name the fallback source, never call it an enable failure. Confirm only what the tool result returned.',
'COUNTING CONTRACT — what "near" means. (1) While Contacts is ACTIVE, "near / nearby / how many aircraft" means the Contacts window: answer from contactsWindow in the tool result — those are the exact numbers on the user\'s panel. set_context_mode, analyst_query, and get_current_view_state carry it after Contacts settles. For "Open Contacts and tell me how many aircraft are within 250 km", call set_context_mode{mode:"contacts"} first and answer from contactsWindow.aircraft; do not answer from a pre-Contacts analyst query. analyst_query\'s own count measures currently-loaded records and is usually lower; never give it as the window count. CENTER PRECEDENCE for a nearby/how-many ask, in order: an explicit place in the question ("over Texas", "near Austin") always wins and ignores Contacts state; else the CONTACTS SUBJECT when Contacts is active and has one — a selected datacenter, dam, fire, or cable does NOT silently become the center; else an entity the user explicitly names ("around this datacenter"); else the current view, said aloud ("nothing is selected, so this is the current view"). With Contacts active but NO subject yet, use the view and say so; never read an empty panel. (2) With Contacts OFF, "nearby" means in view; "near <place>" means a radius around that place. (3) EVERY count names its scope in words — "42 in your window", "8 in view", "about 30 within 250 km of Austin" — never a bare number; analyst_query returns scopeLabel for exactly this. Two different numbers with named scopes are not a contradiction; say both if asked. (4) State counts VERBATIM — never estimate, round, or hedge ("a few", "less than a dozen"): if a tool returns 46, say 46. (5) When it matters, add once: counts cover loaded data, and the flights layer loads where you look.',
'While Cockpit is active, navigate with control_cockpit (next/previous, optionally targetLayer or aircraftClass). track_entity and fly_to_location are REFUSED by design while Cockpit owns the camera — that refusal is correct, not an error to retry. To go somewhere else, exit Cockpit first. control_cockpit enter establishes Contacts itself, so do not call set_context_mode before or after it.',
'When the target layer is unknown, OMIT layerId in track_entity so it searches all enabled layers. Passing the wrong layerId ("flights" for a military contact) returns "Nothing matched" even though the contact is loaded.',
'READING THE VIEW: with no selected object or overlay entities, answer from get_entity_context\'s basemap context (tiles source, centre target, reverse-geocoded place, altitude, style, enabled layers); say there is nothing only when that is unavailable too. Prefer, in order: a nearby knownLandmarks name ("Eiffel Tower"); at local zoom nearbyPlaces, place.labels and viewportPlaces visible and street labels; viewportPlaces dominantCountry/Region/Locality over raw coordinates; the majority of viewportSamples over one reverse-geocoded address. Legible labels in an attached screenshot count too, but at global or regional viewScale never name a precise street or city from one centre pixel.',
'If get_entity_context has no selected object or overlay entities, use its basemap context: Google Photorealistic 3D Tiles/Cesium source, center target coordinates, reverse-geocoded place, camera altitude, active style, and enabled layers. Do not say there is nothing unless the basemap target is also unavailable.',
'If basemap context includes knownLandmarks, prefer the nearest known landmark by name for "what am I looking at" answers. For example, if knownLandmarks includes Eiffel Tower, say Eiffel Tower.',
'At local zoom, use basemap nearbyPlaces, place.labels, viewportPlaces.visibleLabels, and viewportPlaces.streetLabels to identify the building, premises, roads, and named places visible around the screen target.',
'If basemap context includes viewportPlaces, prefer dominantCountry, dominantRegion, and dominantLocality over raw coordinates.',
'When basemap context includes viewportSamples or an inferred country, trust that over a single reverse-geocoded address. If most samples indicate Iran, say Iran, not the United States.',
'When a viewport screenshot is attached after get_entity_context, read clearly legible street, building, and place labels from it and combine them with structured label context. Respect scene viewScale: at global/continental/regional scale, avoid naming a precise street/city from one center pixel.',
'Do not mention disabled layers or stale selections.',
'POINTING: pointer_context says what the cursor is on (pointing:<kind>: resolved by the tools; target "none": not pointing). For "this/that/here/that one" with no name, pass "pointer": get_entity_context scope, track_entity/fly_to_location query, analyst_query scope.kind ("around here" is pointer, not view), annotate_map target. "The second/last one" is referent:n from the last numbered list. Without a pointer, ask or follow WHERE.',
'PREAMBLES: for instant commands (layers, styles, HUD, detection, zoom, panels, basemap, radio controls, camera nudges, and lookups such as get_entity_context) call the tool without speaking first. Only for slow or multi-step work (annotate_map, a searched fly_to_location, select_nearest_aircraft, analyst_query over a named place, or two or more tools) you may first say ONE sentence of at most 12 words naming the actual steps ("Finding the Capitol, then outlining its grounds."), then call the tools at once in the same response.',
'When one request contains MULTIPLE changes, call ALL the corresponding tools before the confirmation. Never confirm a partial subset; if one fails, say which.',
'AFTER TOOLS: speak the result\'s say line once, lightly rephrased at most, never adding to it and never repeating your preamble; if say is null, confirm briefly or continue. Sources, notes and caveats are on the voice card: speak a caveat only when it changes the answer, or when asked why, how sure or which source. Lower bounds ("at least 500") and partial answers are part of the answer — always say them. No filler or hedges ("let me check", "keep in mind", "it seems", "based on the data available"), and never add "approximately" to an exact tool number.',
'When a request requires a tool call, do not speak in the same response as the tool call. Call the tool first.',
'When a single user request contains MULTIPLE changes (e.g. "switch to operator layout, use balanced detection at density 50, and switch to Bing aerial"), call ALL the corresponding tools — multiple tool calls in sequence — before speaking. Never confirm a partial subset. If a later tool fails, say which parts succeeded and which failed.',
'After receiving tool output, speak exactly one short confirmation. Do not repeat the confirmation.',
'For "show/open/turn on" layer requests, enable the matching layer. For "hide/close/turn off", disable it.',
// INSTRUCTION-ONLY mapping for the two globe-scale named views.
//
@@ -45,23 +49,28 @@ function realtimeInstructions(
// string is the whole rollback.
'NAMED VIEWS are shorthand for tool calls you already have — there is no "mode" tool for them. Treat ONLY these as the shorthand: "infrastructure mode" / "the infrastructure view" / "show me global infrastructure" means three set_layer_visibility calls (local-datacenters, local-dams, telegeography-submarine-cables) plus zoom_to_globe; "environmental mode" / "earth watch" / "active events", said as the name of a view, means set_layer_visibility for local-firms and earthquakes plus zoom_to_globe. Anything vaguer is NOT this shorthand — an open-ended question about the world or the news is an ordinary question: answer it, or use analyst_query over the layers already on. Never switch a whole view on to answer a question nobody asked to see. When you do run one, make every call before speaking, then give one confirmation naming the resulting state; if the fires layer comes back unavailable because no FIRMS key is configured, say so plainly — the earthquakes still loaded. "Live contacts" and "space missions" are NOT this pattern: they stay set_context_mode{mode:"contacts"} and set_context_mode{mode:"space-missions"}.',
'For visual filter requests, call set_visual_style with one of the allowed style IDs.',
'Disambiguation table — basemap vs layer vs style: basemap switching requires an explicit stack name — "Bing aerial" means set_map_stack bing-aerial, "aerial with labels" means bing-labels, "OSM"/"road map" means osm, "Esri"/"Esri imagery" means esri-imagery, "Google 3D"/"photorealistic" means photoreal. Any other mention of "satellite" or "satellites" means the satellites DATA LAYER via set_layer_visibility, never a basemap — except "recent/latest satellite imagery" (the recent-imagery layer) and "weather satellite"/clouds (weather-satellite). "surveillance"/"night vision"/"thermal" are visual STYLES via set_visual_style.',
'SHIPS vs STYLES: "vessels" is easily heard as "visuals". "Show me vessels", "turn on vessels", "the vessels", "visuals" or "vestibules" said WITHOUT a style name means the ships layer: set_layer_visibility{layerId:"ais-live-vessels"}, exactly as "ships" or "boats" does. Call set_visual_style only when the user names a style (retro/CRT, surveillance/night vision, thermal, anime, noir, snow, normal) or says filter, effect or look. Never turn on a different data layer (such as fires) for a ships request.',
'Disambiguation table — basemap vs layer vs style: basemap switching requires an explicit stack name — "Bing aerial" means set_map_stack bing-aerial, "aerial with labels" means bing-labels, "OSM"/"road map" means osm, "Esri"/"Esri imagery" means esri-imagery, "Google 3D"/"photorealistic" means photoreal. Any mention of "satellite" or "satellites" ALWAYS means the satellites DATA LAYER via set_layer_visibility, never a basemap. "surveillance"/"night vision"/"thermal" are visual STYLES via set_visual_style.',
'HUD requests ("hud on/off", "switch to operator/minimal/tactical/cyber layout") use set_hud. Detection requests ("detection on", "dense mode", "balanced mode", "sparse mode", "set density to 25", "use weighted allocation") use set_detection. Density snaps to 0/25/50/75/100 and derives Sparse/Balanced/Dense; panoptic is a legacy alias for Dense.',
'Cyber sonar requests use set_cyber_sonar: enabled, rings, rangePct, intensityPct (Power), opacityPct, sectorDeg. For settings questions read get_current_view_state.controls.hud.sonar. Sonar is not detection. Do not silently switch themes, reveal the HUD or enable sonar when changing a slider. If the user explicitly requests Cyber plus sonar settings, call set_hud first, wait for success, then set_cyber_sonar. Confirm returned values; mapSweepActive and contactSweepActive distinguish saved settings from active map effects, including in Cockpit or reduced motion. Do not claim visible contacts solely because enabled is true.',
'Bloom/sharpen requests use set_post_processing. Scene requests ("play orbital watch", "stop the scene", "what scenes are there") use control_scene. CCTV camera requests ("next camera", "nearest camera", "select the Congress camera", "show coverage") use control_cctv — the CCTV layer must be enabled first.',
'Radio playback requests use control_radio. "Turn on/start the radio" means action=play; action=enable only reveals Radio markers and must be reserved for explicit "show/enable the Radio layer/markers" requests. After a prepared playback result, briefly confirm any other completed actions and say "Turning on the radio"—never claim it is already playing. The client keeps Radio muted until playback is verified, then closes voice before restoring Radio volume. Examples: "play news near Austin" → select category=news locationId=austin; "play US news" → select category=news country=US; "Radio volume 30" → volume; pause/resume/stop/next/previous use the matching action. Radio selection never moves the camera.',
'"Track/follow <something specific>" (a callsign, ship name, satellite name) uses track_entity. "Take me to the biggest fire" uses track_entity with query "biggest fire" (the fires layer must be enabled). Bare "orbit" means camera orbit of the current landmark. "Stop following/tracking" uses stop_tracking.',
'"Show me which planes are overhead"/"frame the ships"/"show me the satellites above" use frame_overhead with the matching target.',
"After frame_overhead, speak ONLY from the tool result's count field — e.g. 'Framed fourteen aircraft, labels on'; never reassess or second-guess the count aloud.",
'Confirmations echo the RESULTING state, never the request: "HUD operator layout", "Density twenty-five percent", "Bing aerial imagery", "Tracking UAL428", "Framed fourteen aircraft". On ok=false, state the failure plainly: "Nothing matched UAL999", "No ships within 120 kilometers". Never claim an action without ok=true in the tool result.',
'For destination requests such as "take me to Italy", "go to NYC", or "show me the Eiffel Tower", call fly_to_location. Prefer known city IDs when available; otherwise pass the plain place query.',
'Navigation-only requests ("take me to X", "go to X", "fly to X") are NOT descriptions: call fly_to_location alone and do NOT also call annotate_map, unless the user explicitly asks to mark the place or you go on to explain specific places there. Never drop a point pin on a region-scale natural feature (a mountain range, desert, sea, or forest) — a single point in the middle of the Rockies is meaningless. If the user explicitly asks to mark such a region, prefer type=area.',
'For country and city destinations, omit rangeM so GEV frames the whole country or city in view. For landmarks and buildings, omit rangeM so GEV chooses a close landmark view.',
'Only supply rangeM when the user asks for a particular numeric height, distance, closer view, or wider view.',
'For relative requests such as "zoom out a little", "pull back", "zoom in more", or "get closer", always call adjust_camera_zoom. But "globe view", "whole earth", "the whole planet", or "zoom all the way out" is an ABSOLUTE framing: call zoom_to_globe once instead — repeated adjust_camera_zoom calls can never reach the globe. Never claim the camera moved without the tool returning ok=true.',
'WHITEBOARD THE WORLD: when you describe a specific place, building, campus, district or boundary, or how places relate, call annotate_map as you talk. type=area traces a building, campus, compound, park, district or boundary outline; pin marks a spot; highlight pulses a spot with no footprint; arrow joins two places ("it sits next to the Marina"). Use place names; screenX/screenY only for a spot you can see in the latest screenshot but cannot name; never invent coordinates. Set entityKind whenever you know what the target is.',
'Keep spoken confirmations short, e.g. "Opening datacenters" or "Flying to London".',
'WHITEBOARD THE WORLD: whenever you describe or explain a specific place, building, campus, district, boundary, or a spatial relationship between places, call annotate_map to mark it visually as you talk — like sketching on the map. To call out a specific building, campus, compound, park, or district, use type=area (it traces and encloses the real footprint — a building gets a glowing volume, a district gets a draped outline). Use type=highlight only for a transient pulse on a precise spot that has no meaningful footprint, and type=pin to drop a labeled marker. Examples: "what is the Palace of Fine Arts?" → an AREA on it; "the old military base next to it" → an AREA on the Presidio; "ILM is right here" → a pin; "it sits next to the Marina" → an arrow from one to the other. Prefer place NAMES so the app resolves real positions and outlines; never invent coordinates or pixel locations.',
'On every annotation, also set entityKind to what the thing IS when you know it: building (one structure), compound (campus/grounds/mall/park), district (neighborhood/area of a city), street (a named road), or point_feature (a monument, statue, memorial, plaque, fountain, or other small point landmark). entityKind is a FACT about the target, independent of the mark type you chose — monuments and statues are point_feature even when you use type=area; the app then anchors them as precise points instead of guessing at a footprint.',
annotationGuidance,
'annotate_map results: work any say line in naturally (places not found, a straight-line fallback with no travel time, or a full map — then ask before clearing). outlinePending means the boundary is still being traced: never claim it is drawn. A later map_annotation_outline system item reports each outcome; use it to quietly confirm or correct yourself.',
'ROUTES vs DISTANCES: "walking/driving route from A to B" → type=route with the ordered points and the mode; the app draws the street path and reports distance and time you can read aloud. "How far is X from Y" or "X is next to Y" → type=arrow (straight-line distance), never route.',
'If an annotate_map result has partial:true or any failedLabels, do not pretend those places appeared — briefly work into your narration that you could not pinpoint them (e.g. "I couldn\'t place X"). If a route comes back as a direct line (no street route was found), describe it as a straight-line distance, not a walking/driving time. If an annotate_map result has capped:true, the map is full — ASK the user whether to clear before drawing more; do not clear unprompted. outlinePending:true is NOT a failure, but it is also NOT an outline: the anchor mark is placed and the boundary is still being traced in the background. Narrate it in progress — e.g. "tracing the boundary now" — and NEVER state the outline is already drawn or visible; it may yet come back as just a point. A later system item of type map_annotation_outline reports the final outcome per mark (status resolved or failed, with its label): use it to quietly confirm, or to correct yourself if you implied a boundary that stayed a point — an honest miss beats a misleading guess.',
'PREFER NAMES. Only when you cannot name or geocode a place but you can clearly SEE the exact spot in the most recent viewport screenshot, fall back to screenX/screenY (normalized 0..1 from that image) to point at it; the app converts the pixel to a real world point. Never use screenX/screenY for something you could name.',
'PATHS vs DISTANCES: for "walking/driving route from A to B" (or through several stops), use type=route with the ordered points and the matching mode (walking/driving/cycling) — the app draws the real street-following path on the map and reports distance and travel time, which you can read aloud. For "how far is X from Y", "is it nearby", or "X is next to Y", use type=arrow between the two — it draws a floating connector and shows the straight-line distance. Do NOT use route for a simple distance/proximity question.',
].join('\n');
}
+24 -11
View File
@@ -1,33 +1,46 @@
import { makeOptInRateLimiter, clientKey } from '../common/rate-limit.js';
import { makeCostRateLimiter, clientKey } from '../common/rate-limit.js';
/**
* Requests/min/IP applied when GEV_RATELIMIT_OPENAI_PER_MIN is unset. Both
* places that already name a value agree on it: `.env.example` recommends 30
* and the Pinokio build ships 30 (pinokio/_ENVIRONMENT), so the packaged app
* is unaffected and only an unconfigured server changes. The HUD asks for a
* summary every 15s (HUD_SUMMARY_INTERVAL_MS), so an open tab spends 4/min and
* a voice session adds one token mint; the cap clears ordinary use several
* times over and bites only a caller working the endpoint far harder than the
* app does.
*/
const OPENAI_DEFAULT_PER_MIN = 30;
// Built LAZILY on first request, NOT at module load: `.env` values are applied to process.env later
// (the plugin config hook calls loadEnv → process.env, AFTER this module is imported), so reading
// process.env here at import time would always see them unset and silently stay unlimited even when
// process.env here at import time would always see them unset and fall back to the default even when
// configured via .env. Building on first request (like the OPENAI_API_KEY reads) sees the loaded env;
// the result is cached so the limiter's per-IP window state persists. `null` = unlimited (default).
// the result is cached so the limiter's per-IP window state persists. `null` = the explicit 0 opt-out.
let _openAiRateLimiter;
/** OpenAI cost endpoints (realtime/token + hud-summary). Null = unlimited (default). */
/** OpenAI cost endpoints (realtime/token + hud-summary). Null only when set to 0. */
function openAiRateLimiter() {
if (_openAiRateLimiter === undefined)
_openAiRateLimiter = makeOptInRateLimiter(
_openAiRateLimiter = makeCostRateLimiter(
process.env.GEV_RATELIMIT_OPENAI_PER_MIN,
OPENAI_DEFAULT_PER_MIN,
);
return _openAiRateLimiter;
}
/**
* Apply an opt-in limiter to a request, writing a 429 when over the cap.
* When `limiter` is null (unlimited, the default) this is a no-op returning
* `true`, so the handler proceeds exactly as before.
* Apply the limiter to a request, writing a 429 when over the cap.
* When `limiter` is null — the explicit `0` opt-out — this is a no-op
* returning `true`, so the handler proceeds unthrottled.
*
* @param {((key:string)=>boolean)|null} limiter
* @param {import('http').IncomingMessage} req
* @param {import('http').ServerResponse} res
* @returns {boolean} True if the request may proceed; false if a 429 was sent.
*/
function enforceOptInRateLimit(limiter, req, res) {
if (!limiter) return true; // unlimited (default) — no behavior change
function enforceRateLimit(limiter, req, res) {
if (!limiter) return true; // explicit 0 opt-out — unthrottled
if (limiter(clientKey(req))) return true;
res.statusCode = 429;
res.setHeader('Content-Type', 'application/json');
@@ -36,4 +49,4 @@ function enforceOptInRateLimit(limiter, req, res) {
return false;
}
export { enforceOptInRateLimit, openAiRateLimiter };
export { enforceRateLimit, openAiRateLimiter, OPENAI_DEFAULT_PER_MIN };
+5 -19
View File
@@ -1,4 +1,4 @@
import { enforceOptInRateLimit, openAiRateLimiter } from './rate-limit.js';
import { enforceRateLimit, openAiRateLimiter } from './rate-limit.js';
import {
resolveVoiceModel,
isKnownVoiceTier,
@@ -10,7 +10,6 @@ import {
OPENAI_REALTIME_REASONING_DEFAULT,
OPENAI_REALTIME_CONTEXT_TOKENS_DEFAULT,
OPENAI_REALTIME_CONTEXT_RETENTION_DEFAULT,
OPENAI_REALTIME_TRANSCRIBE_MODEL_DEFAULT,
} from './constants.js';
import { realtimeInstructions } from './instructions.js';
import { GEV_REALTIME_TOOLS } from './tools.js';
@@ -21,6 +20,7 @@ function createRealtimeTokenHandler({
fetchImpl = (...args) => fetch(...args),
resolveApiKey = () => process.env.OPENAI_API_KEY,
models = {},
tools = GEV_REALTIME_TOOLS,
} = {}) {
return async (req, res) => {
res.setHeader('Cache-Control', 'no-store');
@@ -31,8 +31,8 @@ function createRealtimeTokenHandler({
return;
}
// Opt-in per-IP throttle (GEV_RATELIMIT_OPENAI_PER_MIN). No-op when unset.
if (!enforceOptInRateLimit(openAiRateLimiter(), req, res)) return;
// Per-IP throttle (GEV_RATELIMIT_OPENAI_PER_MIN). On by default; 0 disables.
if (!enforceRateLimit(openAiRateLimiter(), req, res)) return;
const apiKey = resolveApiKey();
if (!apiKey) {
@@ -89,14 +89,6 @@ function createRealtimeTokenHandler({
OPENAI_REALTIME_CONTEXT_RETENTION_DEFAULT,
),
);
// User captions for the voice card. Billed separately from the session.
const transcribeModel =
process.env.OPENAI_REALTIME_TRANSCRIBE_MODEL ||
OPENAI_REALTIME_TRANSCRIBE_MODEL_DEFAULT;
const transcription =
transcribeModel.toLowerCase() === 'off'
? {}
: { transcription: { model: transcribeModel } };
const sessionConfig = {
session: {
type: 'realtime',
@@ -112,7 +104,6 @@ function createRealtimeTokenHandler({
audio: {
input: {
noise_reduction: { type: 'near_field' },
...transcription,
turn_detection: {
type: 'semantic_vad',
eagerness: 'low',
@@ -123,7 +114,7 @@ function createRealtimeTokenHandler({
output: { voice },
},
instructions: realtimeInstructions(annotationGuidance),
tools: GEV_REALTIME_TOOLS,
tools,
tool_choice: 'auto',
},
};
@@ -148,11 +139,6 @@ function createRealtimeTokenHandler({
// case where a bogus ?tier= was silently downgraded to standard.
res.setHeader('X-GEV-Voice-Tier', tier);
res.setHeader('X-GEV-Voice-Model', model);
// Captions are billed separately; the client meters them with this id.
res.setHeader(
'X-GEV-Voice-Transcribe-Model',
transcription.transcription ? transcribeModel : 'off',
);
if (requestedTier && !isKnownVoiceTier(requestedTier)) {
res.setHeader('X-GEV-Voice-Tier-Fallback', '1');
}
@@ -1,5 +1,3 @@
import { voiceLayerAliasHint, voiceQueryFieldHint } from './layerManifest.js';
export const ACTION_DESCRIPTIONS = {
fly_to_location: {
description:
@@ -14,11 +12,7 @@ export const ACTION_DESCRIPTIONS = {
},
query: {
description:
'Plain place search query, e.g. "London", "Eiffel Tower", or "Dubai Marina". "pointer" = the spot under the cursor.',
$position: 1,
},
referent: {
description: 'Numbered item n from the last result list.',
'Plain place search query, e.g. "London", "Eiffel Tower", or "Dubai Marina".',
$position: 1,
},
viewMode: {
@@ -82,13 +76,13 @@ export const ACTION_DESCRIPTIONS = {
$position: 1,
},
set_layer_visibility: {
description:
"Enable or disable one God's Eye View data layer. Every shipped layer is in the enum; never tell the user a listed layer does not exist.",
description: "Enable or disable one registered God's Eye View data layer.",
$position: 1,
parameters: {
properties: {
layerId: {
description: `Spoken names: ${voiceLayerAliasHint()}.`,
description:
'Common-name mapping for the non-obvious ids: space mission(s) → rocket-launches; fires/wildfires/active fires → local-firms (NASA FIRMS); ships/vessels/boats → ais-live-vessels; undersea/submarine cables → telegeography-submarine-cables; datacenters → local-datacenters; dams → local-dams; bikes/bike share → bikeshare; street traffic/congestion → traffic; traffic cameras → cctv; internet radio/stations → radio; ALPR/license plate readers/Flock cameras → alpr-cameras; local ADS-B/my receiver/my antenna (aircraft heard by a local RTL-SDR receiver) → local-adsb.',
$position: 1,
},
},
@@ -160,13 +154,9 @@ export const ACTION_DESCRIPTIONS = {
properties: {
scope: {
description:
'Use auto by default. selected returns the clicked/selected entity; in_view returns visible entities near the screen center; pointer returns what the cursor is on.',
'Use auto by default. selected returns the clicked/selected entity; in_view returns visible entities near the screen center.',
$position: 2,
},
referent: {
description: 'Numbered item n from the last result list.',
$position: 1,
},
layerId: {
description: 'Optional layer filter for visible entity context.',
$position: 2,
@@ -359,17 +349,12 @@ export const ACTION_DESCRIPTIONS = {
properties: {
query: {
description:
'Callsign, ship name, satellite name, ICAO hex, MMSI, or NORAD id. "pointer" = the contact under the cursor.',
$position: 1,
},
referent: {
description:
'Numbered item n from the last result list (-1 = last); query may then be its label.',
'Callsign, ship name, satellite name, ICAO hex, MMSI, or NORAD id.',
$position: 1,
},
layerId: {
description:
'Optional: flights, military, ais-live-vessels or satellites. Other layers are not trackable; for fires use query "biggest fire".',
'Optional layer hint: flights | military | ais-live-vessels | satellites.',
$position: 1,
},
},
@@ -396,7 +381,7 @@ export const ACTION_DESCRIPTIONS = {
},
annotate_map: {
description:
'Mark places on the 3D map as you talk about them (see WHITEBOARD): pins, highlights, footprints and boundaries, arrows, routes and labels. Give place names (or explicit lat/lng); the app resolves real positions and outlines. Several places may go in one call.',
'Draw annotations on the 3D map to visually point out what you are talking about — like sketching on a whiteboard over the world. Use this whenever you mention a specific place, building, campus, boundary, district, or a relationship between two places, so the user can SEE what you mean. Give place NAMES (preferred) or explicit lat/lng; the app resolves them to real-world positions and real building/area outlines — never guess pixel positions. Call this as you begin describing something, and you may mark several places in one call.',
$position: 1,
parameters: {
properties: {
@@ -413,7 +398,7 @@ export const ACTION_DESCRIPTIONS = {
},
target: {
description:
'Place name to resolve ("Palace of Fine Arts, San Francisco"), or "pointer" for the spot under the cursor. A monument inside a larger landmark gets its own name + city ("Tejano Monument, Austin"), never "X at the Texas State Capitol" (that stacks them on one centroid).',
'Place name to resolve, e.g. "Palace of Fine Arts, San Francisco", "the Pentagon", "Presidio of San Francisco". Preferred over coordinates. For a specific monument/statue/feature that sits within a larger landmark, use its OWN name + city ("Tejano Monument, Austin", "Texas African American History Memorial, Austin") — do NOT phrase it as "X at the Texas State Capitol", which makes the geocoder collapse several of them onto the same centroid so they stack on one spot.',
$position: 2,
},
points: {
@@ -465,24 +450,27 @@ export const ACTION_DESCRIPTIONS = {
},
entityKind: {
description:
'What the target IS (a fact, not a style): building = one structure; compound = campus/grounds/mall/park; district = neighborhood; street = a named road; point_feature = monument/statue/memorial/fountain, anchored as a precise point. Set it whenever you know it.',
'What KIND of thing the target IS — a fact, not a style choice: building = one structure; compound = campus/grounds/mall/park; district = neighborhood or area of a city; street = a named road/corridor; point_feature = monument/statue/memorial/plaque/fountain or other small point landmark. Set it whenever you know it — it routes the resolver to the right footprint source (point_feature anchors monuments as precise points instead of adopting a nearby building outline).',
$position: 2,
},
screenX: {
description:
'Fallback only, for a spot visible in the latest screenshot that you cannot name: normalized x (0=left, 1=right).',
'Fallback only: when you cannot name/geocode the place but can SEE it in the latest viewport screenshot, the normalized horizontal position (0=left, 1=right) of the spot. The app converts it back to a real world point under that pixel.',
$position: 3,
},
screenY: {
description: 'Fallback only: normalized y (0=top, 1=bottom).',
description:
'Fallback only: normalized vertical position (0=top, 1=bottom) of the spot in the latest viewport screenshot.',
$position: 3,
},
toScreenX: {
description: 'For type=arrow: destination x (pixel fallback).',
description:
'For type=arrow: normalized x of the arrow destination from the screenshot (pixel fallback).',
$position: 3,
},
toScreenY: {
description: 'For type=arrow: destination y (pixel fallback).',
description:
'For type=arrow: normalized y of the arrow destination from the screenshot (pixel fallback).',
$position: 3,
},
},
@@ -540,18 +528,18 @@ export const ACTION_DESCRIPTIONS = {
},
analyst_query: {
description:
'Counts, lists, superlatives and attribute filters over records loaded by ENABLED layers — "how many flights over Texas", "biggest fire near LA", "ships headed to Oakland", "next rocket launch". complete:false means the count is a floor ("at least") and rankings cover the records examined; partial names unanswered layers. Items carry lat/lon for a follow-up fly_to_location. ok:false codes: LAYER_OFF (offer to enable), NOT_READY (loading), FOLLOW_UP_MISMATCH (ask again without followUp), UNKNOWN_FIELD/UNKNOWN_SCOPE/BAD_VALUE (retry with the allowed values).',
'Answer questions ABOUT the data currently loaded on the map — counts, lists, superlatives, and attribute filters over live layers (flights, military, ships, fires, earthquakes, satellites, datacenters, dams). Examples: "how many flights over Texas", "biggest fire near LA", "which ships are headed to Oakland", "anything above 40,000 feet", "fastest thing in view". For satellites and infrastructure, counts and ranks cover only bounded examined loaded records; omitted records can change nearest/count. Queries ONLY client-side data from ENABLED layers — if the needed layer is off, say so and offer to enable it. For a follow-up about the previous answer\'s set ("which of those is closest?"), set followUp=true and send only the new filters/sort.',
$position: 1,
parameters: {
properties: {
layers: {
description:
'Layers to query. fires → local-firms; ships → ais-live-vessels; buses/trains → transit; hurricanes → weather-cyclones; bases → military-installations; my receiver → local-adsb. Units differ per layer, so rank speed within one layer.',
'Layers to query. fires/wildfires → local-firms; ships/vessels → ais-live-vessels.',
$position: 2,
},
scope: {
description:
'Always set it (see WHERE). view = a radius around the view centre ("in view", "nearby"); pointer = "around here/this" while pointing (radius at the cursor); region = "over Texas"; radius with center {lat, lon} = "near <place>"; anywhere = no place ("biggest anywhere", "today"). There is no drawn-area scope yet: say so rather than answering for the view.',
'Spatial scope. Default: view (near the camera). Use kind=region for "over Texas"-style asks; kind=anywhere for global questions.',
$position: 2,
properties: {
name: {
@@ -560,23 +548,23 @@ export const ACTION_DESCRIPTIONS = {
$position: 1,
},
km: {
description: 'For kind=radius or pointer.',
description: 'For kind=radius.',
$position: 1,
},
},
},
filters: {
description: `ANDed; values keep their type (numbers, true/false). Altitude is meters (40,000 ft = 12192). Time fields take ISO; ageHours = hours ago. Every layer has id, lat, lon. Fields — ${voiceQueryFieldHint()}.`,
description:
'Attribute predicates, ANDed. ALTITUDE IS METERS (40,000 ft = 12192). Fields: altitudeM, speedMps, military, onGround, aircraftClass, callsign, operator, routeOrigin, routeDestination, originCountry (flights); speedKts, shipType, destination (ships); frp, confidence (fires); magnitude, depthKm, place (earthquakes).',
$position: 1,
},
sortBy: {
description:
'Field to rank by, or "distance" (nearest first unless sortDir=desc).',
description: 'Field to rank by, or "distance" for nearest-first.',
$position: 1,
},
followUp: {
description:
'true = re-filter the last answer (its layers) instead of fresh data.',
'true = re-query the PREVIOUS result set instead of fresh data.',
$position: 1,
},
},
@@ -584,7 +572,7 @@ export const ACTION_DESCRIPTIONS = {
},
next_iss_pass: {
description:
'Next ISS pass over the camera location (or lat/lon): rise time, direction, peak elevation, duration and estimated visibility. Needs the satellites layer to have loaded once; if not, ask the user to enable it.',
"When the user asks when the ISS / the space station will next fly over: returns the next geometric ISS pass with estimated visibility for the current camera location (or an explicit lat/lon) — rise time (ISO + minutes from now), rise compass direction, peak elevation, and duration. Requires the satellites layer to have loaded its catalog at least once this session; if it hasn't, tell the user to enable the satellites layer and try again.",
$position: 1,
parameters: {
properties: {
+13 -1
View File
@@ -1,4 +1,16 @@
import { createActionTools } from '../../../src/voice/actionSchemas.js';
import { ACTION_DESCRIPTIONS } from '../../../src/voice/actionDescriptions.js';
import { ACTION_DESCRIPTIONS } from './toolDescriptions.js';
export const GEV_REALTIME_TOOLS = createActionTools(ACTION_DESCRIPTIONS);
/**
* The session's tools: the app actions, then additional function tools
* whose names no action already uses.
*/
export function realtimeSessionTools(additional = []) {
const names = new Set(GEV_REALTIME_TOOLS.map((tool) => tool.name));
return [
...GEV_REALTIME_TOOLS,
...additional.filter((tool) => !names.has(tool.name)),
];
}
+1 -1
View File
@@ -7,7 +7,7 @@ export {
export { installRouteMiddleware } from './places/routes.js';
export {
makeRateLimiter,
makeOptInRateLimiter,
makeCostRateLimiter,
clientKey,
} from './common/rate-limit.js';
export { haversineKm } from './common/geo.js';
+25 -6
View File
@@ -2,21 +2,35 @@ import {
googleServerApiKey,
keylessGooglePlacesResponse,
} from './google-key.js';
import { makeOptInRateLimiter, clientKey } from '../common/rate-limit.js';
import { makeCostRateLimiter, clientKey } from '../common/rate-limit.js';
import { admitSameSite } from '../common/same-site.js';
import {
projectNearbyPlaces,
projectTextSearchPlaces,
} from '../../../src/data/placeProviderPayloads.js';
// Construct lazily after the standalone environment has loaded.
// undefined = not built yet; null = unlimited; fn = active limiter
// undefined = not built yet; null = the explicit 0 opt-out; fn = active limiter
let _googleRateLimiter;
/** Google cost endpoint (nearby-places). Null = unlimited (default). */
/**
* Requests/min/IP applied when GEV_RATELIMIT_GOOGLE_PER_MIN is unset. This is
* the value the Pinokio build already ships (pinokio/_ENVIRONMENT), so the
* packaged app keeps behaving exactly as it does today and only an
* unconfigured server changes — from unlimited to what the product already
* runs with. Places calls are user-driven (a search, a nearby lookup, an
* installation probe) rather than polled, so this sits far above what the app
* generates and bites only a caller enumerating the endpoint. `.env.example`
* suggests a tighter 60 when the host is not localhost; that still applies.
*/
export const GOOGLE_DEFAULT_PER_MIN = 120;
/** Google cost endpoints (nearby-places + text-search). Null only when set to 0. */
function googleRateLimiter() {
if (_googleRateLimiter === undefined)
_googleRateLimiter = makeOptInRateLimiter(
_googleRateLimiter = makeCostRateLimiter(
process.env.GEV_RATELIMIT_GOOGLE_PER_MIN,
GOOGLE_DEFAULT_PER_MIN,
);
return _googleRateLimiter;
}
@@ -50,6 +64,9 @@ export function googlePlacesContextProxy({
} = {}) {
function install(middlewares) {
middlewares.use('/api/google/nearby-places', async (req, res) => {
// Gate first, like the OpenAI routes: a cross-site caller learns nothing
// about this endpoint's method surface.
if (admitSameSite(req, res)) return;
if (req.method !== 'GET') {
res.statusCode = 405;
res.setHeader('Content-Type', 'application/json');
@@ -80,7 +97,7 @@ export function googlePlacesContextProxy({
}
const { latitude, longitude } = coordinates;
// Opt-in per-IP throttle (GEV_RATELIMIT_GOOGLE_PER_MIN). No-op when unset.
// Per-IP throttle (GEV_RATELIMIT_GOOGLE_PER_MIN). On by default; 0 disables.
// Inlined (not the shared helper) so the 429 body keeps this endpoint's
// `places: []` contract that the client expects on every error response.
const _grl = googleRateLimiter();
@@ -161,6 +178,8 @@ export function googlePlacesContextProxy({
// a view-biased Text Search lands on the actual feature. Same key, field
// mask, throttle, and `places: []` error contract as nearby-places above.
middlewares.use('/api/google/text-search', async (req, res) => {
// Paid like nearby-places, so gated the same way, first.
if (admitSameSite(req, res)) return;
if (req.method !== 'GET') {
res.statusCode = 405;
res.setHeader('Content-Type', 'application/json');
@@ -200,7 +219,7 @@ export function googlePlacesContextProxy({
}
const { latitude, longitude } = coordinates;
// Opt-in per-IP throttle (GEV_RATELIMIT_GOOGLE_PER_MIN). No-op when unset.
// Per-IP throttle (GEV_RATELIMIT_GOOGLE_PER_MIN). On by default; 0 disables.
// Inlined (like nearby-places) so the 429 body keeps the `places: []`
// contract the client expects on every error response.
const _grl = googleRateLimiter();
+9 -2
View File
@@ -1,8 +1,11 @@
import { fileURLToPath } from 'node:url';
import { defineConfig, loadEnv } from 'vite';
import { resolveAllowedHosts } from '../../build/allowedHosts.js';
import { createBrowserViteConfig } from '../../build/vite.js';
import { localProviderPlugins } from '../providers/local.js';
import { localMcpPlugin } from '../mcp/plugin.js';
import { apiNotFoundPlugin } from './api-not-found.js';
import { standaloneVoiceTools } from './voiceTools.js';
const root = fileURLToPath(new URL('../../', import.meta.url));
@@ -13,12 +16,16 @@ export default defineConfig(({ command, mode }) => {
if (process.env[key] === undefined) process.env[key] = value;
}
return createBrowserViteConfig({
plugins: [...localProviderPlugins(), apiNotFoundPlugin()],
plugins: [
...localProviderPlugins({ realtime: { tools: standaloneVoiceTools() } }),
localMcpPlugin(),
apiNotFoundPlugin(),
],
googleApiKey: process.env.GOOGLE_MAPS_API_KEY,
cesiumToken: process.env.CESIUM_ION_TOKEN,
host: process.env.HOST,
port: process.env.PORT,
naturalVoice: process.env.GEV_NATURAL_VOICE,
allowedHosts: resolveAllowedHosts(process.env.GEV_ALLOWED_HOSTS),
command,
});
});
+18
View File
@@ -0,0 +1,18 @@
/**
* The voice session's tools for the standalone server: the app actions plus
* the catalog tools voice offers (see src/tools/surfaces.js), which the
* browser runs through the same catalog.
*/
import {
coreTools,
toFunctionTools,
toolsForSurface,
} from '../../src/tools/index.js';
import { realtimeSessionTools } from '../providers/openai/tools.js';
export function standaloneVoiceTools() {
return realtimeSessionTools(
toFunctionTools(toolsForSurface(coreTools, 'voice')),
);
}
+305
View File
@@ -0,0 +1,305 @@
/**
* Embed mode and views in the running app.
*
* With `?embed=1` the app shows only the globe: clean view, with the HUD,
* welcome and setup prompts hidden. A page that frames it changes what it
* shows by posting `{ type: 'gev:view', id, view }`; the app applies the view
* through its own actions and answers `{ type: 'gev:view-applied', id, ok,
* steps }`. It announces `{ type: 'gev:ready' }` once it can take views.
*
* Annotations in a share link are drawn once the link has been restored,
* embedded or not.
*/
import * as Cesium from 'cesium';
import { announceNavigationAuthority } from '../navigationPolicy.js';
import { annotationsFromParams, createView } from '../view/index.js';
export const EMBED_VIEW_MESSAGE = 'gev:view';
export const EMBED_APPLIED_MESSAGE = 'gev:view-applied';
export const EMBED_READY_MESSAGE = 'gev:ready';
const FOLLOW_LAYERS = {
aircraft: 'flights',
military_aircraft: 'military',
satellite: 'satellites',
};
const FOLLOW_ATTEMPTS = 20;
const FOLLOW_RETRY_MS = 1000;
const FLIGHT_SECONDS = 2;
/**
* Whether this page shows the app embedded: `?embed=1` when another page
* frames it, or inline when a panel page loads the app into itself and sets
* `globalThis.GEV_EMBED_INLINE` first.
*/
export function isEmbedded(location = globalThis.location) {
return (
isEmbeddedInline() ||
new URLSearchParams(location?.search || '').get('embed') === '1'
);
}
/** Whether a panel page loaded the app into itself. */
export function isEmbeddedInline() {
return globalThis.GEV_EMBED_INLINE === true;
}
// A panel keeps drawing at about this rate when its host stops animation
// frames; see keepPanelRendering.
const PANEL_FRAME_MS = 33;
const MISSED_FRAMES_MS = 250;
/** A render error's details, including the plain objects workers report. */
function describeError(error) {
if (error instanceof Error)
return `${error.name}: ${error.message}\n${error.stack}`;
try {
return JSON.stringify(error, Object.getOwnPropertyNames(error ?? {}));
} catch {
return String(error);
}
}
/**
* Keep the globe drawing in an inline panel. Some hosts report a panel on
* screen as hidden, which stops the browser's animation frames and with them
* Cesium's render loop; while frames stop arriving, draw from a timer.
* Returns a function that stops it.
*/
export function keepPanelRendering(
viewer,
{ windowRef = globalThis.window, now = () => performance.now() } = {},
) {
let lastFrame = now();
let frameRequest = null;
const onFrame = () => {
lastFrame = now();
frameRequest = windowRef.requestAnimationFrame(onFrame);
};
frameRequest = windowRef.requestAnimationFrame(onFrame);
// Cesium stops drawing after a render error, and so does the timer.
// Report the error in full: workers report plain objects, which Cesium
// prints as [object Object].
let failed = false;
const removeErrorListener = viewer.scene?.renderError?.addEventListener(
(_scene, error) => {
failed = true;
console.error(
"[God's Eye View panel] render error:",
describeError(error),
);
},
);
const timer = windowRef.setInterval(() => {
if (
failed ||
now() - lastFrame < MISSED_FRAMES_MS ||
viewer.isDestroyed?.()
)
return;
viewer.resize();
viewer.render();
}, PANEL_FRAME_MS);
return () => {
windowRef.clearInterval(timer);
windowRef.cancelAnimationFrame(frameRequest);
removeErrorListener?.();
};
}
const delay = (ms, signal) =>
new Promise((resolve) => {
const timer = setTimeout(resolve, ms);
signal?.addEventListener(
'abort',
() => {
clearTimeout(timer);
resolve();
},
{ once: true },
);
});
/** Fly the camera to a view's camera and resolve when the flight ends. */
function flyTo(viewer, camera) {
announceNavigationAuthority('embed-view');
return new Promise((resolve) => {
viewer.camera.flyTo({
destination: Cesium.Cartesian3.fromDegrees(
camera.lon,
camera.lat,
camera.altitude_m,
),
orientation: {
heading: Cesium.Math.toRadians(camera.heading_deg),
pitch: Cesium.Math.toRadians(camera.pitch_deg),
roll: 0,
},
duration: FLIGHT_SECONDS,
complete: () => resolve(true),
cancel: () => resolve(false),
});
});
}
/**
* Make the app show a view: style and map, exactly the view's layers,
* annotations, the camera, and the followed entity. Each step runs through the
* app's own actions (`run(name, args)`); a failed step is reported and the
* rest still run. Resolves to the steps and whether each succeeded.
*/
export async function applyView(
view,
{ viewer, dataManager, run, signal, retryMs = FOLLOW_RETRY_MS },
) {
const steps = [];
const act = async (name, args) => {
let result = null;
try {
result = await run(name, args);
} catch (error) {
result = { ok: false, error: error?.message || String(error) };
}
steps.push({
step: name,
ok: result?.ok !== false,
...(result?.error ? { error: result.error } : {}),
});
return result;
};
if (view.style) await act('set_visual_style', { style: view.style });
if (view.map) await act('set_map_stack', { stack: view.map });
const wanted = new Set(view.layers);
for (const layer of dataManager.getAll()) {
if (layer.enabled && !wanted.has(layer.id))
await act('set_layer_visibility', { layerId: layer.id, enabled: false });
}
for (const layerId of view.layers) {
if (!dataManager.isEnabled(layerId))
await act('set_layer_visibility', { layerId, enabled: true });
}
if (!view.follow) await act('stop_tracking', {});
// Annotations go first: drawing them may frame the marks, and the view's
// own camera should have the last word.
await act('clear_annotations', {});
if (view.annotations.length)
await act('annotate_map', { annotations: [...view.annotations] });
if (!view.follow) {
steps.push({ step: 'camera', ok: await flyTo(viewer, view.camera) });
return steps;
}
// A followed entity owns the camera, and a camera flight would end the
// follow, so fly to the view only while the entity is not there yet. It
// appears once its layer has data; retry until then.
const layerId = FOLLOW_LAYERS[view.follow.kind];
const follow = async () => {
try {
return (
(await run('track_entity', { query: view.follow.id, layerId }))?.ok ===
true
);
} catch {
return false;
}
};
let followed = await follow();
if (!followed) {
steps.push({ step: 'camera', ok: await flyTo(viewer, view.camera) });
for (let attempt = 1; attempt < FOLLOW_ATTEMPTS && !followed; attempt++) {
if (signal?.aborted) break;
await delay(retryMs, signal);
followed = await follow();
}
}
steps.push({ step: 'follow', ok: followed });
if (followed && view.follow.cockpit)
await act('control_cockpit', { action: 'enter', targetLayer: layerId });
return steps;
}
/**
* Install embed mode when the page asks for it, and draw any annotations the
* opening link carries. Returns a function that removes the message handler.
*/
export function installViews({
shell,
viewer,
dataManager,
run,
signal,
location = globalThis.location,
windowRef = globalThis.window,
}) {
const ready = Promise.resolve(shell.initialRestorePromise).catch(() => {});
const linked = annotationsFromParams(
new URLSearchParams(String(location?.hash || '').replace(/^#/, '')),
);
if (linked.length)
void ready.then(() => {
if (!signal?.aborted) void run('annotate_map', { annotations: linked });
});
if (!isEmbedded(location)) return () => {};
windowRef.document.body.classList.add('ui-embed');
shell.setCleanView?.(true);
const stopRendering = isEmbeddedInline()
? keepPanelRendering(viewer, { windowRef })
: () => {};
// A framing page talks to the app across frames; an inline panel shares
// the page with it and talks through the page's own window.
const peer = isEmbeddedInline() ? windowRef : windowRef.parent;
// Answers go back only to the origin that asked. An inline panel posts to
// its own window, and a sandboxed framing page has no origin to name.
const post = (message, origin = '*') => {
if (peer && (peer !== windowRef || isEmbeddedInline()))
peer.postMessage(message, origin);
};
// Views apply one at a time, in the order they arrive.
let queue = ready;
const onMessage = (event) => {
if (event.source !== peer || event.data?.type !== EMBED_VIEW_MESSAGE)
return;
const { id = null } = event.data;
const replyOrigin =
isEmbeddedInline() || !event.origin || event.origin === 'null'
? '*'
: event.origin;
let view;
try {
view = createView(event.data.view);
} catch (error) {
post(
{
type: EMBED_APPLIED_MESSAGE,
id,
ok: false,
error: error.message,
},
replyOrigin,
);
return;
}
queue = queue.then(async () => {
if (signal?.aborted) return;
const steps = await applyView(view, { viewer, dataManager, run, signal });
post(
{
type: EMBED_APPLIED_MESSAGE,
id,
ok: steps.every((step) => step.ok),
steps,
},
replyOrigin,
);
});
};
windowRef.addEventListener('message', onMessage);
void ready.then(() => {
if (!signal?.aborted) post({ type: EMBED_READY_MESSAGE });
});
return () => {
stopRendering();
windowRef.removeEventListener('message', onMessage);
};
}
+325
View File
@@ -0,0 +1,325 @@
import assert from 'node:assert/strict';
import test from 'node:test';
import {
applyView,
installViews,
isEmbedded,
keepPanelRendering,
} from './embed.js';
import { createView } from '../view/index.js';
const fakeViewer = () => {
const flights = [];
return {
flights,
camera: {
flyTo(options) {
flights.push(options);
options.complete();
},
},
};
};
const fakeLayers = (enabled) => ({
getAll: () =>
['flights', 'earthquakes', 'ais-live-vessels'].map((id) => ({
id,
enabled: enabled.has(id),
})),
isEnabled: (id) => enabled.has(id),
});
const recorder = (answers = {}) => {
const calls = [];
const run = async (name, args) => {
calls.push([name, args]);
const answer = answers[name];
return typeof answer === 'function'
? answer(calls)
: (answer ?? { ok: true });
};
return { calls, run };
};
test('embed mode is opt-in by query parameter', () => {
assert.equal(isEmbedded({ search: '?embed=1' }), true);
assert.equal(isEmbedded({ search: '?welcome=0' }), false);
});
test('a view is applied through the app actions in order', async () => {
const viewer = fakeViewer();
const { calls, run } = recorder();
const view = createView({
camera: { lat: 25, lon: 121, altitude_m: 300000, pitch_deg: -40 },
layers: ['ais-live-vessels'],
style: 'thermal',
map: 'osm',
annotations: [{ type: 'pin', target: 'Taipei 101' }],
});
const steps = await applyView(view, {
viewer,
dataManager: fakeLayers(new Set(['earthquakes'])),
run,
});
assert.deepEqual(
calls.map(([name, args]) => [
name,
args.layerId ?? args.style ?? args.stack ?? null,
args.enabled ?? null,
]),
[
['set_visual_style', 'thermal', null],
['set_map_stack', 'osm', null],
['set_layer_visibility', 'earthquakes', false],
['set_layer_visibility', 'ais-live-vessels', true],
['stop_tracking', null, null],
['clear_annotations', null, null],
['annotate_map', null, null],
],
);
assert.equal(viewer.flights.length, 1);
assert.ok(steps.every((step) => step.ok));
assert.ok(steps.some((step) => step.step === 'camera'));
});
test('a followed entity is retried until its layer has it', async () => {
let tries = 0;
const { run } = recorder({
track_entity: () => ({ ok: ++tries >= 3 }),
});
const view = createView({
camera: { lat: 0, lon: 0 },
follow: { kind: 'aircraft', id: 'abc123' },
});
const steps = await applyView(view, {
viewer: fakeViewer(),
dataManager: fakeLayers(new Set()),
run,
retryMs: 1,
});
assert.equal(tries, 3);
assert.deepEqual(steps.at(-1), { step: 'follow', ok: true });
});
test('an embedded page takes views only from its parent and answers it', async () => {
const posted = [];
const targets = [];
const parent = {
postMessage: (message, target) => {
posted.push(message);
targets.push(target);
},
};
const windowRef = new EventTarget();
windowRef.parent = parent;
windowRef.document = { body: { classList: new Set() } };
windowRef.document.body.classList.add = Set.prototype.add;
const shell = {
initialRestorePromise: Promise.resolve(),
clean: null,
setCleanView(value) {
this.clean = value;
},
};
const { calls, run } = recorder();
const remove = installViews({
shell,
viewer: fakeViewer(),
dataManager: fakeLayers(new Set()),
run,
location: { search: '?embed=1', hash: '' },
windowRef,
});
await new Promise((resolve) => setTimeout(resolve, 0));
assert.equal(shell.clean, true);
assert.deepEqual(posted, [{ type: 'gev:ready' }]);
const send = (source, data, origin = 'https://host.example') => {
const event = new Event('message');
Object.assign(event, { source, data, origin });
windowRef.dispatchEvent(event);
};
send({}, { type: 'gev:view', id: 1, view: { camera: { lat: 1, lon: 2 } } });
send(parent, { type: 'gev:view', id: 2, view: { camera: {} } });
send(parent, {
type: 'gev:view',
id: 3,
view: { camera: { lat: 1, lon: 2 }, layers: ['flights'] },
});
await new Promise((resolve) => setTimeout(resolve, 10));
assert.deepEqual(posted[1], {
type: 'gev:view-applied',
id: 2,
ok: false,
error: 'A view needs a camera lat and lon',
});
assert.equal(posted[2].id, 3);
assert.equal(posted[2].ok, true);
assert.ok(
calls.some(
([name, args]) =>
name === 'set_layer_visibility' && args.layerId === 'flights',
),
);
assert.equal(posted.filter((message) => message.id === 1).length, 0);
// Ready names no data and goes to any parent; answers go only to the
// origin that asked.
assert.deepEqual(targets, [
'*',
'https://host.example',
'https://host.example',
]);
remove();
});
test('an app loaded inline into a panel page talks through its own window', async () => {
const posted = [];
const windowRef = new EventTarget();
windowRef.parent = windowRef;
windowRef.postMessage = (message) => posted.push(message);
windowRef.document = { body: { classList: new Set() } };
windowRef.document.body.classList.add = Set.prototype.add;
Object.assign(windowRef, {
requestAnimationFrame: () => 1,
cancelAnimationFrame() {},
setInterval: () => 1,
clearInterval() {},
});
globalThis.GEV_EMBED_INLINE = true;
try {
assert.equal(isEmbedded({ search: '' }), true);
const remove = installViews({
shell: { initialRestorePromise: Promise.resolve() },
viewer: fakeViewer(),
dataManager: fakeLayers(new Set()),
run: recorder().run,
location: { search: '', hash: '' },
windowRef,
});
await new Promise((resolve) => setTimeout(resolve, 0));
assert.deepEqual(posted, [{ type: 'gev:ready' }]);
const event = new Event('message');
Object.assign(event, {
source: windowRef,
data: { type: 'gev:view', id: 7, view: { camera: { lat: 1, lon: 2 } } },
});
windowRef.dispatchEvent(event);
await new Promise((resolve) => setTimeout(resolve, 10));
assert.equal(posted[1].id, 7);
assert.equal(posted[1].ok, true);
remove();
} finally {
delete globalThis.GEV_EMBED_INLINE;
}
});
test('annotations in a link are drawn after it restores, embedded or not', async () => {
const { calls, run } = recorder();
const an = encodeURIComponent(
JSON.stringify([{ type: 'pin', target: 'Austin' }]),
);
installViews({
shell: { initialRestorePromise: Promise.resolve() },
viewer: fakeViewer(),
dataManager: fakeLayers(new Set()),
run,
location: { search: '', hash: `#v=2&lat=1&lon=2&an=${an}` },
windowRef: new EventTarget(),
});
await new Promise((resolve) => setTimeout(resolve, 0));
assert.deepEqual(calls, [
['annotate_map', { annotations: [{ type: 'pin', target: 'Austin' }] }],
]);
});
test('a panel keeps drawing from a timer while animation frames stop', () => {
let clock = 0;
let tick = null;
let frame = null;
const windowRef = {
requestAnimationFrame: (fn) => ((frame = fn), 1),
cancelAnimationFrame: () => (frame = null),
setInterval: (fn) => ((tick = fn), 1),
clearInterval: () => (tick = null),
};
const drawn = [];
const viewer = {
resize: () => drawn.push('resize'),
render: () => drawn.push('render'),
};
const stop = keepPanelRendering(viewer, { windowRef, now: () => clock });
// Frames arrive: the render loop draws, so the timer does nothing.
clock = 100;
frame();
tick();
assert.deepEqual(drawn, []);
// Frames stop, as in a panel its host reports hidden: the timer draws.
clock = 400;
tick();
assert.deepEqual(drawn, ['resize', 'render']);
stop();
assert.equal(tick, null);
assert.equal(frame, null);
});
test('a panel reports a render error in full and stops drawing', () => {
let listener = null;
let tick = null;
const windowRef = {
requestAnimationFrame: () => 1,
cancelAnimationFrame() {},
setInterval: (fn) => ((tick = fn), 1),
clearInterval() {},
};
const drawn = [];
const viewer = {
resize() {},
render: () => drawn.push('render'),
scene: {
renderError: {
addEventListener: (fn) => ((listener = fn), () => (listener = null)),
},
},
};
const logged = [];
const original = console.error;
console.error = (...args) => logged.push(args.join(' '));
try {
let clock = 0;
const stop = keepPanelRendering(viewer, { windowRef, now: () => clock });
listener(viewer.scene, { name: 'RuntimeError', message: 'worker failed' });
assert.match(logged[0], /render error: .*"message":"worker failed"/);
clock = 1000;
tick();
assert.deepEqual(drawn, []);
stop();
assert.equal(listener, null);
} finally {
console.error = original;
}
});
test('a followed aircraft owns the camera, and cockpit view is entered after it', async () => {
const viewer = fakeViewer();
const { calls, run } = recorder();
const steps = await applyView(
createView({
camera: { lat: 32.7, lon: -117.2 },
follow: { kind: 'military_aircraft', id: 'AE1234', cockpit: true },
}),
{ viewer, dataManager: fakeLayers(new Set()), run, retryMs: 0 },
);
// Following worked at once, so no camera flight ended it.
assert.equal(viewer.flights.length, 0);
assert.deepEqual(calls.slice(-2), [
['track_entity', { query: 'ae1234', layerId: 'military' }],
['control_cockpit', { action: 'enter', targetLayer: 'military' }],
]);
assert.ok(steps.every((step) => step.ok));
assert.throws(
() =>
createView({
camera: { lat: 0, lon: 0 },
follow: { kind: 'satellite', id: '25544', cockpit: true },
}),
/no cockpit view/,
);
});
+662
View File
@@ -0,0 +1,662 @@
/**
* The God's Eye View panel's script, run inside the panel page a host shows.
* It is sent as source text (see src/tools/globePanel.js), so it must not use
* imports or anything outside the function.
*
* The page lives on the host's site, and hosts may refuse other addresses,
* such as a server on the user's machine. So the panel never requests the
* app's server itself: every request for the app's own paths goes to the
* panel's MCP server as a call to `config.toolName`, which requests the path
* from the app's server. The app's built page, scripts, styles, images and
* data all arrive that way; map imagery from providers loads directly.
*/
export function panelRuntime(config) {
const status = document.getElementById('status');
const open = document.getElementById('open');
const expand = document.getElementById('expand');
const say = (text) => {
if (status.isConnected) status.textContent = text;
};
let blocked = null;
document.addEventListener('securitypolicyviolation', (event) => {
blocked = `${event.effectiveDirective} blocked ${event.blockedURI || 'a resource'}`;
});
// JSON-RPC with the host.
let nextId = 1;
const pending = new Map();
const send = (message) =>
window.parent.postMessage({ jsonrpc: '2.0', ...message }, '*');
const request = (method, params) =>
new Promise((resolve, reject) => {
const id = nextId++;
pending.set(id, { resolve, reject });
send({ id, method, params });
});
const notify = (method, params) => send({ method, params });
// Requests to the app's server, through the MCP server.
const MAX_CALLS = 6;
let calls = 0;
const waiting = [];
async function callTool(args) {
if (calls >= MAX_CALLS) await new Promise((go) => waiting.push(go));
calls += 1;
try {
const result = await request('tools/call', {
name: config.toolName,
// The server refuses the panel's requests without its key.
arguments: { ...args, key: config.panelKey },
});
if (result?.isError || !result?.structuredContent)
throw new Error(result?.content?.[0]?.text || 'The request failed');
return result.structuredContent;
} finally {
calls -= 1;
waiting.shift()?.();
}
}
const toBase64 = (bytes) => {
let binary = '';
for (let start = 0; start < bytes.length; start += 0x8000)
binary += String.fromCharCode(...bytes.subarray(start, start + 0x8000));
return btoa(binary);
};
const fromBase64 = (text) =>
Uint8Array.from(atob(text), (char) => char.charCodeAt(0));
async function bodyBytes(body) {
if (body == null) return null;
if (typeof body === 'string') return new TextEncoder().encode(body);
if (body instanceof ArrayBuffer) return new Uint8Array(body);
if (ArrayBuffer.isView(body))
return new Uint8Array(body.buffer, body.byteOffset, body.byteLength);
return new Uint8Array(await new Response(body).arrayBuffer());
}
const NULL_BODY = new Set([101, 204, 205, 304]);
const held = new Map();
/** Request a path on the app's server; resolves to a Response. */
function appFetch(path, init = {}) {
const method = (init.method || 'GET').toUpperCase();
// Built files never change while the panel runs, so each loads once.
const shared = method === 'GET' && path.startsWith(config.panelBase);
if (shared && held.has(path)) return held.get(path).then(respond);
const loading = (async () => {
const headers = {};
new Headers(init.headers || {}).forEach((value, name) => {
headers[name] = value;
});
const bytes = await bodyBytes(init.body);
let part = await callTool({
path,
method,
headers,
...(bytes ? { body: toBase64(bytes) } : {}),
});
const parts = [fromBase64(part.body)];
while (part.nextOffset !== undefined) {
if (init.signal?.aborted) break;
part = await callTool({ id: part.id, offset: part.nextOffset });
parts.push(fromBase64(part.body));
}
let body = new Blob(parts);
if (part.encoding === 'gzip')
body = await new Response(
body.stream().pipeThrough(new DecompressionStream('gzip')),
).blob();
return {
body: await body.arrayBuffer(),
status: part.status,
statusText: part.statusText,
headers: part.headers,
};
})();
if (shared) {
held.set(path, loading);
loading.catch(() => held.delete(path));
}
const result = loading.then(respond);
if (!init.signal) return result;
return Promise.race([
result,
new Promise((_, reject) => {
const abort = () =>
reject(new DOMException('The request was aborted', 'AbortError'));
if (init.signal.aborted) abort();
init.signal.addEventListener('abort', abort, { once: true });
}),
]);
}
function respond({ body, status, statusText, headers }) {
return new Response(NULL_BODY.has(status) ? null : body.slice(0), {
status,
statusText,
headers,
});
}
// The app's own paths: relative ones, absolute ones on this page's site
// (matched by scheme and host, since a host may use its own scheme), and
// ones under the app's address in the page, an https address for code
// that needs one (`config.appBaseUrl`).
const page = new URL(document.baseURI);
const appOrigin = new URL(config.appBaseUrl).origin;
function appPath(value) {
let url;
try {
url = new URL(String(value), document.baseURI);
} catch {
return null;
}
const own =
url.origin === appOrigin ||
(url.protocol === page.protocol && url.host === page.host);
return own ? url.pathname + url.search : null;
}
const fileUrls = new Map();
/**
* A data: URL holding an app file, for elements that load by URL. Hosts
* allow data: images where some refuse blob: ones. Files of the panel
* build never change and are kept; others, such as live camera frames
* that change address on every refresh, are not.
*/
function appFileUrl(path) {
if (fileUrls.has(path)) return fileUrls.get(path);
const loading = appFetch(path)
.then((response) => response.blob())
.then(
(blob) =>
new Promise((resolve, reject) => {
const reader = new FileReader();
reader.onload = () => resolve(reader.result);
reader.onerror = () => reject(reader.error);
reader.readAsDataURL(blob);
}),
);
if (path.startsWith(config.panelBase)) {
fileUrls.set(path, loading);
loading.catch(() => fileUrls.delete(path));
}
return loading;
}
function installRequests() {
const nativeFetch = window.fetch.bind(window);
window.fetch = async (input, init = {}) => {
const isRequest = input instanceof Request;
const path = appPath(isRequest ? input.url : input);
if (path === null) return nativeFetch(input, init);
if (!isRequest) return appFetch(path, init);
return appFetch(path, {
method: init.method || input.method,
headers: init.headers || input.headers,
body:
init.body ??
(['GET', 'HEAD'].includes(input.method)
? undefined
: await input.arrayBuffer()),
signal: init.signal || input.signal,
});
};
// XMLHttpRequest for app paths: answered from appFetch, with the
// instance's state replaced, and the usual events dispatched.
const xhr = XMLHttpRequest.prototype;
const nativeOpen = xhr.open;
const nativeSend = xhr.send;
const nativeHeader = xhr.setRequestHeader;
const nativeAbort = xhr.abort;
xhr.open = function (method, url, ...rest) {
const path = appPath(url);
if (path === null) {
this.__gev = null;
return nativeOpen.call(this, method, url, ...rest);
}
this.__gev = { method, path, headers: {}, controller: null };
setState(this, { readyState: 1 });
this.dispatchEvent(new Event('readystatechange'));
};
xhr.setRequestHeader = function (name, value) {
if (!this.__gev) return nativeHeader.call(this, name, value);
this.__gev.headers[name] = value;
};
xhr.abort = function () {
if (!this.__gev) return nativeAbort.call(this);
this.__gev.controller?.abort();
};
xhr.send = function (body) {
const state = this.__gev;
if (!state) return nativeSend.call(this, body);
state.controller = new AbortController();
appFetch(state.path, {
method: state.method,
headers: state.headers,
body,
signal: state.controller.signal,
})
.then(async (response) => {
const type = this.responseType;
const value =
type === 'arraybuffer'
? await response.arrayBuffer()
: type === 'blob'
? await response.blob()
: await response.text();
const text = typeof value === 'string' ? value : null;
let parsed = value;
if (type === 'json') {
try {
parsed = JSON.parse(text);
} catch {
parsed = null;
}
} else if (type === 'document')
parsed = new DOMParser().parseFromString(text, 'text/html');
const headers = [...response.headers]
.map(([name, val]) => `${name}: ${val}`)
.join('\r\n');
setState(this, {
readyState: 4,
status: response.status,
statusText: response.statusText,
response: parsed,
responseText: type === '' || type === 'text' ? text : '',
responseURL: new URL(state.path, document.baseURI).href,
getAllResponseHeaders: () => headers,
getResponseHeader: (name) => response.headers.get(name),
});
for (const name of ['readystatechange', 'load', 'loadend'])
this.dispatchEvent(
name === 'readystatechange'
? new Event(name)
: new ProgressEvent(name),
);
})
.catch((error) => {
setState(this, { readyState: 4, status: 0 });
const name = error?.name === 'AbortError' ? 'abort' : 'error';
for (const event of ['readystatechange', name, 'loadend'])
this.dispatchEvent(
event === 'readystatechange'
? new Event(event)
: new ProgressEvent(event),
);
});
};
function setState(target, values) {
for (const [name, value] of Object.entries(values))
Object.defineProperty(target, name, {
configurable: true,
...(typeof value === 'function' ? { value } : { get: () => value }),
});
}
const nativeBeacon = navigator.sendBeacon?.bind(navigator);
if (nativeBeacon)
navigator.sendBeacon = (url, data) => {
const path = appPath(url);
if (path === null) return nativeBeacon(url, data);
appFetch(path, { method: 'POST', body: data }).catch(() => {});
return true;
};
// Images name app files by URL; give them the file as a data: URL.
const image = Object.getOwnPropertyDescriptor(
HTMLImageElement.prototype,
'src',
);
Object.defineProperty(HTMLImageElement.prototype, 'src', {
...image,
set(value) {
const path = appPath(value);
if (path === null) return image.set.call(this, value);
appFileUrl(path).then(
(url) => image.set.call(this, url),
() => this.dispatchEvent(new Event('error')),
);
},
});
// Workers on the app's server start from a small worker here that is
// handed the script once it has loaded; messages wait until then.
const NativeWorker = window.Worker;
const BOOT =
'const early=[];let ready=false;' +
"addEventListener('message',e=>{if(e.data&&e.data.__gevScript){" +
'const go=()=>{ready=true;for(const d of early)dispatchEvent(new MessageEvent("message",{data:d}))};' +
'if(e.data.module)import(e.data.__gevScript).then(go);else{importScripts(e.data.__gevScript);go()}' +
'e.stopImmediatePropagation();return}' +
'if(!ready){early.push(e.data);e.stopImmediatePropagation()}});';
window.Worker = function Worker(url, options) {
const path = appPath(url);
if (path === null || String(url).startsWith('blob:'))
return new NativeWorker(url, options);
const worker = new NativeWorker(
URL.createObjectURL(new Blob([BOOT], { type: 'text/javascript' })),
options,
);
appFetch(path)
.then((response) => response.blob())
.then((blob) => URL.createObjectURL(blob))
.then((script) =>
worker.postMessage({
__gevScript: script,
module: options?.type === 'module',
}),
);
return worker;
};
window.Worker.prototype = NativeWorker.prototype;
// Some hosts drop the GPU contents of a panel's 2D canvases while they
// consider the panel off screen, and the overlays then cover the globe
// in black. Keep 2D canvases in memory instead, which hosts do not drop.
const getContext = HTMLCanvasElement.prototype.getContext;
HTMLCanvasElement.prototype.getContext = function (type, options) {
return type === '2d'
? getContext.call(this, type, { ...options, willReadFrequently: true })
: getContext.call(this, type, options);
};
// Markup the app adds as HTML text would request its images from this
// page's site at once; hold their addresses until the files are here.
const htmlWithHeldImages = (html) =>
String(html).replace(
/(<(?:img|source)\b[^>]*?\s)src=(["'])([^"']*)\2/gi,
(tag, before, quote, value) =>
appPath(value) === null
? tag
: `${before}data-gev-src=${quote}${value}${quote}`,
);
const fillHeldImages = (root) => {
for (const element of root?.querySelectorAll?.('[data-gev-src]') ?? []) {
const path = appPath(element.getAttribute('data-gev-src'));
element.removeAttribute('data-gev-src');
appFileUrl(path).then((url) => element.setAttribute('src', url));
}
};
const innerHtml = Object.getOwnPropertyDescriptor(
Element.prototype,
'innerHTML',
);
Object.defineProperty(Element.prototype, 'innerHTML', {
...innerHtml,
set(value) {
innerHtml.set.call(this, htmlWithHeldImages(value));
fillHeldImages(this);
},
});
const insertHtml = Element.prototype.insertAdjacentHTML;
Element.prototype.insertAdjacentHTML = function (position, html) {
insertHtml.call(this, position, htmlWithHeldImages(html));
fillHeldImages(this.parentNode ?? this);
};
// Markup and styles the app adds later name app files too.
new MutationObserver((records) => {
for (const record of records)
for (const node of record.addedNodes) {
if (node.nodeType !== 1) continue;
if (node.nodeName === 'STYLE') void inlineStyleUrls(node);
for (const element of [node, ...node.querySelectorAll('img[src]')]) {
if (element.nodeName !== 'IMG') continue;
const path = appPath(element.getAttribute('src'));
if (path !== null)
appFileUrl(path).then((url) => element.setAttribute('src', url));
}
}
}).observe(document.documentElement, { childList: true, subtree: true });
}
/** Replace url() references to app files in CSS with data: URLs. */
async function cssWithFiles(text, baseUrl) {
const found = new Map();
for (const match of text.matchAll(/url\(\s*(['"]?)([^'")]+)\1\s*\)/g)) {
const reference = match[2];
if (/^(data|blob):/.test(reference)) continue;
const path = appPath(new URL(reference, baseUrl).href);
if (path !== null && !found.has(reference))
found.set(
reference,
appFileUrl(path).catch(() => null),
);
}
let result = text;
for (const [reference, loading] of found) {
const url = await loading;
if (url) result = result.split(reference).join(url);
}
return result;
}
async function inlineStyleUrls(style) {
if (!/url\(/.test(style.textContent)) return;
style.textContent = await cssWithFiles(style.textContent, document.baseURI);
}
async function addStylesheet(href) {
const path = appPath(href);
const style = document.createElement('style');
document.head.appendChild(style);
if (path === null) {
const link = document.createElement('link');
link.rel = 'stylesheet';
link.href = href;
style.replaceWith(link);
return;
}
const text = await (await appFetch(path)).text();
style.textContent = await cssWithFiles(
text,
new URL(path, document.baseURI).href,
);
}
function runScript(text, type) {
const script = document.createElement('script');
if (type) script.type = type;
script.textContent = text;
document.body.appendChild(script);
}
/**
* Load God's Eye View into this page from the app's panel build, in
* inline embed mode, starting at the first view as its link would.
*/
async function startApp(url, view) {
window.GEV_EMBED_INLINE = true;
window.GEV_APP_BASE_URL = config.appBaseUrl;
try {
history.replaceState(
null,
'',
location.href.split('#')[0] + new URL(url).hash,
);
// The app restores this view from its link, following included, so
// it needs applying again only for what links cannot carry: cockpit.
if (queued === view && !view.follow?.cockpit) queued = null;
} catch {
// The view still arrives once the app is ready.
}
installRequests();
const response = await appFetch(config.panelBase);
if (!response.ok)
throw new Error(
response.status === 404
? 'the app has no panel build (npm run build:panel)'
: `the app answered ${response.status}`,
);
const built = new DOMParser().parseFromString(
await response.text(),
'text/html',
);
window.CESIUM_BASE_URL = `${config.panelBase}cesium/`;
const [prelude] = await Promise.all([
appFetch(config.panelBase + config.workerPreludePath).then((r) =>
r.text(),
),
...[...built.head.querySelectorAll('link[rel="stylesheet"]')].map(
(link) => addStylesheet(link.getAttribute('href')),
),
]);
// Cesium's script sets CESIUM_WORKERS, its workers as one script it
// starts from memory. Workers request files themselves, which this page
// cannot answer, so the prelude, holding those files, runs ahead of it.
let workers;
Object.defineProperty(window, 'CESIUM_WORKERS', {
configurable: true,
get: () => workers,
set: (script) => {
workers = prelude + script;
},
});
// Images in the markup load their files once added; give them the
// files instead of letting them request this page's site.
const images = [];
for (const image of built.body.querySelectorAll('img[src]')) {
const path = appPath(image.getAttribute('src'));
if (path === null) continue;
image.removeAttribute('src');
images.push([image, path]);
}
for (const node of [...built.body.childNodes]) {
if (node.nodeName !== 'SCRIPT')
document.body.insertBefore(document.adoptNode(node), status);
}
for (const [image, path] of images)
appFileUrl(path).then((url) => image.setAttribute('src', url));
const scripts = await Promise.all(
[...built.querySelectorAll('script')].map(async (script) => ({
type: script.type,
text: script.getAttribute('src')
? await (await appFetch(appPath(script.getAttribute('src')))).text()
: script.textContent,
})),
);
for (const { type, text } of scripts) runScript(text, type);
// Browsers cap live 3D contexts across pages and take one back when
// there are too many; say so rather than show no map.
document.addEventListener(
'webglcontextlost',
() => {
const notice = document.createElement('div');
notice.id = 'status';
notice.textContent =
"This panel lost its 3D graphics, likely because other God's Eye " +
"View panels in this conversation hold them. Use Open in God's " +
'Eye View above, or show it in a new conversation.';
document.body.appendChild(notice);
},
{ capture: true, once: true },
);
}
// Tool results: the first view starts the app, later ones move it.
let started = false;
let ready = false;
let queued = null;
let currentUrl = null;
const postView = (view) =>
window.postMessage({ type: 'gev:view', id: nextId++, view }, '*');
function show(result) {
const data = result?.structuredContent;
const view = data?.view;
const url = data?.url || view?.url;
if (!view || !url) return;
currentUrl = url;
open.hidden = false;
if (ready) return postView(view);
queued = view;
if (started) return;
started = true;
say("Loading God's Eye View…");
startApp(url, view).catch((error) =>
say(`God's Eye View could not load here: ${error?.message || error}.`),
);
setTimeout(() => {
if (!ready)
say(
"God's Eye View did not load here" +
(blocked ? ` (${blocked})` : '') +
". Use Open in God's Eye View above.",
);
}, config.loadTimeoutMs);
}
window.addEventListener('message', (event) => {
const message = event.data;
if (event.source === window) {
if (message?.type === 'gev:ready') {
ready = true;
status.remove();
if (queued) postView(queued);
queued = null;
}
return;
}
if (event.source !== window.parent || message?.jsonrpc !== '2.0') return;
if (
message.id !== undefined &&
pending.has(message.id) &&
!message.method
) {
const { resolve, reject } = pending.get(message.id);
pending.delete(message.id);
if (message.error) reject(message.error);
else resolve(message.result);
return;
}
if (message.method === 'ui/notifications/tool-result') show(message.params);
if (message.method === 'ui/notifications/host-context-changed')
showDisplayMode(message.params);
});
// Fullscreen, where the host offers it, gives the globe the room.
let displayMode = 'inline';
const showDisplayMode = (context) => {
if (context?.displayMode) displayMode = context.displayMode;
if (context?.availableDisplayModes)
expand.hidden = !context.availableDisplayModes.includes('fullscreen');
expand.textContent = displayMode === 'fullscreen' ? 'Collapse' : 'Expand';
};
expand.addEventListener('click', () => {
request('ui/request-display-mode', {
mode: displayMode === 'fullscreen' ? 'inline' : 'fullscreen',
}).then(
(result) => showDisplayMode({ displayMode: result?.mode }),
() => {},
);
});
open.addEventListener('click', () => {
if (!currentUrl) return;
request('ui/open-link', { url: currentUrl }).catch(() =>
window.open(currentUrl, '_blank', 'noopener'),
);
});
// The handshake the MCP Apps SDK's App.connect performs: hosts validate
// these parameters, and keep a view hidden until it is initialized.
request('ui/initialize', {
appCapabilities: { availableDisplayModes: ['inline', 'fullscreen'] },
appInfo: {
name: 'gods-eye-view',
title: "God's Eye View",
version: '1.0.0',
},
protocolVersion: config.protocolVersion,
}).then(
(result) => {
showDisplayMode(result?.hostContext);
send({ method: 'ui/notifications/initialized' });
notify('ui/notifications/size-changed', {
width: document.body.clientWidth,
height: config.panelHeight,
});
},
(error) =>
say(
"This client did not accept the God's Eye View panel" +
(error?.message ? `: ${error.message}` : '.'),
),
);
}
+14 -7
View File
@@ -4,6 +4,7 @@ import { initDrawTool } from '../annotations/drawTool.js';
import { initImageryBoxTool } from '../ui/imageryBoxTool.js';
import { createRecentImageryPanel } from '../ui/recentImagery.js';
import { initGevVoiceCommands } from '../voice/gevRealtime.js';
import { installViews, isEmbeddedInline } from './embed.js';
import { installScopeMask, destroyScopeMask } from '../scopeMask.js';
import {
installRenderGovernor,
@@ -118,7 +119,9 @@ export function createApplicationTools({
// GPU. Holder/data state is untouched, so return is seamless: restore
// the loop, refresh the one DOM surface we gated, render a frame.
const syncVisibilitySuspension = () => {
const hidden = document.hidden;
// A panel's host may report it hidden while it is on screen; the panel
// keeps drawing itself (see keepPanelRendering in embed.js).
const hidden = document.hidden && !isEmbeddedInline();
viewer.useDefaultRenderLoop = !hidden;
cockpitCloudEffects?.setSuspended?.(hidden);
if (!hidden) {
@@ -157,13 +160,8 @@ export function createApplicationTools({
defer(() => {
if (window.__godsEyeView === debug) delete window.__godsEyeView;
});
let voiceCommands = initGevVoiceCommands({
const voiceCommands = initGevVoiceCommands({
...voice,
// Choosing Cloud or On-device rebuilds voice; track the live controls.
onReplace: (next) => {
voiceCommands = next;
debug.voiceCommands = next;
},
floorServices: operations.surface.groundFloor,
annotationResolver: operations.annotationResolver,
searchNavigation: operations.searchAndFlyTo,
@@ -181,5 +179,14 @@ export function createApplicationTools({
delete window.__gevVoiceCommands;
});
debug.voiceCommands = voiceCommands;
defer(
installViews({
shell: styleManager,
viewer,
dataManager,
run: (name, args) => voiceCommands.runner(name, args, { signal }),
signal,
}),
);
return { sceneDirector, annotations, voiceCommands };
}
+33 -9
View File
@@ -86,23 +86,47 @@ test('Cockpit heading tape leaves the bottom exit row unobstructed', () => {
assert.doesNotMatch(css, /cockpit-compass-label/);
});
test('Cockpit vision cycle exposes exactly five real visual styles without NONE', () => {
assert.match(cycleVisionMode.toString(), /const modes = COCKPIT_VISION_MODES;/);
assert.match(setVisionMode.toString(), /const labels = \{\s*optical: inherited,\s*crt: 'CRT',\s*nvg: 'NVG',\s*thermal: 'FLIR',\s*noir: 'NOIR',?\s*\};/);
test('Cockpit vision cycle exposes every map preset exactly once', () => {
assert.match(
cycleVisionMode.toString(),
/const modes = COCKPIT_VISION_MODES;/,
);
assert.match(
setVisionMode.toString(),
/const labels = \{\s*optical: 'NORMAL',\s*crt: 'CRT',\s*nvg: 'NVG',\s*thermal: 'FLIR',\s*anime: 'ANIME',\s*noir: 'NOIR',\s*snow: 'SNOW',?\s*\};/,
);
assert.doesNotMatch(setVisionMode.toString(), /none: 'NONE'/);
assert.match(ui, /getInheritedVisionLabel: \(\) =>\s*\(?[\s\S]*?STYLE_STATUS_LABELS\[this\.activeStyle\]/);
assert.doesNotMatch(ui, /getInheritedVisionLabel/);
assert.match(html, /id="cockpit-vision-current-label"[^>]*>NORMAL<\/strong>/);
assert.match(ui, /const target = applyCockpitVisionStageIntensities\(\s*this\.stages,\s*next,\s*this\._cockpitVisionRestore,?\s*\);/);
assert.match(ui, /this\._cockpitVisionRestore = captureCockpitVisionBaseline\(\s*this\.stages,\s*this\.transitions,?\s*\);/);
assert.match(
cockpitEnter.toString(),
/this\.setVisionMode\(this\.getInitialVisionMode\(\)\)/,
'Cockpit must enter on the mode matching the active map preset',
);
assert.match(
readRadioSource(
new URL('./ui/cockpitCoordinator.js', import.meta.url),
'utf8',
),
/getInitialVisionMode:\s*\(\)\s*=>\s*cockpitVisionModeForStyle\(this\.activeStyle\)/,
);
assert.match(
ui,
/if \(next === 'optical'\) \{[\s\S]*?applyCockpitVisionStageIntensities\(\s*this\.stages,\s*next,\s*this\._cockpitVisionRestore,?\s*\);[\s\S]*?return;[\s\S]*?const target = applyCockpitVisionStageIntensities/,
'the inherited entry must restore the map shader while CRT, NVG, FLIR, and NOIR remain temporary Cockpit overrides',
/const target = applyCockpitVisionStageIntensities\(\s*this\.stages,\s*next,?\s*\);/,
);
assert.match(
ui,
/this\._cockpitVisionRestore = captureCockpitVisionBaseline\(\s*this\.stages,\s*this\.transitions,?\s*\);/,
);
assert.match(
ui,
/if \(next === 'optical'\) \{[\s\S]*?applyCockpitVisionStageIntensities\(\s*this\.stages,\s*next,?\s*\);[\s\S]*?this\._updateSliderPanel\(null,[\s\S]*?return;[\s\S]*?const target = applyCockpitVisionStageIntensities/,
'Normal must clear the inherited map shader while the other map presets remain temporary Cockpit overrides',
);
assert.match(
ui,
/_syncCockpitInheritedStyle\(\)[\s\S]*?name === this\.activeStyle \? 1 : 0[\s\S]*?this\.transitions\.delete\(name\)[\s\S]*?setVisionMode\(this\.cockpitView\.visionMode\)/,
'changing the map preset in Cockpit must refresh both the inherited label and restore baseline',
'changing the map preset in Cockpit must refresh the exit restore baseline without replacing Normal',
);
assert.match(ui, /setStyle\([\s\S]*?this\._syncCockpitInheritedStyle\(\);/);
});
+22 -5
View File
@@ -3,21 +3,40 @@ export const COCKPIT_VISION_MODES = Object.freeze([
'crt',
'nvg',
'thermal',
'anime',
'noir',
'snow',
]);
const TARGET_STYLE_BY_MODE = Object.freeze({
crt: 'retro',
nvg: 'surveillance',
thermal: 'thermal',
anime: 'anime',
noir: 'noir',
snow: 'snow',
});
/** Normalize a requested Cockpit vision mode to the inherited preset entry. */
const COCKPIT_MODE_BY_STYLE = Object.freeze({
normal: 'optical',
retro: 'crt',
surveillance: 'nvg',
thermal: 'thermal',
anime: 'anime',
noir: 'noir',
snow: 'snow',
});
/** Normalize a requested Cockpit vision mode to the unfiltered Normal entry. */
export function normalizeCockpitVisionMode(mode) {
return COCKPIT_VISION_MODES.includes(mode) ? mode : 'optical';
}
/** Start Cockpit on the equivalent supported map preset. */
export function cockpitVisionModeForStyle(style) {
return COCKPIT_MODE_BY_STYLE[String(style || '').toLowerCase()] || 'optical';
}
/** Settle pending map-style crossfades and return their intended final intensities. */
export function captureCockpitVisionBaseline(stages, transitions) {
const baseline = {};
@@ -34,12 +53,10 @@ export function captureCockpitVisionBaseline(stages, transitions) {
* Apply Cockpit-only stage intensities without changing any shader parameters.
* Returns the temporary style whose parameters should be shown, or null.
*/
export function applyCockpitVisionStageIntensities(stages, mode, restore = {}) {
export function applyCockpitVisionStageIntensities(stages, mode) {
const next = normalizeCockpitVisionMode(mode);
if (next === 'optical') {
for (const [name, intensity] of Object.entries(restore)) {
if (stages[name]) stages[name].uniforms.intensity = intensity;
}
for (const stage of Object.values(stages)) stage.uniforms.intensity = 0;
return null;
}
+75 -23
View File
@@ -3,6 +3,7 @@ import test from 'node:test';
import {
applyCockpitVisionStageIntensities,
captureCockpitVisionBaseline,
cockpitVisionModeForStyle,
COCKPIT_VISION_MODES,
normalizeCockpitVisionMode,
} from './cockpitVisionPolicy.js';
@@ -14,12 +15,31 @@ const createStages = () => ({
thermal: { uniforms: { intensity: 0, heat: 0.8 } },
});
test('Cockpit vision order exposes inherited, CRT, NVG, FLIR, and NOIR modes', () => {
assert.deepEqual(COCKPIT_VISION_MODES, ['optical', 'crt', 'nvg', 'thermal', 'noir']);
test('Cockpit vision order mirrors the unique map preset order', () => {
assert.deepEqual(COCKPIT_VISION_MODES, [
'optical',
'crt',
'nvg',
'thermal',
'anime',
'noir',
'snow',
]);
assert.equal(normalizeCockpitVisionMode('none'), 'optical');
assert.equal(normalizeCockpitVisionMode('unknown'), 'optical');
});
test('Cockpit entry selects the vision mode equivalent to the active map style', () => {
assert.equal(cockpitVisionModeForStyle('normal'), 'optical');
assert.equal(cockpitVisionModeForStyle('retro'), 'crt');
assert.equal(cockpitVisionModeForStyle('surveillance'), 'nvg');
assert.equal(cockpitVisionModeForStyle('thermal'), 'thermal');
assert.equal(cockpitVisionModeForStyle('anime'), 'anime');
assert.equal(cockpitVisionModeForStyle('noir'), 'noir');
assert.equal(cockpitVisionModeForStyle('snow'), 'snow');
assert.equal(cockpitVisionModeForStyle('unsupported'), 'optical');
});
test('Cockpit settles pending map crossfades before a temporary preset takes ownership', () => {
const stages = createStages();
const transitions = new Map([
@@ -29,49 +49,81 @@ test('Cockpit settles pending map crossfades before a temporary preset takes own
const restore = captureCockpitVisionBaseline(stages, transitions);
assert.deepEqual(restore, { noir: 1, retro: 0, surveillance: 0, thermal: 0 });
assert.equal(transitions.size, 0);
applyCockpitVisionStageIntensities(stages, 'nvg', restore);
applyCockpitVisionStageIntensities(stages, 'nvg');
assert.equal(stages.surveillance.uniforms.intensity, 1);
applyCockpitVisionStageIntensities(stages, 'optical', restore);
assert.equal(stages.noir.uniforms.intensity, 1);
applyCockpitVisionStageIntensities(stages, 'optical');
assert.equal(stages.noir.uniforms.intensity, 0);
assert.equal(stages.retro.uniforms.intensity, 0);
});
test('returning from a temporary preset restores the exact inherited intensities', () => {
test('Normal clears every temporary and inherited visual stage', () => {
const stages = createStages();
const restore = Object.fromEntries(
Object.entries(stages).map(([name, stage]) => [name, stage.uniforms.intensity]),
Object.entries(stages).map(([name, stage]) => [
name,
stage.uniforms.intensity,
]),
);
applyCockpitVisionStageIntensities(stages, 'thermal', restore);
applyCockpitVisionStageIntensities(stages, 'optical', restore);
applyCockpitVisionStageIntensities(stages, 'thermal');
applyCockpitVisionStageIntensities(stages, 'optical');
assert.deepEqual(
Object.fromEntries(Object.entries(stages).map(([name, stage]) => [name, stage.uniforms.intensity])),
restore,
Object.fromEntries(
Object.entries(stages).map(([name, stage]) => [
name,
stage.uniforms.intensity,
]),
),
{ noir: 0, retro: 0, surveillance: 0, thermal: 0 },
);
assert.deepEqual(restore, {
noir: 0.72,
retro: 0.18,
surveillance: 0,
thermal: 0,
});
});
test('temporary styles replace each other without changing the inherited restore snapshot', () => {
const stages = createStages();
const restore = Object.fromEntries(
Object.entries(stages).map(([name, stage]) => [name, stage.uniforms.intensity]),
Object.entries(stages).map(([name, stage]) => [
name,
stage.uniforms.intensity,
]),
);
applyCockpitVisionStageIntensities(stages, 'thermal', restore);
assert.equal(applyCockpitVisionStageIntensities(stages, 'crt', restore), 'retro');
applyCockpitVisionStageIntensities(stages, 'thermal');
assert.equal(applyCockpitVisionStageIntensities(stages, 'crt'), 'retro');
assert.equal(stages.retro.uniforms.intensity, 1);
assert.equal(stages.noir.uniforms.intensity, 0);
applyCockpitVisionStageIntensities(stages, 'optical', restore);
assert.equal(stages.noir.uniforms.intensity, 0.72);
assert.equal(stages.retro.uniforms.intensity, 0.18);
applyCockpitVisionStageIntensities(stages, 'optical');
assert.equal(stages.noir.uniforms.intensity, 0);
assert.equal(stages.retro.uniforms.intensity, 0);
assert.deepEqual(restore, {
noir: 0.72,
retro: 0.18,
surveillance: 0,
thermal: 0,
});
});
test('NOIR is a temporary Cockpit override and inherited restores the captured map style', () => {
test('NOIR is a temporary Cockpit override and Normal clears it', () => {
const stages = createStages();
const restore = Object.fromEntries(
Object.entries(stages).map(([name, stage]) => [name, stage.uniforms.intensity]),
Object.entries(stages).map(([name, stage]) => [
name,
stage.uniforms.intensity,
]),
);
assert.equal(applyCockpitVisionStageIntensities(stages, 'noir', restore), 'noir');
assert.equal(applyCockpitVisionStageIntensities(stages, 'noir'), 'noir');
assert.equal(stages.noir.uniforms.intensity, 1);
assert.equal(stages.retro.uniforms.intensity, 0);
applyCockpitVisionStageIntensities(stages, 'optical', restore);
assert.equal(stages.noir.uniforms.intensity, 0.72);
assert.equal(stages.retro.uniforms.intensity, 0.18);
applyCockpitVisionStageIntensities(stages, 'optical');
assert.equal(stages.noir.uniforms.intensity, 0);
assert.equal(stages.retro.uniforms.intensity, 0);
assert.deepEqual(restore, {
noir: 0.72,
retro: 0.18,
surveillance: 0,
thermal: 0,
});
});
+281
View File
@@ -0,0 +1,281 @@
import assert from 'node:assert/strict';
import test from 'node:test';
import path from 'node:path';
import { fileURLToPath } from 'node:url';
import { Readable } from 'node:stream';
import { readPinokioEnvironment } from '../scripts/pinokio-environment.mjs';
import { openAiRealtimeProxy } from 'gods-eye-view/server/providers/openai';
import {
makeCostRateLimiter,
resolvePerMinuteCap,
} from '../server/providers/common/rate-limit.js';
import { OPENAI_DEFAULT_PER_MIN } from '../server/providers/openai/rate-limit.js';
import { handleHudSummary } from '../server/providers/openai/hud-summary.js';
import {
googlePlacesContextProxy,
GOOGLE_DEFAULT_PER_MIN,
} from '../server/providers/places/google.js';
// Both limiters are cached at module scope on first use, so the env value that
// matters is the one in place when a route is first exercised — restoring the
// variable afterwards does not rebuild the limiter. Each route is therefore
// driven by exactly one test here; a second test wanting a different cap would
// need its own process, not just a different env value.
/** Set an env var for one test and put the old value back afterwards. */
function env(t, name, value) {
const old = process.env[name];
if (value === undefined) delete process.env[name];
else process.env[name] = value;
t.after(() => {
if (old === undefined) delete process.env[name];
else process.env[name] = old;
});
}
function install(plugin) {
const routes = new Map();
plugin.configureServer({
middlewares: { use: (route, handler) => routes.set(route, handler) },
restart: async () => {},
});
return routes;
}
function request(
handler,
{ url = '/', method = 'POST', remoteAddress = '127.0.0.1' } = {},
) {
return new Promise((resolve, reject) => {
const req = Readable.from([]);
Object.assign(req, {
method,
url,
headers: { host: 'localhost:4173', 'content-type': 'application/json' },
socket: { remoteAddress },
});
const headers = {};
const res = {
statusCode: 200,
setHeader(name, value) {
headers[name.toLowerCase()] = value;
},
end(body = '') {
resolve({ status: this.statusCode, headers, body: String(body) });
},
};
Promise.resolve(handler(req, res)).catch(reject);
});
}
test('an unset throttle resolves to the documented default, not to unlimited', () => {
// The defect #16 reports: the cap was opt-in, so an unconfigured server let
// anyone who could reach it mint tokens on the operator's key without bound.
assert.equal(resolvePerMinuteCap(undefined, 30), 30);
assert.equal(resolvePerMinuteCap('', 30), 30);
assert.equal(resolvePerMinuteCap(' ', 30), 30);
assert.ok(
typeof makeCostRateLimiter(undefined, 30) === 'function',
'an unconfigured cost endpoint must still be throttled',
);
});
test('an explicit 0 is the documented way to run unthrottled', () => {
assert.equal(resolvePerMinuteCap('0', 30), 0);
assert.equal(
makeCostRateLimiter('0', 30),
null,
'operators who mean "no throttle" must still be able to say so',
);
});
test('a configured value wins, and a fractional one floors', () => {
assert.equal(resolvePerMinuteCap('45', 30), 45);
assert.equal(resolvePerMinuteCap(45, 30), 45);
assert.equal(resolvePerMinuteCap('30.7', 30), 30);
});
test('a positive fraction keeps the guard rather than flooring to unlimited', () => {
for (const fraction of ['0.5', '0.99', 0.01, '1e-9']) {
assert.equal(resolvePerMinuteCap(fraction, 30), 1, String(fraction));
assert.equal(
typeof makeCostRateLimiter(fraction, 30),
'function',
`${fraction} must still throttle`,
);
}
// Only an explicit zero turns the limiter off.
assert.equal(makeCostRateLimiter('0', 30), null);
assert.equal(makeCostRateLimiter('0.0', 30), null);
});
test('an unreadable value keeps the guard instead of removing it', () => {
// `3O` (letter O) and `60/min` are the shapes a hand-edited .env produces.
// Reading either as "unlimited" would disarm the spend guard on a typo —
// the opposite of what the operator was reaching for when they typed it.
for (const typo of ['3O', '60/min', 'thirty', '-5', 'NaN']) {
assert.equal(
resolvePerMinuteCap(typo, 30),
30,
`${typo} must fall back to the default, not to unlimited`,
);
}
});
test('the default cap clears the app’s own demand with room to spare', () => {
// src/hud.js HUD_SUMMARY_INTERVAL_MS = 15000 -> 4 summaries/min per open
// tab, plus one token mint per voice session. The cap has to sit well above
// that or it would throttle the app rather than an abuser.
const hudRequestsPerMinutePerTab = 60_000 / 15_000;
assert.ok(
OPENAI_DEFAULT_PER_MIN >= hudRequestsPerMinutePerTab * 4,
`default ${OPENAI_DEFAULT_PER_MIN}/min leaves too little headroom over ${hudRequestsPerMinutePerTab}/min of ordinary use`,
);
});
test('the realtime token route throttles by default, and says so in a 429', async (t) => {
env(t, 'GEV_RATELIMIT_OPENAI_PER_MIN', undefined);
env(t, 'OPENAI_API_KEY', 'fixture-upstream-secret');
let upstreamCalls = 0;
t.mock.method(globalThis, 'fetch', async () => {
upstreamCalls += 1;
return Response.json({ value: 'fixture-ephemeral' });
});
const handler = install(openAiRealtimeProxy({})).get('/api/realtime/token');
const statuses = [];
for (let i = 0; i < OPENAI_DEFAULT_PER_MIN + 1; i += 1) {
statuses.push((await request(handler)).status);
}
assert.deepEqual(
statuses.slice(0, OPENAI_DEFAULT_PER_MIN),
Array(OPENAI_DEFAULT_PER_MIN).fill(200),
'the default must not throttle ordinary use',
);
const overLimit = await request(handler);
assert.equal(overLimit.status, 429);
assert.equal(overLimit.headers['retry-after'], '5');
assert.equal(JSON.parse(overLimit.body).error, 'Rate limit exceeded');
assert.equal(
overLimit.body.includes('fixture-upstream-secret'),
false,
'a throttled reply must not leak the server-held key',
);
assert.ok(
upstreamCalls <= OPENAI_DEFAULT_PER_MIN,
'a throttled request must be refused before it reaches OpenAI, or it still costs money',
);
});
test('the HUD summary route throttles the paid path, never the free one', async (t) => {
env(t, 'OPENAI_API_KEY', 'fixture-upstream-secret');
let upstreamCalls = 0;
t.mock.method(globalThis, 'fetch', async () => {
upstreamCalls += 1;
return Response.json({ output_text: 'five word hud summary here' });
});
// Its own client address: the limiter is cached at module scope and shared
// across the OpenAI routes, so the realtime case above has already spent
// 127.0.0.1's budget inside this process.
const from = '10.9.9.9';
const statuses = [];
for (let i = 0; i < OPENAI_DEFAULT_PER_MIN; i += 1)
statuses.push(
(await request(handleHudSummary, { remoteAddress: from })).status,
);
assert.deepEqual(
statuses,
Array(OPENAI_DEFAULT_PER_MIN).fill(200),
'the default must not throttle ordinary HUD use',
);
const overLimit = await request(handleHudSummary, { remoteAddress: from });
assert.equal(overLimit.status, 429);
assert.equal(overLimit.headers['retry-after'], '5');
assert.equal(JSON.parse(overLimit.body).error, 'Rate limit exceeded');
assert.equal(
overLimit.body.includes('fixture-upstream-secret'),
false,
'a throttled reply must not leak the server-held key',
);
assert.ok(
upstreamCalls <= OPENAI_DEFAULT_PER_MIN,
'a throttled request must be refused before it reaches OpenAI, or it still costs money',
);
// The keyless HUD answer is deliberately NOT throttled: it is composed
// locally, spends nothing upstream, and capping it would only take the free
// HUD away from an unconfigured install. Same exhausted client, no key.
delete process.env.OPENAI_API_KEY;
const free = await request(handleHudSummary, { remoteAddress: from });
assert.equal(free.status, 200, 'the free path answers past a spent budget');
assert.equal(JSON.parse(free.body).configured, false);
assert.equal(
upstreamCalls <= OPENAI_DEFAULT_PER_MIN,
true,
'the free path reaches no provider',
);
});
test('the Google places route throttles by default and keeps its places[] contract', async (t) => {
env(t, 'GEV_RATELIMIT_GOOGLE_PER_MIN', undefined);
let upstreamCalls = 0;
const handler = install(
googlePlacesContextProxy({
resolveApiKey: () => 'fixture-google-server-key',
fetchImpl: async () => {
upstreamCalls += 1;
return Response.json({ places: [] });
},
}),
).get('/api/google/nearby-places');
const url = '/?lat=37.77493&lon=-122.41942';
const statuses = [];
for (let i = 0; i < GOOGLE_DEFAULT_PER_MIN; i += 1) {
statuses.push((await request(handler, { url, method: 'GET' })).status);
}
assert.ok(
statuses.every((s) => s === 200),
'the default must not throttle ordinary use',
);
const overLimit = await request(handler, { url, method: 'GET' });
assert.equal(overLimit.status, 429);
assert.equal(overLimit.headers['retry-after'], '5');
// Every error response on this route carries places[]; the client reads it
// unconditionally, so a throttled reply must not be the one shape that omits it.
assert.deepEqual(JSON.parse(overLimit.body).places, []);
assert.equal(
upstreamCalls,
GOOGLE_DEFAULT_PER_MIN,
'a throttled request must be refused before it reaches Google, or it still costs money',
);
});
test('the code defaults match the caps the Pinokio build ships', () => {
// These two numbers used to live only in pinokio/_ENVIRONMENT, and the value
// this repo suggested elsewhere had already drifted away from them. Now that
// they are the server's defaults, the packaged app and an unconfigured
// checkout must agree, or a Pinokio user's throttle silently stops matching
// the one everybody else gets. If you deliberately want them to differ, this
// assertion is the place to say so.
const shipped = readPinokioEnvironment(
path.join(
path.dirname(fileURLToPath(import.meta.url)),
'..',
'pinokio',
'_ENVIRONMENT',
),
);
assert.equal(
Number(shipped.GEV_RATELIMIT_OPENAI_PER_MIN),
OPENAI_DEFAULT_PER_MIN,
);
assert.equal(
Number(shipped.GEV_RATELIMIT_GOOGLE_PER_MIN),
GOOGLE_DEFAULT_PER_MIN,
);
});
+3 -21
View File
@@ -58,30 +58,18 @@ export function normalizeAdminName(value) {
const PACKS = {
countries: {
url: new URL('./local_data/natural_earth/countries.json', import.meta.url),
importJson: () =>
import('./local_data/natural_earth/countries.json', {
with: { type: 'json' },
}),
},
admin1: {
url: new URL(
'./local_data/natural_earth/states_provinces.json',
import.meta.url,
),
importJson: () =>
import('./local_data/natural_earth/states_provinces.json', {
with: { type: 'json' },
}),
},
counties: {
url: new URL(
'./local_data/us_census_counties/counties.json',
import.meta.url,
),
importJson: () =>
import('./local_data/us_census_counties/counties.json', {
with: { type: 'json' },
}),
},
};
@@ -297,7 +285,7 @@ function addKey(index, key, entry, tier) {
const loadAdmin1 = createRetryableLoader(async () => {
packLoads.admin1 += 1;
const pack = await loadBundledJson(PACKS.admin1.url, PACKS.admin1.importJson);
const pack = await loadBundledJson(PACKS.admin1.url);
const decimals = pack.meta?.decimals ?? 3;
const index = new Map();
for (const feature of pack.features || []) {
@@ -323,10 +311,7 @@ const loadAdmin1 = createRetryableLoader(async () => {
const loadCounties = createRetryableLoader(async () => {
packLoads.counties += 1;
const pack = await loadBundledJson(
PACKS.counties.url,
PACKS.counties.importJson,
);
const pack = await loadBundledJson(PACKS.counties.url);
const decimals = pack.meta?.decimals ?? 4;
const byFull = new Map();
const byName = new Map();
@@ -348,10 +333,7 @@ const loadCounties = createRetryableLoader(async () => {
const loadCountries = createRetryableLoader(async () => {
packLoads.countries += 1;
const pack = await loadBundledJson(
PACKS.countries.url,
PACKS.countries.importJson,
);
const pack = await loadBundledJson(PACKS.countries.url);
const index = new Map();
for (const feature of pack.features || []) {
const entry = {
+3
View File
@@ -5,6 +5,9 @@ import { createAisStreamSource } from '../sources/live/standalone.js';
const aisLiveVesselsLayer = createApplicationVessels({
source: createAisStreamSource({
apiUrl: import.meta.env?.VITE_AIS_LIVE_API_URL || '/api/ais-live',
// Resolved against the document's address, which a panel host may serve
// from its own scheme.
origin: () => globalThis.document?.baseURI ?? 'http://localhost',
}),
options: {
maxRows: import.meta.env?.VITE_AIS_LIVE_MAX_ROWS,
+136 -835
View File
File diff suppressed because it is too large. Load diff
+5 -255
View File
@@ -262,132 +262,20 @@ test('helpers: haversine sanity + scope radius', () => {
assert.deepEqual(scoped.map((f) => f.id).sort(), ['GND1', 'SWA1']);
});
test('capped cohorts report returned/total/truncated and keep it on follow-up', async () => {
test('bounded loaded cohorts disclose truncation before filtering and retain it on follow-up', async () => {
const engine = createAnalystEngine({
getRecords: () => [{ id: 'sample dam', lat: 0, lon: 0, name: 'sample' }],
getRecordCoverage: () => ({ total: 3000, truncated: true }),
getRecordCoverage: () => ({ basis: 'bounded-loaded-records', recordsExamined: 1, loadedCount: 3000, sourceTruncated: true }),
getViewContext: () => ({ lat: 0, lon: 0, viewRadiusKm: 25 }),
});
const result = await engine.query({ layers: ['local-dams'], scope: { kind: 'anywhere' }, sortBy: 'distance' });
assert.equal(result.count, 1);
const layer = result.coverage.layersQueried[0];
assert.deepEqual({ returned: layer.returned, total: layer.total, truncated: layer.truncated }, { returned: 1, total: 3000, truncated: true });
assert.deepEqual(result.coverage.records, { returned: 1, total: 3000, truncated: true });
assert.match(result.display.caveat, /counted 1 of 3,000 loaded records/);
assert.equal(result.coverage.layersQueried[0].sourceTruncated, true);
assert.match(result.coverage.note, /omitted records may change the nearest item or count/);
const followUp = await engine.query({ followUp: true });
assert.equal(followUp.coverage.layersQueried[0].total, 3000);
assert.equal(followUp.coverage.layersQueried[0].loadedCount, 3000);
});
test('an uncapped cohort is complete and carries no caveat', async () => {
const result = await makeEngine().query({ layers: ['flights'], scope: { kind: 'anywhere' } });
assert.deepEqual(result.coverage.records, { returned: 4, total: 4, truncated: false });
assert.equal(result.display.caveat, undefined);
assert.equal(result.display.scope, 'anywhere in the loaded data');
});
test('unknown fields, operators, scopes and wrong-typed values are refused with the allowed values', async () => {
const engine = makeEngine();
const field = await engine.query({ layers: ['flights'], filters: [{ field: 'time', op: 'gte', value: 5 }] });
assert.equal(field.ok, false);
assert.equal(field.code, 'UNKNOWN_FIELD');
assert.ok(field.allowed.includes('altitudeM'));
const op = await engine.query({ layers: ['flights'], filters: [{ field: 'altitudeM', op: 'wat', value: 5 }] });
assert.equal(op.code, 'BAD_OPERATOR');
assert.deepEqual(op.allowed, ['gt', 'gte', 'lt', 'lte', 'eq', 'neq']);
const flag = await engine.query({ layers: ['flights'], filters: [{ field: 'military', op: 'eq', value: 'false' }] });
assert.equal(flag.code, 'BAD_VALUE', 'the string "false" must not match the TRUE record');
const num = await engine.query({ layers: ['flights'], filters: [{ field: 'altitudeM', op: 'gt', value: 'high' }] });
assert.equal(num.code, 'BAD_VALUE');
const scope = await engine.query({ layers: ['flights'], scope: { kind: 'polygon' } });
assert.equal(scope.code, 'UNKNOWN_SCOPE');
assert.deepEqual(scope.allowed, ['view', 'region', 'radius', 'anywhere']);
const sort = await engine.query({ layers: ['flights'], sortBy: 'loudness' });
assert.equal(sort.code, 'UNKNOWN_FIELD');
assert.equal(engine.hasMemory(), false, 'refusals never become follow-up memory');
});
test('a typed boolean filter still matches exactly', async () => {
const r = await makeEngine().query({ layers: ['flights'], scope: { kind: 'anywhere' }, filters: [{ field: 'military', op: 'eq', value: false }] });
assert.deepEqual(r.items.map((i) => i.id).sort(), ['GND1', 'N123', 'SWA1']);
});
test('distance honours sortDir: desc is farthest first', async () => {
const far = await makeEngine().query({ layers: ['flights'], scope: { kind: 'anywhere' }, sortBy: 'distance', sortDir: 'desc', limit: 1 });
assert.equal(far.items[0].id, 'N123');
const near = await makeEngine().query({ layers: ['flights'], scope: { kind: 'anywhere' }, sortBy: 'distance', limit: 1 });
assert.equal(near.items[0].id, 'SWA1');
});
test('a follow-up without a previous answer is refused, not a fresh flights query', async () => {
const r = await makeEngine().query({ followUp: true, filters: [] });
assert.equal(r.ok, false);
assert.equal(r.code, 'NO_RESULT_CONTEXT');
});
test('follow-ups never rewrite the remembered rows', async () => {
const engine = makeEngine();
await engine.query({ layers: ['flights'], scope: { kind: 'anywhere' } });
await engine.query({ followUp: true, sortBy: 'distance' });
const again = await engine.query({ followUp: true });
assert.equal(again.count, 4, 'a follow-up keeps the remembered set, not the view');
assert.ok(again.items.every((item) => item.distanceKm === undefined));
});
test('a superseded slow region query cannot overwrite the newer follow-up memory', async () => {
let releaseSlow;
const engine = createAnalystEngine({
getRecords: (key) => ({ flights: FLIGHTS, 'ais-live-vessels': SHIPS }[key] || []),
resolveRegionRing: () => new Promise((resolve) => { releaseSlow = () => resolve(TEXLAND); }),
getViewContext: () => ({ lat: 30.27, lon: -97.74, viewRadiusKm: 150 }),
});
let generation = 0;
const slowGeneration = ++generation;
const slow = engine.query({ layers: ['flights'], scope: { kind: 'region', name: 'Texland' } }, { isCurrent: () => generation === slowGeneration });
const newGeneration = ++generation;
const fresh = await engine.query({ layers: ['ais-live-vessels'], scope: { kind: 'anywhere' } }, { isCurrent: () => generation === newGeneration });
assert.equal(fresh.ok, true);
releaseSlow();
const stale = await slow;
assert.equal(stale.code, 'CANCELLED');
const followUp = await engine.query({ followUp: true });
assert.deepEqual(followUp.items.map((i) => i.layerKey), ['ais-live-vessels', 'ais-live-vessels']);
});
test('off, loading and unavailable layers are refused distinctly; a real zero stays ok', async () => {
const engineFor = (snapshot, rows = [], warming = false) => createAnalystEngine({
getRecords: () => rows,
getLayerSnapshot: () => snapshot,
isWarming: () => warming,
getViewContext: () => ({ lat: 0, lon: 0, viewRadiusKm: 25 }),
});
assert.equal((await engineFor({ enabled: false, feedState: 'off' }).query({ layers: ['earthquakes'] })).code, 'LAYER_OFF');
assert.equal((await engineFor({ enabled: true, feedState: 'loading' }).query({ layers: ['earthquakes'] })).code, 'NOT_READY');
assert.equal((await engineFor({ enabled: true, feedState: 'nominal' }, [], true).query({ layers: ['earthquakes'] })).code, 'NOT_READY');
assert.equal((await engineFor({ enabled: true, feedState: 'unavailable' }).query({ layers: ['earthquakes'] })).code, 'FEED_UNAVAILABLE');
const zero = await engineFor({ enabled: true, feedState: 'nominal' }, [{ id: 'q', lat: 50, lon: 50, magnitude: 2 }]).query({ layers: ['earthquakes'], filters: [{ field: 'magnitude', op: 'gte', value: 5 }] });
assert.equal(zero.ok, true);
assert.equal(zero.count, 0);
assert.equal(zero.coverage.layersQueried[0].status, 'ok');
});
test('time fields filter by ISO and derive ageHours; the feed window is exposed', async () => {
const now = Date.parse('2026-09-23T12:00:00Z');
const engine = createAnalystEngine({
now: () => now,
getRecords: () => [
{ id: 'recent', lat: 0, lon: 0, magnitude: 4, timeMs: now - 2 * 3_600_000 },
{ id: 'older', lat: 0, lon: 0, magnitude: 5, timeMs: now - 20 * 3_600_000 },
],
getViewContext: () => ({ lat: 0, lon: 0, viewRadiusKm: 25 }),
});
const iso = await engine.query({ layers: ['earthquakes'], scope: { kind: 'anywhere' }, filters: [{ field: 'timeMs', op: 'gte', value: '2026-09-23T06:00:00Z' }] });
assert.deepEqual(iso.items.map((i) => i.id), ['recent']);
const age = await engine.query({ layers: ['earthquakes'], scope: { kind: 'anywhere' }, filters: [{ field: 'ageHours', op: 'lte', value: 6 }] });
assert.deepEqual(age.items.map((i) => i.id), ['recent']);
assert.equal(age.items[0].ageHours, 2);
assert.equal(age.display.window, 'last 24 h');
assert.equal(age.coverage.layersQueried[0].window, 'last 24 h');
});
test('follow-up provenance stays attached to old rows after a feed recovers', async () => {
let state = 'stale';
@@ -405,141 +293,3 @@ test('follow-up provenance stays attached to old rows after a feed recovers', as
assert.equal(engine.hasMemory(), false);
assert.equal((await engine.query()).coverage.feedProvenance.overall, 'nominal');
});
test('a large loaded set sorts and summarizes without overflowing the stack', async () => {
const rows = Array.from({ length: 200_000 }, (_, i) => ({ id: `FIRE-${i}`, lat: 0, lon: 0, frp: i }));
const engine = createAnalystEngine({
getRecords: () => rows,
getViewContext: () => ({ lat: 0, lon: 0, viewRadiusKm: 25 }),
});
const result = await engine.query({ layers: ['local-firms'], scope: { kind: 'anywhere' }, sortBy: 'frp', limit: 1 });
assert.equal(result.count, 200_000);
assert.equal(result.items[0].frp, 199_999);
assert.equal(result.summary.frpMin, 0);
});
test('an enabled but empty layer says why on screen', async () => {
const engine = createAnalystEngine({
getRecords: () => [],
getLayerSnapshot: () => ({ enabled: true, feedState: 'nominal' }),
getEmptyNote: () => 'Fly below 400 km to a covered region',
getViewContext: () => ({ lat: 0, lon: 0, viewRadiusKm: 25 }),
});
const result = await engine.query({ layers: ['transit'] });
assert.equal(result.ok, true);
assert.equal(result.count, 0);
assert.equal(result.coverage.layersQueried[0].status, 'empty');
assert.match(result.display.caveat, /transit: Fly below 400 km/);
});
test('a radius around a named place without a centre is refused, not measured around the camera', async () => {
const r = await makeEngine().query({ layers: ['ais-live-vessels'], scope: { kind: 'radius', name: 'Singapore', km: 200 } });
assert.equal(r.ok, false);
assert.equal(r.code, 'BAD_SCOPE');
const ok = await makeEngine().query({ layers: ['ais-live-vessels'], scope: { kind: 'radius', name: 'Houston', km: 200, center: { lat: 29.6, lon: -95.0 } } });
assert.equal(ok.ok, true);
});
test('a field copied with its hinted unit still resolves', async () => {
const r = await makeEngine().query({ layers: ['local-firms'], scope: { kind: 'anywhere' }, sortBy: 'frp(MW)', filters: [{ field: 'frp (MW)', op: 'gt', value: 100 }] });
assert.equal(r.ok, true);
assert.deepEqual(r.items.map((i) => i.id), ['FIRE-3', 'FIRE-1']);
});
test('a written unit converts to the field unit or is refused', async () => {
const engine = makeEngine();
const feet = await engine.query({ layers: ['flights'], scope: { kind: 'anywhere' }, filters: [{ field: 'altitudeM(ft)', op: 'gt', value: 40000 }] });
assert.equal(feet.ok, true);
assert.deepEqual(feet.items.map((i) => i.id), ['RCH01'], '40,000 ft is 12,192 m, not 40,000 m');
const same = await engine.query({ layers: ['flights'], scope: { kind: 'anywhere' }, filters: [{ field: 'altitudeM (m)', op: 'gt', value: 12192 }] });
assert.deepEqual(same.items.map((i) => i.id), ['RCH01']);
const knots = await engine.query({ layers: ['flights'], scope: { kind: 'anywhere' }, filters: [{ field: 'speedMps(kts)', op: 'gt', value: 470 }] });
assert.deepEqual(knots.items.map((i) => i.id), ['RCH01'], '470 kn is ~242 m/s');
const wrong = await engine.query({ layers: ['flights'], filters: [{ field: 'altitudeM(kts)', op: 'gt', value: 1 }] });
assert.equal(wrong.code, 'BAD_UNIT');
const unitless = await engine.query({ layers: ['flights'], filters: [{ field: 'callsign(ft)', op: 'eq', value: 'X' }] });
assert.equal(unitless.code, 'BAD_UNIT');
const hinted = await engine.query({ layers: ['local-firms'], scope: { kind: 'anywhere' }, sortBy: 'frp(MW)' });
assert.equal(hinted.ok, true);
});
test('unambiguous numeric strings are numbers; blanks, booleans and words are not', async () => {
const engine = makeEngine();
const text = await engine.query({ layers: ['flights'], scope: { kind: 'anywhere' }, filters: [{ field: 'altitudeM', op: 'gt', value: '12192' }] });
assert.equal(text.ok, true);
assert.deepEqual(text.items.map((i) => i.id), ['RCH01']);
for (const value of ['', ' ', true, 'high', '12k'])
assert.equal((await engine.query({ layers: ['flights'], filters: [{ field: 'altitudeM', op: 'gt', value }] })).code, 'BAD_VALUE', JSON.stringify(value));
});
test('any supplied centre must be a real coordinate', async () => {
const engine = makeEngine();
for (const center of [{ lat: 0 }, { lat: 999, lon: 0 }, { lat: 10, lon: 200 }, { lat: '10', lon: 20 }])
assert.equal((await engine.query({ layers: ['flights'], scope: { kind: 'radius', km: 100, center } })).code, 'BAD_SCOPE', JSON.stringify(center));
assert.equal((await engine.query({ layers: ['flights'], scope: { kind: 'radius', km: 100, center: { lat: 30.2, lon: -97.7 } } })).ok, true);
});
test('mixed off/loading/unavailable layers refuse; a partly answerable query says so', async () => {
const snapshots = { earthquakes: { enabled: true, feedState: 'unavailable' }, 'local-firms': { enabled: true, feedState: 'loading' }, flights: { enabled: false, feedState: 'off' } };
const engine = createAnalystEngine({
getRecords: (key) => (key === 'ais-live-vessels' ? SHIPS : []),
getLayerSnapshot: (key) => snapshots[key] || { enabled: true, feedState: 'nominal' },
getViewContext: () => ({ lat: 0, lon: 0, viewRadiusKm: 25 }),
});
const none = await engine.query({ layers: ['earthquakes', 'local-firms', 'flights'] });
assert.equal(none.ok, false);
assert.equal(none.code, 'NOT_READY');
assert.deepEqual(none.layerStatus.map((l) => l.status), ['error', 'loading', 'off']);
assert.equal(engine.hasMemory(), false, 'a refusal never becomes follow-up memory');
const down = await engine.query({ layers: ['earthquakes', 'flights'] });
assert.equal(down.code, 'FEED_UNAVAILABLE');
const partial = await engine.query({ layers: ['ais-live-vessels', 'earthquakes'], scope: { kind: 'anywhere' } });
assert.equal(partial.ok, true);
assert.equal(partial.partial, true);
assert.deepEqual(partial.unanswered, ['earthquakes']);
assert.match(partial.display.caveat, /earthquakes error/);
});
test('returned items are copies: mutating them cannot change the remembered answer', async () => {
const engine = makeEngine();
const first = await engine.query({ layers: ['flights'], scope: { kind: 'anywhere' }, sortBy: 'altitudeM' });
for (const item of first.items) {
item.altitudeM = -1;
item.id = 'MUTATED';
}
const again = await engine.query({ followUp: true, sortBy: 'altitudeM' });
assert.deepEqual(again.items.map((i) => i.id), ['RCH01', 'SWA1', 'N123', 'GND1']);
assert.equal(FLIGHTS[1].altitudeM, 13500, 'the provider rows are untouched');
});
test('ranking 250k rows stays within a budget and yields to the page', async () => {
const rows = Array.from({ length: 250_000 }, (_, i) => ({ id: `F${i}`, lat: (i % 170) - 85, lon: (i % 350) - 175, frp: (i * 7919) % 100_003 }));
const engine = createAnalystEngine({ getRecords: () => rows, getViewContext: () => ({ lat: 0, lon: 0, viewRadiusKm: 25 }) });
let ticks = 0;
const timer = setInterval(() => ticks++, 0);
const started = performance.now();
const result = await engine.query({ layers: ['local-firms'], scope: { kind: 'anywhere' }, sortBy: 'frp', limit: 1 });
const elapsed = performance.now() - started;
clearInterval(timer);
assert.equal(result.count, 250_000);
assert.equal(result.items[0].frp, 100_002);
assert.ok(elapsed < 400, `took ${Math.round(elapsed)} ms`);
assert.ok(ticks > 0, 'the scan yielded between slices');
let current = true;
const pending = engine.query({ layers: ['local-firms'], scope: { kind: 'anywhere' }, sortBy: 'distance' }, { isCurrent: () => current });
current = false;
assert.equal((await pending).code, 'CANCELLED', 'a superseded scan stops at the next slice');
});
test('analyst: a follow-up naming other layers than the last answer is refused', async () => {
const engine = createAnalystEngine({
getRecords: (key) => ({ flights: FLIGHTS, 'local-firms': FIRES }[key] || []),
resolveRegionRing: async () => null,
getViewContext: () => ({ lat: 30.27, lon: -97.74, viewRadiusKm: 150 }),
});
await engine.query({ layers: ['flights'], scope: { kind: 'anywhere' } });
const r = await engine.query({ layers: ['local-firms'], followUp: true });
assert.equal(r.ok, false);
assert.equal(r.code, 'FOLLOW_UP_MISMATCH');
assert.equal(engine.hasMemory(), true, 'the earlier answer is still the last answer');
});
+22 -8
View File
@@ -4,24 +4,38 @@
* The browser fetches the file from `url` — a `new URL('./….json',
* import.meta.url)` written in the calling module, which Vite serves as-is in
* dev and emits as an asset in the production build. Under Node (node:test)
* that URL is a `file:` URL, which `fetch` cannot read, so the caller's
* JSON-attributed import runs instead.
* that URL is a `file:` URL, which `fetch` cannot read, so the file is read
* from disk instead.
*
* The attributed import cannot serve the browser: the Vite dev server answers
* it with a JavaScript module, which the browser rejects for a JSON import.
* Nothing here imports the JSON as a module. A JSON-attributed import cannot
* serve the browser (the Vite dev server answers it with a JavaScript module,
* which the browser rejects), and a literal `import('./….json')` makes the
* production build emit a JavaScript copy of every pack that the browser
* never loads.
*
* @module data/bundledJson
*/
/**
* @param {URL} url - Resolved against the calling module's `import.meta.url`.
* @param {() => Promise<{default: any}>} importJson - The caller's literal
* `import('./….json', { with: { type: 'json' } })`, used only under Node.
* @param {{signal?: AbortSignal}} [options]
* @returns {Promise<any>} The parsed JSON.
*/
export async function loadBundledJson(url, importJson, { signal } = {}) {
if (url.protocol === 'file:') return (await importJson()).default;
export async function loadBundledJson(url, { signal } = {}) {
signal?.throwIfAborted();
if (url.protocol === 'file:') {
// Node's built-in, reached without an import, so the browser build never
// bundles or rewrites it; only Node, reading `file:` URLs, gets here. The
// read is synchronous so the result settles as promptly as the cached
// module import it replaces; callers already memoize each pack.
const getBuiltinModule = globalThis.process?.getBuiltinModule;
if (typeof getBuiltinModule !== 'function')
throw new Error(
`God's Eye View needs Node 24.14 or newer to read ${url.pathname.split('/').pop()} (running ${globalThis.process?.version ?? 'an unknown runtime'})`,
);
const { readFileSync } = getBuiltinModule('node:fs');
return JSON.parse(readFileSync(url, 'utf8'));
}
const response = await fetch(url, { signal });
if (!response.ok) {
throw new Error(`HTTP ${response.status} for ${url.pathname}`);
+57 -10
View File
@@ -2,13 +2,14 @@ import { test } from 'node:test';
import assert from 'node:assert/strict';
import { loadBundledJson } from './bundledJson.js';
const MARINE = new URL('./local_data/natural_earth/marine.json', import.meta.url);
const importMarine = () =>
import('./local_data/natural_earth/marine.json', { with: { type: 'json' } });
const MARINE = new URL(
'./local_data/natural_earth/marine.json',
import.meta.url,
);
test('bundled JSON: a file URL uses the JSON import under Node', async (t) => {
test('bundled JSON: a file URL is read from disk under Node', async (t) => {
const fetchMock = t.mock.method(globalThis, 'fetch');
const pack = await loadBundledJson(MARINE, importMarine);
const pack = await loadBundledJson(MARINE);
assert.ok(pack.features.some((ft) => ft.name === 'Gulf of Mexico'));
assert.equal(fetchMock.mock.callCount(), 0);
});
@@ -21,16 +22,62 @@ test('bundled JSON: a served URL is fetched as plain JSON', async (t) => {
headers: { 'content-type': 'application/json' },
});
});
const url = new URL('http://localhost/src/data/local_data/natural_earth/marine.json');
const importJson = () => assert.fail('the browser path must not import JSON');
assert.deepEqual(await loadBundledJson(url, importJson), { features: [] });
const url = new URL(
'http://localhost/src/data/local_data/natural_earth/marine.json',
);
assert.deepEqual(await loadBundledJson(url), { features: [] });
assert.deepEqual(requests, [url.href]);
});
test('bundled JSON: an HTTP error rejects so the retryable loader can retry', async (t) => {
t.mock.method(globalThis, 'fetch', async () => new Response('', { status: 404 }));
t.mock.method(
globalThis,
'fetch',
async () => new Response('', { status: 404 }),
);
await assert.rejects(
loadBundledJson(new URL('http://localhost/missing.json'), importMarine),
loadBundledJson(new URL('http://localhost/missing.json')),
/HTTP 404/,
);
});
test('bundled JSON: an aborted load rejects before reading', async () => {
const controller = new AbortController();
controller.abort();
await assert.rejects(loadBundledJson(MARINE, { signal: controller.signal }), {
name: 'AbortError',
});
});
test('bundled JSON: no data pack is imported as a module', async () => {
// A literal JSON import in app code makes the production build emit a
// JavaScript copy of the pack beside the fetched JSON, which the browser
// never loads. Tests are not bundled, so they may still import JSON.
const { spawnSync } = await import('node:child_process');
const result = spawnSync(
'git',
[
'grep',
'-nE',
String.raw`import\(\s*['"][^'"]*local_data/[^'"]*\.json`,
'--',
'src',
':!*.test.mjs',
],
{ cwd: new URL('../../', import.meta.url), encoding: 'utf8' },
);
// git grep exits 1 when nothing matches.
assert.equal(result.status, 1, result.stdout);
});
test('bundled JSON: an older Node says which version is needed', async (t) => {
// Node before 20.16 / 22.3 has no process.getBuiltinModule.
const original = process.getBuiltinModule;
t.after(() => {
process.getBuiltinModule = original;
});
process.getBuiltinModule = undefined;
await assert.rejects(loadBundledJson(MARINE), {
message: /needs Node 24\.14 or newer to read marine\.json/,
});
});
+25
View File
@@ -128,6 +128,14 @@ test('the catalog applies the fair cap to configured packs end to end', async ()
sources.slice(0, 3).map((s) => s.id),
['env-0', 'env-1', 'env-2'],
);
assert.equal(sources[0].pack, 'env');
const [trimmed] = getSources.trimmedPacks();
assert.deepEqual(
{ ...trimmed, region: undefined },
{ pack: 'env', available: 12, served: 9, region: undefined },
);
assert.equal(trimmed.region.south, 30);
assert.ok(Math.abs(trimmed.region.north - 30.11) < 1e-9);
} finally {
for (const key of Object.keys(process.env)) {
if (!(key in saved)) delete process.env[key];
@@ -135,3 +143,20 @@ test('the catalog applies the fair cap to configured packs end to end', async ()
Object.assign(process.env, saved);
}
});
test('a pack trimmed to its nearest cameras reports how many it offered', async () => {
const { prioritizeSources } =
await import('../../server/providers/cctv/normalize.js');
const cameras = Array.from({ length: 5 }, (_, i) => ({
id: `c${i}`,
lat: i,
lon: 0,
}));
const kept = prioritizeSources(cameras, 2, [{ lat: 0, lon: 0 }]);
assert.deepEqual(
kept.map((camera) => camera.id),
['c0', 'c1'],
);
assert.equal(kept.available, 5);
assert.deepEqual(kept.region, { west: 0, south: 0, east: 0, north: 4 });
});
-1
View File
@@ -25,7 +25,6 @@ test('firms analyst record: full record maps every contract field', () => {
frp: 1520.4,
confidence: 0.9,
satellite: 'N21',
sensor: 'VIIRS',
acqTime: 1_753_600_000_000,
});
});
+3 -7
View File
@@ -1,12 +1,8 @@
import { createLocalGeoJsonLayer } from './localGeojsonCore.js';
import { INFRASTRUCTURE_DATA_URLS } from '../sources/infrastructureData.js';
// Resolved by Vite in builds and relative to this module in other consumers.
const datacentersUrl = new URL(
'./local_data/datacenters/datacenters.geojsonl',
import.meta.url,
).href;
const damsUrl = new URL('./local_data/dams/dams.geojsonl', import.meta.url)
.href;
const datacentersUrl = INFRASTRUCTURE_DATA_URLS['local-datacenters'];
const damsUrl = INFRASTRUCTURE_DATA_URLS['local-dams'];
/**
* Create fresh datacenter and dam layers without starting or loading them.
+1
View File
@@ -192,6 +192,7 @@ test('consumer build includes only infrastructure code and resolves assets under
const sources = Object.keys(entry.modules).filter((id) => id.endsWith('.js'));
assert.deepEqual(sources.map((id) => id.split('/').at(-1)).sort(), [
'infrastructure.js',
'infrastructureData.js',
'infrastructureOverlayEntry.js',
'localGeojsonCore.js',
'localGeojsonLod.js',
+2 -723
View File
@@ -11,8 +11,6 @@ export const LAYER_STATE_TOKEN_ALPHABET =
'0123456789abcdefghijklmnopqrstuvwxyz';
const LAYER_STATE_TOKEN_PATTERN = /^[a-z0-9]{1,2}$/;
const NEW_SINGLE_CHARACTER_TOKEN_PATTERN = /^[0-9]$/;
/** Re-check cadence while a shared subject waits for its feed row to arrive. */
const PENDING_TRACKING_POLL_MS = 1_000;
/**
* A tracking ID is a transponder address, not free text: 6 hex digits for an
* ICAO24, with slack for TIS-B (`~abc123`) and similar prefixed forms. Bounding
@@ -433,12 +431,6 @@ const OPTION_GROUPS = Object.freeze({
]),
});
const TRACKING_OPTION_KEY_BY_LAYER = Object.freeze({
flights: 'selectedFlightsTrackingId',
military: 'selectedMilitaryTrackingId',
satellites: 'selectedSatTrackingId',
});
export const SHARE_TRACKING_RESTORE_POLICIES = Object.freeze({
flights: Object.freeze({
optionOwner: 'flights',
@@ -740,7 +732,7 @@ export function validateLayerStateAllocations(
return true;
}
const REGISTRY_BY_ID = new Map(
export const REGISTRY_BY_ID = new Map(
LAYER_STATE_REGISTRY.map((entry) => [entry.id, entry]),
);
const REGISTRY_BY_TOKEN = new Map(
@@ -752,7 +744,7 @@ const OPTION_OWNER_IDS = Object.freeze([
),
]);
function optionSpecs(ownerId) {
export function optionSpecs(ownerId) {
return OPTION_GROUPS[ownerId] || [];
}
@@ -1055,716 +1047,3 @@ export function layerOptionsForRestore(state, layerId) {
if (!entry?.optionOwner) return null;
return { ...normalizeLayerState(state).options[entry.optionOwner] };
}
function safeStorage() {
try {
return globalThis.localStorage || null;
} catch {
return null;
}
}
function currentLayerOutcome(dataManager, layerId) {
const state = dataManager.getLayerLifecycleState?.(layerId);
return {
settledEnabled: Boolean(state?.enabled),
lifecycleState: state?.lifecycleState || 'missing',
lifecycleUncertain: Boolean(state?.uncertain),
};
}
/**
* Owns durable user layer preferences independently from transient runtime
* choreography, and coordinates passive post-registration restoration.
*/
export class LayerStateCoordinator {
constructor(
dataManager,
shareLinkManager,
{
storage = safeStorage(),
restoreGate = null,
onDurableStateChange = null,
onTrackingRestoreStatus = null,
now = () => Date.now(),
// Injectable so the pending-window behavior is deterministically testable
// without sleeping out a 90 s expiry.
setTimer = (fn, ms) => setTimeout(fn, ms),
clearTimer = (handle) => clearTimeout(handle),
} = {},
) {
if (!dataManager?.registrationsFinalized) {
throw new Error(
'Layer state requires finalized data-layer registrations',
);
}
this.dataManager = dataManager;
this.shareLinkManager = shareLinkManager || null;
this.storage = storage;
this.restoreGate = restoreGate;
this.onDurableStateChange = onDurableStateChange;
this.onTrackingRestoreStatus = onTrackingRestoreStatus;
this.now = now;
this.setTimer = setTimer;
this.clearTimer = clearTimer;
this._durableState = createDefaultLayerState();
this._source = 'defaults';
this._destroyed = false;
this._restoreControllers = new Map();
this._shareCreatedAtMs = null;
this._trackingRestoreController = null;
this._trackingRestoreGeneration = 0;
this._pendingTrackingTimer = null;
this._pendingTrackingContext = null;
this._unsubscribe = this.dataManager.subscribe((change) =>
this._handleManagerChange(change),
);
this._unsubscribeVisibilityRequests =
this.dataManager.subscribeVisibilityRequests((change) =>
this._handleVisibilityRequest(change),
);
this.restorePromise = Promise.resolve([]);
this.lastRestoreResults = [];
}
start({
shareLayerState = null,
allowLocalState = true,
shareCreatedAtMs = null,
} = {}) {
if (this._destroyed)
throw new Error('Layer-state coordinator is destroyed');
let selected = shareLayerState
? normalizeLayerState(shareLayerState)
: null;
if (selected) {
this._source = 'share';
this._shareCreatedAtMs = Number.isFinite(shareCreatedAtMs)
? shareCreatedAtMs
: null;
} else if (allowLocalState) {
let stored = null;
try {
stored = parseStoredLayerState(
this.storage?.getItem?.(LAYER_STATE_STORAGE_KEY),
);
} catch {
/* best effort */
}
if (stored) {
selected = stored;
this._source = 'local';
}
} else {
// A valid historical camera/style share with no v2 layer payload keeps
// the exact legacy default-layer behavior. It must not inherit an
// unrelated recipient's saved local layer preferences.
this._source = 'legacy-share';
}
this._durableState = selected || createDefaultLayerState();
this.shareLinkManager?.setLayerStateProvider?.(() =>
this.getDurableState(),
);
this.shareLinkManager?.onLayerStateChange?.();
this._notifyDurableState();
if (!selected) return this.restorePromise;
this.restorePromise = this._restoreSelectedState(
this._source === 'share'
? LAYER_RESTORE_ORIGINS.share
: LAYER_RESTORE_ORIGINS.local,
);
return this.restorePromise;
}
get source() {
return this._source;
}
getDurableState() {
return cloneLayerState(this._durableState);
}
_notifyDurableState() {
try {
this.onDurableStateChange?.(this.getDurableState());
} catch {
/* UI sync is best effort */
}
}
_handleVisibilityRequest(change) {
if (!isExplicitLayerStateOrigin(change?.origin)) return;
this._restoreControllers
.get(change.layerId)
?.abort('superseded-by-explicit-visibility');
if (SHARE_TRACKING_RESTORE_POLICIES[change.layerId]) {
this._revokePendingTrackingWatch('superseded-by-explicit-visibility');
}
}
/** Revoke every passive restore before explicit navigation can be reclaimed. */
cancelPendingRestores(reason = 'superseded-by-explicit-navigation') {
for (const controller of this._restoreControllers.values())
controller.abort(reason);
this._revokePendingTrackingWatch(reason);
}
/**
* Revoke a pending shared Follow. Physical navigation may also clear only
* the exact passive selection, without writing recipient preferences.
*/
cancelPendingShareTracking(
reason = 'superseded-by-explicit-navigation',
{ clearSelection = false } = {},
) {
this._revokePendingTrackingWatch(reason);
if (!clearSelection) return false;
const selected = this._selectedShareTrackingTarget();
return selected ? this._passivelyClearTrackingSelection(selected) : false;
}
_handleManagerChange(change) {
if (!change || this._destroyed) return;
// Parameter and visibility ownership are independent. A newer explicit
// option request may replace passive share options, but it must not abort
// the same layer's visibility lifecycle.
if (change.type === 'params-requested') {
if (
isExplicitLayerStateOrigin(change.origin) &&
SHARE_TRACKING_RESTORE_POLICIES[change.layerId]
) {
this._revokePendingTrackingWatch('superseded-by-explicit-params');
}
return;
}
// A layer that goes away takes its latch with it, at ANY origin — a
// programmatic disable or teardown never reaches the explicit-intent path
// below, so revoke here before that early return.
if (
change.type === 'visibility' &&
change.enabled === false &&
SHARE_TRACKING_RESTORE_POLICIES[change.layerId]
) {
this._revokePendingTrackingWatch('owner-layer-disabled');
}
if (!isExplicitLayerStateOrigin(change.origin)) return;
if (change.type === 'visibility') {
this._restoreControllers
.get(change.layerId)
?.abort('superseded-by-explicit-visibility');
if (SHARE_TRACKING_RESTORE_POLICIES[change.layerId]) {
this._revokePendingTrackingWatch('superseded-by-explicit-visibility');
}
const enabled = new Set(this._durableState.enabledLayerIds);
if (change.enabled) enabled.add(change.layerId);
else enabled.delete(change.layerId);
this._commitExplicit({
...this._durableState,
enabledLayerIds: [...enabled],
});
return;
}
if (change.type !== 'params') return;
const entry = REGISTRY_BY_ID.get(change.layerId);
if (!entry?.optionOwner) return;
const ownerId = entry.optionOwner;
const nextOwnerOptions = { ...this._durableState.options[ownerId] };
const requestedParams = change.requestedParams || {};
const trackingOptionKey =
TRACKING_OPTION_KEY_BY_LAYER[change.layerId] || null;
let changed = false;
for (const spec of optionSpecs(ownerId)) {
// Only persist keys present in this explicit request. getLayerParams()
// can return a wider live snapshot containing transient or passively
// changed values that this user action did not own. Tracking is the one
// exception: an unrelated explicit option cancels a pending restoration
// in that family, so its wider live value (active ID or null) must replace
// the formerly durable pending ID instead of allowing reload resurrection.
const explicitlyRequested = Object.hasOwn(requestedParams, spec.key);
const implicitTrackingSync =
!explicitlyRequested && spec.key === trackingOptionKey;
if (!explicitlyRequested && !implicitTrackingSync) continue;
const value = spec.normalize(change.params[spec.key]);
if (value === null) {
if (implicitTrackingSync) {
nextOwnerOptions[spec.key] = null;
changed = true;
continue;
}
if (spec.defaultValue !== null || change.params[spec.key] !== null)
continue;
}
nextOwnerOptions[spec.key] = value;
changed = true;
}
if (!changed) return;
this._commitExplicit({
...this._durableState,
options: { ...this._durableState.options, [ownerId]: nextOwnerOptions },
});
}
_commitExplicit(candidate) {
this._durableState = normalizeLayerState(candidate);
const serialized = serializeStoredLayerState(this._durableState);
try {
if (this.storage?.getItem?.(LAYER_STATE_STORAGE_KEY) !== serialized) {
this.storage?.setItem?.(LAYER_STATE_STORAGE_KEY, serialized);
}
} catch {
/* storage can be unavailable or quota-limited */
}
this.shareLinkManager?.onLayerStateChange?.();
this._notifyDurableState();
}
async _waitForRestoreGate() {
if (!this.restoreGate) return;
await (typeof this.restoreGate === 'function'
? this.restoreGate()
: this.restoreGate);
}
async _restoreSelectedState(origin) {
for (const entry of LAYER_STATE_REGISTRY) {
this._restoreControllers.set(entry.id, new AbortController());
}
try {
await this._waitForRestoreGate();
const enabled = new Set(this._durableState.enabledLayerIds);
const settled = await Promise.allSettled(
LAYER_STATE_REGISTRY.map(async (entry) => {
const controller = this._restoreControllers.get(entry.id);
const targetEnabled = enabled.has(entry.id);
const options = layerOptionsForRestore(this._durableState, entry.id);
if (origin === LAYER_RESTORE_ORIGINS.share && options) {
for (const trackingKey of Object.values(
TRACKING_OPTION_KEY_BY_LAYER,
)) {
delete options[trackingKey];
}
}
if (this._destroyed || controller?.signal.aborted) {
return {
layerId: entry.id,
targetEnabled,
origin,
phase: 'reserved',
...currentLayerOutcome(this.dataManager, entry.id),
appliedOptions: {},
cancellationReason: this._destroyed ? 'destroyed' : 'superseded',
errorClass: 'cancelled',
persistenceWrite: false,
succeeded: false,
};
}
// Reserve passive option state before any asynchronous lifecycle work.
// A later explicit params intent then wins on its own lane without
// cancelling or being overwritten by the visibility restore.
const paramsSucceeded =
!options ||
Object.keys(options).length === 0 ||
this.dataManager.setLayerParams(entry.id, options, { origin });
return this.dataManager
.restoreLayerState(
entry.id,
{
enabled: targetEnabled,
params: null,
},
{ origin, signal: controller.signal },
)
.then((result) => ({
...result,
appliedOptions: paramsSucceeded && options ? options : {},
errorClass: paramsSucceeded
? result.errorClass
: 'ParamsRejected',
succeeded: paramsSucceeded && result.succeeded,
}));
}),
);
this.lastRestoreResults = settled.map((result, index) => {
if (result.status === 'fulfilled') return result.value;
const entry = LAYER_STATE_REGISTRY[index];
return {
layerId: entry.id,
targetEnabled: enabled.has(entry.id),
origin,
phase: 'coordinator',
...currentLayerOutcome(this.dataManager, entry.id),
appliedOptions: {},
cancellationReason: null,
errorClass: result.reason?.name || 'Error',
error: String(result.reason?.message || result.reason),
persistenceWrite: false,
succeeded: false,
};
});
return this.lastRestoreResults.map((result) => ({ ...result }));
} finally {
this._restoreControllers.clear();
this._notifyDurableState();
}
}
_selectedShareTrackingTarget() {
if (this._source !== 'share') return null;
for (const [layerId, policy] of Object.entries(
SHARE_TRACKING_RESTORE_POLICIES,
)) {
const targetId =
this._durableState.options?.[policy.optionOwner]?.[policy.optionKey];
if (targetId !== null && targetId !== undefined && targetId !== '') {
return { layerId, targetId, ...policy };
}
}
return null;
}
_passivelyClearTrackingSelection(selected) {
const current =
this._durableState.options?.[selected.optionOwner]?.[selected.optionKey];
if (String(current) !== String(selected.targetId)) return false;
const ownerOptions = {
...this._durableState.options[selected.optionOwner],
[selected.optionKey]: null,
};
this._durableState = normalizeLayerState({
...this._durableState,
options: {
...this._durableState.options,
[selected.optionOwner]: ownerOptions,
},
});
this.dataManager.setLayerParams?.(
selected.layerId,
{ [selected.optionKey]: null },
{ origin: LAYER_RESTORE_ORIGINS.share },
);
this.shareLinkManager?.onLayerStateChange?.();
this._notifyDurableState();
return true;
}
/**
* `atMs` is the moment the subject was first found ABSENT, not the moment the
* verdict is delivered. Waiting out the pending window must not by itself
* push a fresh link into the "expired" wording — that word describes the
* SHARE's age, not how long this client watched for the subject.
*/
_classifyMissingTrackingTarget(selected, atMs = this.now()) {
const copiedAt = this._shareCreatedAtMs;
if (!Number.isFinite(copiedAt)) return 'unavailable';
const ageMs = atMs - copiedAt;
return ageMs > selected.expiryWindowMs ? 'expired' : 'unavailable';
}
/** Whether the owning layer currently follows the shared subject. */
_trackingTargetLatched(selected) {
const params = this.dataManager.getLayerParams?.(selected.layerId);
const active = params?.[selected.optionKey];
return (
active !== null &&
active !== undefined &&
String(active) === String(selected.targetId)
);
}
/** Stop watching a pending shared subject without deciding its fate. */
_cancelPendingTrackingWatch() {
if (this._pendingTrackingTimer !== null)
this.clearTimer(this._pendingTrackingTimer);
this._pendingTrackingTimer = null;
const pending = this._pendingTrackingContext;
if (pending?.signal && pending.abortHandler) {
pending.signal.removeEventListener('abort', pending.abortHandler);
pending.abortHandler = null;
}
}
/** Publish a share-follow lifecycle update without allowing UI errors to own state. */
_publishTrackingRestoreStatus(status) {
try {
this.onTrackingRestoreStatus?.(status);
} catch {
/* status UI is best effort */
}
}
/**
* Revoke a pending shared Follow wholesale.
*
* The watch and the LAYER's own deferred-restore latch are two halves of one
* mechanism, so they must die together. Aborting only the restore controller
* left the timer alive: the controller has already settled by the time the
* watch exists, so the abort was a no-op and the orphaned timer went on to
* announce "Shared … unavailable" for a subject whose latch had been
* cancelled — a notice about work no longer being attempted.
*/
_revokePendingTrackingWatch(reason) {
this._trackingRestoreController?.abort(reason);
this._trackingRestoreGeneration += 1;
this._cancelPendingTrackingWatch();
const pending = this._pendingTrackingContext;
this._pendingTrackingContext = null;
if (pending) {
this.dataManager.cancelPendingLayerRestore?.(pending.selected.layerId, {
origin: LAYER_RESTORE_ORIGINS.share,
reason: String(reason || 'cancelled'),
});
this._publishTrackingRestoreStatus({
...pending.probe,
...pending.selected,
status: 'cancelled',
classification: 'cancelled',
reason: String(reason || 'cancelled'),
cleared: false,
});
}
}
/**
* Hold a not-yet-arrived shared subject PENDING instead of declaring it gone.
*
* A recipient's first authoritative refresh routinely lands without a given
* contact — the feed is polled, coverage is partial, and rendering trails the
* snapshot by a poll. Reload-from-local already survives this: its restore
* arms the layer's own deferred-restore latch, which re-attempts on every
* later poll. The shared path used to decide on that single refresh, clear
* the subject from durable state AND from the URL, then post a failure notice
* seconds into startup — so the same link healed on reload but never on the
* share.
*
* Arm the SAME latch, then watch it in the background: the caller is never
* blocked (startup must not wait out a 90 s window before it may write the
* URL again), and the terminal verdict is deferred until the source-specific
* window has genuinely expired. The existing wordings are unchanged.
*/
async _beginPendingTrackingRestore(selected, probe, signal = null) {
const generation = this._trackingRestoreGeneration;
const absentAtMs = this.now();
// Arm the layer's deferred-restore latch under the passive share origin, so
// it re-attempts each poll and never rewrites recipient preferences.
let armed = false;
let armError = null;
try {
armed =
(await this.dataManager.setLayerParams?.(
selected.layerId,
{ [selected.optionKey]: selected.targetId },
{ origin: LAYER_RESTORE_ORIGINS.share },
)) === true;
} catch (error) {
armError = error;
}
if (
this._destroyed ||
generation !== this._trackingRestoreGeneration ||
signal?.aborted
) {
if (armed) {
this.dataManager.cancelPendingLayerRestore?.(selected.layerId, {
origin: LAYER_RESTORE_ORIGINS.share,
reason: String(signal?.reason || 'superseded'),
});
}
return {
...probe,
...selected,
status: 'cancelled',
classification: 'cancelled',
reason: String(signal?.reason || 'superseded'),
cleared: false,
};
}
if (!armed) {
const terminal = {
...probe,
...selected,
status: 'source-unavailable',
classification: 'source-unavailable',
reason: String(
armError?.message || armError || 'tracking restore latch rejected',
),
cleared: this._passivelyClearTrackingSelection(selected),
};
this._publishTrackingRestoreStatus(terminal);
return terminal;
}
const deadline = this.now() + selected.expiryWindowMs;
const settle = (terminal) => {
if (this._pendingTrackingContext?.generation !== generation) return;
this._cancelPendingTrackingWatch();
this._pendingTrackingContext = null;
this._publishTrackingRestoreStatus(terminal);
};
const poll = () => {
this._pendingTrackingTimer = null;
if (this._destroyed || generation !== this._trackingRestoreGeneration)
return;
// The layer that owns the latch may have gone away since the last tick
// (disable, teardown, replacement). There is nothing left attempting this
// restore, so abandon it silently rather than announcing a verdict.
if (this.dataManager.isEffectivelyEnabled?.(selected.layerId) === false) {
this._revokePendingTrackingWatch('owner-layer-disabled');
return;
}
if (this._trackingTargetLatched(selected)) {
settle({
...probe,
...selected,
status: 'found',
classification: 'followed',
cleared: false,
});
return;
}
if (this.now() >= deadline) {
// The window really has elapsed — only now does the verdict apply.
const classification =
probe.status === 'missing'
? this._classifyMissingTrackingTarget(selected, absentAtMs)
: 'source-unavailable';
const cleared = this._passivelyClearTrackingSelection(selected);
settle({ ...probe, ...selected, classification, cleared });
return;
}
this._pendingTrackingTimer = this.setTimer(
poll,
PENDING_TRACKING_POLL_MS,
);
this._pendingTrackingTimer?.unref?.();
};
this._cancelPendingTrackingWatch();
const pending = {
...probe,
...selected,
status: 'pending',
classification: 'pending',
cleared: false,
};
const pendingContext = {
generation,
selected,
probe,
signal,
abortHandler: null,
};
if (signal) {
pendingContext.abortHandler = () => {
if (this._pendingTrackingContext?.generation !== generation) return;
this._revokePendingTrackingWatch(signal.reason || 'aborted');
};
signal.addEventListener('abort', pendingContext.abortHandler, {
once: true,
});
}
this._pendingTrackingContext = pendingContext;
this._publishTrackingRestoreStatus(pending);
this._pendingTrackingTimer = this.setTimer(poll, PENDING_TRACKING_POLL_MS);
this._pendingTrackingTimer?.unref?.();
return pending;
}
/**
* Refresh and restore the one shareable tracked target after destination
* camera and ordinary layer restoration have settled.
*/
async restoreShareTrackingSelection({ signal = null } = {}) {
const selected = this._selectedShareTrackingTarget();
if (!selected || this._destroyed)
return { status: 'skipped', reason: 'no-shared-target' };
this._revokePendingTrackingWatch('superseded-by-newer-restore');
const controller = new AbortController();
const combinedSignal = signal
? AbortSignal.any([signal, controller.signal])
: controller.signal;
this._trackingRestoreController = controller;
const generation = ++this._trackingRestoreGeneration;
let result;
try {
result = await this.dataManager.resolveLayerTrackingTarget(
selected.layerId,
selected.targetId,
{ signal: combinedSignal, origin: LAYER_RESTORE_ORIGINS.share },
);
} catch (error) {
result = combinedSignal.aborted
? {
status: 'cancelled',
reason: String(combinedSignal.reason || 'aborted'),
}
: {
status: 'source-unavailable',
reason: String(error?.message || error),
};
}
if (
generation !== this._trackingRestoreGeneration ||
this._destroyed ||
combinedSignal.aborted
) {
if (this._trackingRestoreController === controller)
this._trackingRestoreController = null;
return {
...result,
status: 'cancelled',
reason: String(combinedSignal.reason || 'superseded'),
};
}
if (this._trackingRestoreController === controller)
this._trackingRestoreController = null;
if (result.status === 'found') {
const terminal = {
...result,
...selected,
classification: 'followed',
cleared: false,
};
this._publishTrackingRestoreStatus(terminal);
return terminal;
}
if (['cancelled', 'superseded', 'destroyed'].includes(result.status))
return result;
// A subject that is simply not here YET is not a subject that is gone. Hold
// it on the layer's own deferred-restore latch for its source-specific
// window before any verdict is reached or shown. `unsupported` layers have
// no latch to arm, so they still decide immediately.
if (result.status === 'missing' || result.status === 'source-unavailable') {
return this._beginPendingTrackingRestore(
selected,
result,
combinedSignal,
);
}
const classification =
result.status === 'missing'
? this._classifyMissingTrackingTarget(selected)
: 'source-unavailable';
const cleared = this._passivelyClearTrackingSelection(selected);
const terminal = { ...result, ...selected, classification, cleared };
this._publishTrackingRestoreStatus(terminal);
return terminal;
}
destroy() {
if (this._destroyed) return;
this._destroyed = true;
for (const controller of this._restoreControllers.values())
controller.abort('coordinator-destroyed');
this._restoreControllers.clear();
this._revokePendingTrackingWatch('coordinator-destroyed');
this._trackingRestoreController = null;
this._unsubscribe?.();
this._unsubscribe = null;
this._unsubscribeVisibilityRequests?.();
this._unsubscribeVisibilityRequests = null;
this.shareLinkManager?.setLayerStateProvider?.(null);
this.onDurableStateChange = null;
this.onTrackingRestoreStatus = null;
}
}
+1 -1
View File
@@ -11,7 +11,6 @@ import {
LAYER_STATE_STORAGE_KEY,
LAYER_STATE_TOKEN_ALPHABET,
LAYER_STATE_TOKEN_RESERVATIONS,
LayerStateCoordinator,
REGISTERED_LAYER_IDS,
SHARE_TRACKING_RESTORE_POLICIES,
createDefaultLayerState,
@@ -24,6 +23,7 @@ import {
validateLayerStateAllocations,
validateLayerStateRegistry,
} from './layerState.js';
import { LayerStateCoordinator } from './layerStateCoordinator.js';
import radioLayer from './radio.js';
import { stampInitialShareGesture } from '../navigationPolicy.js';
+742
View File
@@ -0,0 +1,742 @@
/**
* Applies layer state to the running application: restores shared and stored
* layer choices and persists changes to browser storage.
*/
import {
LAYER_RESTORE_ORIGINS,
LAYER_STATE_REGISTRY,
LAYER_STATE_STORAGE_KEY,
REGISTRY_BY_ID,
SHARE_TRACKING_RESTORE_POLICIES,
cloneLayerState,
createDefaultLayerState,
isExplicitLayerStateOrigin,
layerOptionsForRestore,
normalizeLayerState,
optionSpecs,
parseStoredLayerState,
serializeStoredLayerState,
} from './layerState.js';
/** Re-check cadence while a shared subject waits for its feed row to arrive. */
const PENDING_TRACKING_POLL_MS = 1_000;
const TRACKING_OPTION_KEY_BY_LAYER = Object.freeze({
flights: 'selectedFlightsTrackingId',
military: 'selectedMilitaryTrackingId',
satellites: 'selectedSatTrackingId',
});
function safeStorage() {
try {
return globalThis.localStorage || null;
} catch {
return null;
}
}
function currentLayerOutcome(dataManager, layerId) {
const state = dataManager.getLayerLifecycleState?.(layerId);
return {
settledEnabled: Boolean(state?.enabled),
lifecycleState: state?.lifecycleState || 'missing',
lifecycleUncertain: Boolean(state?.uncertain),
};
}
/**
* Owns durable user layer preferences independently from transient runtime
* choreography, and coordinates passive post-registration restoration.
*/
export class LayerStateCoordinator {
constructor(
dataManager,
shareLinkManager,
{
storage = safeStorage(),
restoreGate = null,
onDurableStateChange = null,
onTrackingRestoreStatus = null,
now = () => Date.now(),
// Injectable so the pending-window behavior is deterministically testable
// without sleeping out a 90 s expiry.
setTimer = (fn, ms) => setTimeout(fn, ms),
clearTimer = (handle) => clearTimeout(handle),
} = {},
) {
if (!dataManager?.registrationsFinalized) {
throw new Error(
'Layer state requires finalized data-layer registrations',
);
}
this.dataManager = dataManager;
this.shareLinkManager = shareLinkManager || null;
this.storage = storage;
this.restoreGate = restoreGate;
this.onDurableStateChange = onDurableStateChange;
this.onTrackingRestoreStatus = onTrackingRestoreStatus;
this.now = now;
this.setTimer = setTimer;
this.clearTimer = clearTimer;
this._durableState = createDefaultLayerState();
this._source = 'defaults';
this._destroyed = false;
this._restoreControllers = new Map();
this._shareCreatedAtMs = null;
this._trackingRestoreController = null;
this._trackingRestoreGeneration = 0;
this._pendingTrackingTimer = null;
this._pendingTrackingContext = null;
this._unsubscribe = this.dataManager.subscribe((change) =>
this._handleManagerChange(change),
);
this._unsubscribeVisibilityRequests =
this.dataManager.subscribeVisibilityRequests((change) =>
this._handleVisibilityRequest(change),
);
this.restorePromise = Promise.resolve([]);
this.lastRestoreResults = [];
}
start({
shareLayerState = null,
allowLocalState = true,
shareCreatedAtMs = null,
} = {}) {
if (this._destroyed)
throw new Error('Layer-state coordinator is destroyed');
let selected = shareLayerState
? normalizeLayerState(shareLayerState)
: null;
if (selected) {
this._source = 'share';
this._shareCreatedAtMs = Number.isFinite(shareCreatedAtMs)
? shareCreatedAtMs
: null;
} else if (allowLocalState) {
let stored = null;
try {
stored = parseStoredLayerState(
this.storage?.getItem?.(LAYER_STATE_STORAGE_KEY),
);
} catch {
/* best effort */
}
if (stored) {
selected = stored;
this._source = 'local';
}
} else {
// A valid historical camera/style share with no v2 layer payload keeps
// the exact legacy default-layer behavior. It must not inherit an
// unrelated recipient's saved local layer preferences.
this._source = 'legacy-share';
}
this._durableState = selected || createDefaultLayerState();
this.shareLinkManager?.setLayerStateProvider?.(() =>
this.getDurableState(),
);
this.shareLinkManager?.onLayerStateChange?.();
this._notifyDurableState();
if (!selected) return this.restorePromise;
this.restorePromise = this._restoreSelectedState(
this._source === 'share'
? LAYER_RESTORE_ORIGINS.share
: LAYER_RESTORE_ORIGINS.local,
);
return this.restorePromise;
}
get source() {
return this._source;
}
getDurableState() {
return cloneLayerState(this._durableState);
}
_notifyDurableState() {
try {
this.onDurableStateChange?.(this.getDurableState());
} catch {
/* UI sync is best effort */
}
}
_handleVisibilityRequest(change) {
if (!isExplicitLayerStateOrigin(change?.origin)) return;
this._restoreControllers
.get(change.layerId)
?.abort('superseded-by-explicit-visibility');
if (SHARE_TRACKING_RESTORE_POLICIES[change.layerId]) {
this._revokePendingTrackingWatch('superseded-by-explicit-visibility');
}
}
/** Revoke every passive restore before explicit navigation can be reclaimed. */
cancelPendingRestores(reason = 'superseded-by-explicit-navigation') {
for (const controller of this._restoreControllers.values())
controller.abort(reason);
this._revokePendingTrackingWatch(reason);
}
/**
* Revoke a pending shared Follow. Physical navigation may also clear only
* the exact passive selection, without writing recipient preferences.
*/
cancelPendingShareTracking(
reason = 'superseded-by-explicit-navigation',
{ clearSelection = false } = {},
) {
this._revokePendingTrackingWatch(reason);
if (!clearSelection) return false;
const selected = this._selectedShareTrackingTarget();
return selected ? this._passivelyClearTrackingSelection(selected) : false;
}
_handleManagerChange(change) {
if (!change || this._destroyed) return;
// Parameter and visibility ownership are independent. A newer explicit
// option request may replace passive share options, but it must not abort
// the same layer's visibility lifecycle.
if (change.type === 'params-requested') {
if (
isExplicitLayerStateOrigin(change.origin) &&
SHARE_TRACKING_RESTORE_POLICIES[change.layerId]
) {
this._revokePendingTrackingWatch('superseded-by-explicit-params');
}
return;
}
// A layer that goes away takes its latch with it, at ANY origin — a
// programmatic disable or teardown never reaches the explicit-intent path
// below, so revoke here before that early return.
if (
change.type === 'visibility' &&
change.enabled === false &&
SHARE_TRACKING_RESTORE_POLICIES[change.layerId]
) {
this._revokePendingTrackingWatch('owner-layer-disabled');
}
if (!isExplicitLayerStateOrigin(change.origin)) return;
if (change.type === 'visibility') {
this._restoreControllers
.get(change.layerId)
?.abort('superseded-by-explicit-visibility');
if (SHARE_TRACKING_RESTORE_POLICIES[change.layerId]) {
this._revokePendingTrackingWatch('superseded-by-explicit-visibility');
}
const enabled = new Set(this._durableState.enabledLayerIds);
if (change.enabled) enabled.add(change.layerId);
else enabled.delete(change.layerId);
this._commitExplicit({
...this._durableState,
enabledLayerIds: [...enabled],
});
return;
}
if (change.type !== 'params') return;
const entry = REGISTRY_BY_ID.get(change.layerId);
if (!entry?.optionOwner) return;
const ownerId = entry.optionOwner;
const nextOwnerOptions = { ...this._durableState.options[ownerId] };
const requestedParams = change.requestedParams || {};
const trackingOptionKey =
TRACKING_OPTION_KEY_BY_LAYER[change.layerId] || null;
let changed = false;
for (const spec of optionSpecs(ownerId)) {
// Only persist keys present in this explicit request. getLayerParams()
// can return a wider live snapshot containing transient or passively
// changed values that this user action did not own. Tracking is the one
// exception: an unrelated explicit option cancels a pending restoration
// in that family, so its wider live value (active ID or null) must replace
// the formerly durable pending ID instead of allowing reload resurrection.
const explicitlyRequested = Object.hasOwn(requestedParams, spec.key);
const implicitTrackingSync =
!explicitlyRequested && spec.key === trackingOptionKey;
if (!explicitlyRequested && !implicitTrackingSync) continue;
const value = spec.normalize(change.params[spec.key]);
if (value === null) {
if (implicitTrackingSync) {
nextOwnerOptions[spec.key] = null;
changed = true;
continue;
}
if (spec.defaultValue !== null || change.params[spec.key] !== null)
continue;
}
nextOwnerOptions[spec.key] = value;
changed = true;
}
if (!changed) return;
this._commitExplicit({
...this._durableState,
options: { ...this._durableState.options, [ownerId]: nextOwnerOptions },
});
}
_commitExplicit(candidate) {
this._durableState = normalizeLayerState(candidate);
const serialized = serializeStoredLayerState(this._durableState);
try {
if (this.storage?.getItem?.(LAYER_STATE_STORAGE_KEY) !== serialized) {
this.storage?.setItem?.(LAYER_STATE_STORAGE_KEY, serialized);
}
} catch {
/* storage can be unavailable or quota-limited */
}
this.shareLinkManager?.onLayerStateChange?.();
this._notifyDurableState();
}
async _waitForRestoreGate() {
if (!this.restoreGate) return;
await (typeof this.restoreGate === 'function'
? this.restoreGate()
: this.restoreGate);
}
async _restoreSelectedState(origin) {
for (const entry of LAYER_STATE_REGISTRY) {
this._restoreControllers.set(entry.id, new AbortController());
}
try {
await this._waitForRestoreGate();
const enabled = new Set(this._durableState.enabledLayerIds);
const settled = await Promise.allSettled(
LAYER_STATE_REGISTRY.map(async (entry) => {
const controller = this._restoreControllers.get(entry.id);
const targetEnabled = enabled.has(entry.id);
const options = layerOptionsForRestore(this._durableState, entry.id);
if (origin === LAYER_RESTORE_ORIGINS.share && options) {
for (const trackingKey of Object.values(
TRACKING_OPTION_KEY_BY_LAYER,
)) {
delete options[trackingKey];
}
}
if (this._destroyed || controller?.signal.aborted) {
return {
layerId: entry.id,
targetEnabled,
origin,
phase: 'reserved',
...currentLayerOutcome(this.dataManager, entry.id),
appliedOptions: {},
cancellationReason: this._destroyed ? 'destroyed' : 'superseded',
errorClass: 'cancelled',
persistenceWrite: false,
succeeded: false,
};
}
// Reserve passive option state before any asynchronous lifecycle work.
// A later explicit params intent then wins on its own lane without
// cancelling or being overwritten by the visibility restore.
const paramsSucceeded =
!options ||
Object.keys(options).length === 0 ||
this.dataManager.setLayerParams(entry.id, options, { origin });
return this.dataManager
.restoreLayerState(
entry.id,
{
enabled: targetEnabled,
params: null,
},
{ origin, signal: controller.signal },
)
.then((result) => ({
...result,
appliedOptions: paramsSucceeded && options ? options : {},
errorClass: paramsSucceeded
? result.errorClass
: 'ParamsRejected',
succeeded: paramsSucceeded && result.succeeded,
}));
}),
);
this.lastRestoreResults = settled.map((result, index) => {
if (result.status === 'fulfilled') return result.value;
const entry = LAYER_STATE_REGISTRY[index];
return {
layerId: entry.id,
targetEnabled: enabled.has(entry.id),
origin,
phase: 'coordinator',
...currentLayerOutcome(this.dataManager, entry.id),
appliedOptions: {},
cancellationReason: null,
errorClass: result.reason?.name || 'Error',
error: String(result.reason?.message || result.reason),
persistenceWrite: false,
succeeded: false,
};
});
return this.lastRestoreResults.map((result) => ({ ...result }));
} finally {
this._restoreControllers.clear();
this._notifyDurableState();
}
}
_selectedShareTrackingTarget() {
if (this._source !== 'share') return null;
for (const [layerId, policy] of Object.entries(
SHARE_TRACKING_RESTORE_POLICIES,
)) {
const targetId =
this._durableState.options?.[policy.optionOwner]?.[policy.optionKey];
if (targetId !== null && targetId !== undefined && targetId !== '') {
return { layerId, targetId, ...policy };
}
}
return null;
}
_passivelyClearTrackingSelection(selected) {
const current =
this._durableState.options?.[selected.optionOwner]?.[selected.optionKey];
if (String(current) !== String(selected.targetId)) return false;
const ownerOptions = {
...this._durableState.options[selected.optionOwner],
[selected.optionKey]: null,
};
this._durableState = normalizeLayerState({
...this._durableState,
options: {
...this._durableState.options,
[selected.optionOwner]: ownerOptions,
},
});
this.dataManager.setLayerParams?.(
selected.layerId,
{ [selected.optionKey]: null },
{ origin: LAYER_RESTORE_ORIGINS.share },
);
this.shareLinkManager?.onLayerStateChange?.();
this._notifyDurableState();
return true;
}
/**
* `atMs` is the moment the subject was first found ABSENT, not the moment the
* verdict is delivered. Waiting out the pending window must not by itself
* push a fresh link into the "expired" wording — that word describes the
* SHARE's age, not how long this client watched for the subject.
*/
_classifyMissingTrackingTarget(selected, atMs = this.now()) {
const copiedAt = this._shareCreatedAtMs;
if (!Number.isFinite(copiedAt)) return 'unavailable';
const ageMs = atMs - copiedAt;
return ageMs > selected.expiryWindowMs ? 'expired' : 'unavailable';
}
/** Whether the owning layer currently follows the shared subject. */
_trackingTargetLatched(selected) {
const params = this.dataManager.getLayerParams?.(selected.layerId);
const active = params?.[selected.optionKey];
return (
active !== null &&
active !== undefined &&
String(active) === String(selected.targetId)
);
}
/** Stop watching a pending shared subject without deciding its fate. */
_cancelPendingTrackingWatch() {
if (this._pendingTrackingTimer !== null)
this.clearTimer(this._pendingTrackingTimer);
this._pendingTrackingTimer = null;
const pending = this._pendingTrackingContext;
if (pending?.signal && pending.abortHandler) {
pending.signal.removeEventListener('abort', pending.abortHandler);
pending.abortHandler = null;
}
}
/** Publish a share-follow lifecycle update without allowing UI errors to own state. */
_publishTrackingRestoreStatus(status) {
try {
this.onTrackingRestoreStatus?.(status);
} catch {
/* status UI is best effort */
}
}
/**
* Revoke a pending shared Follow wholesale.
*
* The watch and the LAYER's own deferred-restore latch are two halves of one
* mechanism, so they must die together. Aborting only the restore controller
* left the timer alive: the controller has already settled by the time the
* watch exists, so the abort was a no-op and the orphaned timer went on to
* announce "Shared … unavailable" for a subject whose latch had been
* cancelled — a notice about work no longer being attempted.
*/
_revokePendingTrackingWatch(reason) {
this._trackingRestoreController?.abort(reason);
this._trackingRestoreGeneration += 1;
this._cancelPendingTrackingWatch();
const pending = this._pendingTrackingContext;
this._pendingTrackingContext = null;
if (pending) {
this.dataManager.cancelPendingLayerRestore?.(pending.selected.layerId, {
origin: LAYER_RESTORE_ORIGINS.share,
reason: String(reason || 'cancelled'),
});
this._publishTrackingRestoreStatus({
...pending.probe,
...pending.selected,
status: 'cancelled',
classification: 'cancelled',
reason: String(reason || 'cancelled'),
cleared: false,
});
}
}
/**
* Hold a not-yet-arrived shared subject PENDING instead of declaring it gone.
*
* A recipient's first authoritative refresh routinely lands without a given
* contact — the feed is polled, coverage is partial, and rendering trails the
* snapshot by a poll. Reload-from-local already survives this: its restore
* arms the layer's own deferred-restore latch, which re-attempts on every
* later poll. The shared path used to decide on that single refresh, clear
* the subject from durable state AND from the URL, then post a failure notice
* seconds into startup — so the same link healed on reload but never on the
* share.
*
* Arm the SAME latch, then watch it in the background: the caller is never
* blocked (startup must not wait out a 90 s window before it may write the
* URL again), and the terminal verdict is deferred until the source-specific
* window has genuinely expired. The existing wordings are unchanged.
*/
async _beginPendingTrackingRestore(selected, probe, signal = null) {
const generation = this._trackingRestoreGeneration;
const absentAtMs = this.now();
// Arm the layer's deferred-restore latch under the passive share origin, so
// it re-attempts each poll and never rewrites recipient preferences.
let armed = false;
let armError = null;
try {
armed =
(await this.dataManager.setLayerParams?.(
selected.layerId,
{ [selected.optionKey]: selected.targetId },
{ origin: LAYER_RESTORE_ORIGINS.share },
)) === true;
} catch (error) {
armError = error;
}
if (
this._destroyed ||
generation !== this._trackingRestoreGeneration ||
signal?.aborted
) {
if (armed) {
this.dataManager.cancelPendingLayerRestore?.(selected.layerId, {
origin: LAYER_RESTORE_ORIGINS.share,
reason: String(signal?.reason || 'superseded'),
});
}
return {
...probe,
...selected,
status: 'cancelled',
classification: 'cancelled',
reason: String(signal?.reason || 'superseded'),
cleared: false,
};
}
if (!armed) {
const terminal = {
...probe,
...selected,
status: 'source-unavailable',
classification: 'source-unavailable',
reason: String(
armError?.message || armError || 'tracking restore latch rejected',
),
cleared: this._passivelyClearTrackingSelection(selected),
};
this._publishTrackingRestoreStatus(terminal);
return terminal;
}
const deadline = this.now() + selected.expiryWindowMs;
const settle = (terminal) => {
if (this._pendingTrackingContext?.generation !== generation) return;
this._cancelPendingTrackingWatch();
this._pendingTrackingContext = null;
this._publishTrackingRestoreStatus(terminal);
};
const poll = () => {
this._pendingTrackingTimer = null;
if (this._destroyed || generation !== this._trackingRestoreGeneration)
return;
// The layer that owns the latch may have gone away since the last tick
// (disable, teardown, replacement). There is nothing left attempting this
// restore, so abandon it silently rather than announcing a verdict.
if (this.dataManager.isEffectivelyEnabled?.(selected.layerId) === false) {
this._revokePendingTrackingWatch('owner-layer-disabled');
return;
}
if (this._trackingTargetLatched(selected)) {
settle({
...probe,
...selected,
status: 'found',
classification: 'followed',
cleared: false,
});
return;
}
if (this.now() >= deadline) {
// The window really has elapsed — only now does the verdict apply.
const classification =
probe.status === 'missing'
? this._classifyMissingTrackingTarget(selected, absentAtMs)
: 'source-unavailable';
const cleared = this._passivelyClearTrackingSelection(selected);
settle({ ...probe, ...selected, classification, cleared });
return;
}
this._pendingTrackingTimer = this.setTimer(
poll,
PENDING_TRACKING_POLL_MS,
);
this._pendingTrackingTimer?.unref?.();
};
this._cancelPendingTrackingWatch();
const pending = {
...probe,
...selected,
status: 'pending',
classification: 'pending',
cleared: false,
};
const pendingContext = {
generation,
selected,
probe,
signal,
abortHandler: null,
};
if (signal) {
pendingContext.abortHandler = () => {
if (this._pendingTrackingContext?.generation !== generation) return;
this._revokePendingTrackingWatch(signal.reason || 'aborted');
};
signal.addEventListener('abort', pendingContext.abortHandler, {
once: true,
});
}
this._pendingTrackingContext = pendingContext;
this._publishTrackingRestoreStatus(pending);
this._pendingTrackingTimer = this.setTimer(poll, PENDING_TRACKING_POLL_MS);
this._pendingTrackingTimer?.unref?.();
return pending;
}
/**
* Refresh and restore the one shareable tracked target after destination
* camera and ordinary layer restoration have settled.
*/
async restoreShareTrackingSelection({ signal = null } = {}) {
const selected = this._selectedShareTrackingTarget();
if (!selected || this._destroyed)
return { status: 'skipped', reason: 'no-shared-target' };
this._revokePendingTrackingWatch('superseded-by-newer-restore');
const controller = new AbortController();
const combinedSignal = signal
? AbortSignal.any([signal, controller.signal])
: controller.signal;
this._trackingRestoreController = controller;
const generation = ++this._trackingRestoreGeneration;
let result;
try {
result = await this.dataManager.resolveLayerTrackingTarget(
selected.layerId,
selected.targetId,
{ signal: combinedSignal, origin: LAYER_RESTORE_ORIGINS.share },
);
} catch (error) {
result = combinedSignal.aborted
? {
status: 'cancelled',
reason: String(combinedSignal.reason || 'aborted'),
}
: {
status: 'source-unavailable',
reason: String(error?.message || error),
};
}
if (
generation !== this._trackingRestoreGeneration ||
this._destroyed ||
combinedSignal.aborted
) {
if (this._trackingRestoreController === controller)
this._trackingRestoreController = null;
return {
...result,
status: 'cancelled',
reason: String(combinedSignal.reason || 'superseded'),
};
}
if (this._trackingRestoreController === controller)
this._trackingRestoreController = null;
if (result.status === 'found') {
const terminal = {
...result,
...selected,
classification: 'followed',
cleared: false,
};
this._publishTrackingRestoreStatus(terminal);
return terminal;
}
if (['cancelled', 'superseded', 'destroyed'].includes(result.status))
return result;
// A subject that is simply not here YET is not a subject that is gone. Hold
// it on the layer's own deferred-restore latch for its source-specific
// window before any verdict is reached or shown. `unsupported` layers have
// no latch to arm, so they still decide immediately.
if (result.status === 'missing' || result.status === 'source-unavailable') {
return this._beginPendingTrackingRestore(
selected,
result,
combinedSignal,
);
}
const classification =
result.status === 'missing'
? this._classifyMissingTrackingTarget(selected)
: 'source-unavailable';
const cleared = this._passivelyClearTrackingSelection(selected);
const terminal = { ...result, ...selected, classification, cleared };
this._publishTrackingRestoreStatus(terminal);
return terminal;
}
destroy() {
if (this._destroyed) return;
this._destroyed = true;
for (const controller of this._restoreControllers.values())
controller.abort('coordinator-destroyed');
this._restoreControllers.clear();
this._revokePendingTrackingWatch('coordinator-destroyed');
this._trackingRestoreController = null;
this._unsubscribe?.();
this._unsubscribe = null;
this._unsubscribeVisibilityRequests?.();
this._unsubscribeVisibilityRequests = null;
this.shareLinkManager?.setLayerStateProvider?.(null);
this.onDurableStateChange = null;
this.onTrackingRestoreStatus = null;
}
}
+7 -82
View File
@@ -1,4 +1,9 @@
import * as Cesium from 'cesium';
import {
layerTitle,
mapAnalystRecord,
parseGeojsonLines,
} from '../sources/infrastructureData.js';
import { createInfrastructureOverlayEntry } from './infrastructureOverlayEntry.js';
import { isPointerFree } from './inputOwnership.js';
import {
@@ -86,78 +91,6 @@ export function localInfrastructureOverlayCopy(properties, layerId) {
return { title, details };
}
/**
* Map one local infrastructure feature to a JSON-safe analyst record
* (analyst query engine seam). Pure — no Cesium types. Missing/unknown
* fields are null, never NaN/undefined. Layer-specific fields that do not
* apply (river/output on datacenters, capacity on dams) stay null so a
* shared compact payload can copy them without inventing values.
* Names are unclamped — overlay cards shorten for paint; queries need the
* full source string ("Usina Hidrelétrica de Itaipu").
* @param {Object|null|undefined} raw - {id, lat, lon, properties}.
* @param {string} [layerId] Local layer id (`local-datacenters` / `local-dams`).
* @returns {{id: string, name: string|null, lat: number|null, lon: number|null,
* operator: string|null, capacity: string|null, river: string|null,
* output: string|null}}
*/
export function mapAnalystRecord(raw, layerId = '') {
const num = (v) => (Number.isFinite(v) ? v : null);
const text = (v) => {
const t = String(v ?? '').trim();
return t && t !== 'undefined' && t !== 'null' ? t : null;
};
const props =
raw?.properties &&
typeof raw.properties === 'object' &&
!Array.isArray(raw.properties)
? raw.properties
: {};
const tags =
props.tags && typeof props.tags === 'object' && !Array.isArray(props.tags)
? props.tags
: {};
const name =
text(props.name) ||
text(tags.name) ||
text(tags['name:en']) ||
text(tags.official_name) ||
null;
const operator =
text(tags.operator) ||
text(props.operator) ||
text(tags['operator:short']) ||
null;
const capacity =
layerId === 'local-datacenters'
? text(tags['capacity:it_load']) ||
text(tags.it_load) ||
text(tags.capacity) ||
text(props.capacity)
: null;
const river =
layerId === 'local-dams'
? text(tags.associated_river) ||
text(props.associated_river) ||
text(tags.river) ||
text(props.river) ||
text(tags['river:name'])
: null;
const output =
layerId === 'local-dams'
? text(props.output) || text(tags['plant:output:electricity'])
: null;
return {
id: name || text(raw?.id) || layerTitle(layerId),
name,
lat: num(raw?.lat),
lon: num(raw?.lon),
operator,
capacity,
river,
output,
};
}
/**
* Produce one normalized-contract input owned by a local infrastructure layer.
* The host revalidates the authoritative `source` value while normalizing it.
@@ -664,11 +597,7 @@ export function createLocalGeoJsonLayer(
}
const text = await response.text();
if (_destroyed) return;
const lines = text
.split('\n')
.filter((l) => l.trim().length > 0);
features = lines.map((line) => JSON.parse(line));
features = parseGeojsonLines(text);
_cachedFeatures = features;
}
@@ -1360,8 +1289,4 @@ function clampCardLine(value) {
return text.length > 48 ? `${text.slice(0, 45)}...` : text;
}
function layerTitle(layerId) {
if (layerId === 'local-datacenters') return 'Datacenter';
if (layerId === 'local-dams') return 'Dam';
return 'Feature';
}
export { mapAnalystRecord };
+11 -7
View File
@@ -14,10 +14,6 @@ export function createMilitaryNamesLoader({
loadPack = (signal) =>
loadBundledJson(
new URL('./local_data/osm_military_names/names.json', import.meta.url),
() =>
import('./local_data/osm_military_names/names.json', {
with: { type: 'json' },
}),
{ signal },
),
timeoutMs = MILITARY_NAMES_TIMEOUT_MS,
@@ -93,8 +89,16 @@ export function nameMilitaryFragment(fragment, names) {
};
}
/** Area-ranked spatial thinning of visible label points, including dateline views. */
export function militaryNamesInView(names, box, cap = MILITARY_POINT_CAP) {
/**
* Area-ranked spatial thinning of visible label points, including dateline
* views. With `thinned` false every point in the box is kept, up to `cap`.
*/
export function militaryNamesInView(
names,
box,
cap = MILITARY_POINT_CAP,
thinned = true,
) {
const width =
box.east >= box.west ? box.east - box.west : 360 + box.east - box.west;
const height = box.north - box.south;
@@ -107,7 +111,7 @@ export function militaryNamesInView(names, box, cap = MILITARY_POINT_CAP) {
continue;
count++;
const cell = `${Math.min(31, Math.floor((x / width) * 32))}:${Math.min(15, Math.floor(((record.latitude - box.south) / height) * 16))}`;
if (cells.has(cell) || records.length >= cap) continue;
if ((thinned && cells.has(cell)) || records.length >= cap) continue;
cells.add(cell);
records.push(record);
}
+10
View File
@@ -217,3 +217,13 @@ test('a close view keeps the parent name inside its polygon when the usual label
hole,
);
});
test('queries can take every named site in a box instead of one per display cell', async () => {
const names = await loadMilitaryNames();
const box = { west: -2, south: 50, east: 2, north: 54 };
const thinned = militaryNamesInView(names, box);
const all = militaryNamesInView(names, box, Infinity, false);
assert.equal(all.count, thinned.count);
assert.equal(all.records.length, all.count);
assert.ok(all.records.length > thinned.records.length);
});
+2 -10
View File
@@ -124,17 +124,9 @@ let _entries = null;
const PACKS = {
regions: {
url: new URL('./local_data/natural_earth/regions.json', import.meta.url),
importJson: () =>
import('./local_data/natural_earth/regions.json', {
with: { type: 'json' },
}),
},
marine: {
url: new URL('./local_data/natural_earth/marine.json', import.meta.url),
importJson: () =>
import('./local_data/natural_earth/marine.json', {
with: { type: 'json' },
}),
},
};
@@ -178,8 +170,8 @@ function buildEntries(pack, kind) {
*/
const loadIndex = createRetryableLoader(async () => {
const [regions, marine] = await Promise.all([
loadBundledJson(PACKS.regions.url, PACKS.regions.importJson),
loadBundledJson(PACKS.marine.url, PACKS.marine.importJson),
loadBundledJson(PACKS.regions.url),
loadBundledJson(PACKS.marine.url),
]);
_entries = [
...buildEntries(regions, 'natural'),
+1 -5
View File
@@ -22,10 +22,6 @@ const CITY_FILES = [
'./local_data/neighborhoods/san-francisco.json',
import.meta.url,
),
importJson: () =>
import('./local_data/neighborhoods/san-francisco.json', {
with: { type: 'json' },
}),
},
];
@@ -112,7 +108,7 @@ function cityLoader(city) {
let loader = _cityLoaders.get(city.id);
if (!loader) {
loader = createRetryableLoader(async () => {
const fc = await loadBundledJson(city.url, city.importJson);
const fc = await loadBundledJson(city.url);
return Array.isArray(fc.features) ? fc.features : [];
});
_cityLoaders.set(city.id, loader);
-17
View File
@@ -65,23 +65,6 @@ export function unregisterPickOwner(layerId) {
_owners.delete(layerId);
}
/**
* The registered layer that owns a picked id, or null.
* @param {string} pickedId - Canonical pick id from {@link resolvePickId}.
* @returns {string|null} Owning layer id.
*/
export function findPickOwner(pickedId) {
if (!pickedId) return null;
for (const [ownerId, predicate] of _owners) {
try {
if (predicate(pickedId)) return ownerId;
} catch {
// a broken predicate must never break picking
}
}
return null;
}
/**
* True when some OTHER registered layer owns the picked id.
* @param {string} layerId - The asking layer's id (excluded from the scan).
-128
View File
@@ -1,128 +0,0 @@
import { getContextStore, isContextRecordActive } from './contextStore.js';
import { findPickOwner, resolvePickId } from './pickRegistry.js';
/**
* Resolve a `scene.pick()` result to the identity a person would name: which
* layer, which record, and its display label.
*
* Two lanes, in order:
* 1. Context-store carriers. Entity-backed layers (infrastructure GeoJSON,
* selected vessels, fires, installations) stamp `__gevContextId` on the
* picked object, and the store record already carries the label and
* position voice uses everywhere else.
* 2. Pick owners. Primitive-backed layers (aircraft, satellites, vessels)
* register an ownership predicate in `pickRegistry`; the owning layer's
* `findByQuery` turns the pick id into a live descriptor.
*
* Trails claim their pick ids only to make clicks inert, so they never
* resolve. Disabled layers never resolve: what the user cannot see is not
* what they are pointing at.
*/
const KIND_BY_LAYER = Object.freeze({
flights: 'aircraft',
military: 'aircraft',
'local-adsb': 'aircraft',
'ais-live-vessels': 'vessel',
satellites: 'satellite',
'local-firms': 'fire',
});
const IGNORED_OWNERS = new Set(['trails']);
function text(value) {
return String(value ?? '').trim();
}
function finite(value) {
if (value === null || value === undefined || value === '') return null;
const number = Number(value);
return Number.isFinite(number) ? number : null;
}
/** Kind of thing a layer holds, for deixis ("this aircraft", "this place"). */
export function pickedKindForLayer(layerId) {
return KIND_BY_LAYER[layerId] || 'feature';
}
function contextCarrierRecord(picked, dataManager) {
const store = getContextStore();
for (const carrier of [
picked?.id,
picked?.primitive?.id,
picked?.primitive,
]) {
const contextId =
carrier && typeof carrier === 'object' ? carrier.__gevContextId : null;
if (!contextId) continue;
const record = store.entities.get(contextId);
if (record && isContextRecordActive(record, dataManager)) return record;
}
return null;
}
function fromRecord(record) {
const layerId = record.layerId || null;
const id =
layerId === 'ais-live-vessels' && record.properties?.mmsi
? String(record.properties.mmsi)
: String(record.id);
return {
layerId,
id,
label:
text(record.label) ||
text(record.properties?.name) ||
text(record.layerName) ||
id,
kind: pickedKindForLayer(layerId),
lat: finite(record.latitude),
lon: finite(record.longitude),
};
}
function descriptorLabel(found, fallback) {
return (
text(found?.callsign) ||
text(found?.registration) ||
text(found?.name) ||
text(found?.icao24) ||
text(found?.mmsi) ||
text(found?.noradId) ||
fallback
);
}
/**
* @param {object|null} picked Result of `scene.pick()`.
* @param {object} [options]
* @param {object} [options.dataManager] Layer lifecycle (enabled state, modules).
* @returns {{layerId: string, id: string, label: string, kind: string, lat: number|null, lon: number|null, altitudeM?: number|null}|null}
*/
export function resolvePickedContext(picked, { dataManager = null } = {}) {
if (!picked) return null;
const record = contextCarrierRecord(picked, dataManager);
if (record) return fromRecord(record);
const pickId = resolvePickId(picked);
if (!pickId) return null;
const layerId = findPickOwner(pickId);
if (!layerId || IGNORED_OWNERS.has(layerId)) return null;
if (dataManager?.isEnabled && !dataManager.isEnabled(layerId)) return null;
const layer = dataManager?.layers?.get?.(layerId);
let found = null;
try {
found = layer?.module?.findByQuery?.(pickId) || null;
} catch {
found = null;
}
return {
layerId,
id: pickId,
label: descriptorLabel(found, text(layer?.name) || pickId),
kind: pickedKindForLayer(layerId),
lat: finite(found?.latitude),
lon: finite(found?.longitude),
altitudeM: finite(found?.altitudeM),
};
}
-75
View File
@@ -1,75 +0,0 @@
import assert from 'node:assert/strict';
import test from 'node:test';
globalThis.window ||= globalThis;
globalThis.CustomEvent ||= class CustomEvent {
constructor(type, init) {
this.type = type;
this.detail = init?.detail;
}
};
globalThis.dispatchEvent ||= () => true;
const { registerEntityContext } = await import('./contextStore.js');
const { registerPickOwner, unregisterPickOwner, findPickOwner } = await import('./pickRegistry.js');
const { resolvePickedContext, pickedKindForLayer } = await import('./pickedContext.js');
function manager({ enabled = new Set(['flights', 'local-datacenters']), modules = {} } = {}) {
return {
isEnabled: (id) => enabled.has(id),
layers: new Map(Object.entries(modules).map(([id, module]) => [id, { name: id, module }])),
};
}
test('an aircraft billboard resolves through its pick owner to a live label', () => {
registerPickOwner('flights', (id) => id === 'a1b2c3');
const dataManager = manager({
modules: {
flights: {
findByQuery: (q) => (q === 'a1b2c3' ? { icao24: 'a1b2c3', callsign: 'UPS793', latitude: 30.2, longitude: -97.7, altitudeM: 9000 } : null),
},
},
});
const picked = { id: 'a1b2c3', primitive: {} };
assert.equal(findPickOwner('a1b2c3'), 'flights');
assert.deepEqual(resolvePickedContext(picked, { dataManager }), {
layerId: 'flights',
id: 'a1b2c3',
label: 'UPS793',
kind: 'aircraft',
lat: 30.2,
lon: -97.7,
altitudeM: 9000,
});
unregisterPickOwner('flights');
});
test('an entity-backed feature resolves through the context store', () => {
const entity = { show: true };
registerEntityContext(entity, {
id: 'local-datacenters:7',
layerId: 'local-datacenters',
label: 'Data Center 7',
latitude: 30.1,
longitude: -97.6,
});
const out = resolvePickedContext({ id: entity }, { dataManager: manager() });
assert.deepEqual(out, {
layerId: 'local-datacenters',
id: 'local-datacenters:7',
label: 'Data Center 7',
kind: 'feature',
lat: 30.1,
lon: -97.6,
});
});
test('disabled layers, trails and empty picks never resolve', () => {
registerPickOwner('military', (id) => id === 'm1');
assert.equal(resolvePickedContext({ id: 'm1' }, { dataManager: manager() }), null, 'military is disabled');
unregisterPickOwner('military');
assert.equal(resolvePickedContext({ id: 'gev-trail:1' }, { dataManager: manager() }), null);
assert.equal(resolvePickedContext(null), null);
assert.equal(resolvePickedContext({ id: 'unknown' }, { dataManager: manager() }), null);
assert.equal(pickedKindForLayer('ais-live-vessels'), 'vessel');
});
+3
View File
@@ -211,6 +211,9 @@ export function shouldShowFirstRun({
if (hasShareState) return false;
const params = new URLSearchParams(location?.search || '');
if (params.get('welcome') === '0') return false;
// An embedded globe never shows the welcome launcher.
if (params.get('embed') === '1' || globalThis.GEV_EMBED_INLINE === true)
return false;
// The demo/support escape hatch outranks both suppressions on purpose.
if (params.get('welcome') === '1') return true;
if (readStored('local', storage, FIRST_RUN_STORAGE_KEY) === 'suppressed')
+4 -6
View File
@@ -667,14 +667,12 @@ test('the voice TOOL SCHEMA matches the pinned release — the mission mapping i
// Cyber deliberately adds one layout; first-run missions still change no tools.
hudLayout.enum = hudLayout.enum.filter((layout) => layout !== 'cyber');
const block = JSON.stringify(legacyTools);
// Re-derived for the voice layer manifest (generated layer enums, alias and
// field hints), point-and-ask's pointer/referent arguments and the prompt
// consolidation (shorter analyst, annotate_map and ISS wording); the
// missions still ride existing tools.
assert.equal(block.length, 28800, 'serialized tool schema length drifted');
// Re-derived for the additive `local-adsb` set_layer_visibility value and
// its common-name mapping; the missions still ride existing tools.
assert.equal(block.length, 27432, 'serialized tool schema length drifted');
assert.equal(
crypto.createHash('sha256').update(block).digest('hex'),
'7e346dadcc6e6b520c77a5c8cae501c9ee9a2b6e6c61babbe1b0f564cd396db3',
'a2a4a787f4528f75b01f3f42caec636f29c37452c0d45b11f4f986171d6be57d',
'the first-run missions must ride EXISTING tools: no schema edit, no cache bust',
);
const instructions = fs.readFileSync(new URL('../server/providers/openai/instructions.js', import.meta.url), 'utf8');
+5 -1
View File
@@ -318,7 +318,11 @@ export async function initKeySetup({
globalThis.location?.hash?.slice(1) || '',
);
if (next !== null)
globalThis.history?.replaceState?.(null, '', `#${next}`);
globalThis.history?.replaceState?.(
null,
'',
new URL(`#${next}`, globalThis.location.href).href,
);
} catch {
// Continuity is a nicety, never a blocker.
}
+24 -22
View File
@@ -1,7 +1,8 @@
/**
* Key setup ("POWER UP") — the pure core.
*
* One registry, three pure functions, zero dependencies. The dev server's
* One registry and pure functions; its only import is the proxy-header check
* shared with the other local gates (localRequestGate.mjs). The dev server's
* /api/setup endpoints (vite.config.js) and the in-app panel (keySetup.js)
* are both thin shells over this module, so what a key is called, what it
* unlocks, and how a .env line is written each live in exactly one place.
@@ -11,6 +12,8 @@
* behavior below unit-testable.
*/
import { hasProxySignals } from './localRequestGate.mjs';
/** Longest accepted key/token value. Real provider keys are all far shorter. */
export const KEY_SETUP_VALUE_LIMIT = 512;
@@ -184,6 +187,24 @@ export function parseWindowsUserSid(stdout) {
: null;
}
/**
* Whether the launcher has sharing on. Every sharing signal the launcher
* recognizes (scripts/pinokio-preflight.mjs) counts, so the two sets cannot
* drift apart. One DELIBERATE divergence: preflight is a boot check that
* treats an empty PINOKIO_SHARE_VAR as sharing-on (fail closed before Start),
* but here an empty/unset value is the NORMAL git-clone and Pinokio state, so
* a bare/sentinel value is not sharing; only a real tunnel var is.
*/
export function isSharingEnabled(env = {}) {
const shareVar = String(env.PINOKIO_SHARE_VAR ?? '').trim();
return (
['PINOKIO_SHARE_CLOUDFLARE', 'PINOKIO_SHARE_LOCAL'].some((name) =>
/^(1|true)$/i.test(String(env[name] || '').trim()),
) ||
(shareVar !== '' && shareVar !== '__gev_sharing_disabled__')
);
}
/**
* The admission gate for the Provider Settings endpoints — pure, exported so
* every refusal below is pinned by a unit assertion rather than a review note.
@@ -219,20 +240,7 @@ export function admitKeySetupRequest({
// on this machine, whatever its socket says. Refuse them outright as defense
// in depth — the shipped tunnel (Pinokio) is force-closed at boot, so these
// only appear when someone has deliberately fronted the dev server.
const PROXY_SIGNALS = [
'forwarded',
'via',
'x-forwarded-for',
'x-forwarded-host',
'x-forwarded-port',
'x-forwarded-proto',
'x-real-ip',
'cf-connecting-ip',
'cf-ray',
];
if (
PROXY_SIGNALS.some((name) => String(proxyHeaders[name] || '').trim() !== '')
) {
if (hasProxySignals(proxyHeaders)) {
return {
ok: false,
status: 403,
@@ -249,13 +257,7 @@ export function admitKeySetupRequest({
// tunnel var is. This is defense in depth regardless: the loopback+Host checks
// below independently refuse LAN/tunnel traffic, and under Pinokio the launcher
// refuses to boot at all when sharing is genuinely on.
const shareVar = String(env.PINOKIO_SHARE_VAR ?? '').trim();
const sharingEnabled =
['PINOKIO_SHARE_CLOUDFLARE', 'PINOKIO_SHARE_LOCAL'].some((name) =>
/^(1|true)$/i.test(String(env[name] || '').trim()),
) ||
(shareVar !== '' && shareVar !== '__gev_sharing_disabled__');
if (sharingEnabled) {
if (isSharingEnabled(env)) {
return {
ok: false,
status: 403,
+1
View File
@@ -9,6 +9,7 @@ export * from '../../data/cockpitContactDot.js';
export * from '../../data/contactMatch.js';
export * from '../../data/iconOrientation.js';
export * from '../../data/layerState.js';
export * from '../../data/layerStateCoordinator.js';
export * from '../../data/modelVisualAnchor.js';
export * from '../../data/motionModel.js';
export * from '../../data/renderAltitude.js';
-31
View File
@@ -392,37 +392,6 @@ export function createAlprCamerasLayer({ source, services } = {}) {
],
};
},
/**
* Snapshot loaded ALPR cameras (the nearby query box) as plain JSON-safe records for the analyst
* query engine. On demand only (once per spoken query); [] while the
* layer is off.
* @param {number} [maxCount=Infinity] Most records to return.
* @returns {Array<Object>} {id, lat, lon, operator, manufacturer, cameraType, directionDeg}.
*/
getAnalystRecords(maxCount = Infinity) {
if (!state.enabled) return [];
const result = [];
for (const record of state.records || []) {
if (result.length >= maxCount) break;
if (
!Number.isFinite(record.latitude) ||
!Number.isFinite(record.longitude)
)
continue;
result.push({
id: record.id,
lat: record.latitude,
lon: record.longitude,
operator: record.operator ?? null,
manufacturer: record.manufacturer ?? null,
cameraType: record.cameraType ?? null,
directionDeg: Number.isFinite(record.directionDeg)
? record.directionDeg
: null,
});
}
return result;
},
getStats() {
return {
count: state.dataSource?.entities.values.length || 0,
-33
View File
@@ -21,39 +21,6 @@ export function createControls({ state: layerState, services, parts, source }) {
return parts.queries.collectDetectableStations(options);
},
/**
* Snapshot loaded bikeshare stations as plain JSON-safe records for the analyst
* query engine. On demand only (once per spoken query); [] while the
* layer is off.
* @param {number} [maxCount=Infinity] Most records to return.
* @returns {Array<Object>} {id, name, city, lat, lon, bikesAvailable, docksAvailable, capacity, renting}.
*/
getAnalystRecords(maxCount = Infinity) {
if (!layerState._enabled) return [];
const num = (v) => (Number.isFinite(v) ? v : null);
const result = [];
for (const record of layerState._stationRenderMap.values()) {
if (result.length >= maxCount) break;
const info = layerState._stationInfoCache
.get(record.cityId)
?.get(record.stationId);
if (!Number.isFinite(info?.lat) || !Number.isFinite(info?.lon))
continue;
result.push({
id: record.stationName || `Dock ${record.stationId}`,
name: record.stationName || null,
city: record.cityId || null,
lat: info.lat,
lon: info.lon,
bikesAvailable: num(record.bikesAvailable),
docksAvailable: num(record.docksAvailable),
capacity: num(record.capacity),
renting: record.isRenting !== false,
});
}
return result;
},
/**
* Return current layer statistics for the UI status display.
* @returns {{ count: number, lastUpdate: number|null, loading: boolean, loadingLabel?: string, error?: string }}
Loaded 100 of 309 files, more files were not shown because too many files have changed in this diff. Show more