Compare commits

..
Author SHA1 Message Date
Sameh Khamis aa16b7c3b0 Merge pull request #902 from bilawalsidhu/release/v0.2.1
chore(release): prepare v0.2.1
2026-10-02 22:36:12 -07:00
Sameh Khamis 80fc8f0e1c chore(release): prepare v0.2.1
Date the changes since v0.2.0 as 0.2.1, move the live CCTV video entry
into the 0.2.0 section where it shipped, and align the package metadata.
2026-10-02 22:19:04 -07:00
Sameh Khamis 05df7083b9 Merge pull request #873 from manjunath22466/fix/cyber-voice-popups
fix(ui): harden Cyber panel layout and preset restoration
2026-10-02 21:51:37 -07:00
eaa51a24fe test(qa): cover the seven-mode Cockpit vision carousel
The Cockpit utility gate still expected five modes and an inherited NOIR
entry. Cycle all seven modes, check each drives only its own shader and that
Normal clears them all, enter on a Noir map preset, and restore it through
Exit Cockpit and Reset.

Co-authored-by: Sameh Khamis <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-02 21:43:12 -07:00
Sameh KhamisandClaude Opus 5.5 b33cae4b74 Merge main into #873
Resolve the CHANGELOG and CURRENT-STATE conflicts: the PR's entries go under
Unreleased, after the 0.2.0 release.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-02 21:18:15 -07:00
Sameh Khamis 2a612b7315 Merge pull request #901 from bilawalsidhu/fix/build-hygiene
build: ship each bundled data pack once
2026-10-02 21:12:40 -07:00
Sameh KhamisandClaude Opus 5.5 9c87fabf88 build: ship each bundled data pack once
loadBundledJson's callers passed a literal JSON import as a Node fallback,
which made the production build emit a JavaScript copy of every pack beside
the JSON the browser fetches. The loader now reads file: URLs itself under
Node (through process.getBuiltinModule, synchronously, as promptly as the
cached import it replaces), and the callers pass only the URL. On a Node too
old to provide getBuiltinModule it says that Node 24.14 or newer is needed.
dist/ drops from 54 MB to 42 MB. A test keeps app code from importing a pack
as a module.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-02 21:08:11 -07:00
Sameh Khamis 992412fc9a Merge pull request #900 from bilawalsidhu/fix/security-consolidation
Harden the local server: cross-site gate, CSP, default throttles, LAN host check
2026-10-02 20:26:01 -07:00
Sameh KhamisandClaude Opus 5.5 0646630dec fix(server): let the CSP load the Nepal event's embedded media
The Content-Security-Policy blocked the YouTube player API, the Facebook
video SDK and X's post widget that src/data/bhoteKoshiEmbeddedMedia.js
loads, along with their players and posts in frames. script-src now allows
exactly those three script origins and a new frame-src allows YouTube,
youtube-nocookie, Facebook and X embeds. A test ties the policy to the
frames the media module builds and pins the allowed script origins.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-02 20:21:23 -07:00
Sameh KhamisandClaude Opus 5.5 913d45bc07 fix(server): keep cost throttles on for positive fractional limits
A limit such as 0.5 floored to 0 and disabled the throttle, although only an
explicit 0 is meant to. Positive values below 1 now count as 1.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-02 20:17:21 -07:00
Sameh KhamisandClaude Opus 5.5 9b4ccbff62 fix(server): gate Google text search against cross-site requests
/api/google/text-search spends the same Google quota as nearby-places but
was not behind the cross-site gate. It now refuses cross-site requests
before calling Google, and a test covers both paid Google routes.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-02 20:16:39 -07:00
Sameh KhamisandClaude Opus 5.5 e6f1dd70a1 fix(server): apply the Host check before the app's own routes
Vite installs its Host check after the middleware plugins add in
configureServer, so the provider and /api routes answered any Host,
including a DNS-rebinding name. hostCheckPlugin runs first on the dev and
preview servers and applies Vite's rule with the same allowed hosts: IP
addresses, localhost and *.localhost, the configured server host, and the
listed names. A test drives a real Vite server with an unlisted Host.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-02 20:15:54 -07:00
Sameh KhamisandClaude Opus 5.5 81513f9aae fix(mcp): refuse panel requests to routes reached through a dot suffix
The server routes a mounted path when a dot follows it, so /mcp.json reached
the MCP endpoint while panel_request refused only /mcp and /mcp/...; the same
held for the provider-settings, credential and model routes. The refusals now
match a dot after the route as well. A regression test drives panel_request
against a real Vite server.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-02 20:10:47 -07:00
Sameh Khamis 9954e9f991 Merge pull request #899 from bilawalsidhu/fix/mcp-embed-hardening
Tighten the local MCP server and embed mode
2026-10-02 20:04:56 -07:00
Sameh KhamisandClaude Opus 5.5 6cdd6c8af9 refactor(server): share one proxy-header check across the local gates
localRequestGate.mjs owns the reverse-proxy header list and hasProxySignals;
Provider Settings, the cross-site gate and /mcp all use it instead of two
copies of the same nine names. The changelog credits the cross-site gate.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
2026-10-02 20:02:48 -07:00
bbf4976f5e fix(server): keep debug-log timestamps server-owned and send nosniff
The voice debug log writes the server's timestamp after the client's record,
so a record can no longer replace loggedAt. The dev and preview servers also
send X-Content-Type-Options: nosniff with their shared browser headers.

From the findings in #710.

Co-authored-by: Sunil <[email protected]>
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
2026-10-02 20:00:11 -07:00
98b8c6c581 fix(server): keep the Host-header check when binding to all interfaces
Binding HOST=0.0.0.0 no longer sets Vite's allowedHosts to true. The dev and
preview servers accept IP addresses, localhost and the LAN hostnames listed
in GEV_ALLOWED_HOSTS; suffix and wildcard entries are ignored. The
configuration factory takes the resolved list as an input, so build/vite.js
still reads no environment.

Ported from #97 to the current build/vite.js layout. Fixes #21.

Co-authored-by: Sameh Khamis <[email protected]>
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
2026-10-02 19:59:02 -07:00
bb74071dc2 fix(server): throttle the cost-bearing proxies by default
The OpenAI endpoints (/api/realtime/token, /api/openai/hud-summary) now
allow 30 requests per minute per client IP and the Google Places endpoints
120, the caps the Pinokio build already ships. GEV_RATELIMIT_OPENAI_PER_MIN
and GEV_RATELIMIT_GOOGLE_PER_MIN still override them and exactly 0 disables
them; an unreadable value falls back to the default rather than to
unlimited.

Salvaged from #683 onto current main.

Co-authored-by: Sameh Khamis <[email protected]>
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
2026-10-02 19:57:29 -07:00
7aa9bbd3be fix(server): gate cost/log endpoints against cross-site requests and ship a full CSP
The cost-bearing endpoints (/api/realtime/token, /api/openai/hud-summary,
/api/google/nearby-places) and the voice debug log refuse cross-site browser
requests: a foreign or opaque Origin, a Sec-Fetch-Site other than
same-origin/none, or proxy forwarding headers. Loopback tools without an
Origin and the LAN opt-in keep working. The dev and preview servers send one
shared Content-Security-Policy, and embed-mode framing rewrites only the
frame-ancestors directive of it.

Salvaged from #242 onto current main.

Co-authored-by: Sameh Khamis <[email protected]>
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
2026-10-02 19:56:42 -07:00
Sameh KhamisandClaude Opus 5.5 42faed3aea docs: describe the MCP server and panel in the security guide
SECURITY.md gains a section on what /mcp accepts, the panel's request key,
the browser keys the panel build carries and embed-mode framing. TOOLS.md
now describes what panel_request may request, and the changelog records
the recent MCP and embed changes.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
2026-10-02 19:47:37 -07:00
Sameh KhamisandClaude Opus 5.5 857767842f fix(mcp): bound panel requests, request bodies and cancelled calls
Panels on one MCP server share six requests in flight; further requests
wait their turn (up to 256 waiting) instead of failing, and a waiting
request leaves the queue when its caller cancels. Held responses belong to
their own MCP server rather than to the process. /mcp request bodies must
arrive within 30 seconds. Over stdio, a client's notifications/cancelled
aborts the named request, which then gets no response.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
2026-10-02 19:47:37 -07:00
Sameh KhamisandClaude Opus 5.5 fec93aa817 fix(mcp): accept only direct local requests at /mcp
The /mcp route now applies the same proxy and sharing checks as Provider
Settings: requests carrying forwarding headers, or arriving while launcher
sharing is on, are refused. The Host must name the port the request reached
and a browser Origin must match that host exactly. The proxy and sharing
checks move into shared helpers in keySetupCore; Provider Settings behaves
as before.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
2026-10-02 19:47:37 -07:00
Sameh KhamisandClaude Opus 5.5 f7d3db5875 fix(embed): make embed-mode framing opt-in
Embed-mode documents now keep the server's framing protection unless
GEV_EMBED_FRAME_ANCESTORS names the pages that may frame them (or * for any
page). The MCP Apps panel loads the app into its own page and needs no
framing. View answers are posted to the origin that sent the view rather
than to any origin.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
2026-10-02 19:47:37 -07:00
Sameh KhamisandClaude Opus 5.5 3318390e3f fix(mcp): require the panel key for panel requests
panel_request now requires a key that each MCP server generates and places
in its panel page, which keeps the tool from clients that list it to the
model without loading the panel. The key is not access control: any MCP
client can read the panel page. Routes the panel never loads (provider
settings, credential and model endpoints, the MCP endpoint and development
server internals) are refused to every caller, regardless of letter case or
encoding.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
2026-10-02 19:47:37 -07:00
Sameh Khamis b1a958242c Merge pull request #897 from bilawalsidhu/release/v0.2.0
chore(release): prepare v0.2.0
2026-10-02 19:00:29 -07:00
Sameh Khamis b86239aa6d chore(release): prepare v0.2.0
Group the changes since v0.1.1 under a dated 0.2.0 section and align the
package metadata.
2026-10-02 18:47:29 -07:00
Sameh Khamis 74213d18da Merge pull request #896 from bilawalsidhu/feat/tool-catalog
Add a shared tool catalog and MCP server, with God's Eye View inside Claude and ChatGPT
2026-10-02 18:45:08 -07:00
Sameh Khamis a55d260551 docs: drop the server check from the MCP setup 2026-10-02 18:29:42 -07:00
Sameh Khamis bcbc350354 docs: add a one-page MCP setup for Claude and Codex
Steps people and coding agents can follow to run the app, add the MCP
server to Claude Code, Claude Desktop, the Codex CLI and the Codex and
ChatGPT desktop apps, and check it works, linked from the README and the
tools guide.
2026-10-02 18:29:42 -07:00
Sameh Khamis b69d34bad2 Frame a view that only follows an aircraft where the aircraft is
Asked for an aircraft's cockpit view, assistants passed only the follow,
which failed for want of an area or camera, so every such request took
a second call. show_in_gods_eye_view now looks the aircraft up and frames
the view on it, and says so when the aircraft is not reported.
2026-10-02 18:29:42 -07:00
Sameh Khamis ae00f009e6 Log why a tool call failed in the stdio server's log
A failed call's reason went only to the client, so client logs showed a
failure without its cause.
2026-10-02 18:29:42 -07:00
Sameh Khamis e82ab7440d Run the worker prelude ahead of Cesium's own workers script
Cesium's script sets CESIUM_WORKERS itself when it loads, so the panel's
combined workers script was never used, and neither was the embedded
terrain-heights file: the worker still requested it from the panel
page's site. The panel build now writes only the prelude, and the panel
puts it ahead of whatever Cesium assigns to CESIUM_WORKERS. The separate
esbuild bundle of Cesium's workers is gone.
2026-10-02 18:29:42 -07:00
Sameh Khamis 9e5ef78f91 Embed the files Cesium's workers load in the panel's workers script
The ground-polyline worker loads approximateTerrainHeights.json itself,
and a worker's requests go to the panel page's own site, where they
failed with 404 and stopped rendering once ground-clamped lines such as
military installations appeared. The panel build now embeds that file in
the workers script, which answers the worker's request from memory.
2026-10-02 18:29:42 -07:00
Sameh Khamis d4311a9d54 Let a view show an aircraft's cockpit view
A view's follow may ask for cockpit view, which the app enters after it
follows the aircraft; links cannot carry it and open the app following.
Applying a view no longer flies the camera when the followed entity is
already there, since the flight ended the follow; it flies only while
waiting for the entity. The panel applies its first view again only
when it asks for cockpit view.
2026-10-02 18:29:42 -07:00
Sameh Khamis 89a448ee83 Do not apply a panel's first view twice
The panel starts the app from the first view's link, which the app
restores, following included. Sending the same view again once the app
was ready flew the camera to it, which ended the follow and left cockpit
view. The panel now sends only views that arrive while the app loads.
2026-10-02 18:29:42 -07:00
Sameh Khamis 9bc031c753 docs: describe God's Eye View in conversations in the changelog and current state 2026-10-02 18:29:42 -07:00
Sameh Khamis abca609708 Keep panel requests on the app's server and bounded
panel_request resolved paths with a check for a leading //, but URL
parsing also reads /\host as another host, so it could request other
servers. It now resolves the path against the app's address, refuses
any other origin, matches refused routes decoded as well, and does not
follow redirects. Responses are capped at 64 MiB while reading, with a
deadline; held responses make room on arrival and expire by time only,
so a response being read keeps its parts.

The panel keeps converted images only for its build files, so live
camera frames no longer accumulate, and failed build loads are retried.
The dev server answers a malformed /panel/ path with 400 instead of
crashing.
2026-10-02 18:29:42 -07:00
Sameh Khamis efdee8d255 Clean up the panel after host testing
Drop the render restarts added while chasing the black overlays; keep
the full render-error report and stop the timer after an error, as
Cesium does. Correct comments the investigation outran, share the
panel_request name, and document the host differences the panel works
around.
2026-10-02 18:29:42 -07:00
Sameh Khamis d0da4b0722 Keep the panel's 2D canvases in memory
In Claude, the app's 2D overlay canvases covered the globe in black
while the host treated the panel as off screen, and the globe showed
once they were hidden. The panel now asks for 2D canvases backed by
memory rather than the GPU.
2026-10-02 18:29:42 -07:00
Sameh Khamis dbbdef0f78 Offer fullscreen in the panel, and hold images in inserted HTML
Where the host offers fullscreen, an Expand button asks for it; the page
fills the room it is given. Images in markup the app inserts as HTML
text now wait for their files instead of requesting the panel host's
site, which ended the mic.svg and Cesium credit 404s.
2026-10-02 18:29:42 -07:00
Sameh Khamis 5ad0dd0004 Report a panel's render errors in full and restart drawing
Cesium stops drawing after a render error and prints worker failures as
[object Object]. An inline panel now logs the error's details and starts
drawing again, up to three times.
2026-10-02 18:29:42 -07:00
Sameh Khamis 47742f335d Keep an inline panel drawing when its host reports it hidden
Claude reports a panel on screen as hidden, which stopped the app's
render loop and the browser's animation frames, so no map was drawn. An
inline panel no longer suspends rendering when hidden, and draws from a
timer while animation frames stop arriving.
2026-10-02 18:29:42 -07:00
Sameh Khamis 62468c547f Fit the panel to Codex's and Claude's panel rules
Images and stylesheet files load as data: URLs, which hosts allow where
some refuse blob: images; the scene-asset source gets an https app
address the panel serves, since Codex pages use their own scheme; and a
panel whose 3D context the browser takes back says so.
2026-10-02 18:29:42 -07:00
Sameh Khamis 996b95eb45 Load the God's Eye View panel through the MCP server
Hosts serve panels from their own sites and may refuse other addresses;
Codex refuses any address on the user's machine. The panel now never
requests the app's server: it loads the app's panel build and data
through panel_request, a tool only the panel may call, which requests
the path from the app's server. One path in every host; only map
providers load directly.

npm run build:panel writes the panel build to dist/panel, served at
/panel/: one app script, one stylesheet, Cesium's script, and Cesium's
workers as one script Cesium runs from memory. Tools may now be marked
app-only. The panel CORS rules are gone.
2026-10-02 18:29:42 -07:00
Sameh Khamis b11d54d8b8 Support panel hosts that serve the page from their own scheme
Codex serves panels from codex-sandbox:// pages, where origins are
opaque: the panel recognizes the app's own addresses by scheme and host,
the vessels feed resolves against the document address, and the dev
server answers Codex's sandbox origins with CORS.
2026-10-02 15:21:37 -07:00
Sameh Khamis 64c1b1951f Answer panel origins only through the CORS plugin 2026-10-02 15:21:37 -07:00
Sameh Khamis 282efac862 Start the panel's app at the first view instead of the default place 2026-10-02 15:21:37 -07:00
Sameh Khamis 5b555d7708 Load images the panel redirects to the app's server with CORS
The app treats those images as its own site's and loads them without
CORS, so once they come from the app's server WebGL refuses them and
Cesium stops rendering.
2026-10-02 15:21:37 -07:00
Sameh Khamis 06460020d0 Send the panel's app requests to the app's server without <base>
Claude refuses <base> in panels, so relative addresses reached the
host's site: styles, Cesium assets and API calls all failed. The panel
now points fetch, XHR, beacons, images, media, links, workers and added
style url()s at the app's server, and declares its policy under OpenAI's
widget key as well.
2026-10-02 15:21:37 -07:00
Sameh Khamis d9ecd126c8 Let the app run under a panel page's document base
A panel page loads the app with <base> pointing at the app's server, so
addresses the app builds resolve against the document base instead of
the page location, and hash updates stay on the page's own address.
The dev server answers panel origins with CORS on the API routes too.
2026-10-02 15:21:37 -07:00
Sameh Khamis 63341b947e Run God's Eye View inside the MCP Apps panel instead of framing it
Hosts such as Claude do not let a panel frame other sites, so the panel
now loads the app's page, styles and scripts into itself in inline embed
mode, and declares the app and map providers as resource and connect
domains. The dev server answers panel origins with CORS.
2026-10-02 15:21:37 -07:00
Sameh Khamis cb04fd5b19 feat: say in the panel what a host's security policy blocked
When a host's content security policy refuses part of the panel, such as
its frame, the panel now names the blocked directive and address instead of
waiting until its load timeout.
2026-10-02 15:21:37 -07:00
Sameh Khamis d368df533d fix: complete the MCP Apps handshake the way hosts validate it
The panel sent clientInfo and capabilities in ui/initialize, where the MCP
Apps schema requires appInfo, so hosts such as Claude rejected the handshake
and never showed the panel. It now sends exactly appCapabilities, appInfo and
protocolVersion, sends initialized without parameters, says so when a host
rejects it, and offers the link when God's Eye View does not load within 20
seconds. Checked against the MCP Apps SDK's own host bridge.
2026-10-02 15:21:37 -07:00
Sameh Khamis ba751d9433 fix: also name a tool's UI resource with the flat ui/resourceUri key
Some MCP Apps hosts read the earlier flat _meta key; the MCP Apps SDK sends
it alongside _meta.ui.resourceUri, and so does the adapter now.
2026-10-02 15:21:37 -07:00
Sameh Khamis 5775eb9053 feat: log local MCP requests to stderr by method
The stdio server writes each request's method, and the tool or resource it
names, to stderr, which clients such as Claude Desktop copy into their logs.
Arguments and results are never logged.
2026-10-02 15:21:36 -07:00
Sameh Khamis 47ef168815 refactor: name the view tool show_in_gods_eye_view
The tool and its panel now say "Show in God's Eye View" rather than
"show on the globe", in tool names, descriptions, server instructions and
docs.
2026-10-02 15:21:36 -07:00
Sameh Khamis 216a75ac71 feat: show views as the live globe in MCP Apps clients
show_on_globe replaces open_in_gods_eye_view and names an MCP Apps view, a
ui:// panel that loads the app in embed mode and posts each later view to it,
so the globe in the conversation changes without reloading. It also takes a
view another answer returned. Tools can declare a UI resource, and the MCP
adapter serves resources/list and resources/read and lists each tool's
resource in _meta.ui.resourceUri.
2026-10-02 15:21:36 -07:00
Sameh Khamis 60c40865e9 feat: add an embed mode that shows only the globe
?embed=1 shows the globe alone: clean view with the HUD, panels, welcome and
setup prompts hidden, and provider attribution kept. The page that frames it
sends views with postMessage; the app applies each through its own actions
and reports every step. Annotations in any link are drawn once it restores.
Only embed-mode documents may be framed; every other page keeps
X-Frame-Options DENY and frame-ancestors 'none', and
GEV_EMBED_FRAME_ANCESTORS can restrict who frames them.
2026-10-02 15:21:36 -07:00
Sameh Khamis f68aa988ec feat: carry annotations in views and aim tilted views at the area
A view can carry the marks the app's annotate_map action draws, and links
carry them in the an parameter, bounded in count and size. A tilted view of
an area now places the camera behind its center so the area stays in the
middle of the frame.
2026-10-02 15:21:36 -07:00
Sameh Khamis af8856cedf feat: lead the MCP tool list with what the globe shows
MCP no longer lists place search, routing, plain weather and wind, the
regional brief, the HUD caption, radio, bike share and transit. Assistants
already cover these or they add little without the globe; voice keeps them,
and tools that combine others still reach them.
2026-10-02 15:21:36 -07:00
Sameh Khamis d6473b6049 feat: return the view that shows each answer
Tool answers with something to show now include data.view: the view with
the matching layers on, the area framed from above or a single aircraft or
satellite followed, and the link that opens it. Area framing moves to
src/tools/views.js so links and answers frame alike.
2026-10-02 15:21:36 -07:00
Sameh Khamis 96a8a9addb feat: describe what the app shows as a view
A view is a camera, data layers, visual style, map imagery and an aircraft,
military aircraft or satellite to follow, written in the share-link format
the app already restores. gods-eye-view/view builds, encodes and decodes one,
and share links take their style names from it. open_in_gods_eye_view now
builds its link from a full view: an area or camera, layers, style, map and
something to follow.
2026-10-02 15:21:36 -07:00
Sameh Khamis de1e0836b3 fix: send "vessels" to the ships layer in voice
Voice hears audio directly, and "vessels" is easily heard as "visuals",
which the instructions route to visual styles, so asking for vessels turned
on other layers or a CRT style. The instructions now say a vessels request
without a style name means the ships layer, and a style needs its name.
2026-10-02 15:21:36 -07:00
Sameh Khamis 759d05db4e fix: name only tools voice offers in its instructions
The voice instructions recommended aircraft_in_area, vessels_in_area and
get_earthquakes, which voice no longer offers, and the docs said voice gets
nearly every query. The instructions now name tools voice has and send
aircraft, ship and hazard questions to analyst_query; a test keeps the
instructions to tools the voice session lists.
2026-10-02 15:21:36 -07:00
Sameh Khamis 33445aec73 fix: keep each aircraft feed's freshness in search answers
find_aircraft merged the records of the feeds that answered and dropped their
freshness, so a stale OpenSky snapshot read as a current "not reported".
Answers now list each feed with its freshness and coverage and say which
feed's data may be stale.
2026-10-02 15:21:36 -07:00
Sameh Khamis 41d1352930 fix: search every vessel the server retains
Vessel tools read the source's default of the newest 12,000 vessels while
the server keeps up to 50,000, so a lookup could miss a vessel still held.
They now ask for more than any server keeps and receive everything it has.
2026-10-02 15:21:36 -07:00
Sameh Khamis ad9a3976a9 fix: say the camera catalog has no cameras rather than none exist
An empty camera answer outside the catalog's regions read as "0 public
cameras", as if the area had none. It now says the catalog has no cameras
there.
2026-10-02 15:21:36 -07:00
Sameh Khamis f610de77dc fix: judge camera catalog trimming by region, not by returned cameras
Coverage was checked only for packs among the cameras found, so an area
whose cameras were all trimmed away answered zero cameras, complete. The
catalog now reports where each trimmed pack's cameras are, and the tool
marks an answer partial whenever its area overlaps that region.
2026-10-02 15:21:36 -07:00
Sameh Khamis 7f56679a46 fix: show the whole weather map when a window cannot cover the area
The map window was moved inside the product's bounds without checking that
it still covered the requested area, so an area across the antimeridian got
an image of only one side. A window must now hold the whole area, clipped to
the product, or the tool returns the product's full extent.
2026-10-02 15:21:36 -07:00
Sameh Khamis 4cfca57cfb fix: bound large boxes by their farthest point on the sphere
A box's radius was the distance to its farthest corner, which falls short for
large boxes: a whole-world box measured about 10,000 km, so bike-share and
transit selection never considered systems beyond it. The radius now checks
whether the box holds the point opposite its center and otherwise samples
every edge.
2026-10-02 15:21:36 -07:00
Sameh Khamis e1ac5ac270 fix: cancel shared place lookups nobody is waiting for
The place cache shared one lookup between callers but started it without a
signal or deadline, so a cancelled request kept running and stayed cached.
A shared lookup now has its own deadline, is cancelled when every caller has
given up, and only a successful answer is kept.
2026-10-02 15:21:36 -07:00
Sameh Khamis 66b07418bd fix: say when the camera catalog serves only part of a region
Camera packs keep their nearest cameras and the catalog cap thins them
further, so London answered 250 cameras while TfL offers many more. The
catalog now records each pack's full count, tags cameras with their pack,
and serves the packs it trimmed; find_cctv_cameras names them.
2026-10-02 15:21:36 -07:00
Sameh Khamis e779432925 fix: report stale and partial data in more tool answers
A sweep for the same pattern found more answers presented as current and
complete: launches and satellite orbits served from a stale proxy cache, a
regional aircraft fallback read as worldwide, license plate cameras trimmed
or partly loaded, and a radio directory of popular stations read as every
station. The launch and satellite sources now report the proxies' stale
markers, and each answer says what it covers.
2026-10-02 15:21:36 -07:00
Sameh Khamis 6a45a8f6a2 fix: check only own properties when validating tool arguments
Inherited names such as constructor, toString and __proto__ passed as
declared properties despite additionalProperties: false, and a schema-valued
additionalProperties was accepted but never enforced. Validation now uses own
properties, and definitions must give additionalProperties as a boolean.
2026-10-02 15:21:36 -07:00
Sameh Khamis c4a4247f70 fix: cancel a local MCP tool call when its client disconnects
The /mcp route built its Request without a signal, so a tool kept working
after the client left. The request's signal now aborts when the connection
closes before the answer is sent.
2026-10-02 15:21:36 -07:00
Sameh Khamis 13f8fa7c8e fix: wait briefly for installation names before answering
Installation tile results can arrive before the name pack, and the tool
answered with generic names such as "Military area" where a moment later
it would say "Camp Mabry". It now waits up to five seconds for the named
result and says when names are still loading.
2026-10-02 15:21:36 -07:00
Sameh Khamis 9e4cc9095b fix: measure traffic inside the requested circle and report missing tiles
Traffic figures for a radius area counted roads anywhere in its bounding
box, and a tile that failed to load silently changed them. Roads are now cut
to the circle, and the flow source reports per request whether any tile
failed so the answer can say its figures are partial.
2026-10-02 15:21:36 -07:00
Sameh Khamis 302a1c1995 fix: say when bike share, transit and place limits leave results out
Bike share and transit read only the three nearest systems, and place
search returns at most five matches within 50 km, while answers read as
complete. Answers now name the systems not searched and say when a place
search limit may have left places out.
2026-10-02 15:21:36 -07:00
Sameh Khamis 56f0d2e203 fix: carry weather and fire freshness into tool answers
get_weather dropped the proxy's stale status and get_active_fires dropped
FIRMS staleness and failed satellite sources, so answers and the HUD caption
read them as current. Both now report them, and the caption context marks
such sections stale or degraded.
2026-10-02 15:21:36 -07:00
Sameh Khamis dd8ba562d2 fix: find fire perimeters that overlap an area
Fire perimeters were placed at their average vertex, so a fire enclosing the
whole requested area was missed. Polygons now count when a vertex is inside
the area, an edge crosses it, or the polygon contains it outside any hole.
2026-10-02 15:21:36 -07:00
Sameh Khamis 4af15aa6a9 fix: keep a named place's own point in situation briefs
Brief sections received a named place as its bounding box, so they centered
on the box's middle: Tokyo's aircraft were sorted from a point in the
Pacific. Sections now receive the place name and resolve it like a direct
query, and place lookups are cached briefly so a brief resolves it once.
2026-10-02 15:21:36 -07:00
Sameh Khamis 1b3aa8de8b fix: stop reading MCP request bodies past the size limit
The HTTP handler read the whole body before checking its size, so a body
without Content-Length could be buffered without bound. It now reads the
stream and stops as soon as the body passes the limit.
2026-10-02 15:21:36 -07:00
Sameh Khamis 6f56c84b0b fix: find cables that cross an area between vertices
Cable searches only tested vertices, so a cable crossing an area with no
vertex inside it was missed. Lines now test each segment against the area's
bounds, across the antimeridian too, and against a radius area's great-circle
distance.
2026-10-02 15:21:36 -07:00
Sameh Khamis 5b73b70d19 fix: bound radius areas by the exact spherical cap
The longitude span divided the angular radius by the center's cosine, which
is too narrow for large radii and near the poles. Areas now use the exact
tangent bound and span every longitude when the circle reaches a pole.
2026-10-02 15:21:36 -07:00
Sameh Khamis cac5ba3b72 fix: derive HUD caption feed states from section results
The caption request declared every answered section nominal, so a stale
aircraft snapshot reached the caption as live. Each section now reports
stale or unavailable from its own result, and the provenance envelope is
built with the app's feedProvenanceEnvelope.
2026-10-02 15:21:36 -07:00
Sameh Khamis 853c60583f fix: report stale transit feeds and drop expired positions
get_transit_vehicles returned positions from a feed the proxy served from
its error cache, and hour-old fixes, as current. It now applies the transit
layer's rules: a feed is stale when the proxy says so or the operator has
not answered recently, and fixes older than the vehicle limit are dropped
and counted.
2026-10-02 15:21:36 -07:00
Sameh Khamis 61974848fe fix: keep aircraft search results when one feed is down
find_aircraft failed whenever either feed failed. It now answers from the
feeds that responded, names the ones that did not, and searches only the
military feed for registrations, the only feed that reports them.
2026-10-02 15:21:36 -07:00
Sameh Khamis 963c791a5c fix: run composite tool sections through the catalog
situation_brief, military_awareness and the HUD caption called other tools
directly, so replaced tools and interceptors did not apply to their sections.
Tools now reach each other through the catalog, and interceptors see these
calls with the calling tool as parent. Surfaces become a view over the
composed catalog, so a composite offered on voice keeps sections that voice
does not list.
2026-10-02 15:21:36 -07:00
Sameh Khamis 7ddee426e7 fix: count every mapped installation in tool answers
The installation source keeps one named site per display cell for the map.
Queries now ask for every site and the exact box, and report when the
source returned only part of the mapped sites.
2026-10-02 15:21:36 -07:00
Sameh Khamis 2e071b62d3 fix: repeat MCP tool data as JSON text for clients without structured content 2026-10-02 15:21:36 -07:00
Sameh Khamis df0e0f32e8 feat: choose per tool whether MCP, voice or both offer it
src/tools/surfaces.js lists the tools a surface leaves out, and the MCP
server, the voice session's tool list and the browser catalog voice calls
all filter through it. Voice leaves out tools that answer with images, link
to the app, or repeat what its app actions already answer.
2026-10-01 14:36:12 -07:00
Sameh Khamis b4067f9859 fix: time-limit OpenSky snapshot requests and retry once
A stalled global snapshot held the request for over a minute. Each attempt
now gets 10 seconds, a timed-out or failed attempt is retried once, and a
second failure is answered from the stale cache or the regional fallback.
The OAuth token request gets the same limit.
2026-10-01 14:26:20 -07:00
Sameh Khamis ec1adbdc19 fix: tighten tool answers found in local testing
Unknown aircraft tracks read as empty instead of a server error, aircraft
info names the aircraft and flight, the HUD caption sends the app's caption
context, and satellite pass, satellites overhead and nearby places take a
place name or point like the other tools.
2026-10-01 14:01:25 -07:00
Sameh Khamis 1a03e7483d feat: share the tool catalog with voice 2026-10-01 13:52:36 -07:00
Sameh Khamis 0953ebb026 feat: add a Bhote Koshi flood event tool 2026-10-01 13:44:41 -07:00
Sameh Khamis a747b8a4e0 feat: add a military awareness tool 2026-10-01 13:41:30 -07:00
Sameh Khamis 28cfd39d1a feat: add a datacenter and dam tool 2026-10-01 13:38:38 -07:00
Sameh Khamis a8e5ff9390 refactor: move infrastructure data parsing into a portable source module
The datacenter and dam layers parsed their bundled GeoJSON Lines and
built analyst records inside the Cesium-backed layer module. Move the
data URLs, the line parser, mapAnalystRecord and layerTitle unchanged to
src/sources/infrastructureData.js; the layer module imports them and
still exports mapAnalystRecord. The consumer-build pin lists the new
module.
2026-10-01 13:34:26 -07:00
Sameh Khamis 0a454815cb feat: add a license plate reader camera tool
Add find_alpr_cameras over the layer's OpenStreetMap camera extracts:
cameras in a US or Canadian area of at most 3° per side, nearest first,
with operator, manufacturer and facing direction, and the source's
attribution. Areas outside coverage or too wide are reported.
2026-10-01 13:31:31 -07:00
Sameh Khamis c1957d884e feat: add a submarine cable tool
Add find_submarine_cables over the bundled TeleGeography data: cables
passing through or landing in an area, or matching a name, with landing
points and the dataset's attribution. The local MCP server's fetch reads
the bundled data files a browser would load by URL, and nothing outside
that directory.
2026-10-01 13:28:02 -07:00
Sameh Khamis 9be90fff44 feat: add a recent satellite imagery tool
Add get_recent_imagery. It searches the recent Harmonized Landsat and
Sentinel-2 catalog for an area, picks the image the layer would show
with the layer's own ranking, and returns it with its date, sensor and
cloud cover, falling back to the VIIRS daily overview.
2026-10-01 13:25:00 -07:00
Sameh Khamis a3aa058569 feat: add weather map and wind tools
Add get_weather_map, which returns the latest NOAA radar, satellite or
lightning map over an area as an image, choosing a window the image
route accepts and reporting areas outside the product's coverage, and
get_wind, which samples GFS or IFS wind at a location with the wind
layer's own grid sampling.
2026-10-01 13:22:16 -07:00
Sameh Khamis c43c94ba5f feat: read weather image frames through the weather source
Add getImage() to the weather source. It reads one frame through the
bounded image route the layer already uses and returns its content type
and bytes, so code outside the layer can show the current map.
2026-10-01 13:18:01 -07:00
Sameh Khamis fe69f22346 feat: turn layers on in links to an area
open_in_gods_eye_view accepts layers and writes them with the app's own
share-link layer codec, so the link opens with those layers on. Layer
names are limited to the registered layer ids.
2026-10-01 13:17:15 -07:00
Sameh Khamis 9de2705eaf refactor: move the layer state coordinator into its own module
layerState.js held both the share-link layer codec and the coordinator
that restores layer choices into the running app and persists them to
browser storage. Move the coordinator, unchanged, to
layerStateCoordinator.js so the codec stays a pure data module that code
outside the browser can use to write share links. The registry stays in
layerState.js, where the token tooling reads it, and the aircraft
barrel re-exports both.
2026-10-01 13:16:19 -07:00
Sameh Khamis a8b5860e9f refactor: share the default tool services between surfaces
Move the composition of the default tool services from the Node MCP
server into gods-eye-view/tools/services, a portable export. Given a
fetch that resolves the sources' /api paths, it builds the same services
in a browser as on the local MCP server, so other surfaces can compose
the same catalog. Queries still read only their services.
2026-10-01 13:12:34 -07:00
Sameh Khamis 0dbd6375fc fix: center named places on their own location
A place's bounds can reach far from the place itself; Tokyo's include
remote Pacific islands, so the middle of its box is open ocean. Keep the
geocoder's point for a named place and use it as the area's center,
while bounds still decide what lies inside the area.
2026-10-01 12:50:15 -07:00
Sameh Khamis 469b1e8d89 feat: serve the MCP server at /mcp on the local servers
Mount the local MCP server at /mcp on the development and preview
servers for clients that connect by URL. The route accepts only
requests from this machine that name a loopback host and, when an
Origin is sent, come from a loopback origin, and drains oversized
bodies so the 413 response is delivered. The provider-order regression
test now pins the route between the providers and the API fallback.
2026-10-01 12:47:44 -07:00
Sameh Khamis b6af8b34e0 feat: add a tool that links to an area in the app
Add open_in_gods_eye_view. It builds a version 2 share link looking
straight down on an area, with altitude chosen from the area's size, on
the app address the application supplies.
2026-10-01 12:42:08 -07:00
Sameh Khamis 9d12da0a79 feat: add a traffic flow tool
Add get_traffic_flow over the traffic source's TomTom flow tiles. It
reports average speed as a share of free-flow speed, congested and
closed road length, and a breakdown by road category, stepping down a
zoom level when an area needs more tiles than the source reads at once.
2026-10-01 12:36:00 -07:00
Sameh Khamis e6cbcc31ad feat: add ship tools
Add vessels_in_area, find_vessel and get_vessel_track over the live
vessel source, and a ships section in situation_brief. Without an AIS
key the source's own message reaches clients as an unavailable feed.
2026-10-01 12:32:09 -07:00
Sameh Khamis 0a74e36ce8 feat: add weather, hazard, terrain and situation tools
Add get_weather, get_regional_brief, get_cyclones, get_fire_perimeters,
get_terrain_height, find_military_installations and get_map_features
over the application request services and the layers' sources.

situation_brief combines weather, earthquakes, fires, aircraft and
cyclones for an area by reusing those tools, marking sections that are
unavailable instead of failing. get_hud_caption returns the app's
heads-up display caption for the same overview.
2026-10-01 12:28:34 -07:00
Sameh Khamis 8c97fed92d feat: add bike-share and transit tools
Add get_bike_share over the public GBFS systems and get_transit_vehicles
over the GTFS-Realtime feeds. Each finds the systems or feeds whose
coverage reaches the area, reads up to three, and reports any that were
unavailable instead of failing the whole answer. Transit results carry
each feed's attribution and license.
2026-10-01 12:22:13 -07:00
Sameh Khamis 2ba751ace8 feat: read the transit feed catalog through the transit source
Add getFeeds() to the transit source. It reads the public catalog route
the server already serves, so code outside the layer can find feeds
near a location without loading the layer.
2026-10-01 12:19:19 -07:00
Sameh Khamis eb12303f46 refactor: move GBFS station parsing into a portable source module
The bike-share layer parsed station documents inside its model factory.
Move the parsers unchanged to src/sources/gbfsStations.js and have the
model import them, so code outside the layer can read GBFS stations
without constructing the layer.
2026-10-01 12:18:48 -07:00
Sameh Khamis e680a59bf5 feat: add place search and routing tools
Add search_places and places_nearby over the app's Google Places routes,
and plan_route over its OpenStreetMap routing. The place search service
reports when no search key is configured, so an unconfigured server is
not mistaken for an empty result, and rate limits become retry_later.
Route endpoints accept a place name or coordinates.
2026-10-01 12:16:01 -07:00
Sameh Khamis 85fee59da1 feat: add public camera and radio tools
Add find_cctv_cameras, get_cctv_snapshot and find_radio_stations. A
snapshot returns the camera's current frame as image content with its
credit. Tool results may now carry base64 images, which the MCP adapter
sends as image content.
2026-10-01 12:11:00 -07:00
Sameh Khamis d65402eae3 feat: read current camera frames through the CCTV source
Add getFrame(camera) to the CCTV source. It reads one frame through the
registered frame endpoint the layer already uses and returns its content
type and bytes, so code outside the layer can show a camera's view.
2026-10-01 12:11:00 -07:00
Sameh Khamis 22c3fe40b0 feat: add aviation and satellite tools
Add aircraft_in_area, find_aircraft, get_aircraft_track and
get_aircraft_info over the live aircraft sources, with an optional
military feed, and next_satellite_pass and satellites_overhead over the
satellite catalog and the existing pass predictor.

Source failures from the live contract now become tool errors with the
matching code, so a rate limit reaches clients as retry_later with a
wait time. Tool schemas may use pattern.
2026-10-01 12:05:50 -07:00
Sameh Khamis 9895942633 refactor: move TLE text parsing into a portable source module
The satellites layer parsed catalog text inside its Cesium-backed orbit
owner. Move the parser unchanged to src/sources/tle.js, add a catalog
number helper, and have the layer delegate to it so code outside the
layer can read TLE text without loading rendering.
2026-10-01 12:05:41 -07:00
Sameh Khamis c3dfa8e853 feat: add a tool catalog and local MCP server
Define tools that answer questions from the app's data once, independent
of the surface that exposes them. A tool declares its arguments and the
services it reads; applications compose tools with their own services and
optional call interceptors. Queries reuse the layers' portable source
factories, so they run wherever an application routes /api requests.

The first queries are get_earthquakes, get_active_fires and
get_recent_launches, with a shared area argument (place name, bounding
box, or point and radius) and capped lists.

gods-eye-view/tools/mcp adapts a catalog to the Model Context Protocol
with a small JSON-RPC handler and a stateless HTTP transport, adding no
dependencies. npm run mcp serves Core's tools over stdio to local clients
such as Claude Code, reading from a running app. Both exports are
portable and checked by check:boundaries; the app does not import them.
2026-10-01 11:51:59 -07:00
Manjunath 1dbc8a4452 fix(ui): preserve Cyber and Cockpit visual state 2026-10-01 17:00:10 +05:30
Manjunath e03bf008f3 fix(ui): harden Cyber panel layout and preset restoration 2026-10-01 01:50:31 +05:30
602 changed files with 20783 additions and 33201 deletions

No files matched your search

+34 -45
View File
@@ -34,16 +34,19 @@ GOOGLE_MAPS_API_KEY=
# single shared key keeps working — splitting is opt-in.
# SERVER-SIDE ONLY.
GOOGLE_MAPS_SERVER_API_KEY=
# Optional: opt-in per-IP rate limit for the Google Places cost endpoint
# (/api/google/nearby-places), in requests per minute.
# DEFAULT IS UNLIMITED — unset (or 0) means no throttling, unchanged behavior.
# Set a positive integer to cap per-IP Places quota spend when self-hosting.
# Recommended whenever HOST is not localhost. NOTE: this is a per-IP,
# process-local guard (in-memory, resets on restart; over-limit requests get a
# sanitized 429) — it is NOT a billing cap. For hard spend protection set
# provider-side budget alerts: Google Cloud Console -> Billing -> Budgets & alerts
# (plus per-API quotas under APIs & Services -> Quotas).
# GEV_RATELIMIT_GOOGLE_PER_MIN=60
# Per-IP rate limit for the Google Places cost endpoints
# (/api/google/nearby-places + /api/google/text-search), in requests per minute.
# DEFAULTS TO 120, the value the Pinokio build already ships. Raise or lower it
# with a positive integer; 60 is a sensible tightening whenever HOST is not
# localhost. Set it to exactly 0 to run unthrottled. An unreadable value (a
# typo, a stray unit) falls back to the default rather than to unlimited, so a
# slip cannot quietly disarm the guard. A positive value below 1 counts as 1.
# NOTE: this is a per-IP, process-local guard (in-memory, resets on restart;
# over-limit requests get a sanitized 429) — it is NOT a billing cap. For hard
# spend protection set provider-side budget alerts: Google Cloud Console ->
# Billing -> Budgets & alerts (plus per-API quotas under APIs & Services ->
# Quotas).
# GEV_RATELIMIT_GOOGLE_PER_MIN=60 # uncomment to tighten from the default 120
# Optional: Google Photorealistic 3D Tiles through Cesium ion, Bing world
# imagery, and Cesium World Terrain. The free Community plan is for eligible
@@ -64,23 +67,20 @@ OPENAI_REALTIME_REASONING_EFFORT=low
# Keep only a short conversational window. Current map state is fetched live.
OPENAI_REALTIME_CONTEXT_TOKENS=3000
OPENAI_REALTIME_CONTEXT_RETENTION=0.5
# Voice-card captions of what you said. Billed separately by OpenAI and
# counted in the session cost meter and cap; off disables them.
OPENAI_REALTIME_TRANSCRIBE_MODEL=gpt-4o-mini-transcribe
# The voice debug log (.gev-logs/realtime-conversations.jsonl, rotated at
# 32 MB, never expired) omits transcripts and typed or spoken text by default.
# Set to 1 to keep them while debugging; delete the log when you are done.
GEV_VOICE_LOG_CONTENT=
OPENAI_HUD_SUMMARY_MODEL=gpt-5-nano
# Optional: opt-in per-IP rate limit for the OpenAI cost endpoints
# Per-IP rate limit for the OpenAI cost endpoints
# (/api/realtime/token + /api/openai/hud-summary), in requests per minute.
# DEFAULT IS UNLIMITED — unset (or 0) means no throttling, unchanged behavior.
# Set a positive integer to cap per-IP spend when self-hosting on a network.
# Recommended whenever HOST is not localhost. NOTE: this is a per-IP,
# process-local guard (in-memory, resets on restart; over-limit requests get a
# sanitized 429) — it is NOT a billing cap. For hard spend protection set a
# provider-side budget: OpenAI platform -> Settings -> Limits (usage limits).
# GEV_RATELIMIT_OPENAI_PER_MIN=30
# DEFAULTS TO 30, the value the Pinokio build also ships. That clears ordinary
# use several times over: an open tab asks for a HUD summary every 15s (4/min)
# and a voice session mints one token. Raise or lower it with a positive
# integer; set it to exactly 0 to run unthrottled. An unreadable value (a typo,
# a stray unit) falls back to the default rather than to unlimited, so a slip
# cannot quietly disarm the guard. A positive value below 1 counts as 1.
# NOTE: this is a per-IP, process-local guard (in-memory, resets on restart;
# over-limit requests get a sanitized 429) — it is NOT a billing cap. For hard
# spend protection set a provider-side budget: OpenAI platform -> Settings ->
# Limits (usage limits).
# GEV_RATELIMIT_OPENAI_PER_MIN=30 # the default; uncomment only to change it
# Optional: OpenSky auth mode (oauth | basic | auto | anon)
# Default: oauth — requires client credentials below
@@ -108,6 +108,11 @@ PORT=4173
# networks you trust, and consider the GEV_RATELIMIT_* throttles above.
# HOST=localhost
# HOST=0.0.0.0
# Optional: comma-separated LAN hostnames to accept, in addition to localhost
# and IP addresses, for example GEV_ALLOWED_HOSTS=globe.lan,globe.internal.
# Binding HOST=0.0.0.0 alone does NOT trust every Host header; suffix and
# wildcard entries are ignored.
# GEV_ALLOWED_HOSTS=
# Optional: NASA FIRMS live active fires. SERVER-SIDE ONLY (never reaches the
# browser — the client polls the /api/firms proxy). Free key:
@@ -194,23 +199,7 @@ AISSTREAM_API_KEY=
# Optional: comma-separated Overpass URLs; use only instances you run or pay for.
OVERPASS_UPSTREAMS=
# Optional: place-lookup endpoints. Lookups try bundled data first, then the
# endpoint set here, or the public service under its usage limits when unset.
# NOMINATIM_URL: search endpoint for last-resort place search and voice
# outlines (e.g. https://nominatim.example.org/search). A configured URL
# replaces the public service: nothing falls back to nominatim.openstreetmap.org,
# and an invalid value disables Nominatim. Unset uses the public
# nominatim.openstreetmap.org: one request at a time, 1.1 s apart, a short
# queue, cached answers (outlines 90 days, places 30 days, no-results 24 h in
# .gev-cache/nominatim) and a daily cap per install. Set it empty to disable.
# The public service's limit applies to all installs of this app together; the
# per-install cap bounds each install, not the total, so heavy or shared
# deployments should point this at their own server.
# NOMINATIM_URL=
# Public-instance requests per UTC day, outlines and search together. Not
# applied to your own NOMINATIM_URL.
# NOMINATIM_DAILY_CAP=50
# PHOTON_URL: keyless place search endpoint used by the browser
# (e.g. https://photon.example.org/api/). Unset uses photon.komoot.io; set it
# empty to disable.
# PHOTON_URL=
# Optional: who may frame the app in embed mode (?embed=1), as CSP
# frame-ancestors sources such as "https://example.com", or * for any page.
# Unset, no page may frame it. Other pages of the app can never be framed.
# GEV_EMBED_FRAME_ANCESTORS=
+136 -117
View File
@@ -1,37 +1,106 @@
# Changelog
- The voice card shows analyst counts ("At least 250,000 aircraft …") with
their scope, feed window and caveats, and marks partial answers. Analyst
speech uses the same deterministic headline, preserves lower bounds and says
when an answer is partial. Follow-ups retain unanswered-layer coverage, and
keep the original scope label when they narrow a remembered result.
Contacts-window answers read one immutable panel snapshot, mark a 20,000-row
layer cap as a lower bound, and commit that displayed cohort only after
cancellation checks pass. Concurrent analyst calls from one model response
remain independent; a stopped or replaced user turn still cancels them.
- Voice answers are shorter and fill slow work instead of going silent. Slow
or multi-step requests may open with one short spoken plan; instant ones
just happen. Tool results carry a code-built `say` line and a `display`
card: sources and notes stay on screen, a caveat is spoken only when it
changes the answer, and "at least" and partial answers are always said. A card above the mic shows captions, the plan's steps and the
result. When a tool that reports steps (layers, place lookups, outlines,
nearest-aircraft) is still running after 1.5 s its current step is spoken
once; others say "still working" once after 8 s. A soft
tick plays in push-to-talk. Holding Space while the assistant talks cuts it
off. Each turn writes latency and speech figures to the debug log
(`node scripts/voice-turn-spans.mjs`). Captions of what you say are
counted in the voice cost meter; the debug log leaves out what was said
unless `GEV_VOICE_LOG_CONTENT=1`.
- Point and ask: hold Space with the cursor on an aircraft, ship or spot on
the map and say "what's that", "track that one" or "how many flights
around here". A reticle marks the point and the voice card shows what
"this" meant. In open mic it works when you have just moved the cursor
there. "The second one" picks from the last numbered list. Asking about
"the last one" resolves to the last of the five rows actually shown, and
referent-only tracking calls are accepted while empty targets are rejected.
Asking about
bare ground up close gives the assistant a small crop of the map around the
point.
## [Unreleased]
## [0.2.1] — 2026-10-02 — God's Eye View through Your Agent (with security fixes and other features)
- Ship each bundled data pack once. The region, marine, admin-boundary,
county, military-name and neighborhood packs were emitted twice by the
production build, as the JSON the browser fetches and as an unused
JavaScript copy; `dist/` drops from 54 MB to 42 MB. Under Node the loader
reads the JSON file directly (reporting that Node 24.14 or newer is needed
on a runtime too old to do so), and a test keeps app code from importing a
pack as a module.
- Apply the Host check before the app's own routes. Vite installs its Host
check after the middleware that plugins add, so the provider and `/api`
routes used to answer any Host, including a DNS-rebinding name, in every
mode. A first-running middleware now applies the same allowed hosts on the
dev and preview servers.
- Refuse cross-site browser requests to the cost-bearing and log endpoints
(`/api/realtime/token`, `/api/openai/hud-summary`,
`/api/google/nearby-places`, `/api/google/text-search`,
`/api/realtime/debug-log`): a foreign or opaque
Origin, a cross-site `Sec-Fetch-Site`, or proxy forwarding headers get 403,
while loopback tools and LAN use keep working. The dev and preview servers
send one shared Content-Security-Policy (James Sumpter, #242).
- The voice debug log records the server's own time: a record can no longer
supply its `loggedAt`. The dev and preview servers also send
`X-Content-Type-Options: nosniff` (findings by Sunil, #710).
- Keep the Host-header check when binding to all interfaces. `HOST=0.0.0.0`
used to accept any Host; the dev and preview servers now accept IP
addresses, `localhost` and the LAN hostnames listed in `GEV_ALLOWED_HOSTS`
(suffix and wildcard entries are ignored), which also keeps DNS-rebinding
names out in LAN mode. The `.local` suffix is no longer accepted by default;
list such a name explicitly (Puspo Aditya, #97, fixes #21).
- Throttle the cost-bearing proxies by default instead of on request. The
OpenAI endpoints (`/api/realtime/token`, `/api/openai/hud-summary`) now allow
30 requests per minute per client IP and the Google Places endpoints
(`/api/google/nearby-places`, `/api/google/text-search`) 120 — the caps the
Pinokio build already ships, so the packaged app is unaffected and only an
unconfigured server changes, from unlimited to what the product already runs
with. `GEV_RATELIMIT_OPENAI_PER_MIN` and `GEV_RATELIMIT_GOOGLE_PER_MIN` still
override the caps, and exactly `0` disables them; a value that cannot be read
as a number now falls back to the default rather than to unlimited, so a typo
cannot silently disarm the guard (daikaginza, #683).
- Tighten the local MCP server and embed mode. The panel's `panel_request`
needs the key in its MCP server's panel page, and refuses Provider
Settings, credential and model endpoints, `/mcp` and development server
routes in any letter case, encoding or dot suffix; panels on one server share six
requests in flight, and further requests wait their turn. `/mcp` applies Provider Settings' proxy and sharing checks,
requires a Host on the port it reached and an Origin from that host, and
times out request bodies after 30 seconds; stdio honors
`notifications/cancelled`. Embed mode is no longer framable unless
`GEV_EMBED_FRAME_ANCESTORS` allows the framing page, and view answers go
only to the origin that sent the view. SECURITY.md describes the MCP surface
and the panel's browser keys.
- Cockpit now enters on the existing matching map style while keeping one fixed,
duplicate-free carousel over `Normal / CRT / NVG / FLIR / Anime / Noir / Snow`.
Normal remains a real unfiltered option; Cockpit-only choices still restore
the captured map style through both Exit Cockpit and Reset.
- Cyber's compact right-rail and Cockpit utility buttons now center their glyphs
vertically and share the same inset and edge alignment.
- Keep Cyber Voice help/error popups and Location/Visual Presets pins clear of
their decorative frames. Leave space above attribution for its full logo row.
- Add an intentionally future-facing, opt-in panel surface contract for new
panels to inherit compatible Normal, Cyber and Cockpit styling. Its first
production adopter will land separately after this change.
- Restore the user's previous visual preset when they explicitly switch from
Cyber to another HUD layout, without overriding scene or shared-link state.
- Move the aligned Cyber side-panel rails upward on desktop so the left stack
clears the lower coordinate card; keep Cockpit's independent visor layout.
## [0.2.0] — 2026-10-02 — God's Eye View through Your Agent
- Show God's Eye View inside AI conversations. Answers that can be shown
include a view (camera, layers, style, map, marks, something to follow),
and `show_in_gods_eye_view` displays it as the live globe in clients that
support MCP Apps, such as Claude Desktop and the Codex and ChatGPT desktop
apps, and as a link everywhere else. The panel loads the app's panel build
(`npm run build:panel`) and its data through the MCP server, so a local
server needs no HTTPS or tunnel. `?embed=1` shows the globe alone for pages
that frame the app.
- Add tools for language-model clients and a local MCP server, served over
stdio by `npm run mcp` and over HTTP at `/mcp` on the development and preview
servers for local requests only. `npm run mcp`
serves earthquake, active-fire, launch, aircraft, ship, satellite, camera, radio,
place, routing, bike-share, transit, traffic, weather, cyclone, fire-perimeter,
terrain, installation and map-feature queries, plus a situation brief, over
stdio to clients such as Claude Code, reading
from a running app. Tools are defined
once in `gods-eye-view/tools`, reuse the layers' source factories, and are
exposed through the protocol adapter in `gods-eye-view/tools/mcp`. The
catalog also covers weather maps, wind, recent imagery, submarine cables,
license plate reader cameras, datacenters and dams, the Bhote Koshi flood,
military awareness and links that open the app over an area with layers on.
Voice offers the same queries next to its app actions.
- Public Overpass instances are no longer used by default. Street Traffic
roads come from TomTom flow tiles, OpenFreeMap vector tiles, or both, chosen
@@ -182,7 +251,6 @@
geocode and admin-boundary fallback answers `region-timeout` after 3 s
instead of holding the reply.
- Transit and Directions rows repaint as soon as their data lands again:
`refreshLayerStats()` now lives on the layer lifecycle, not only on the
compatibility facade. `scripts/qa-radio.mjs` uses it instead of a private
@@ -216,57 +284,7 @@
(`src/ui/imagerySplit.js`, `src/maps/imageryComparison.js`), and
`MapSourceController.subscribe()` reports every settled map switch.
## Unreleased — voice reaches every layer; honest analyst answers
- Voice can switch on every shipped data layer (the two scene-driven
Bhote Koshi layers stay scene-only), including transit, Recent
Imagery, wind, the weather layers, cyclones and mapped installations, and
understands their spoken names ("buses", "hurricanes", "military bases").
One voice manifest (`src/voice/layerManifest.js`) generates the tool enums
and aliases, and a unit test fails when a new layer has no entry.
- `analyst_query` counts up to 250,000 loaded records per layer instead of the
first 2,000, and says when a layer went past that (`complete: false`). It
also answers over bikeshare stations, transit vehicles, rocket launches,
cyclones, ALPR cameras, mapped installations, local ADS-B aircraft and fire perimeters; fire
records carry their sensor (VIIRS or MODIS).
- Unknown fields, operators, units, value types, scope kinds and invalid
centres are refused with the allowed values instead of answering zero or widening to anywhere. A
layer that is off, still loading or unavailable is reported as such, not
as a count of zero; a partly answerable query is marked `partial`.
Distance sorts honour `sortDir`, a written unit such as `altitudeM(ft)`
converts, and large rankings no longer sort the whole set.
- Results carry `lat`/`lon` for each item and a `display` object with the
scope, the feed window (earthquakes and fires cover the last 24 h) and any
caveat for the screen. The model is asked to answer in one short phrase.
- The weather and Recent Imagery panels can be opened by voice.
## Unreleased — voice outlines without Overpass
- Voice outlines work again without Overpass. Bundled US Census places and
Who's On First neighborhoods (lazy-loaded; `src/data/local_data/`) answer
first, then streets, grounds and pointed-at buildings from OpenFreeMap
tiles, then Nominatim for one explicit ask at a time. Traffic and outlines
share pending fetch/decode ownership and a projection-aware retained-byte
ceiling for the same immutable tile; consumer caches and request lifetimes
remain independent, and eviction or teardown retires queued idle projections.
Public-provider answers are cached and capped per
install. Public Nominatim requests now reserve their daily slot and paced
start atomically across processes sharing that installation's state file,
and fail closed when durable state is unavailable or invalid. Operator-selected
endpoints remain outside the public cap, spacing and state policy, and an
upstream cooldown is isolated to the endpoint that returned it. Anything else
stays a point marked "outline unavailable". Administrative Nominatim
outlines carry the selected record's verified country/admin-1/admin-2
identity; ambiguous, missing, contradictory or retired evidence is not
promoted into a verified level.
- Who's On First rebuilds accept only exact raw registry licence values in the
reviewed open-licence allowlist. New, unknown, restricted or noncommercial
source licences stay excluded until they are reviewed explicitly.
- New settings: `NOMINATIM_URL` (your own Nominatim instead of the public
one; empty disables it), `NOMINATIM_DAILY_CAP` (public requests per day,
default 50) and `PHOTON_URL` (browser place search; empty disables it).
## Unreleased — local receiver feeds
### Local receiver feeds
- The Local ADS-B layer also reads local 1090 MHz and 978 MHz UAT decoder
feeds: the `aircraft.json` that dump1090-fa, readsb, tar1090 or skyaware978
@@ -295,7 +313,7 @@
- Records carry `band` (`1090`/`978`) and `source` (`webusb`/`feed`).
- See `docs/LOCAL-RECEIVERS.md`.
## Unreleased — local RTL-SDR and Local ADS-B
### Local RTL-SDR and Local ADS-B
- Add a Local RTL-SDR card to the Radio panel. It connects a USB RTL-SDR in
desktop Chrome or Edge through WebUSB and receives broadcast FM (tune, seek,
@@ -320,7 +338,7 @@
- Add `@jtarrio/webrtlsdr` and `@jtarrio/signals` (Apache-2.0); see
`THIRD_PARTY_NOTICES.md`.
## Unreleased — weather review
### Weather review
- On 3D Tiles, draw a 4096×2048 detail window around the view on each
observed-weather shell except global infrared, sampled by the shell's own
@@ -586,26 +604,26 @@ Analyst records for loaded satellites, datacenters and dams, with explicit bound
- Split application scene, controls, catalog, tools and HTML into reusable components; configure application request services and sources without changing global fetch. Preserve standalone markup and voice behavior. Explicit annotation navigation may resolve a distant named target.
## Satellite pass prediction
### Satellite pass prediction
- Bisect pass rise/set to ~0.2 s and fit peak elevation with a parabola.
- Mark passes visible from Earth-shadow and civil-twilight checks.
- Add `getNextSatellitePass(noradId, options)` for any loaded catalog satellite.
- `next_iss_pass` retains the next geometric pass and adds visibility metadata. `next_satellite_pass` adds bounded loaded-catalog name/NORAD lookup and optional visible-only filtering (Rehaan Delmotra, #451; maintainer adaptation).
## Voice component boundaries
### Voice component boundaries
- Separate voice controls, Realtime connection requests and the action runner.
- Allow compatible endpoints and server-selected models through construction options.
- Cancel pending token/SDP requests on Stop or teardown and reject expired secrets.
## Configurable geospatial services
### Configurable geospatial services
- Compose geocoding, place context and routes through independent providers.
- Allow compatible endpoint configuration without changing voice tools or annotation behavior.
- Isolate configured source caches and reject results after cancellation.
## ALPR camera locations
### ALPR camera locations
- Port Manjunath's (@manjunath22466) cyan camera badges, coral selection brackets,
gradient direction wedges and animated tactical labels into the reusable ALPR
@@ -625,32 +643,32 @@ Analyst records for loaded satellites, datacenters and dams, with explicit bound
- Separate the request adapter, camera model, presentation, and instance lifecycle.
Source cancellation also guards late response bodies and rejects invalid query bounds.
## Release disabled infrastructure rendering
### Release disabled infrastructure rendering
- Remove built Data Center, Dam and Submarine Cable entities when their layers
are disabled, avoiding retained visualizer work and entity memory.
- Keep parsed datasets cached for re-enable; rebuild entities without refetching.
## Camera layer components
### Camera layer components
- Separate camera source requests, placement, frames, projection, cards and calibration.
- Own visibility listeners and pending initialization within each layer lifetime.
- Preserve existing camera catalogs, URL families, geometry and playback behavior.
## Traffic and bikeshare components
### Traffic and bikeshare components
- Separate traffic loading, animation, styling and lifecycle into factory-owned components.
- Give each flow source its own bounded decode cache and cancellation checks.
- Separate bikeshare registry, station requests, rendering, selection and proximity handling.
## Installation and context components
### Installation and context components
- Separate mapped-site requests, records, placement, selection and viewport lifecycle.
- Separate proximity queries, subject tracking, navigation/history, panel and direction rendering.
- Retain source and ground-floor ownership in standalone composition; reject malformed
installation snapshots and ignore failures from cancelled requests.
## Satellite and mission layer components
### Satellite and mission layer components
- Separate catalog loading, orbit calculations, display, tracking and interaction
into instance-owned satellite components.
@@ -658,20 +676,20 @@ Analyst records for loaded satellites, datacenters and dams, with explicit bound
operations, retaining existing layer controls and satellite coordination.
- Cancel late mission source work and reject malformed launch snapshots.
## Fire layer components
### Fire layer components
- Split fire source loading, state, rendering, cards, selection and viewport work
into reusable components with application-owned scene services.
- Cancel late refreshes, retain good data after malformed responses, and preserve
selection identity without repeating a user-selection notification on refresh.
## Earthquake components
### Earthquake components
- Separate earthquake snapshot loading, record validation, and display ownership.
- Cancel pending earthquake refreshes on disable or destruction, retaining the
last good snapshot after malformed or failed refreshes.
## September 8, 2026
### September 8, 2026
Earthquake refreshes validate the complete feed and construct replacement entities before clearing the previous snapshot. Malformed rows and duplicate rendered IDs retain the last good entities, overlays, count and timestamp and report a malformed response; unknown magnitude is excluded from M2.5+ rendering.
@@ -682,7 +700,7 @@ Launch payloads with missing records now say PAYLOAD DATA UNAVAILABLE. Missing n
This changelog records public product changes. For the authoritative description
of current runtime behavior, see [`docs/CURRENT-STATE.md`](docs/CURRENT-STATE.md).
## [Unreleased]
### Other changes
- Add ECMWF IFS model selection to Wind (#464, thanks @beneduzi), with model-scoped forecast-step caches, cancellation of replaced requests, and separate issue/valid timestamps.
@@ -703,7 +721,7 @@ of current runtime behavior, see [`docs/CURRENT-STATE.md`](docs/CURRENT-STATE.md
- Separate submarine cable sources and rendering components, and export bundled geography lookup modules.
### Added
#### Added
- DISPLAY ▸ Draw: draw on the world by hand. Pick Area, Line or Pin, click the
vertices, double-click or press Enter to finish, label and colour it; Backspace
@@ -713,7 +731,7 @@ of current runtime behavior, see [`docs/CURRENT-STATE.md`](docs/CURRENT-STATE.md
together. While you are drawing, the draw tool owns the pointer and no layer
selects what you click through (#235 — thanks @cora-fresh-labs).
### Fixed
#### Fixed
- Keep traffic-road bounds crossing the antimeridian monotonic and inside the
longitude range accepted by the Overpass request path, preserving the small
@@ -780,7 +798,7 @@ of current runtime behavior, see [`docs/CURRENT-STATE.md`](docs/CURRENT-STATE.md
toggle. Camera departure cancels pending work, arrival checks the final view,
and superseded requests cannot keep a newer view loading.
### Added
#### Added
- Add MODIS NRT (Terra+Aqua, ~1 km) active fires to the FIRMS layer, sharing the
existing `FIRMS_MAP_KEY` and 30-minute cache.
@@ -908,9 +926,9 @@ of current runtime behavior, see [`docs/CURRENT-STATE.md`](docs/CURRENT-STATE.md
- Extract shared surface keyboard handling for the welcome launcher and Provider
Settings, preserving Tab/Escape behavior and releasing the listener on teardown.
### Added
#### Added
### Security
#### Security
- The CCTV media route no longer forwards a client `Range` header to the upstream
camera host as it arrived. A single `bytes=` range is canonicalized and
@@ -941,7 +959,7 @@ of current runtime behavior, see [`docs/CURRENT-STATE.md`](docs/CURRENT-STATE.md
- Cancel the active location lookup when its controls are disposed.
### Fixed
#### Fixed
- `DATA_SOURCES.md` states what the project does with camera frame content: a
successful upstream response is relayed as the provider served it, nothing in
@@ -1020,7 +1038,7 @@ of current runtime behavior, see [`docs/CURRENT-STATE.md`](docs/CURRENT-STATE.md
fetching, AIS records/tracks and shared request helpers; preserve existing
routes, local setup, fallback behavior and rendering.
### Added
#### Added
- **Directions layer** — keyless A→B directions without a geocoder or a
microphone (thanks @spcpza). The row's chips arm a globe click for A and B
@@ -1048,7 +1066,7 @@ of current runtime behavior, see [`docs/CURRENT-STATE.md`](docs/CURRENT-STATE.md
voice routes since launch, previously uncredited), with the OpenStreetMap
credit and the "fix the map" link the service's usage policy asks for.
### Changed
#### Changed
- The interface asks Google Fonts for only the icon glyphs it draws, instead of
the whole variable icon font, and no longer requests a second icon family that
@@ -1063,7 +1081,7 @@ of current runtime behavior, see [`docs/CURRENT-STATE.md`](docs/CURRENT-STATE.md
- Rename standalone browser startup to `src/standalone/` and add a Node-only
`gods-eye-view/build/vite` export with checked package ownership.
### Development
#### Development
- The CCTV launcher and preview-server tests resolve their temporary fixture
root through `fs.realpath`, so they pass on macOS, where the system temp
@@ -1097,7 +1115,7 @@ of current runtime behavior, see [`docs/CURRENT-STATE.md`](docs/CURRENT-STATE.md
infrastructure modules and their consumer tests. Package boundary checks keep
those exports separate from app startup and local Node services.
### Fixed
#### Fixed
- Reduce terrain-height timeouts when Re:Earth slows down. Batches are
sized against measured response latency on both browser and server to reduce
@@ -1188,7 +1206,7 @@ of current runtime behavior, see [`docs/CURRENT-STATE.md`](docs/CURRENT-STATE.md
and upstream error bodies; response statuses and cache fallback remain intact.
Includes the security fixes contributed by Tom-Neverwinter in PR #171.
### Fixed
#### Fixed
- Map Source keyboard opening retries focus until the selected tile is visible.
Leaving the disclosure, pointer interaction, or closing the tray cancels the
@@ -1230,7 +1248,7 @@ of current runtime behavior, see [`docs/CURRENT-STATE.md`](docs/CURRENT-STATE.md
that had since recovered. A miss is now cached only when every source
consulted actually returned a verdict.
### Added
#### Added
- Keyless place search. The LOCATION search box and the `fly_to_location` voice
tool now resolve place names through Photon (komoot, over OpenStreetMap) when
@@ -1250,6 +1268,13 @@ of current runtime behavior, see [`docs/CURRENT-STATE.md`](docs/CURRENT-STATE.md
to Puppeteer 25.10.0 and Sharp 0.35.4. Cesium remains on 1.138.0.
Browser QA awaits the new asynchronous executable-path lookup.
### Live CCTV video
- Live HLS video shares one decoder between the camera panel and projection,
with a DelDOT HTTPS source pack. Credit: Daniel Slay (@Danielslay86), PR #489.
- Maintainer adjustments bound sessions and downloads, remove disk/subprocess
remuxing, reject redirects, and clean up playback on switching or disabling.
## [0.1.1] — 2026-09-01 — Installation and live-data fixes
### Changed
@@ -1487,9 +1512,3 @@ represent previously published GitHub Releases.
- Initial project version.
### Live CCTV integration candidate
- Live HLS video shares one decoder between the camera panel and projection,
with a DelDOT HTTPS source pack. Credit: Daniel Slay (@Danielslay86), PR #489.
- Maintainer adjustments bound sessions and downloads, remove disk/subprocess
remuxing, reject redirects, and clean up playback on switching or disabling.
+1 -4
View File
@@ -96,8 +96,7 @@ The optional Local ADS-B layer and the broadcast-FM receiver use radio you recei
- **Re:Earth Terrain.** Keyless (no API key). Used two ways: (1) `src/mapStackController.js` swaps in a `Cesium.CesiumTerrainProvider` pointed at Re:Earth's `cesium-mesh/ellipsoid` quantized-mesh endpoint for globe stacks without a Cesium ion token (e.g. OSM), replacing a flat `EllipsoidTerrainProvider`; falls back to the flat provider if the endpoint can't be reached. (2) The server-side `/api/terrain/heights` proxy (disk-cached, serve-stale) resolves per-point ellipsoidal ground height for entity placement. Both are best-effort with a keyless-safe fallback (bundled EGM96 geoid math) if Re:Earth is unreachable.
- **OSRM on the FOSSGIS routing servers.** Keyless (no API key). Every route is requested by a person: clicking A and B in the Directions layer, or a voice "route from A to B". The server-side `/api/route` proxy sends an identifying `User-Agent`, pins the upstream host, accepts 2-12 coordinates, rejects any leg over 600 km or a total over 2,500 km, caps the buffered response at 8 MB, times the upstream out at 12 s, rate-limits per client, caches each route for 10 minutes, and coalesces identical requests that are in flight at the same time. Turn-by-turn maneuvers are asked for only when a caller wants them (`steps=1`), so the callers that do not read them send the smaller request. The "fix the map" link the policy asks for is in the Data attribution popover beside the OpenStreetMap credit.
- **Mapped installations.** OpenFreeMap military polygons use a bundled Overture/OSM name index; wide views show named points. Optional operator-configured Overpass and explicit Google Places searches keep their own source paths. Mapped data does not establish current activity or complete coverage.
- **Place search.** Named-place fly-to uses Google Geocoding when a Maps key is configured, then Photon (`PHOTON_URL`; unset uses the public instance, empty disables it), then `/api/geocode` (Nominatim search) when neither answers. `/api/geocode` and the voice outline route `/api/geocode/outline` share one Nominatim queue (`NOMINATIM_URL`; unset uses the public instance, empty disables it): one request at a time at least 1.1 s apart, an identifying `User-Agent` and `Referer`, a short queue that refuses a burst, a pause after 429/403/5xx answers (honouring `Retry-After` in full), at most one retry after a network failure, and a per-install daily cap on public-instance requests (`NOMINATIM_DAILY_CAP`, default 50). Processes using the same installation state file atomically reserve the next public start and daily slot; restarts share that count, spacing and any pause. Public access fails closed if the state file cannot be read, locked, written or semantically validated. This coordination does not span independent installations or state directories, and the public service's total limit still applies across all installs, so heavy or shared deployments should set their own `NOMINATIM_URL`. An operator-selected endpoint is explicitly outside the public cap, spacing and durable-state requirement; an upstream cooldown is kept with the endpoint that returned it rather than leaking across operator URLs. Answers are cached on disk (outlines 90 days, places 30 days, no-results 24 hours); failures are not cached. Outlines are looked up only for an explicit ask, after bundled data and any configured Overpass, and only results whose type suits the ask are used. Administrative outline responses expose country/admin-1/admin-2 identity only when the selected Nominatim row has a recognized, unambiguous address type consistent with its own name; numeric rank, ambiguous types, missing fields and contradictory names do not manufacture identity. It is not used for systematic or bulk queries.
- **Street and building outlines.** Voice street highlights, pointed-at buildings and grounds use the OpenFreeMap vector tiles the traffic layer already loads. Traffic and outlines share one bounded pending fetch/decode owner per immutable XYZ tile; materialized outline geometry is charged to the retained-byte ceiling and capped by coordinate count. Consumer result caches and request lifetimes remain independent, while eviction or explicit teardown cancels queued idle projections and releases their decoded owner. No Overpass request is made unless `OVERPASS_UPSTREAMS` is configured.
- **Place search.** Named-place fly-to uses Google Geocoding when a Maps key is configured, then Photon's public instance, then `/api/geocode` (Nominatim search) when neither answers. `/api/geocode` uses the shared Nominatim forward-search queue at no more than one request per second, sends an identifying `User-Agent` and `Referer`, caches answers for five minutes, shares one upstream call between identical searches already in flight, bounds the queue so a burst is refused rather than held, and drops a queued search whose caller has given up. It asks for one result per search and is not used for systematic or bulk queries.
- **Cockpit regional briefing.** `/api/regional-brief` rounds aircraft coordinates into 0.1° cache cells, caches results for five minutes, and resolves physical regions from bundled public-domain Natural Earth polygons without reverse-geocoding requests. Google News RSS is queried with the resolved locality/region first; GDELT is used only when that RSS query fails or is empty. Google's published Google News terms restrict that source to personal, noncommercial use, so commercial deployments must disable/replace it or obtain separate permission; GDELT permits commercial dataset use with citation. The Data attribution popover identifies the active headline sources; article links retain publisher attribution. Headlines are location-query matches, not verified incidents, risk rankings, or evidence that a location is safe. Empty, partial, stale, and unavailable source states remain distinct. Open-Meteo supplies current conditions independently of the news source. `WX OFF` disables cockpit weather rendering only; the Local Info briefing still fetches its source-backed weather values and displays the required linked Open-Meteo credit.
- **Dynamic weather presentation.** While cockpit mode is active, `/api/weather-effects` requests current Open-Meteo observations for the aircraft/camera location, rounds coordinates into 0.1° cache cells, caches results for five minutes, and may retain a stale observation for up to 30 minutes during a transient outage. WMO condition code selects the visual family; observed cloud cover, precipitation, visibility, wind speed, and wind direction bound its strength and motion. Missing or expired weather renders no synthetic atmospheric effect, and normal globe view never renders the weather overlay.
@@ -119,8 +118,6 @@ The [Bhote Koshi event pack](public/events/bhote-koshi-2026/README.md), under `p
| **DataSF Analysis Neighborhoods** (41 SF neighborhood polygons) | `neighborhoods/` | **PDDL 1.0** (public domain) | ✅ (no restrictions) | "City & County of San Francisco — DataSF" (courtesy — not legally required) |
| **Natural Earth countries, states and provinces** (260 admin-0/map units + 4,587 admin-1 polygons) | `natural_earth/` | **Public domain** | ✅ (no restrictions) | "Made with Natural Earth" (courtesy credit — not legally required) |
| **US Census Bureau counties** (3,235 county polygons) | `us_census_counties/` | **Public domain** | ✅ (no restrictions) | "U.S. Census Bureau" (courtesy — not legally required) |
| **US Census Bureau places** (32,629 city, town and CDP polygons) | `us_census_places/` | **Public domain** | ✅ (no restrictions) | "U.S. Census Bureau" (courtesy — not legally required) |
| **Who's On First neighborhoods** (61,755 polygons) | `wof_neighborhoods/` | Per source: CC BY, ODC-By, Open Government licences, CC0, PDDL, public domain (folder `ATTRIBUTION.md`) | ✅ (attribution) | "Neighborhood boundaries: Who's On First and contributors"; per-source credits in the folder |
| **CCTV ground heights** (3,445 cameras) | `cctv_ground_heights/` | Precomputed camera placement heights, aligned to work with Google Photorealistic 3D Tiles (folder README) | — | — |
### ⚠️ TeleGeography is bundled but NonCommercial
+3 -1
View File
@@ -279,6 +279,8 @@ _Ask for radio near anywhere and the globe starts broadcasting — every station
---
**💬 Or ask from Claude and Codex.** God's Eye View is also an MCP server: ask Claude Desktop, Codex or ChatGPT desktop to show a place, and the live globe opens right in the conversation. Setup takes a minute — see [docs/MCP_SETUP.md](docs/MCP_SETUP.md).
## 🛰️ What's on the Globe
Nineteen layers and map sources. **Seventeen have a keyless path.** Some offer additional capabilities with a provider key. (🟢 no key · 🟡 free key · 🔴 metered.)
@@ -510,7 +512,7 @@ Everything above is the deliberately cheap baseline — enough to get a real tas
### 🔒 Sharing an instance
By default nobody else can reach your server — it binds to localhost. To share on your LAN, opt in explicitly (`npm run dev -- --host 0.0.0.0 --port 4173`, or `HOST=0.0.0.0 ./scripts/dev-fresh.sh` on macOS/Linux) — but know that ⚠️ **a LAN-visible server brokers your configured API keys to anyone who can reach it.** Set the per-IP throttles (`GEV_RATELIMIT_OPENAI_PER_MIN`, `GEV_RATELIMIT_GOOGLE_PER_MIN` — see `.env.example`) and, before anything else, **configure provider quotas, usage limits, and billing alerts**: app-level throttles are not billing caps, and a budget alert alone does not stop spending. Full threat model in [SECURITY.md](SECURITY.md).
By default nobody else can reach your server — it binds to localhost. To share on your LAN, opt in explicitly (`npm run dev -- --host 0.0.0.0 --port 4173`, or `HOST=0.0.0.0 ./scripts/dev-fresh.sh` on macOS/Linux). IP addresses work as they are; if you use a LAN hostname, add it with `GEV_ALLOWED_HOSTS=globe.lan`, since binding to `0.0.0.0` does not trust arbitrary Host headers. But know that ⚠️ **a LAN-visible server brokers your configured API keys to anyone who can reach it.** Per-IP throttles on the cost-bearing endpoints are on by default (`GEV_RATELIMIT_OPENAI_PER_MIN`, `GEV_RATELIMIT_GOOGLE_PER_MIN` — see `.env.example` to tune them), but before anything else, **configure provider quotas, usage limits, and billing alerts**: app-level throttles are not billing caps, and a budget alert alone does not stop spending. Full threat model in [SECURITY.md](SECURITY.md).
Provider Settings is disabled when the server is shared, so remote users cannot
access the key-entry panel.
+13 -2
View File
@@ -71,14 +71,16 @@ The data proxies under `server/providers/` are written so the browser cannot tur
- **Sanitized errors** — internal error details are not echoed back to clients.
- **Coalesced OAuth refresh** and cached successful responses only (OpenSky).
- **Redacted debug logging.** The voice debug log (`.gev-logs/`, gitignored) strips API keys, bearer tokens, client secrets, and image data URLs before writing.
- **Cross-site gate on cost/log endpoints.** The cost-bearing (`/api/realtime/token`, `/api/openai/hud-summary`, `/api/google/nearby-places`, `/api/google/text-search`) and log (`/api/realtime/debug-log`) endpoints refuse cross-site browser requests: a foreign or opaque `Origin`, a `Sec-Fetch-Site` other than `same-origin`/`none` (this is what blocks an `<img>`/navigation that carries no `Origin`), or any reverse-proxy/CDN forwarding header returns `403`. Non-browser loopback tools (the repo QA harnesses POST the token endpoint with no `Origin`) and the explicit `HOST=0.0.0.0` LAN opt-in keep working — the gate deliberately does not require a loopback remote address; that stricter requirement belongs only to the credential panel (`admitKeySetupRequest`). The Host check (`allowedHosts`, applied by `hostCheckPlugin` ahead of every route, since Vite's own check runs only after plugin middleware) rejects a rebinding `Host` before any handler runs, in every mode, and the gate is defense in depth behind it: the LAN opt-in keeps that check and accepts only IP addresses, `localhost` and names listed in `GEV_ALLOWED_HOSTS`. **Limit:** a name you list there is trusted like the app's own address, so list only names you control.
- **Full Content-Security-Policy.** The dev and preview servers send a real CSP (`script-src 'self' 'unsafe-eval' blob:` plus the YouTube, Facebook and X scripts the Nepal event's embedded media loads, with a matching `frame-src` for their players and posts: no inline script and no other script origin; `blob:` covers the object URLs Cesium's bundled workers bootstrap through in the built app), alongside the existing `X-Frame-Options: DENY` and `frame-ancestors 'none'`. `'unsafe-eval'` stays because Knockout, bundled inside Cesium's widgets package, resolves the global object with `eval` at load time and the globe does not initialize without it; styles permit `'unsafe-inline'` and Google Fonts because Vite's dev client injects `<style>` elements.
## Network exposure — the operator threat model
The dev server is a **key broker**: every server-side key above is spendable by anyone who can send HTTP requests to it. That shapes the defaults:
- **Local-only by default.** `./scripts/dev-fresh.sh` (and the Vite config itself) bind to `localhost`, so only your machine can reach the server — and only local names are accepted (`allowedHosts` stays restricted, which also blunts DNS-rebinding tricks).
- **Local-only by default.** `./scripts/dev-fresh.sh` (and the Vite config itself) bind to `localhost`, so only your machine can reach the server — and only local names are accepted (`allowedHosts` stays restricted, which also blunts DNS-rebinding tricks). Binding to all interfaces keeps that restriction: IP addresses and `localhost` work, and LAN hostnames must be listed in `GEV_ALLOWED_HOSTS` (for example `GEV_ALLOWED_HOSTS=globe.lan`); suffix and wildcard entries are ignored.
- **LAN exposure is an explicit opt-in**: `HOST=0.0.0.0 ./scripts/dev-fresh.sh`. The launcher prints a prominent warning plus your LAN URL. Understand what opting in means: **every device on that network can drive the proxies and spend your OpenAI / Google / OpenSky / AISStream / TomTom / FIRMS quota** for as long as the server runs. Do this only on networks you trust.
- **App-level throttles (opt-in):** `GEV_RATELIMIT_OPENAI_PER_MIN` and `GEV_RATELIMIT_GOOGLE_PER_MIN` cap the cost-bearing endpoints per client IP per minute (over-limit requests receive a sanitized `429`). They are **per-IP, process-local, in-memory guards** — they reset on restart and are **not billing caps**.
- **App-level throttles (on by default):** `GEV_RATELIMIT_OPENAI_PER_MIN` (default `30`) and `GEV_RATELIMIT_GOOGLE_PER_MIN` (default `120`) — the values the Pinokio build already ships — cap the cost-bearing endpoints per client IP per minute (over-limit requests receive a sanitized `429`). Both defaults sit well above what the app itself generates; set either to exactly `0` to run unthrottled. They are **per-IP, process-local, in-memory guards** — they reset on restart, a caller with several addresses gets a bucket per address, and they are **not billing caps**.
- **Provider-side budgets are the real backstop.** For hard spend protection, configure limits where the money is: OpenAI platform usage limits, Google Cloud budget alerts + per-API quotas, and equivalent controls for any other keyed provider.
- **Pinokio LAN and Cloudflare sharing are refused.** The current supported
Pinokio release re-reads sharing state when an app registers its Open URL and
@@ -90,6 +92,15 @@ The dev server is a **key broker**: every server-side key above is spendable by
loopback only. Use a separately reviewed authentication proxy for remote
access and keep provider-side quotas as the spend backstop.
## MCP server and panel
God's Eye View's tools are also served to MCP clients: over stdio (`npm run mcp`) and at `/mcp` on the development and preview servers.
- **`/mcp` answers only direct local requests**: a loopback connection naming a loopback host on the port it reached, a browser `Origin` (when sent) from that same host, no proxy forwarding headers, and launcher sharing off. It accepts only JSON, so a web page cannot post to it cross-site. This is transport safety, not authentication: any program on your machine can use the tools, and some of them spend the same provider quotas as the app.
- **What the panel's requests can reach.** In clients that display MCP Apps, the panel loads the app through the `panel_request` tool, which requests paths on the app's server. It is marked for the panel only and requires a key that each MCP server puts in its panel page, which keeps it from clients that list tools to the model without loading the panel. That key is not access control: any MCP client can read the panel page, key included, and then reach the app's files and data routes through `panel_request`, much as its other tools reach the data. Provider Settings, credential and model endpoints, `/mcp` and the development server's internal routes are refused to every caller.
- **The panel carries the two browser keys.** The panel build (`npm run build:panel`) includes the same Google Maps key and Cesium ion token as the app, and they run inside the client's panel page, on the client's site. A key restricted by HTTP referrer must also allow that page, or the panel cannot load tiles. To keep the app's own keys narrower, build the panel with separately restricted keys in the environment of `npm run build:panel`.
- **Embed mode is not framable by default.** `?embed=1` documents keep `X-Frame-Options: DENY` and `frame-ancestors 'none'` unless `GEV_EMBED_FRAME_ANCESTORS` names the pages that may frame them; a framing page can change what the app shows. Other documents can never be framed.
## Scope & expectations
- The Vite server is a **development/preview** server. If you expose it beyond localhost, put it behind your own auth/proxy and review the bindings (see the threat model above).
+94
View File
@@ -0,0 +1,94 @@
/**
* Host header allowlist for the Vite dev/preview server. The configuration
* factory passes the configured names in; this module reads no environment.
*
* Binding to a wildcard interface only controls which network interfaces accept
* connections. It must not turn into a wildcard Host-header policy: this server
* brokers configured API keys through local proxy routes.
*/
import { isIP } from 'node:net';
export const DEFAULT_ALLOWED_HOSTS = Object.freeze(['localhost', '127.0.0.1']);
/**
* Resolve Vite's allowedHosts setting from an optional comma-separated list.
* Empty entries and suffix/wildcard entries are ignored; repeated names keep
* their first occurrence. Vite treats leading-dot values as suffix wildcards,
* which would defeat an explicit host policy.
*
* @param {string|undefined|null} configuredHosts
* @returns {string[]}
*/
export function resolveAllowedHosts(configuredHosts) {
const hosts = String(configuredHosts ?? '')
.split(',')
.map((host) => host.trim())
.filter((host) => host && !host.startsWith('.') && !host.includes('*'));
return [...new Set([...DEFAULT_ALLOWED_HOSTS, ...hosts])];
}
/**
* Whether a Host header names a host the server answers, by Vite's own rule:
* IP addresses, `localhost` and `*.localhost` always; then `additional` names
* (the configured server host) and `allowedHosts` (a leading dot allows a
* domain and its subdomains). `allowedHosts === true` allows every host.
*
* @param {string|undefined} hostHeader
* @param {string[]|true} allowedHosts
* @param {string[]} [additional]
* @returns {boolean}
*/
export function isAllowedHost(hostHeader, allowedHosts, additional = []) {
if (allowedHosts === true) return true;
const host = String(hostHeader ?? '').trim();
if (!host) return false;
if (host[0] === '[') {
const end = host.indexOf(']');
return end > 0 && isIP(host.slice(1, end)) === 6;
}
const colon = host.indexOf(':');
const hostname = (colon === -1 ? host : host.slice(0, colon)).toLowerCase();
if (isIP(hostname) === 4) return true;
if (hostname === 'localhost' || hostname.endsWith('.localhost')) return true;
if (additional.includes(hostname)) return true;
return (allowedHosts || []).some(
(allowed) =>
allowed === hostname ||
(allowed[0] === '.' &&
(allowed.slice(1) === hostname || hostname.endsWith(allowed))),
);
}
/**
* Vite plugin that applies the Host check before every other middleware.
* Vite installs its own check after the middleware plugins add in
* `configureServer`, so without this the app's `/api` routes would answer any
* Host, including a DNS-rebinding name. It runs first (`enforce: 'pre'`) and
* uses the same allowed hosts as Vite, on the dev and preview servers.
*/
export function hostCheckPlugin() {
const middleware = (config, preview) => {
const allowed = preview
? config.preview.allowedHosts
: config.server.allowedHosts;
const additional = [config.server.host, config.preview.host].filter(
(value) => typeof value === 'string' && value,
);
return (req, res, next) => {
if (isAllowedHost(req.headers.host, allowed, additional)) return next();
res.writeHead(403, { 'Content-Type': 'text/plain' });
res.end('Blocked request. This host is not allowed.');
};
};
return {
name: 'host-check',
enforce: 'pre',
configureServer(server) {
server.middlewares.use(middleware(server.config, false));
},
configurePreviewServer(server) {
server.middlewares.use(middleware(server.config, true));
},
};
}
+73
View File
@@ -0,0 +1,73 @@
/**
* Lets chosen pages frame the app in embed mode (`?embed=1`). Every
* document keeps the server's `X-Frame-Options: DENY` and
* `frame-ancestors 'none'` unless `GEV_EMBED_FRAME_ANCESTORS` names who may
* frame embed-mode documents: CSP frame-ancestors sources, or `*` for any
* page. A framing page can change what the app shows, so no page may frame
* it by default. Other documents, including Provider Settings, are never
* framable. "Any" sends no framing restriction at all: `frame-ancestors *`
* would still refuse sandboxed frames with an opaque origin.
*/
const FRAMING_HEADERS = new Set(['x-frame-options', 'content-security-policy']);
/** The directives of a CSP header value, minus any frame-ancestors rule. */
function withoutFraming(value) {
return String(value ?? '')
.split(';')
.map((directive) => directive.trim())
.filter(Boolean)
.filter((directive) => !/^frame-ancestors\b/i.test(directive));
}
/** Whether a request is for an embed-mode document. */
export function isEmbedDocumentRequest(url) {
let parsed;
try {
parsed = new URL(url || '/', 'http://localhost');
} catch {
return false;
}
return (
parsed.searchParams.get('embed') === '1' &&
(parsed.pathname === '/' || parsed.pathname.endsWith('.html'))
);
}
/** Vite plugin applying the embed framing policy on dev and preview servers. */
export function embedFramingPlugin({
ancestors = process.env.GEV_EMBED_FRAME_ANCESTORS || '',
} = {}) {
const allowed = ancestors.trim();
// Unset: embed documents keep the server's framing protection.
if (!allowed) return { name: 'embed-framing' };
const anywhere = allowed === '*';
const policy = `frame-ancestors ${allowed}`;
const install = (server) => {
server.middlewares.use((req, res, next) => {
if (!isEmbedDocumentRequest(req.url)) return next();
// The server's own headers are written when the response is sent, so
// intercept them rather than setting ours first.
const setHeader = res.setHeader.bind(res);
res.setHeader = (name, value) => {
const key = String(name).toLowerCase();
if (!FRAMING_HEADERS.has(key)) return setHeader(name, value);
if (key === 'content-security-policy') {
// Keep every other directive of the server's policy (script-src,
// connect-src, ...) and only swap the framing rule.
const kept = withoutFraming(value);
const next = anywhere ? kept : [...kept, policy];
return next.length ? setHeader(name, next.join('; ')) : res;
}
return res;
};
if (!anywhere) setHeader('Content-Security-Policy', policy);
next();
});
};
return {
name: 'embed-framing',
configureServer: install,
configurePreviewServer: install,
};
}
+153
View File
@@ -0,0 +1,153 @@
/**
* The MCP Apps panel build. A panel loads the app through its MCP server,
* one tool call per file (see src/tools/globePanel.js), so the app must be
* few files: one app script, one stylesheet, and Cesium's own script, which
* carries its workers and starts them from memory. The build is served at
* PANEL_BASE next to the app.
*/
// The panel loads these paths (src/tools/globePanel.js); a test keeps the
// two in step.
export const PANEL_BASE = '/panel/';
/** The script the panel runs ahead of Cesium's workers (workerFilesPrelude). */
export const PANEL_WORKER_PRELUDE_PATH = 'cesium/worker-prelude.js';
export const PANEL_OUT_DIR = 'dist/panel';
/** A browser Vite config changed to produce the panel build. */
export function panelBuildConfig(config) {
return {
...config,
base: PANEL_BASE,
build: {
...config.build,
outDir: PANEL_OUT_DIR,
emptyOutDir: true,
modulePreload: false,
cssCodeSplit: false,
rollupOptions: {
...config.build?.rollupOptions,
output: { inlineDynamicImports: true },
},
},
};
}
const CONTENT_TYPES = {
'.css': 'text/css',
'.geojsonl': 'application/geo+json-seq',
'.glb': 'model/gltf-binary',
'.html': 'text/html',
'.jpg': 'image/jpeg',
'.js': 'text/javascript',
'.json': 'application/json',
'.png': 'image/png',
'.svg': 'image/svg+xml',
'.wasm': 'application/wasm',
'.woff2': 'font/woff2',
'.xml': 'application/xml',
};
/**
* Vite plugin serving the panel build at PANEL_BASE on the dev server, as
* the preview server and production builds serve it from the build output.
*/
export function panelBuildPlugin({ outDir = PANEL_OUT_DIR } = {}) {
return {
name: 'panel-build',
configureServer(server) {
server.middlewares.use(async (req, res, next) => {
const { pathname } = new URL(req.url || '/', 'http://localhost');
if (!pathname.startsWith(PANEL_BASE)) return next();
const { readFile } = await import('node:fs/promises');
const { extname, resolve, sep } = await import('node:path');
const root = resolve(server.config.root, outDir);
let relative;
try {
relative = decodeURIComponent(pathname.slice(PANEL_BASE.length));
} catch {
res.statusCode = 400;
res.end('Bad path');
return;
}
const file = resolve(root, relative || 'index.html');
if (file !== root && !file.startsWith(root + sep)) return next();
try {
const body = await readFile(file);
res.setHeader(
'Content-Type',
CONTENT_TYPES[extname(file)] || 'application/octet-stream',
);
res.end(body);
} catch {
res.statusCode = 404;
res.end(
relative ? 'Not found' : 'No panel build; run npm run build:panel',
);
}
});
},
};
}
/**
* Cesium files its workers load themselves, as paths under Cesium's base.
* Workers request them from the panel page's own site, which has none of
* the app's files, so the panel build embeds them in the workers script.
*/
export const PANEL_WORKER_FILES = Object.freeze([
'Assets/approximateTerrainHeights.json',
]);
/**
* A script, run in each worker before Cesium's, that answers the worker's
* requests for `files` (Cesium-relative path → text) from memory and lets
* every other request through.
*/
export function workerFilesPrelude(files) {
const answer = (FILES) => {
const find = (url) => {
const path = String(url).split(/[?#]/)[0];
for (const name of Object.keys(FILES))
if (path.endsWith(`/${name}`)) return FILES[name];
return null;
};
const xhr = XMLHttpRequest.prototype;
const { open, send, setRequestHeader } = xhr;
xhr.open = function (method, url, ...rest) {
this.__gevFile = find(url);
if (this.__gevFile === null) return open.call(this, method, url, ...rest);
};
xhr.setRequestHeader = function (...args) {
if (this.__gevFile === null) return setRequestHeader.apply(this, args);
};
xhr.send = function (body) {
const text = this.__gevFile;
if (text === null || text === undefined) return send.call(this, body);
const type = this.responseType;
const response =
type === 'json'
? JSON.parse(text)
: type === 'arraybuffer'
? new TextEncoder().encode(text).buffer
: text;
const state = {
readyState: 4,
status: 200,
statusText: 'OK',
response,
responseText: typeof response === 'string' ? response : '',
getAllResponseHeaders: () => 'content-type: application/json\r\n',
getResponseHeader: (name) =>
name.toLowerCase() === 'content-type' ? 'application/json' : null,
};
for (const [name, value] of Object.entries(state))
Object.defineProperty(this, name, { configurable: true, value });
setTimeout(() => {
this.dispatchEvent(new Event('readystatechange'));
this.dispatchEvent(new ProgressEvent('load'));
this.dispatchEvent(new ProgressEvent('loadend'));
});
};
};
return `(${answer})(${JSON.stringify(files)});\n`;
}
+67 -15
View File
@@ -1,5 +1,50 @@
import { applicationHtmlPlugin } from './application-html.js';
import cesium from 'vite-plugin-cesium';
import { DEFAULT_ALLOWED_HOSTS, hostCheckPlugin } from './allowedHosts.js';
import { embedFramingPlugin } from './embed-framing.js';
import { panelBuildPlugin } from './panel.js';
/**
* Content-Security-Policy for every document the dev/preview server serves.
* No inline script is permitted, and the only foreign script origins are the
* three that the Bhote Koshi event's embedded media needs (YouTube, Facebook
* and X), which frame-src matches for their players and posts. 'unsafe-eval' is
* required: Knockout (bundled inside @cesium/widgets) resolves the global
* object with `(0, eval)("this")` at module load, and without it the Cesium
* widget never initializes (verified in headless Chrome). It also covers
* Cesium's WASM decoders. `blob:` is required in the built app: Cesium's
* bundled workers bootstrap through `importScripts(blob:...)`, which a worker
* checks against script-src (the dev server loads them by URL instead).
* Widen any other directive only for a real violation.
*/
export const BROWSER_CSP = [
"default-src 'self'",
// The Bhote Koshi event's embedded media: the YouTube player API, the
// Facebook video SDK and X's post widget (src/data/bhoteKoshiEmbeddedMedia.js).
"script-src 'self' 'unsafe-eval' blob: https://www.youtube.com https://connect.facebook.net https://platform.twitter.com",
"style-src 'self' 'unsafe-inline' https://fonts.googleapis.com",
"font-src 'self' data: https://fonts.gstatic.com",
"img-src 'self' data: blob: https:",
"media-src 'self' blob: https:",
"connect-src 'self' blob: data: https: wss: ws:",
"worker-src 'self' blob:",
"child-src 'self' blob:",
// The same media's players and posts, which load in frames.
"frame-src 'self' blob: https://www.youtube-nocookie.com https://www.youtube.com https://www.facebook.com https://platform.twitter.com",
"manifest-src 'self'",
"object-src 'none'",
"base-uri 'self'",
"form-action 'self'",
"frame-ancestors 'none'",
].join('; ');
/** Response headers shared by the dev and preview servers. */
export const BROWSER_HEADERS = Object.freeze({
'X-Frame-Options': 'DENY',
'Content-Security-Policy': BROWSER_CSP,
// Responses are only ever used as the type the server declares.
'X-Content-Type-Options': 'nosniff',
});
/** Build browser assets with explicit inputs; never load environment or providers. */
export function createBrowserViteConfig({
@@ -7,13 +52,21 @@ export function createBrowserViteConfig({
publicDir,
googleApiKey,
cesiumToken,
photonUrl,
host = 'localhost',
port = 4173,
allowedHosts = DEFAULT_ALLOWED_HOSTS,
command,
} = {}) {
return {
plugins: [cesium(), applicationHtmlPlugin(), ...plugins],
plugins: [
// First, so no provider route answers a Host the server does not allow.
hostCheckPlugin(),
cesium(),
applicationHtmlPlugin(),
...plugins,
embedFramingPlugin(),
panelBuildPlugin(),
],
...(publicDir === undefined ? {} : { publicDir }),
// A production build must not clean the dependency cache a running dev
// server is still serving optimized module URLs from.
@@ -32,26 +85,25 @@ export function createBrowserViteConfig({
server: {
host: host || 'localhost',
port: parseInt(port, 10) || 4173,
allowedHosts:
host === '0.0.0.0' || host === '::'
? true
: ['localhost', '127.0.0.1', '.local'],
// A wildcard bind only chooses interfaces; it never disables Vite's
// Host-header check. LAN hostnames are added explicitly (see
// build/allowedHosts.js); IP addresses are always accepted.
allowedHosts: [...allowedHosts],
fs: {
deny: ['.env', '.env.*', '*.{crt,pem}', '**/.git/**', '**/ENVIRONMENT'],
},
// These headers protect the document containing Provider Settings.
headers: {
'X-Frame-Options': 'DENY',
'Content-Security-Policy': "frame-ancestors 'none'",
},
// These headers protect the document containing Provider Settings and
// give the whole page a real Content-Security-Policy (BROWSER_CSP).
// Embed-mode documents are framable instead: embed-framing.js rewrites
// only the frame-ancestors directive and keeps the rest of the policy.
headers: BROWSER_HEADERS,
},
// The preview server serves the same documents, so it carries the same
// framing + CSP hardening (one constant, no drift).
preview: { headers: BROWSER_HEADERS },
define: {
'import.meta.env.GOOGLE_MAPS_API_KEY': JSON.stringify(googleApiKey),
'import.meta.env.CESIUM_ION_TOKEN': JSON.stringify(cesiumToken),
// Only when set: '' disables Photon in the browser, a URL replaces it.
...(photonUrl === undefined
? {}
: { 'import.meta.env.PHOTON_URL': JSON.stringify(String(photonUrl)) }),
},
build: { chunkSizeWarningLimit: 1500 },
};
+87 -95
View File
@@ -1,10 +1,51 @@
# God's Eye View Current State
## Cyber HUD — September 23, 2026
## God's Eye View in conversations — October 2, 2026
Tool answers that can be shown in God's Eye View include a view: camera, layers,
style, map, marks and an aircraft or satellite to follow, written in the
share-link format (`gods-eye-view/view`). `show_in_gods_eye_view` shows a view as live
God's Eye View inside clients that display MCP Apps, and as a link everywhere.
The panel runs the app's panel build (`npm run build:panel`, served at
`/panel/`) and loads it, with its data, through the MCP server, so it works
in Claude Desktop and in the Codex and ChatGPT desktop apps with a local
server and no HTTPS. `?embed=1` shows the app as the globe alone and takes
new views from the page that frames it; framing is off unless
`GEV_EMBED_FRAME_ANCESTORS` allows the framing page. MCP leads with the tools that find
what to show. See [tools and the MCP server](TOOLS.md).
## Tools and local MCP server — October 1, 2026
`gods-eye-view/tools` defines queries that answer questions from the app's data,
and `gods-eye-view/tools/mcp` exposes a composed catalog over the Model Context
Protocol. `npm run mcp` serves Core's tools over stdio to a local MCP client,
reading from a running app's `/api` routes (default `http://localhost:4173`).
The development and preview servers also serve the tools over HTTP at `/mcp`,
accepting only direct local requests: a loopback host on the port reached,
an `Origin` (when sent) from that same host, no proxy forwarding headers, and
launcher sharing off.
Queries cover earthquakes, active fires, recent launches, aircraft (in an
area, by identifier, tracks, and type and route lookups), ships (in an area,
by identifier, and tracks), satellites (next
pass over a point, and those overhead now), public cameras (including a
camera's current image), license plate reader cameras, radio stations, place search, routing, bike-share
stations, transit vehicles, road traffic flow, weather, weather map images
(radar, satellite, lightning), wind, the most recent satellite image of an
area, submarine cables, datacenters and dams, the Bhote Koshi flood event pack, regional briefs, tropical cyclones, fire
perimeters, terrain height, military installations and map features, plus a
combined situation brief, military awareness around a point, the app's heads-up display caption, and a link that
opens the app over an area. Tools reuse the layers' portable source factories, take a
shared `area` argument (place name, bounding box, or point and radius) and cap
lists at 25 rows by default. Voice offers the same queries next to its app
actions: the session lists them, and the browser runs them through the same
catalog, loaded on first use. See [tools and the MCP server](TOOLS.md).
## Cyber HUD — September 30, 2026
Display > HUD > Layout includes Cyber, also available through the HUD voice
action and shared visual state. An explicit first transition into Cyber selects
FLIR with Ironbow 0.42; restored links and subsequent visual tuning remain
FLIR with Ironbow 0.42. An explicit switch back to another HUD layout restores
the visual preset used before Cyber; restored links and scenes remain
authoritative. The skin uses shared red/slate panel treatments in map and cockpit,
with compact 200px collapsed controls and wider expanded panels. Other HUD
layouts retain their existing presentation.
@@ -17,6 +58,21 @@ headers and decorative frames. The narrow-screen rail remains scrollable to reac
each panel. Radio retains the shared nested Context player and compact
header disclosure, without relocating playback controls on theme changes.
Voice help/error popups and Location/Visual Presets pins extend beyond their
Cyber frames without being clipped: the shapes decorate non-interactive
pseudo-elements. The lower-left telemetry card leaves room for attribution's
full logo row. New panels can use the [shared surface contract](panel-surfaces.md)
for theme tokens, rail input and a fixed header with a bounded scroll body.
The contract is intentionally future-facing and opt-in; its first production
adopter will land after this change. Existing owners still control disclosure,
persistence and placement, while small-screen rail popup clipping and other
integration limits remain documented in the contract.
On desktop, Cyber raises the left rail so it clears the lower coordinate card.
`layoutRightPanelRail` reads that rendered top as its measured `baseTop`, which
keeps the right rail aligned. Cockpit retains its separate rail placement for
its visor layout.
Sonar has one contact-highlighting method. Native Cesium points, billboards and
labels are treated in GPU draw commands without replacing their positions,
identities or pick commands. Model and canvas/DOM overlays retain their own
@@ -222,7 +278,7 @@ Consult the linked official advisory for safety decisions.
Voice and HUD snapshots reuse the existing feedState classifier. Analyst follow-ups retain their original data provenance; current-view results append provenance without replacing legacy fields. HUD context and deterministic telemetry include non-nominal feed state.
Analyst records come on demand from each layer's `getAnalystRecords`: flights, military, local ADS-B, vessels, satellites, launches, earthquakes, FIRMS (VIIRS and MODIS, with a `sensor` field), fire perimeters, cyclones, transit, bikeshare, ALPR cameras, mapped installations, datacenters and dams. `analyst_query` counts the whole loaded set (up to 250,000 records per layer; past that `coverage.records` reports `returned`, `total` and `truncated: true` and the result carries `complete: false`). Ranking keeps a top-k and scans in 20,000-row slices that yield to the page and stop when superseded. It refuses unknown fields, operators, units that do not convert (`altitudeM(ft)` converts), wrong-typed values, invalid centres, unknown scope kinds and sort fields with `code` and `allowed`, refuses a follow-up with no previous answer (`NO_RESULT_CONTEXT`) or naming other layers than that answer (`FOLLOW_UP_MISMATCH`), and tells `LAYER_OFF`, `NOT_READY` and `FEED_UNAVAILABLE` apart from a real zero; when only some layers can answer the result is `partial` with `unanswered` ids. The Contacts-window path applies the same filters, ranking and item shape to one immutable panel snapshot. Each aircraft layer retains at most 20,000 rows; reaching that bound sets `complete: false` and the spoken/card count becomes “At least”, never an exact claim. A superseded query never replaces follow-up memory, while sibling analyst calls from one assistant response do not cancel one another. Items carry `lat`/`lon`; the feed window (earthquakes and FIRMS: last 24 h), the precise scope and any caveat sit in `display`, which is on-screen detail rather than something the model recites. "In view" is still a radius around the view centre, not the visible footprint. Satellite distance is ground distance.
Satellite and local infrastructure layers expose on-demand analyst records through their current factory owners. Analyst counts and ranks explicitly cover only bounded examined loaded records (default 2,000 per new layer, core satellite rows before dense extras); omitted records can change nearest/count and satellite distance is ground distance. Existing tools and result fields remain available.
Keyless terrain tiles retry when Re:Earth throttles them. The browser fetches
`terrain.reearth.land/cesium-mesh/ellipsoid/{z}/{x}/{y}.terrain` directly; no
@@ -297,48 +353,6 @@ replacement session; audio meters release failed initialization and reject revok
frames. Delayed action results and post-capture continuations cannot resume a
stopped conversation or send output into a replacement. See [voice ownership](VOICE-OWNERSHIP.md).
Voice speech: `src/voice/speech.js` adds `{ say, display, referents?, schedule? }`
to layer toggles, fly_to_location, frame_overhead, select_nearest_aircraft,
get_entity_context, annotate_map and analyst_query. Analyst speech and the card
share one deterministic count/scope headline; lower bounds use "At least" and
partial answers name the unanswered layers. `get_current_view_state` lists enabled layers only.
The voice card nested in the mic control (`voiceCard.js`) shows captions, plan
steps and the result. `resultDisplay.js` is the one adapter from a result to
the card: builder results show as built, and analyst results show a title
with their scope, window and caveats. Caveats are on the card; the model speaks
one only when it changes the answer, and always says lower bounds and partial
answers. `narration.js` speaks one
code-authored line for the current step of a tool still running at 1.5 s
(out of band, never after the result). Tools that report steps: layer
toggles, place lookups, select_nearest_aircraft and annotate_map (finding,
outlining). Other tools get one "still working" line at 8 s. The
earcon plays in push-to-talk only. A claimed Space hold while the assistant
speaks is a barge-in (WebRTC cancel + audio clear), refused while Radio holds
the speaker (`mayVoiceClaimSpeaker()`). Progress lines stay outside the reply
lifecycle and Radio handoff. `turn.span` debug records carry generation and
playback latency, silence, preamble and word-count figures. Caption
transcription (`OPENAI_REALTIME_TRANSCRIBE_MODEL`, default on) is metered into
the session cost and cap. The debug log omits transcripts and text unless
`GEV_VOICE_LOG_CONTENT=1`.
Point and ask: `pointerContext.js` reads what the cursor is on. A Space hold
is the pointing gesture: its keydown snapshot (released with the key) is sent
as an inert `pointer_context` item with the target, with reticle and chip.
Open-mic speech and typed turns count only a pointer that moved on the map in
the last 5 s and send only the kind (`pointing:"aircraft"`/`"ground"`); the chip appears when a tool
resolves it. Losing the pointer sends one `target:"none"` item. `deixis.js` resolves `'pointer'` in
track_entity/fly_to_location queries, get_entity_context scope, analyst_query
`scope.kind` and annotate_map targets, plus `referent:n` from the latest
numbered result (`referents.js`, cleared at session end). A new result set (an analyst count) replaces
the numbered list even when it is empty. Only the five rows numbered on the
card are resolvable; `-1` means the fifth/last visible row. `track_entity`
accepts either a named query or a referent, while an empty target is rejected. A reticle marks the
point during the turn and the voice card shows "THIS"/"HERE"; both hide in
Clean-UI and recording. A ground-only pointer lookup at local/city scale sends a ringed 512 px crop in
the one retained image slot, dropped if the camera moved or a newer capture or
turn started; screenX/screenY against the crop are mapped back to the canvas. Harness: `qa-voice-routing.mjs --layer deixis`,
`--layer deixis-live`, `--layer behavior --deixis-only`.
The application shell composes focused state owners for navigation, destination
lookup/orbit, Cockpit, visual settings, panel layout, aircraft display and layer
bindings. Keyboard/display subscriptions have a separate lifetime; existing
@@ -405,11 +419,9 @@ Reference feed construction is exported through `sources/reference`; the cable s
Source factories have dedicated `layers/<family>/source` exports. ALPR and earthquake record normalization and CCTV source policy no longer pull rendering into source consumers. Catalog construction and voice feed reads use their focused owners; compatibility entries remain available. Settings filesystem hardening lives under `server/standalone/`. Import-direction checks complement export ownership checks; source behavior, settings policy and rendering are unchanged.
Voice controls bind to a protocol-independent session factory. The default WebRTC adapter preserves the existing Realtime connection, push-to-talk, cost controls and radio handoff. Session subscriptions expose state, transcript, action call/progress/result, interruption, completion and turn-metrics events; stopping or replacing a session cancels pending actions. Alternate adapters can use the same controls and action runner.
Voice controls bind to a protocol-independent session factory. The default WebRTC adapter preserves the existing Realtime connection, push-to-talk, cost controls and radio handoff. Session subscriptions expose state, transcript, action call/result, interruption and completion events; stopping or replacing a session cancels pending actions. Alternate adapters can use the same controls and action runner.
Voice action argument schemas have one portable owner under `src/voice/actionSchemas.js`. The Realtime provider builds 30 tools using separate description-only metadata. Description customization cannot replace argument types, enum values or required fields.
`src/voice/layerManifest.js` declares each catalog layer's voice capabilities: spoken aliases, `toggle`, in-view `context`, `track`, and analyst `query` fields with type and unit, or `noQuery` with the reason. The `set_layer_visibility`, `show_data_layers_menu`, `get_entity_context` and `analyst_query` layer enums, the runner's alias table and the analyst field hints are generated from it. `src/voice/layerManifest.test.mjs` fails when a catalog layer is neither listed nor named in `VOICE_OFF_LAYERS` (today only the two scene-driven Bhote Koshi layers). Global Context is listed with `toggle: false`; `set_context_mode` reaches it. `set_panel_open` also opens `weather-panel` and `recent-imagery-panel`.
Voice action argument schemas have one portable owner under `src/voice/actionSchemas.js`. The Realtime provider builds 29 tools, retaining the existing 28 names and argument contracts apart from the three additive analyst-layer enum values using separate description-only metadata. Description customization cannot replace argument types, enum values or required fields.
Portable source exports provide Radio Browser station normalization, CCTV feed types and regional records independently of HTTP middleware. Radio normalization accepts an explicit URL policy; the standalone directory retains its existing HTTPS rules. Tile coordinate validation accepts explicit zoom bounds with unchanged traffic defaults. Cache, request and rendering owners remain unchanged.
@@ -452,18 +464,7 @@ clears caches and removes map-stack listeners. Standalone setup supplies the
existing service clients. HUD summaries, regional brief, cockpit weather, location
framing and annotation boundaries receive their service instances explicitly.
Compatibility entrypoints retain defaults for direct callers; normal startup does
not change global service slots. Analyst follow-up memory belongs to its voice
runner and ends with the voice session: stopping voice invalidates writes still in
flight and forgets the last answer, the pointer and the numbered list; removing
voice disposes the runner.
Analyst result memory is committed only after any Contacts-window projection
finishes and the call is still current. Cancelled or superseded queries cannot
return success or replace follow-up memory; Contacts follow-ups filter the same
cohort that was shown, and partial/unanswered coverage remains attached when a
remembered result is narrowed.
The narrowed answer retains the remembered scope label and detail, including
the Contacts-window attribution, unless the follow-up supplies a new scope.
not change global service slots. Analyst follow-up memory belongs to its voice runner.
Application startup constructs fresh layer instances from explicit source objects.
Standalone composition selects the existing providers. Controls, launch orbit
@@ -930,10 +931,21 @@ errors.
## Places and CCTV request bounds
With a Google key configured, nearby and text search reject missing, blank,
non-numeric and out-of-range coordinates before the opt-in limiter and upstream
non-numeric and out-of-range coordinates before the per-IP limiter and upstream
request. Text search also requires a nonblank query. Keyless requests retain
their `configured: false` response.
The cost-bearing proxies are throttled per client IP without configuration:
`/api/realtime/token` and `/api/openai/hud-summary` share 30 requests per
minute per IP, `/api/google/nearby-places` and `/api/google/text-search` share
120 — the caps the Pinokio build already sets, so the packaged app is
unaffected. `GEV_RATELIMIT_OPENAI_PER_MIN` and `GEV_RATELIMIT_GOOGLE_PER_MIN` override
those; exactly `0` disables the limiter, while a value that cannot be read as a
number falls back to the default rather than to unlimited. Over-limit requests
receive a sanitized `429` with `Retry-After: 5` and never reach the provider.
The client key is the socket peer address only — a forwarded-for header is not
trusted, so a proxied deployment shares one bucket per upstream hop.
CCTV media waits at most 15 seconds for upstream response headers and returns
504 on timeout. Its timer stops when headers arrive, so live bodies can continue
streaming; a separate 30-second idle deadline then bounds the gap between
@@ -1998,11 +2010,12 @@ down | auth-failed` (plus the unchanged `missing-key`/`unsupported`) with
> registered for lifecycle and restoration but is not duplicated in Data Layers.
> Inside Cockpit, the focused summary card is titled Contact in both visible
> copy and its accessible control labels.
> The top-center Cockpit vision cycle shows the inherited map preset name
> (for example, `NOIR`) followed by CRT, NVG, FLIR, and NOIR. That inherited entry
> leaves the selected map preset unchanged inside Cockpit. NONE is not offered
> in the cycle; CRT, NVG, FLIR, and NOIR temporarily override that preset, while returning to it or exiting
> Cockpit restores the captured map style and its exact shader intensities.
> The top-center Cockpit vision cycle is the fixed, duplicate-free sequence
> Normal, CRT, NVG, FLIR, Anime, Noir, and Snow. Entry selects the map preset
> already in use, so a FLIR map opens on the existing FLIR item while
> Normal remains available exactly once. NONE is not offered. Cockpit choices
> temporarily override the map preset, while Exit Cockpit and Reset both restore
> the captured map style and its exact shader intensities.
> Selecting a Cockpit vision treatment with configurable parameters opens
> Cockpit Display and reveals those parameters through the existing right-side
> accordion; an inherited parameterless Normal preset does not force it open.
@@ -3842,8 +3855,9 @@ and unreachable upstream (502/504) separately from road geometry.
first, activates the Contacts sub-view, and returns the settled 250 km window.
Its `aircraft` count is the exact civilian-plus-military total when both feeds
can answer, or `unknown` when either component is unavailable.
- Cockpit's top vision switch cycles five rendered looks: the inherited map
style, CRT, NVG, FLIR, and Noir. There is no empty `NONE` entry.
- Cockpit's top vision switch cycles all seven map looks: Normal, CRT, NVG,
FLIR, Anime, Noir, and Snow. Entry selects the matching map style without adding
a duplicate carousel item. There is no empty `NONE` entry.
### Live AIS Vessels (June 2026)
@@ -3887,7 +3901,7 @@ and unreachable upstream (502/504) separately from road geometry.
- **Analyst → track handoff carries a key, not just a label** (2026-08-21). `analyst_query` items include `icao24`/`mmsi` alongside the display `id`, and contact lookup uses the shared tiered ranking in `src/data/contactMatch.js`: hex exact → callsign exact → registration exact → callsign prefix → registration prefix → callsign substring → registration substring. The tiers keep an exact callsign ahead of a colliding registration regardless of feed order, and registrations compare separator-insensitively (`G-ABCD`/`GABCD`, `05-8152`/`058152`, `N123AB`/`N-123AB`).
- **A typed command supersedes the turn it interrupts** (2026-08-21). `sendTextCommand` defers its `response.create` behind an active response instead of colliding with it, marks that response superseded so a late function call from it is refused rather than dispatched, and drops the old turn's queued follow-up. A refused call is still ANSWERED — a terminal `{ok:false, superseded:true}` `function_call_output` — because an unanswered `function_call` strands a pending call and deadlocks the model; the refusal creates no response of its own. A burst of typed commands coalesces into one response while keeping both conversation items.
- **Subject reconciliation across satellite catalog rebuilds** (2026-08-21). A dense↔core toggle or TLE refresh clears and repopulates the catalog, so the published subject is re-resolved against the new satrec; a subject that did NOT survive releases the slot. The per-frame refresh cannot do this itself — `_getTrackedFramePosition` returns early once the satellite has no catalog entry — so the reconcile runs at rebuild completion. An empty catalog is a rebuild in flight, not a disappearance.
- **One aircraft-proximity snapshot** (2026-08-22; hardened 2026-09-30). `collectAircraftProximityWindow` (`src/data/militaryAwareness.js`) computes the Contacts panel cohort once. The voice analyst reads the exact retained snapshot and evaluation timestamp instead of re-scanning live layers, so the panel readout and spoken count for one centre cannot cross snapshots. **Invariant: do not re-derive a proximity count anywhere else.** Each layer retains 20,000 rows and samples one row beyond the cap; a saturated cohort reports `complete:false`, `truncated:true`, and is spoken as a lower bound. They diverged before because the panel read live billboard positions while the analyst used last-fix coordinates over a 2,000-record slice — 111 on screen, 15 spoken. Explicit regions, arbitrary points, and a radius different from the panel radius deliberately keep the general record engine. Entity-centred results carry `window: {engine:'contacts-window', centeredOn, radiusKm, flights, military, aircraft, complete, evaluatedAt}`.
- **One aircraft-proximity engine** (2026-08-22). `collectAircraftProximityWindow` (`src/data/militaryAwareness.js`) is the single computation behind both the Contacts panel window and the voice analyst's entity-centred "how many nearby", so the panel readout and the spoken count for one centre are identical by construction. **Invariant: do not re-derive a proximity count anywhere else.** They diverged before because the panel read live billboard positions (20,000 cap) while the analyst used last-fix coordinates over a 2,000-record slice — 111 on screen, 15 spoken. Explicit regions and arbitrary points deliberately keep the general record engine. Entity-centred results carry `window: {engine:'contacts-window', centeredOn, radiusKm, flights, military, aircraft}`.
- **Centre precedence for nearby asks**: explicit place in the question > Contacts subject (a selected non-contact entity never silently becomes the centre) > an entity the user names > the current view, said aloud. Contacts active with no subject uses the view rather than reading an empty panel.
- **Contact-match ties break on hex ascending.** `track_entity` is a mutation fulfilling "follow that one", and the model's observed answer to a non-ok track result is to retry with guesses rather than ask, so the lookup always commits rather than returning an ambiguity. What it owes the caller is stability: hex is unique and always present, so the same query resolves to the same contact for as long as both are loaded.
- **Visual grounding**: at `local` view scale with no structured identity, the client captures the Cesium canvas (≤1200px JPEG, black-frame detection, double-render for freshness) and sends it as `input_image` with a strict "do not invent labels" instruction.
@@ -3904,8 +3918,8 @@ and unreachable upstream (502/504) separately from road geometry.
- **Reliability**: tool-call dedupe (2.5s window across call/item/args keys); response-create queueing that respects active responses and defers follow-ups when the user starts speaking; per-tool follow-up instructions so the agent confirms only what actually happened (zoom confirms only on `ok=true`).
- **Aircraft identity honesty:** “What is this aircraft?” reads callsign, operator, registration, type, and route only from the selected contact context. Missing operator, route, or type enrichment is named explicitly rather than silently omitted or inferred from the callsign.
- **Diagnostics**: every client/server event is posted to `/api/realtime/debug-log` and appended to `.gev-logs/realtime-conversations.jsonl` (gitignored) with secret/image redaction; last 30 errors persist in `localStorage` (`gev-realtime-errors`); `window.__gevVoiceCommands.getDiagnostics()` in the console.
- **Counting semantics ("near")** — a CONTRACT; new count-bearing tool work inherits it. Three honest numbers exist for one question: the Contacts cohort (250 km around the subject, what the panel shows), `analyst_query`'s count of _currently-loaded_ records for the requested scope, and the layer-wide loaded total in `coverage.layersQueried[].total`. They diverge legitimately — the flights layer loads by viewport, so after a camera dive the loaded set can hold a fraction of the cohort (field case: panel 42, analyst 8). The contract:
1. **Contacts ACTIVE** → "near / nearby / how many aircraft" means the **Contacts window** — the panel's numbers, spoken verbatim. Mechanism: `contactsWindow` (`{centeredOn, radiusKm, flights, military, vessels, complete}`), carried by both `analyst_query` and `get_current_view_state`, derived by `contactsWindowFromSnapshot()` from the same snapshot the panel renders so the two cannot drift. Voice filtering and follow-up memory use that immutable snapshot too. A cohort whose feed cannot answer reports `'unknown'`, never a confident zero; a cohort at the 20,000-row retention bound reports a lower bound.
- **Counting semantics ("near")** — a CONTRACT; new count-bearing tool work inherits it. Three honest numbers exist for one question: the Contacts cohort (250 km around the subject, what the panel shows), `analyst_query`'s count of _currently-loaded_ records for the requested scope, and the layer-wide loaded total in `coverage.layersQueried[].records`. They diverge legitimately — the flights layer loads by viewport, so after a camera dive the loaded set can hold a fraction of the cohort (field case: panel 42, analyst 8). The contract:
1. **Contacts ACTIVE** → "near / nearby / how many aircraft" means the **Contacts window** — the panel's numbers, spoken verbatim. Mechanism: `contactsWindow` (`{centeredOn, radiusKm, flights, military, vessels}`), carried by both `analyst_query` and `get_current_view_state`, derived by `contactsWindowFromSnapshot()` from the same snapshot the panel renders so the two cannot drift. A cohort whose feed cannot answer reports `'unknown'`, never a confident zero.
2. **Contacts OFF** → "nearby" means **in view**; "near \<place\>" means a radius around that place. A radius query with Contacts active and no explicit centre is centred on the **active contact**, not the camera.
3. **Every count names its scope in words** — "42 in your window", "8 in view", "about 30 within 250 km of Austin" — never a bare number. `analyst_query` returns `scopeLabel` so this is mechanical. Two different numbers with named scopes are not a contradiction.
4. **The loaded-data caveat is stated once when relevant**: counts cover loaded data, and the flights layer loads where you look (appended to `coverage.note` for radius/view scopes over viewport-loaded layers).
@@ -4002,28 +4016,6 @@ are omitted rather than framing the wrong part of the globe.
Berlin inside Brandenburg). Each pack loads on the first ask that needs it,
never at start-up; the analyst region lookup uses the same packs. Gate:
`scripts/qa-admin-outlines.mjs`.
- Bundled place and neighborhood outlines (`src/data/placeBoundaries.js`):
US Census places (`src/data/local_data/us_census_places/`, one file per
state, public domain, `scripts/build-census-places.mjs`) and Who's On First
neighborhoods (`src/data/local_data/wof_neighborhoods/`, quadtree tiles,
per-source licences, `scripts/build-wof-neighborhoods.py`). After the admin
packs, an area ask named at or near the view is drawn from, in order, a
Census place ("Austin"; a state qualifier narrows the search to that state
wherever the camera is: "Springfield, Illinois"), the San Francisco DataSF
pack, then WOF ("Notting Hill", "Le Marais"). "Near" reaches half the view
radius. A district ask takes a neighborhood first and a place only when the
view is inside it; without an entity kind, the candidate holding the view
wins, else the nearer. "City of X" asks for an incorporated place; when
several places still fit and nothing decides, the geocoder answers.
Compound, grounds, monument and "around" asks skip these, as do names that
end like a landmark ("Central Park") unless asked as a district, and a
qualified neighborhood ("Williamsburg, Virginia") goes to the geocoder. A
geocoded city or neighborhood is drawn from the same packs when one carries
the name and holds the point, before any boundary lookup. Only the index and
the state files or tiles around the point are fetched (files under 4 KB are
inlined into the bundle), and the recently used ones are kept (12 state
files, 24 tiles). Gate:
`node scripts/qa-voice-routing.mjs --layer behavior --behavior outlines`.
- Renderer contract: `src/annotations/hybridAnnotationRenderer.js` routes draped `area`/`route` geometry to world-space Cesium rendering and reticles/pins/arrows/callouts to the screen-space SVG renderer. Area labels are screen-space callouts so all captions share one visual language; progressive outline upgrades convert the existing screen group in place when the anchor snaps to the resolved centroid.
- Tooling: voice has `annotate_map` and `clear_annotations` tools. Annotations accumulate and persist by default; clearing is explicit only. Partial failures, approximate synthesized zones, and route fallbacks are returned as structured tool results so the voice layer can be honest.
- Console/dev API: `window.__gevAnnotations.tour()`, `.demo()`, `.annotate()`, `.clear()`, `.count()`, and `.list()` are the deterministic no-mic test surface.
@@ -4236,7 +4228,7 @@ are omitted rather than framing the wrong part of the globe.
- **Cockpit left-panel clearance:** the Cockpit Contact card and peripheral HUD participate in the adaptive left accordion's live obstacle measurements, including live viewport-height changes. Expanding Layers or Scenes keeps the active panel in the available upper-left corridor with internal scrolling; it does not cover the Contact card, lower Cockpit controls, or Cesium credit line. Outside Cockpit the hidden card does not alter the normal corridor.
- **Cockpit Context scope:** the 250 km radius applies to the air/sea proximity cohorts. Installation counts come only from the currently loaded viewport and are labeled `CURRENT VIEWPORT ONLY` in the cockpit as well as the normal Context panel; neither surface presents them as a complete 250 km installation survey.
- **Cockpit camera anchor:** first-person mode does not write feed-boundary corrections directly into the camera. A cockpit-only inertial anchor advances from the selected aircraft's displayed course and speed, then converges toward the authoritative delayed track with correction capped below forward motion. The displayed kinematics are derived from the same consecutive fix segment as the rendered position, with raw feed speed/course used only as fallback; a transient zero/missing feed speed therefore cannot freeze a visibly moving aircraft after layer enable or a map/cockpit handoff. Rendered altitude continues to come from that interpolated track position. Late ADS-B fixes and short render stalls can remove drift without accelerating or reversing the view. Camera placement runs before scene update/culling at a bounded 20 Hz so a moving cockpit does not force Photoreal 3D Tiles to retraverse on every display frame; textual instruments update at 10 Hz and context/layout work at 4 Hz. Every far Cockpit contact pip shares one stable Cesium texture-atlas entry and skips unused screen-projected course calculations, while only in-range 2D aircraft silhouettes pay the screen-projected rotation cost; ambient glTF collections are hidden/retained rather than synchronously destroyed at cockpit entry, and context rails lay out only on explicit content/state changes and viewport resize. The deliberate 15/30-second layer interpolation delays and per-Cesium-frame position caches remain unchanged.
- **Cockpit route, vision, and view controls:** visible on-screen `COCKPIT`, `RESET`, and `EXIT COCKPIT` controls replace reliance on the `C` shortcut. RESET uses the same canonical globe route as the map and voice actions, exits Cockpit, and releases its camera ownership rather than exposing the hidden map-style top action. When the tracked commercial flight has a plausible ADSBDB route, the top of the right briefing rail shows a compact `FROM → TO` airport strip and the visor shows a centered estimated-destination chevron with its relative bearing; absent or implausible route data hides the strip and cue rather than guessing. The cockpit-local vision control is an interactive `PREV / CURRENT / NEXT` carousel over the inherited map preset, `CRT`, `NVG`, `FLIR`, and `NOIR`; its previous/next actions wrap, and activating the current value advances to the next style. The inherited entry is named directly, such as `NOIR`, and retains that map shader. There is no empty `NONE` entry. CRT, NVG, FLIR, and NOIR temporarily activate the existing Cesium post-process stages, while returning to the inherited entry or exiting Cockpit restores the pre-entry visual style. The regional-news page uses a free Google News RSS locality query first, with the existing GDELT query retained only as a fail-soft fallback; linked headlines remain reporting, not verified incidents or risk intelligence.
- **Cockpit route, vision, and view controls:** visible on-screen `COCKPIT`, `RESET`, and `EXIT COCKPIT` controls replace reliance on the `C` shortcut. RESET uses the same canonical globe route as the map and voice actions, exits Cockpit, and releases its camera ownership rather than exposing the hidden map-style top action. When the tracked commercial flight has a plausible ADSBDB route, the top of the right briefing rail shows a compact `FROM → TO` airport strip and the visor shows a centered estimated-destination chevron with its relative bearing; absent or implausible route data hides the strip and cue rather than guessing. The cockpit-local vision control is an interactive `PREV / CURRENT / NEXT` carousel over `NORMAL`, `CRT`, `NVG`, `FLIR`, `ANIME`, `NOIR`, and `SNOW`; its previous/next actions wrap, and activating the current value advances to the next style. Cockpit enters on the active map preset, so a Cyber FLIR map opens Cockpit on FLIR while Normal remains a distinct option instead of producing a second FLIR entry. Cockpit choices temporarily activate the existing Cesium post-process stages, while exiting Cockpit through either EXIT or RESET restores the pre-entry map style. The regional-news page uses a free Google News RSS locality query first, with the existing GDELT query retained only as a fail-soft fallback; linked headlines remain reporting, not verified incidents or risk intelligence.
- **Cockpit weather status:** the earlier multi-canvas atmospheric compositor remains fail-closed and is not attached to the live viewer. Cockpit clouds are a separate transparent WebGL pass with a capped 520×320 framebuffer, 24 ray steps, three FBM octaves, and a 12 FPS ceiling. It defaults off and starts only when local storage explicitly contains the persisted `WX ON` opt-in (`'1'`). When opted in, observations refresh after five minutes or 25 km of aircraft movement, fail transparent when unavailable or clear, and stop on exit or disable. `WX OFF` governs atmospheric rendering only: the briefing still fetches source-backed Natural Earth region, headline, and Open-Meteo local-information data, aborting and replacing any in-flight request when the selected aircraft changes. No weather effect runs in map mode and no synthetic fallback is shown.
- **Cockpit trail visibility:** entering cockpit hides the selected aircraft's trail body and head so they cannot cross the first-person view; exit restores them. This cockpit-only presentation change does not alter the normal map-mode invariant that aircraft trails render through terrain using their depth-fail material.
- **Aircraft course slew:** civilian and military 3D models retain the 60°/s course limiter, but each rendered frame can consume at most 250 ms of accumulated slew time. A long tile/render stall therefore catches up over multiple visible frames instead of turning one delayed frame into a heading snap.
+71
View File
@@ -0,0 +1,71 @@
# MCP setup
Use God's Eye View from Claude and Codex: tools, links, and the live globe
inside the conversation. Agents: follow these steps in order and check each
one before going on.
## 1. Run the app
```sh
npm install
npm run build:panel # the in-conversation globe; rerun after app changes
npm run dev # http://localhost:5173, keep it running
```
The MCP server reads from this app.
## 2. Add the server to a client
The server is `server/mcp/stdio.js` in this checkout. Use absolute paths
(clients do not load your shell profile), with each argument as its own entry.
| Client | How |
| --- | --- |
| Claude Code | `claude mcp add gods-eye-view -- node /abs/path/gods-eye-view/server/mcp/stdio.js --api-base http://localhost:5173` |
| Codex CLI | `codex mcp add gods-eye-view -- node /abs/path/gods-eye-view/server/mcp/stdio.js --api-base http://localhost:5173` |
| Codex and ChatGPT desktop | Same `~/.codex/config.toml` as the CLI (block below), or Settings → MCP servers → Add server (STDIO). |
| Claude Desktop | Settings → Developer → Edit Config, then add the JSON below. |
```toml
[mcp_servers.gods-eye-view]
command = "node"
args = ["/abs/path/gods-eye-view/server/mcp/stdio.js", "--api-base", "http://localhost:5173"]
```
```json
{
"mcpServers": {
"gods-eye-view": {
"command": "node",
"args": ["/abs/path/gods-eye-view/server/mcp/stdio.js", "--api-base", "http://localhost:5173"]
}
}
}
```
**Windows with the checkout in WSL:** use `"command": "wsl.exe"` and put
`"-e"` and the absolute Linux path to `node` first in `args`, for example
`["-e", "/home/you/.local/share/mise/installs/node/24/bin/node", "/home/you/gods-eye-view/server/mcp/stdio.js", "--api-base", "http://localhost:5173"]`.
## 3. Restart and try it
Fully quit the client (on Windows, from the tray icon) and reopen it, then
start a new chat; clients keep the server they started with. Ask: *"Show San
Diego in God's Eye View with military flights on."*
- Claude Desktop (a chat, not the Code tab), Codex and ChatGPT desktop show
the live globe; the first load takes 10–20 seconds.
- Claude Code and the Codex CLI answer with data and a link.
## Troubleshooting
- **The server does not appear:** run the command from step 2 in a terminal;
it should print `God's Eye View MCP server reading http://localhost:5173`
and wait. A path or `args` mistake fails here.
- **The panel says it could not load:** `npm run build:panel` was not run, or
the dev server is down.
- **Logs:** the server writes each call, and why one failed, to stderr. Claude
Desktop keeps it in its logs folder as `mcp-server-gods-eye-view.log`; in
Codex, right-click the panel → DevTools for the panel's console.
See [tools and the MCP server](TOOLS.md) for what the tools do.
+302
View File
@@ -0,0 +1,302 @@
# Tools and the MCP server
Tools answer questions from God's Eye View data for language-model clients.
They are defined once and exposed through adapters: the Model Context Protocol
(MCP) and function calling, which voice uses.
To use them from Claude or Codex, see [MCP setup](MCP_SETUP.md).
## Layers
| Owner | Responsibility |
| -------------------- | ------------------------------------------------------------------------------------------------- |
| `src/tools/` | Tool definitions, catalog composition, argument validation, shared `area` and result helpers |
| `src/tools/queries/` | Queries, one file per domain, reading only portable source contracts |
| `src/tools/mcp/` | MCP protocol (JSON-RPC) and a stateless HTTP transport; knows the catalog interface, not queries |
| `src/tools/functions.js` | Function-calling adapter: tool records and results for function-calling clients |
| `src/tools/services.js` | The default services: the layers' source factories and request services, given a resolving fetch |
| `server/mcp/` | Node composition: points the sources at a running app's `/api` routes; serves stdio and `/mcp` |
| `server/standalone/voiceTools.js`, `src/standalone/toolCatalog.js` | Standalone voice composition: the session's tool list and the browser catalog |
Dependencies point downward only. `gods-eye-view/tools` and
`gods-eye-view/tools/mcp` are portable exports: they reach no application,
rendering, Node, Cesium or browser-global code, which
`npm run check:boundaries` enforces. In the application, only voice reaches
them: `withToolCatalog` in `src/voice/gevRealtime.js` imports the
function-calling adapter, and the standalone entry supplies the catalog.
## Definitions and composition
`defineTool({ name, kind, title, description, inputSchema, requires, run })`
validates and freezes a tool. `kind` is `query` (answers from data, read-only)
or `action`. `inputSchema` uses a JSON Schema subset that `src/tools/schema.js`
checks completely; unsupported keywords are rejected at definition time.
`run(args, { services, signal })` resolves to `{ summary, data }`: one sentence
for people and a structured object for programs. A tool may also return `images`,
each `{ mimeType, data }` with base64 data; the MCP adapter sends them as image
content. MCP results carry the summary and the data as JSON text, plus the data
as `structuredContent`, for clients that read only one of them.
`composeCatalog({ tools, services, replace, interceptors })` builds a catalog:
- **Tools**: an application adds its own tools to `coreTools`. Reusing a name
fails unless the name is listed in `replace`.
- **Services**: each tool names the services it reads in `requires`. Tools
whose services are not supplied are left out.
- **Interceptors**: `(call, next) => next(call)` functions wrap every call,
outermost first. They can observe, reject or change a call.
- **Composite tools**: `run` receives `tools`, with `has(name)` and
`call(name, args)`, to call other tools through the same catalog, so
replaced tools and interceptors apply. Interceptors see such calls with
`parent`, the calling tool's name.
### Surfaces
`src/tools/surfaces.js` lists which tools MCP and voice offer. A tool is on
both unless `TOOL_SURFACES` turns it off; edit an entry to turn a tool on or
off for one surface. MCP leaves out place search, routing, plain
weather and wind, the regional brief, the HUD caption, radio, bike share and
transit, which assistants already cover or which add little without the
globe. `catalogForSurface(catalog, surface, overrides)` is the
view a surface exposes: it lists and calls only the tools it offers, while
composite tools still reach the whole catalog. `toolsForSurface` gives the
same selection as a list of definitions, such as for the voice session's tool
list. Voice leaves out tools that answer with images, link to the app, or
repeat what its app actions answer.
Expected failures throw `ToolError` with one of `invalid_arguments`,
`unavailable`, `unsupported`, `malformed` or `retry_later`. Other errors are
reported to clients without details.
## Services
`gods-eye-view/tools/services` builds the default set with
`createToolServices({ fetchImpl, appUrl })`. Services are the portable source factories the layers already use, such as
`createUsgsEarthquakeSource`, `createFirmsSource` and `createLaunchSource`,
plus a `places` service with `resolve(name, { signal })`. Sources request
relative `/api/...` paths through an injected `fetchImpl`, so the same tool
code runs wherever an application routes those paths.
The `weather`, `regional`, `terrain`, `summary` and `features` services are the
application request services from `gods-eye-view/application/requests`.
`situation_brief` and `military_awareness` run each section whose services are supplied and mark the
others unavailable. `app` is `{ baseUrl }`, the address links open. The `bikeshare` service is `{ systems, getStations }`: the system registry and
the GBFS source. `createGeocodePlaceService` resolves place names through `/api/geocode`;
`createPlaceSearchService` searches `/api/google/*` and reports when no search key
is configured; `createRouteService` plans routes through `/api/route`.
## Views
`gods-eye-view/view` (`src/view/index.js`) describes what the app shows,
independent of how it is shown: a camera (lat, lon, altitude, heading,
pitch), data layers, visual style, map imagery, and an aircraft, military
aircraft or satellite to follow. `createView` builds and bounds one,
`viewToParams` and `viewFromParams` write and read it in the share-link
format the app restores, and `viewUrl` gives the address that opens it. The
style names are the ones share links use. Ships cannot be followed from a
link yet. An aircraft's `follow` may ask for its cockpit view
(`cockpit: true`); links cannot carry that and open the app following it.
A view can also carry `annotations`, the marks the app's `annotate_map`
action draws (pins, highlights, areas, arrows, routes and labels at a place
name or coordinates). Links carry them in the `an` parameter, and the app
draws them once the link has been restored.
Tools take a view as `VIEW_ARGUMENTS`: an `area` to frame from above, or a
`camera`, plus `layers`, `style`, `map`, `follow` and `annotations`. A
tilted camera over an area looks at its center from behind; camera fields
given with an area override its framing. `resolveViewArguments` turns them
into a view.
### The God's Eye View panel
`show_in_gods_eye_view` names an MCP Apps view (`io.modelcontextprotocol/ui`):
`_meta.ui.resourceUri` is `ui://gods-eye-view/globe`, a `text/html;profile=mcp-app`
resource from `createGlobePanelResource({ runtime })` in
`src/tools/globePanel.js`, with the panel's script from
`src/app/globePanelRuntime.js`. Clients that display apps render it inside
the conversation. The panel completes the MCP Apps handshake
(`ui/initialize`, `ui/notifications/initialized`,
`ui/notifications/size-changed`), and for each
`ui/notifications/tool-result` carrying a view it loads the app in inline
embed mode the first time and posts later views to that same app, so the
globe changes without reloading. Its "Open in God's Eye View" button asks
the host to open the link (`ui/open-link`).
Hosts serve panels from their own sites and may refuse other addresses;
Codex, for one, refuses any address on the user's machine. So the panel
never requests the app's server itself. It loads everything from the app's
own paths through `panel_request`, a tool meant for the panel: it is
marked `_meta.ui.visibility: ["app"]`, and each call must carry the key the
MCP server puts in its panel page. Any client can read that page, so the key
keeps the tool from clients that only list it, and is not access control. The
MCP server requests the path from the app's server and returns the response,
compressed and in parts when large.
The same path works in every host. Only
map imagery, tiles and fonts load directly, from the providers the
resource's `csp` lists. `panel_request` refuses Provider Settings
(`/api/setup`), credential and model endpoints (`/api/realtime`,
`/api/openai`), `/mcp` and the development server's internal routes, in any
letter case, encoding or dot suffix.
The panel loads the app's panel build, which `npm run build:panel` writes to
`dist/panel` and the servers serve at `/panel/`: one app script, one
stylesheet, and Cesium's script, which carries its workers and starts them
from memory (`CESIUM_WORKERS`). Files those workers load themselves are
embedded in a prelude the panel runs ahead of them, since a worker's
requests reach the panel page's own site. Rebuild it after changing
the app.
Hosts differ in ways the panel works around, all inside the panel only:
images and stylesheet files arrive as `data:` URLs, since some hosts refuse
`blob:` images; code that needs an https address for the app gets
`GEV_APP_BASE_URL`, since some hosts serve the page from their own scheme;
the globe keeps drawing from a timer when the host reports the panel hidden
and stops animation frames; and 2D canvases are kept in memory
(`willReadFrequently`), since a host that treats the panel as off screen may
drop their GPU contents and show the overlays as black over the globe.
Tools declare a UI resource with `defineTool({ ui: { resourceUri } })`, and
an app-only tool with `ui: { visibility: ['app'] }`;
`createMcpServer({ resources })` serves `resources/list` and
`resources/read`.
### Embed mode
`?embed=1` shows only the globe: clean view, with the HUD, panels, welcome
and setup prompts hidden; provider attribution stays. A page that frames it
changes the view by posting `{ type: 'gev:view', id, view }` to the frame.
The app applies it through its own actions (style, map, exactly the view's
layers, annotations, then the followed entity, retried until its layer has
it, or the camera when nothing is followed or the entity is not there yet,
since a camera flight would end the follow; then cockpit view when asked) and
answers `{ type: 'gev:view-applied', id, ok, steps }` to the origin that
sent the view. It posts `{ type: 'gev:ready' }` once it can take views, and
only its parent page can send them. See `src/app/embed.js`.
No page may frame the app by default: every document keeps
`X-Frame-Options: DENY` and `frame-ancestors 'none'`. Setting
`GEV_EMBED_FRAME_ANCESTORS` lets the pages it names (CSP frame-ancestors
sources, or `*` for any page) frame embed-mode documents only. The MCP Apps
panel loads the app into its own page and needs no framing.
Answers that have something to show include `data.view`: the view that
shows them, with the matching layers on, an area framed from above, and a
single aircraft or satellite followed, plus `url` to open it (null when the
app's address is not configured). `suggestView` in `src/tools/views.js`
builds one.
## The `area` argument
Location-scoped tools take `area` as exactly one of a `place` name, a `bbox`
(`[west, south, east, north]`, crossing the antimeridian when west exceeds
east), or `lat`, `lon` and `radius_km`. Lists default to 25 rows, at most 200,
and report `total`, `returned` and `truncated`.
## MCP
`createMcpServer({ catalog, name, version, instructions, descriptions, decorate })`
implements `initialize`, `ping`, `tools/list` and `tools/call` for protocol
revisions 2025-11-25, 2025-06-18 and 2025-03-26. `descriptions` overrides a
tool's title or description for this surface; `decorate(definition, tool)`
merges extra fields into each listed definition. `createMcpHttpHandler(server)`
returns a `Request`-to-`Response` handler for stateless Streamable HTTP: one
JSON-RPC message per POST, answered with JSON. The host owns routing and any
access control in front of it.
## Voice
Voice offers the catalog's queries next to its app actions.
`toFunctionTools(tools, { exclude })` turns tools into
`{ type: 'function', name, description, parameters }` records, and
`toFunctionOutput(name, result)` turns a result into
`{ ok, tool, summary, data }`, counting images in `images_omitted` instead of
sending them.
The voice session token endpoint takes its tool list as `realtime.tools`.
`realtimeSessionTools(additional)` appends function tools to the app actions,
skipping names an action already uses, so `next_satellite_pass` stays the
action. The standalone server supplies the core queries that
`src/tools/surfaces.js` offers on voice, which leaves out tools that answer
with images, link to the app, or repeat what voice's app actions answer
(aircraft, ships, earthquakes, fires, datacenters, dams and satellites
overhead, which `analyst_query` covers).
In the browser, `initGevVoiceCommands({ toolCatalog })` takes a function that
resolves a catalog. App action names go to the action runner; other names the
catalog has go to `catalog.call` with the call's abort signal. The standalone
entry composes the catalog with `createToolServices` over the page's fetch and
loads it the first time voice calls a query.
## Running locally
Start the app (`npm run dev` or `npm run preview`), then register the stdio
server with an MCP client, for example Claude Code:
```bash
claude mcp add gods-eye-view -- npm --prefix /path/to/gods-eye-view run --silent mcp
```
`npm run mcp -- --api-base http://localhost:4173` selects another server.
The development and preview servers also serve the same tools over HTTP at
`/mcp`, for clients that connect by URL:
```bash
claude mcp add --transport http gods-eye-view http://localhost:4173/mcp
```
The route accepts only requests from this machine that name a loopback host
on the port they reached and, when a browser sends an `Origin`, come from
that same host. It refuses requests a proxy forwarded and refuses all
requests while launcher sharing is on. This is local transport safety, not
authentication. The
local server's tools request the app's `/api` routes and the public feeds
the sources already use. The panel's `panel_request` also requests the app's
own files and data routes for the panel: it requires the key in the panel's
page, and refuses Provider Settings, credential and model endpoints, `/mcp`
and the development server's internal routes. See SECURITY.md.
## Tools
| Tool | Reads | Returns |
| --------------------- | ------------- | ------------------------------------------------------------- |
| `get_earthquakes` | `earthquakes` | USGS M2.5+ events in the last 24 hours, strongest first |
| `get_active_fires` | `fires` | NASA FIRMS detections in an area, highest radiative power first |
| `get_recent_launches` | `launches` | Launch Library 2 launches in the last 30 days, newest first |
| `aircraft_in_area` | `aircraft` | Aircraft in an area, nearest first; `military: true` reads the `military` feed |
| `find_aircraft` | `aircraft` | Aircraft anywhere by callsign, ICAO address or registration |
| `get_aircraft_track` | `aircraft` | Recent positions of one aircraft, thinned to 200 points |
| `get_aircraft_info` | `aircraft` | Aircraft type and registration, and flight route, from adsbdb |
| `vessels_in_area` | `vessels` | Ships reported by AIS in an area, nearest first, optionally by type |
| `find_vessel` | `vessels` | Ships anywhere by MMSI, IMO number or name |
| `get_vessel_track` | `vessels` | Recent positions of one ship, thinned to 200 points |
| `next_satellite_pass` | `satellites` | Next pass over a place or point (default the ISS), with naked-eye visibility |
| `satellites_overhead` | `satellites` | Satellites in a CelesTrak group above a place or point now, highest first |
| `find_cctv_cameras` | `cctv` | Public cameras in an area, nearest first |
| `get_cctv_snapshot` | `cctv` | The current image from one camera, returned as image content |
| `find_alpr_cameras` | `alpr` | OpenStreetMap-mapped license plate readers in a US/Canadian area up to 3° |
| `find_radio_stations` | `radio` | Radio Browser stations by area and/or search terms, with stream URLs |
| `search_places` | `placeSearch` | Points of interest matching a query within an area (Google Places) |
| `places_nearby` | `placeSearch` | Notable places around a place or point (Google Places) |
| `plan_route` | `routing` | Walking, driving or cycling route over OpenStreetMap, with a simplified path |
| `get_bike_share` | `bikeshare` | Live GBFS stations in an area, with bikes and docks available |
| `get_transit_vehicles` | `transit` | Live GTFS-Realtime vehicle positions in an area, optionally one route |
| `get_traffic_flow` | `traffic` | TomTom flow in a city-sized area: speed vs free flow, congested and closed road |
| `get_weather` | `weather` | Current conditions at a place or point |
| `get_weather_map` | `weatherMaps` | The latest NOAA radar, satellite or lightning map image over an area |
| `get_wind` | `wind` | GFS or IFS model wind 10 m above ground at a location |
| `get_recent_imagery` | `imagery` | The most recent clear Landsat/Sentinel-2 image of an area (VIIRS fallback) |
| `find_submarine_cables` | `cables` | TeleGeography cables and landing points by area or name (CC BY-NC-SA 3.0) |
| `find_infrastructure` | `infrastructure` | OpenStreetMap datacenters or dams in an area, nearest first (ODbL) |
| `get_bhote_koshi_flood` | `events` | The 2026 Bhote Koshi flood: evidence trail in story order, flood path and imagery dates (CC BY-NC 4.0) |
| `get_regional_brief` | `regional` | What and where a location is, its weather and recent headlines |
| `get_cyclones` | `cyclones` | Active NHC/CPHC tropical cyclones, optionally in an area |
| `get_fire_perimeters` | `perimeters` | Mapped WFIGS wildfire perimeters in an area, largest first |
| `get_terrain_height` | `terrain` | Ground, geoid and ellipsoid heights at up to 20 points |
| `find_military_installations` | `installations` | OpenStreetMap military sites in an area of at most 10° per side |
| `get_map_features` | `features` | Administrative areas, named places or monuments at a location (needs Overpass) |
| `situation_brief` | `weather` | Weather, earthquakes, fires, aircraft, ships and cyclones for an area, by section |
| `military_awareness` | `military` | Military and other aircraft, ships and military installations within 250 km of a point, by section |
| `get_hud_caption` | `weather`, `summary` | The app's heads-up display caption for an area |
| `show_in_gods_eye_view` | `app` | A view in God's Eye View: the live panel in clients with MCP Apps, and a link everywhere; takes another answer's view or an area or camera, layers, style, map, marks and something to follow |
| `panel_request` | `app` | Panel only: loads a path from the app's server for the God's Eye View panel |
+1 -7
View File
@@ -14,12 +14,6 @@ The session adapter, action runner and backend interfaces remain unchanged.
| `realtimeCost.js` | Next-session preferences and the current session's model-bound meter |
| `realtimeViewport.js` | One retained image, bounded deletion identities and capture generation |
| `realtimeDiagnostics.js` | Bounded error history and sanitized optional diagnostics |
| `turnMetrics.js` | Per-turn latency and speech spans written to the debug log |
| `narration.js` | Progress lines and the earcon while tools run; spoken through the turns owner |
| `pointerContext.js` | Pointer position, dwell pick and the current turn's pointer snapshot (session-scoped) |
| `referents.js` | Numbered items from the latest result for `referent:n` (session-scoped) |
| `resultDisplay.js` | What the voice card shows for a result and which items it numbers (read by the card and `referents.js`) |
| `gevActions.js` runner | Analyst follow-up memory (session: `resetConversation()` on stop, `dispose()` on removal) |
Pure input policy, preferences and protocol-response policy have separate modules.
Owners receive named readers, operations or focused collaborators. Cross-owner
@@ -35,7 +29,7 @@ Action and post-capture continuations retain their conversation identity before
publishing output, queueing a response or changing status.
Behavioral invariants remain in force: a 500 ms Space hold claims push-to-talk;
short control taps and text entry remain native; a claimed hold while the assistant speaks interrupts it (never during a Radio handoff); a click-started session stays
short control taps and text entry remain native; a click-started session stays
open-mic. Every accepted or refused tool call receives its terminal output once.
New typed input supersedes old intent. Radio waits for the spoken confirmation and
verified muted playback. A live cost meter stays bound to its negotiated model,
+150
View File
@@ -0,0 +1,150 @@
# Panel surface contract
`src/ui/styles/panel-surfaces.css` supplies an opt-in visual surface for new
panels. It uses the application's default color, font, border and radius tokens;
Cyber supplies its existing red/slate palette, monospace typography and shaped
frame. This is CSS shared by the existing panel system, not a panel registry or
a controller. A panel ID does not need a new theme selector.
## Markup
A rail panel uses an outer owner with its existing stable `id`, matching
`data-panel-id`, and `panel-collapsible` class. Its first non-glow child is the
surface. Keep the header and any outward popup outside the scrolling body:
```html
<section
id="example-panel"
data-panel-id="example-panel"
class="panel-collapsible"
>
<div data-panel-surface>
<header data-panel-header>
<h2 class="panel-title panel-surface-title">Example</h2>
<button
class="panel-collapse-btn panel-surface-control"
data-collapse-target="example-panel"
aria-expanded="true"
aria-label="Collapse Example"
>
−
</button>
</header>
<div data-panel-body data-rail-scroller>
<label>Filter <input class="panel-surface-control" type="text" /></label>
<button class="panel-surface-control" type="button">Apply</button>
<p class="panel-surface-error" role="status" hidden></p>
</div>
<aside data-panel-popup hidden aria-label="Example help">
<!-- The panel owns this popup's placement and interactions. -->
</aside>
</div>
</section>
```
Use native controls and accessible names. `.panel-surface-control` is an
optional skin for buttons, text fields, selects and textareas; it provides
hover, focus, disabled and invalid-state styling. Native `disabled` owns actual
disablement; `aria-disabled` alone only describes a state and must be enforced
by the controller. `.panel-surface-error` supplies error color, not announcement
or validation behavior. Keep checkbox/radio semantics and labels intact.
`data-panel-header` stays outside `data-panel-body`, so it remains reachable
when content scrolls. Give the body a bounded height through its containing
surface. The rail's allocation supplies that bound; a dock or dialog owner must
supply its own `max-height` or height. Avoid a flex body whose contents shrink to
fit when those contents should instead scroll.
## Placement and ownership
| Mount | Shared contract | Owner responsibilities |
| ------------------------------------- | --------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Direct child of `#left-panel-stack` | An owner with a direct `[data-panel-surface]` child receives relative positioning and allocated height; inner surface/body skin | Add the wrapper shown above, use the existing layout/disclosure owners, and retain left-rail focus/auto-collapse rules. |
| Direct child of `#right-context-rail` | The same hooks with right-rail allocation and intrinsic measurement | Keep `data-rail-scroller` on the body so measurement preserves its scroll position. Respect Cyber's one-expanded-panel policy. |
| Surface inside `#command-dock` | Surface, controls, popup decoration and inherited dock concealment | Supply geometry, stacking, disclosure, pin behavior if wanted, safe viewport bounds and dismissal. This CSS does not extend the existing Location/Visual Presets tray stack manager to arbitrary panels. |
| `dialog[data-panel-surface]` | The same surface/body skin; a closed dialog remains hidden | Set width/max-height, use `showModal()` or `show()` and `close()`, and own labels, initial/return focus and dismissal. Do not put `panel-collapsible` on a dialog. |
For rails, `--panel-expanded-width` and `--panel-collapsed-width` customize
the desired widths within the rail; otherwise Cyber uses its existing left,
right and collapsed panel widths. `--left-panel-allocated-height` and
`--right-panel-allocated-height` belong
to the existing layout owner, not to individual panels. The surface must be
the wrapper's first non-glow child because left-rail intrinsic measurement
inspects that child. Do not put an absolutely positioned popup before the
surface or inside its body. Position a popup beside or above its surface using
component CSS; the shared rules deliberately do not choose a direction that
could run beyond the viewport.
Rail measurement still considers visible child extents. A popup extending
below the surface can therefore increase its measured height. Prefer a popup
beside or above the surface, or integrate an external overlay host with the
feature's layout owner; this CSS does not change measurement algorithms.
At viewport widths of 720px or less, both rails are intentional scroll
containers. Their overflow can clip outward popups even when the panel's own
surface is unclipped. Use a popup contained within the rail's visible bounds,
or an explicitly owned overlay host outside the rail; the surface contract
does not provide automatic popup escape from these small-screen scroll regions.
For dock disclosure, use a semantic button with `.panel-surface-control` and
the feature's own accessible name, `aria-expanded` state and event binding.
Do not copy the rail example's `.panel-collapse-btn` class into the dock:
existing dock rules intentionally hide that class. The generic surface skin
does not replace the dock's disclosure policy.
The existing `PanelChrome` binds `.panel-collapse-btn[data-collapse-target]`
at initialization and routes state through `setPanelCollapsed`. For additions
present at startup, keep those hooks and the existing chrome lifecycle. A
dynamically mounted panel must bind and dispose its own disclosure using
`bindPanelDisclosure` / `collapsePanelOnEscape` from `panelDisclosure.js`, with
the application owning state changes. Do not rerun the whole application chrome
initializer just to add one panel. `layoutLeftPanelRail` and
`layoutRightPanelRail` accept arbitrary direct `data-panel-id` children; their
host owns layout scheduling, observations and collapse callbacks. Dynamic
content that changes size may require an owner-scheduled layout pass or an
owned `ResizeObserver`; adding CSS attributes does not register new observers.
Persisted/share-link state, pinning, provider loading, keyboard shortcuts and
Cockpit-specific mounting still require deliberate integration with their
current owners. This contract does not add entries to named persistence, dock,
Cockpit or share-state policies. Dispose listeners, observers and timers,
dismiss open popups/dialogs and restore focus before removing a dynamic panel.
## Decoration, popups and visibility
The shared surface reserves `::before` for its non-interactive background. The
Cyber frame clips this pseudo-element only; the surface stays unclipped with
visible overflow. This follows the existing Display and Global Context pattern.
Putting the shape on an interactive ancestor clips all descendant popups even
if their `z-index` is high. A scrolling body also clips descendants by design,
so outward popups must be siblings of the body, or use an explicitly owned
overlay host outside every clipping ancestor.
`data-panel-popup` gives a positioned popup the same decorative and control
tokens. It is not a popover API, focus trap, collision solver or top-layer
manager. The owner must keep it inside the viewport, above relevant siblings,
and reachable by pointer and keyboard. A native modal dialog participates in
the browser top layer; a surface in a rail does not. Transform, containment,
mask, clipping or stacking rules on an additional ancestor can still obstruct
either an ordinary panel or its popup. Arbitrary future markup is not covered.
`hidden`, a closed native dialog and collapsed bodies/popups remain hidden.
Clean UI and recording conceal conforming surfaces, including outward popups;
owners must close modal dialogs when entering these modes so an invisible
modal cannot leave the application inert. Existing Cockpit rail and dock
exclusions remain in force. New left-rail/Cockpit functionality needs an
explicit product integration; the surface attribute grants no mode exception.
Never use theme styling to alter stored preferences or force hidden content
visible.
## Check an addition
Use the real application with the default and Cyber themes. Check open,
collapsed, hidden, disabled/error, theme round trips, clean UI/recording and
the intended Cockpit behavior. Exercise keyboard focus, Escape, pointer input,
the last scrolling item, resize without reload and any pin/unpin controls.
For an outward popup, test visible pixels and pointer hit-testing outside the
surface; a positive bounding box alone does not prove it escaped clipping.
Check each mount the addition actually supports. The CSS contract cannot
establish provider correctness, disclosure lifecycle or responsive placement
without those feature-specific checks.
+2 -2
View File
@@ -1,12 +1,12 @@
{
"name": "gods-eye-view",
"version": "0.1.1",
"version": "0.2.1",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "gods-eye-view",
"version": "0.1.1",
"version": "0.2.1",
"license": "MIT",
"dependencies": {
"@jtarrio/signals": "^0.10.0",
+9 -3
View File
@@ -1,7 +1,7 @@
{
"name": "gods-eye-view",
"private": true,
"version": "0.1.1",
"version": "0.2.1",
"description": "A real-time intelligence console for planet Earth — photorealistic 3D globe, live aircraft/ships/satellites/earthquakes/CCTV, and hands-free voice control. Runs in a browser.",
"type": "module",
"license": "MIT",
@@ -41,12 +41,14 @@
"dev:secure": "./scripts/dev-secure.sh",
"opensky:import": "./scripts/opensky-import-client.sh",
"build": "vite build",
"build:panel": "node scripts/build-panel.mjs",
"preview": "vite preview",
"test": "node scripts/run-unit-tests.mjs",
"test:track": "node scripts/track-regression.mjs",
"qa:map-source-tray": "node scripts/qa-map-source-tray.mjs",
"check:boundaries": "node scripts/check-import-directions.mjs && node scripts/check-package-boundaries.mjs",
"qa:transit": "node scripts/qa-transit.mjs"
"qa:transit": "node scripts/qa-transit.mjs",
"mcp": "node server/mcp/stdio.js"
},
"dependencies": {
"@jtarrio/signals": "^0.10.0",
@@ -250,6 +252,10 @@
"./layers/transit/standalone": "./src/data/transit.js",
"./layers/transit/source": "./src/layers/transit/source.js",
"./sources/transit-service": "./src/sources/transitService.js",
"./sources/map-features": "./src/sources/overpassFeatures.js"
"./sources/map-features": "./src/sources/overpassFeatures.js",
"./tools": "./src/tools/index.js",
"./tools/mcp": "./src/tools/mcp/index.js",
"./tools/services": "./src/tools/services.js",
"./view": "./src/view/index.js"
}
}
+2 -1
View File
@@ -18,7 +18,8 @@ For commercial use, obtain separate permission or exclude both this pack and the
derived coordinate dataset in `src/data/bhoteKoshiFloodPath.js` from the source
and build. Deleting this directory alone does not remove the compiled river
data. Excluding the Nepal scene also requires removing its registrations and
imports before building.
imports before building, including the `get_bhote_koshi_flood` tool in
`src/tools/queries/events.js`.
Sources:
+34 -46
View File
@@ -111,8 +111,8 @@ const cacheDir = path.resolve(
);
const only = args.includes('--only') ? args[args.indexOf('--only') + 1] : null;
export async function fetchPinned(name, source = SOURCES[name]) {
const { url, sha256 } = source;
async function fetchPinned(name) {
const { url, sha256 } = SOURCES[name];
const file = path.join(cacheDir, path.basename(new URL(url).pathname));
let bytes = await readFile(file).catch(() => null);
if (!bytes) {
@@ -137,7 +137,7 @@ const EARTH_RADIUS_KM = 6371;
const toRad = (d) => (d * Math.PI) / 180;
/** Spherical ring area (km²); matches src/data/naturalEarthRegions.js. */
export function ringAreaKm2(ring) {
function ringAreaKm2(ring) {
const n = ring.length;
if (n < 3) return 0;
let sum = 0;
@@ -151,7 +151,7 @@ export function ringAreaKm2(ring) {
}
/** Signed planar (shoelace) area: positive counter-clockwise in lon/lat. */
export function signedArea(ring) {
function signedArea(ring) {
let sum = 0;
for (let i = 0, j = ring.length - 1; i < ring.length; j = i++)
sum += ring[j][0] * ring[i][1] - ring[i][0] * ring[j][1];
@@ -209,16 +209,15 @@ function douglasPeucker(points, tolerance) {
* Simplify a closed ring to an OPEN rounded ring. The ring is split at its
* farthest vertex from the start so Douglas-Peucker has two anchored halves
* (a closed ring has no chord to measure against). Returns null when fewer
* than `minVertices` distinct vertices survive (four by default; a pack that
* keeps triangles passes three).
* than four distinct vertices survive.
*/
function simplifyRing(ring, tolerance, decimals, minVertices = 4) {
function simplifyRing(ring, tolerance, decimals) {
let open = ring.slice();
const first = open[0];
const last = open[open.length - 1];
if (open.length > 1 && first[0] === last[0] && first[1] === last[1])
open = open.slice(0, -1);
if (open.length < minVertices) return null;
if (open.length < 4) return null;
let far = 1;
let farDist = -1;
for (let i = 1; i < open.length; i++) {
@@ -248,7 +247,7 @@ function simplifyRing(ring, tolerance, decimals, minVertices = 4) {
out[0][1] === out[out.length - 1][1]
)
out.pop();
return out.length >= minVertices ? out : null;
return out.length >= 4 ? out : null;
}
/**
@@ -256,17 +255,16 @@ function simplifyRing(ring, tolerance, decimals, minVertices = 4) {
* area floors (the largest part always stays), orient outers
* counter-clockwise and holes clockwise, sort parts by area descending.
*/
export function simplifyPolygons(polygons, params, tolerance, decimals) {
const minVertices = params.minRingVertices ?? 4;
function simplifyPolygons(polygons, params, tolerance, decimals) {
const parts = [];
for (const poly of polygons) {
const outer = simplifyRing(poly[0], tolerance, decimals, minVertices);
const outer = simplifyRing(poly[0], tolerance, decimals);
if (!outer) continue;
const area = ringAreaKm2(outer);
if (signedArea(outer) < 0) outer.reverse();
const holes = [];
for (const hole of poly.slice(1)) {
const ring = simplifyRing(hole, tolerance, decimals, minVertices);
const ring = simplifyRing(hole, tolerance, decimals);
if (!ring || ringAreaKm2(ring) < params.minHoleKm2) continue;
if (signedArea(ring) > 0) ring.reverse();
holes.push(ring);
@@ -289,22 +287,18 @@ export function simplifyPolygons(polygons, params, tolerance, decimals) {
* and a fifth of the minimum tolerance, or failing that unsimplified at two
* more decimals; the caller records `d`.
*/
export function unitTolerance(polygons, params) {
function simplifyUnit(polygons, params) {
const sourceArea = polygons.reduce(
(sum, poly) => sum + ringAreaKm2(poly[0]),
0,
);
return Math.min(
const tolerance = Math.min(
params.maxToleranceDeg,
Math.max(
params.minToleranceDeg,
(params.toleranceFactor * Math.sqrt(sourceArea)) / 111.32,
),
);
}
export function simplifyUnit(polygons, params) {
const tolerance = unitTolerance(polygons, params);
const coarse = simplifyPolygons(polygons, params, tolerance, params.decimals);
if (coarse.length) return { polygons: coarse, decimals: params.decimals };
const fine = simplifyPolygons(
@@ -340,7 +334,7 @@ function encodeRing(ring, decimals) {
return out;
}
export function encodeUnit({ polygons, decimals }) {
function encodeUnit({ polygons, decimals }) {
return polygons.map((poly) => poly.map((ring) => encodeRing(ring, decimals)));
}
@@ -368,7 +362,7 @@ function bboxOf(polygons) {
// ── zip / shapefile / dbf ───────────────────────────────────────────────
/** Read named entries from a zip archive (stored or deflated). */
export function unzip(buffer) {
function unzip(buffer) {
let eocd = buffer.length - 22;
while (eocd >= 0 && buffer.readUInt32LE(eocd) !== 0x06054b50) eocd--;
if (eocd < 0) throw new Error('zip: end of central directory not found');
@@ -400,7 +394,7 @@ export function unzip(buffer) {
}
/** Polygon records (shape type 5) as arrays of closed [lon, lat] rings. */
export function readShp(buffer) {
function readShp(buffer) {
const shapes = [];
let offset = 100;
while (offset < buffer.length) {
@@ -437,7 +431,7 @@ export function readShp(buffer) {
}
/** dBASE records as plain objects (character and numeric fields). */
export function readDbf(buffer) {
function readDbf(buffer) {
const count = buffer.readUInt32LE(4);
const headerLength = buffer.readUInt16LE(8);
const recordLength = buffer.readUInt16LE(10);
@@ -464,7 +458,7 @@ export function readDbf(buffer) {
}
/** Group shapefile rings into polygons: clockwise outers, holes assigned by containment. */
export function shpPolygons(rings) {
function shpPolygons(rings) {
const outers = [];
const holes = [];
for (const ring of rings) {
@@ -481,7 +475,7 @@ export function shpPolygons(rings) {
return polygons;
}
export function pointInRing(ring, x, y) {
function pointInRing(ring, x, y) {
let inside = false;
for (let i = 0, j = ring.length - 1; i < ring.length; j = i++) {
const [xi, yi] = ring[i];
@@ -748,24 +742,18 @@ async function writePack(relative, pack) {
);
}
const isMain =
process.argv[1] &&
path.resolve(process.argv[1]) === fileURLToPath(import.meta.url);
if (isMain) {
if (!only || only === 'countries')
await writePack(
'src/data/local_data/natural_earth/countries.json',
await buildCountries(),
);
if (!only || only === 'admin1')
await writePack(
'src/data/local_data/natural_earth/states_provinces.json',
await buildAdmin1(),
);
if (!only || only === 'counties')
await writePack(
'src/data/local_data/us_census_counties/counties.json',
await buildCounties(),
);
}
if (!only || only === 'countries')
await writePack(
'src/data/local_data/natural_earth/countries.json',
await buildCountries(),
);
if (!only || only === 'admin1')
await writePack(
'src/data/local_data/natural_earth/states_provinces.json',
await buildAdmin1(),
);
if (!only || only === 'counties')
await writePack(
'src/data/local_data/us_census_counties/counties.json',
await buildCounties(),
);
-418
View File
@@ -1,418 +0,0 @@
#!/usr/bin/env node
/**
* Build the bundled US places pack read by `src/data/placeBoundaries.js`:
*
* src/data/local_data/us_census_places/index.json state list, boxes, meta
* src/data/local_data/us_census_places/<ST>.json one file per state
* src/data/local_data/us_census_places/files.js their URLs, for bundlers
*
* Source: US Census Bureau 2025 cartographic boundary places
* (cb_2025_us_place_500k, public domain), pinned by URL and SHA-256, so a
* rerun reproduces the pack byte for byte. The download is cached outside the
* repository (default: the system temp directory).
*
* Geometry uses the county pack's simplification and encoding
* (scripts/build-admin-packs.mjs) with two differences: triangles are kept,
* and a unit whose simplified rings cross themselves or each other is
* simplified again at half the tolerance (finer precision last). Then
* `scripts/pack_geometry.py repair census` checks every feature as a whole
* MultiPolygon under GEOS and repairs the ones that are still invalid (rings
* touching themselves, overlapping parts), so it needs Python with
* shapely==2.1.2 on GEOS 3.13.1.
*
* Usage:
* node scripts/build-census-places.mjs [--cache <dir>] [--python <exe>]
*/
import { execFileSync } from 'node:child_process';
import { mkdir, readdir, rm, writeFile } from 'node:fs/promises';
import path from 'node:path';
import { fileURLToPath } from 'node:url';
import {
PARAMS as ADMIN_PARAMS,
encodeUnit,
fetchPinned,
readDbf,
readShp,
shpPolygons,
simplifyPolygons,
unitTolerance,
unzip,
} from './build-admin-packs.mjs';
const ROOT = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '..');
const OUT = path.join(ROOT, 'src/data/local_data/us_census_places');
export const SOURCE = Object.freeze({
url: 'https://www2.census.gov/geo/tiger/GENZ2025/shp/cb_2025_us_place_500k.zip',
sha256: 'ce0e4019ecd4123d03d53aaa936eed0459b82e3e14b89a3dcd4d5e8b3308627d',
});
/** County-pack parameters; triangles are valid rings here. */
export const PARAMS = Object.freeze({
...ADMIN_PARAMS.counties,
minRingVertices: 3,
maxRepairSteps: 8,
});
/** Census LSAD codes for census designated places (unincorporated). */
const CDP_LSAD = new Set(['55', '57', '62']);
// ── validity ─────────────────────────────────────────────────────────────
function orient(ax, ay, bx, by, cx, cy) {
const v = (bx - ax) * (cy - ay) - (by - ay) * (cx - ax);
return v > 0 ? 1 : v < 0 ? -1 : 0;
}
function onSegment(ax, ay, bx, by, px, py) {
return (
Math.min(ax, bx) <= px &&
px <= Math.max(ax, bx) &&
Math.min(ay, by) <= py &&
py <= Math.max(ay, by)
);
}
/** Whether segments ab and cd share any point (touching counts). */
export function segmentsIntersect(a, b, c, d) {
const o1 = orient(a[0], a[1], b[0], b[1], c[0], c[1]);
const o2 = orient(a[0], a[1], b[0], b[1], d[0], d[1]);
const o3 = orient(c[0], c[1], d[0], d[1], a[0], a[1]);
const o4 = orient(c[0], c[1], d[0], d[1], b[0], b[1]);
if (o1 !== o2 && o3 !== o4) return true;
if (o1 === 0 && onSegment(a[0], a[1], b[0], b[1], c[0], c[1])) return true;
if (o2 === 0 && onSegment(a[0], a[1], b[0], b[1], d[0], d[1])) return true;
if (o3 === 0 && onSegment(c[0], c[1], d[0], d[1], a[0], a[1])) return true;
if (o4 === 0 && onSegment(c[0], c[1], d[0], d[1], b[0], b[1])) return true;
return false;
}
/**
* Whether segments of two different rings cross or overlap. Rings may touch
* at a point (a hole meeting its outer ring at a vertex is valid).
*/
function segmentsCross(s, t) {
const [a, b, c, d] = [s.a, s.b, t.a, t.b];
const o1 = orient(a[0], a[1], b[0], b[1], c[0], c[1]);
const o2 = orient(a[0], a[1], b[0], b[1], d[0], d[1]);
const o3 = orient(c[0], c[1], d[0], d[1], a[0], a[1]);
const o4 = orient(c[0], c[1], d[0], d[1], b[0], b[1]);
if (o1 && o2 && o3 && o4) return o1 !== o2 && o3 !== o4;
if (o1 === 0 && o2 === 0) {
// Collinear: overlapping by more than a point.
const key = a[0] !== b[0] ? 0 : 1;
const lo = Math.max(Math.min(a[key], b[key]), Math.min(c[key], d[key]));
const hi = Math.min(Math.max(a[key], b[key]), Math.max(c[key], d[key]));
return hi > lo;
}
// One endpoint on the other segment: a crossing only when the rings pass
// through each other there; a touch at a point is allowed.
return false;
}
/**
* Whether the rings of one part ([outer, ...holes], open, integer
* coordinates) are simple: no ring crosses or touches itself away from its
* shared vertices, no two rings cross or overlap (they may touch at a point),
* and no ring is flat.
*/
export function partIsValid(rings) {
const segments = [];
for (let r = 0; r < rings.length; r++) {
const ring = rings[r];
if (ring.length < 3) return false;
let twice = 0;
for (let i = 0, j = ring.length - 1; i < ring.length; j = i++)
twice += ring[j][0] * ring[i][1] - ring[i][0] * ring[j][1];
if (twice === 0) return false;
for (let i = 0; i < ring.length; i++) {
const a = ring[i];
const b = ring[(i + 1) % ring.length];
segments.push({
r,
i,
n: ring.length,
a,
b,
minX: Math.min(a[0], b[0]),
maxX: Math.max(a[0], b[0]),
});
}
}
segments.sort((s, t) => s.minX - t.minX);
for (let x = 0; x < segments.length; x++) {
const s = segments[x];
for (let y = x + 1; y < segments.length; y++) {
const t = segments[y];
if (t.minX > s.maxX) break;
if (s.r === t.r) {
const gap = Math.abs(s.i - t.i);
// Neighbours share exactly one vertex; that is not a crossing — unless
// they fold back over each other.
if (gap === 1 || gap === s.n - 1) {
if (s.n > 3 && collinearOverlap(s, t)) return false;
continue;
}
}
if (segmentsCross(s, t)) return false;
}
}
return true;
}
function collinearOverlap(s, t) {
const shared =
s.a === t.a || s.a === t.b ? s.a : s.b === t.a || s.b === t.b ? s.b : null;
if (!shared) return false;
const p = s.a === shared ? s.b : s.a;
const q = t.a === shared ? t.b : t.a;
if (orient(shared[0], shared[1], p[0], p[1], q[0], q[1]) !== 0) return false;
// Collinear and pointing the same way from the shared vertex: a spike.
return (
(p[0] - shared[0]) * (q[0] - shared[0]) +
(p[1] - shared[1]) * (q[1] - shared[1]) >
0
);
}
function toIntegers(polygons, decimals) {
const f = 10 ** decimals;
return polygons.map((poly) =>
poly.map((ring) =>
ring.map(([lon, lat]) => [Math.round(lon * f), Math.round(lat * f)]),
),
);
}
export function polygonsValid(polygons, decimals) {
return toIntegers(polygons, decimals).every(partIsValid);
}
/**
* Simplify a place like `simplifyUnit`, then halve the tolerance until every
* part is valid; then one more decimal; then the source rings at two more.
* @returns {{polygons: Array, decimals: number, repaired: boolean, valid: boolean}}
*/
export function simplifyPlace(source, params = PARAMS) {
let tolerance = unitTolerance(source, params);
const attempts = [];
for (let step = 0; step <= params.maxRepairSteps; step++) {
attempts.push([tolerance, params.decimals]);
tolerance /= 2;
}
attempts.push([params.minToleranceDeg / 5, params.decimals + 1]);
attempts.push([0, params.decimals + 1]);
attempts.push([0, params.decimals + 2]);
let first = null;
for (let i = 0; i < attempts.length; i++) {
const [tol, decimals] = attempts[i];
const polygons = simplifyPolygons(source, params, tol, decimals);
if (!polygons.length) continue;
first ||= { polygons, decimals };
if (polygonsValid(polygons, decimals))
return { polygons, decimals, repaired: i > 0, valid: true };
}
return first
? { ...first, repaired: false, valid: false }
: {
polygons: [],
decimals: params.decimals,
repaired: false,
valid: false,
};
}
// ── names ────────────────────────────────────────────────────────────────
const GOVERNMENT =
/\s+(?:metropolitan government|unified government|consolidated government|metro government|city and county|urban county)\b.*$/i;
/**
* Everyday names for records whose Census name is formal: "Nashville-Davidson
* metropolitan government (balance)" is also "Nashville"; "Urban Honolulu"
* is "Honolulu".
*/
export function placeAliases(name) {
const out = new Set();
let base = String(name).replace(/\s*\(balance\)\s*$/i, '');
if (base !== name) out.add(base);
const stripped = base.replace(GOVERNMENT, '');
if (stripped !== base) {
out.add(stripped);
base = stripped;
}
if (base !== name && /[-/]/.test(base)) out.add(base.split(/[-/]/)[0].trim());
const urban = /^Urban (.+)$/.exec(name);
if (urban) out.add(urban[1]);
out.delete(name);
return [...out].filter(Boolean).sort();
}
// ── pack ─────────────────────────────────────────────────────────────────
function bboxOf(polygons) {
let w = Infinity;
let s = Infinity;
let e = -Infinity;
let n = -Infinity;
for (const poly of polygons)
for (const [lon, lat] of poly[0]) {
if (lon < w) w = lon;
if (lon > e) e = lon;
if (lat < s) s = lat;
if (lat > n) n = lat;
}
return [w, s, e, n];
}
function union(boxes) {
return [
Math.min(...boxes.map((b) => b[0])),
Math.min(...boxes.map((b) => b[1])),
Math.max(...boxes.map((b) => b[2])),
Math.max(...boxes.map((b) => b[3])),
];
}
/**
* The box of a state's places. Alaska crosses the antimeridian: its box is
* taken with western longitudes shifted past 180 when that is narrower.
*/
function unionBox(boxes) {
const plain = union(boxes);
const shifted = union(
boxes.map(([w, s, e, n]) =>
e < 0 ? [w + 360, s, e + 360, n] : [w, s, e, n],
),
);
return shifted[2] - shifted[0] < plain[2] - plain[0] ? shifted : plain;
}
async function build() {
const params = PARAMS;
const source = await fetchPinned('places', SOURCE);
const files = unzip(source.bytes);
const base = path.basename(new URL(SOURCE.url).pathname, '.zip');
const shapes = readShp(files.get(`${base}.shp`));
const records = readDbf(files.get(`${base}.dbf`));
if (shapes.length !== records.length)
throw new Error('places: shp/dbf record counts differ');
const states = new Map();
const stats = { records: records.length, repaired: [], invalid: [] };
records.forEach((record, index) => {
const unit = simplifyPlace(shpPolygons(shapes[index]), params);
if (!unit.polygons.length) throw new Error(`no geometry: ${record.GEOID}`);
if (unit.repaired) stats.repaired.push(record.GEOID);
if (!unit.valid) stats.invalid.push(record.GEOID);
const alt = placeAliases(record.NAME);
const lsad = record.NAMELSAD.startsWith(record.NAME)
? record.NAMELSAD.slice(record.NAME.length).trim()
: '';
let state = states.get(record.STUSPS);
if (!state) {
state = {
st: record.STUSPS,
fips: record.STATEFP,
name: record.STATE_NAME,
features: [],
boxes: [],
};
states.set(record.STUSPS, state);
}
state.boxes.push(bboxOf(unit.polygons));
state.features.push({
geoid: record.GEOID,
name: record.NAME,
...(alt.length ? { alt } : {}),
full: record.NAMELSAD,
...(lsad ? { lsad } : {}),
...(CDP_LSAD.has(record.LSAD) ? { cdp: true } : {}),
...(unit.decimals !== params.decimals ? { d: unit.decimals } : {}),
polygons: encodeUnit(unit),
});
});
if (stats.invalid.length)
throw new Error(`places still invalid: ${stats.invalid.join(', ')}`);
await mkdir(OUT, { recursive: true });
for (const file of await readdir(OUT))
if (/^[A-Z]{2}\.json$/.test(file)) await rm(path.join(OUT, file));
const list = [...states.values()].sort((a, b) => a.st.localeCompare(b.st));
let bytes = 0;
for (const state of list) {
state.features.sort((a, b) => a.geoid.localeCompare(b.geoid));
const text =
JSON.stringify({
st: state.st,
state: state.name,
fips: state.fips,
features: state.features,
}) + '\n';
bytes += Buffer.byteLength(text);
await writeFile(path.join(OUT, `${state.st}.json`), text);
}
const round = (v) => Math.round(v * 1e4) / 1e4;
const index = {
meta: {
title: 'US Census Bureau cartographic boundary places',
source: base,
url: SOURCE.url,
sha256: source.digest,
license: 'Public domain (U.S. Government work, 17 U.S.C. § 105)',
schema:
'states[]: st (USPS), fips, name, count, bbox [w, s, e, n] (e > 180 across the antimeridian). <st>.json: st, state, fips, features[]: geoid, name (NAME), alt?[] (everyday names for formal records), full (NAMELSAD), lsad? (city, town, CDP, …), cdp? (census designated place), d? (decimals when not meta.decimals), polygons[][ring] (first ring outer, rest holes; each ring open and encoded as integers in 10^-d degrees, first vertex absolute then [dLon, dLat] deltas)',
decimals: params.decimals,
curation: params,
script: 'scripts/build-census-places.mjs',
},
states: list.map((s) => ({
st: s.st,
fips: s.fips,
name: s.name,
count: s.features.length,
bbox: unionBox(s.boxes).map(round),
})),
};
await writeFile(
path.join(OUT, 'index.json'),
JSON.stringify(index, null, 1) + '\n',
);
await writeFile(path.join(OUT, 'files.js'), filesModule(list));
const count = list.reduce((n, s) => n + s.features.length, 0);
console.log(
`us_census_places: ${count} places in ${list.length} files, ${bytes} bytes; ` +
`${stats.repaired.length} re-simplified to stay valid`,
);
const args = process.argv.slice(2);
const python = args.includes('--python')
? args[args.indexOf('--python') + 1]
: 'python3';
execFileSync(
python,
[path.join(ROOT, 'scripts/pack_geometry.py'), 'repair', 'census'],
{ stdio: 'inherit', env: { ...process.env, PYTHONDONTWRITEBYTECODE: '1' } },
);
}
/**
* The file list as literal URLs, so a bundler emits every state file as an
* asset and the loader can fetch one by its code.
*/
function filesModule(list) {
const lines = [
'// Generated by scripts/build-census-places.mjs; do not edit.',
"export const INDEX_URL = new URL('./index.json', import.meta.url);",
'export const STATE_URLS = {',
...list.map(
(s) => ` ${s.st}: new URL('./${s.st}.json', import.meta.url),`,
),
'};',
];
return lines.join('\n') + '\n';
}
const isMain =
process.argv[1] &&
path.resolve(process.argv[1]) === fileURLToPath(import.meta.url);
if (isMain) await build();
+46
View File
@@ -0,0 +1,46 @@
#!/usr/bin/env node
/**
* Build the MCP Apps panel: the app under dist/panel, served at /panel/.
* See build/panel.js.
*/
import { readFile, rename, rm, writeFile } from 'node:fs/promises';
import { join } from 'node:path';
import { fileURLToPath } from 'node:url';
import { build } from 'vite';
import standaloneConfig from '../server/standalone/vite.config.js';
import {
PANEL_BASE,
PANEL_OUT_DIR,
PANEL_WORKER_FILES,
PANEL_WORKER_PRELUDE_PATH,
panelBuildConfig,
workerFilesPrelude,
} from '../build/panel.js';
const root = fileURLToPath(new URL('../', import.meta.url));
const outDir = join(root, PANEL_OUT_DIR);
const config = standaloneConfig({ command: 'build', mode: 'production' });
await build({
configFile: false,
root,
...panelBuildConfig(config),
});
// Cesium's plugin copies its files under the base path inside outDir.
const nested = join(outDir, PANEL_BASE, 'cesium');
await rm(join(outDir, 'cesium'), { recursive: true, force: true });
await rename(nested, join(outDir, 'cesium'));
await rm(join(outDir, PANEL_BASE.split('/')[1]), { recursive: true });
// Cesium's workers load these files themselves; embed them in a prelude the
// panel runs ahead of Cesium's workers script.
const files = {};
for (const name of PANEL_WORKER_FILES)
files[name] = await readFile(join(outDir, 'cesium', name), 'utf8');
await writeFile(
join(outDir, PANEL_WORKER_PRELUDE_PATH),
workerFilesPrelude(files),
);
console.log(`Panel build written to ${PANEL_OUT_DIR}`);
-382
View File
@@ -1,382 +0,0 @@
#!/usr/bin/env python3
"""Maintainer-only Who's On First neighborhood pack build (Shapely 2.1).
Reads the pinned Who's On First neighbourhood, macrohood and microhood
archives and the WOF sources registry, keeps current polygon records whose
geometry source matches the reviewed open-licence allowlist, simplifies them
to about 15 m, and
writes src/data/local_data/wof_neighborhoods/:
index.json format, licences per source, tile list with boxes
<tile>.json records whose box centre falls in that quadtree tile
files.js their URLs, for bundlers
ATTRIBUTION.md per-source credits
Usage:
python scripts/build-wof-neighborhoods.py --cache /tmp/wof-neighborhoods [--verify]
Requires shapely==2.1.2 on GEOS 3.13.1 (checked). --verify rebuilds in memory
and fails unless every shipped file matches byte for byte.
Downloads go to --cache and are checked against the pinned SHA-256. The
neighbourhood archive is about 400 MB.
"""
import argparse
import collections
import hashlib
import json
import math
import subprocess
import tarfile
from pathlib import Path
from shapely import make_valid
from shapely.affinity import scale
from shapely.geometry import MultiPolygon, shape
import pack_geometry
ROOT = Path(__file__).resolve().parent.parent
PACK = ROOT / 'src/data/local_data/wof_neighborhoods'
# The legacy distribution is frozen (last published 2025-10-14) and only
# offers `-latest` names; the SHA-256 pins make a changed file fail loudly.
DIST = 'https://data.geocode.earth/wof/dist/legacy'
ARCHIVES = {
'neighbourhood': {
'url': DIST + '/whosonfirst-data-neighbourhood-latest.tar.bz2',
'sha256': '8e5e413aacf2244e4e4167fbd9be860bc407ae6de8b733e47f13abe6d3d10455',
},
'macrohood': {
'url': DIST + '/whosonfirst-data-macrohood-latest.tar.bz2',
'sha256': 'f5fbfabcb8229b83242dbd38530c27e61867e8b3b097e3393d9c841962a99679',
},
'microhood': {
'url': DIST + '/whosonfirst-data-microhood-latest.tar.bz2',
'sha256': 'e8197fa576e8831be8f091ac7e219d29224d89404ab0c56d377339e42a8f29ac',
},
}
# The registry at a fixed commit.
SOURCES = {
'url': 'https://raw.githubusercontent.com/whosonfirst/whosonfirst-sources/'
'e17af153c144fe6e08aa8db2607b4bf8aecd448d/data/sources-spec-latest.json',
'sha256': 'd8a5dd873d26eee11583de4438185c046dec33848d5bde27047ca4bff44d9c39',
}
TOLERANCE_M = 15
PRECISION = 5
TILE_MAX_BYTES = 200_000
TILE_MAX_DEPTH = 16
LICENSE_POLICY = json.loads((ROOT / 'scripts/wof-open-licenses.json').read_text())
ALLOWED_LICENSES = frozenset(LICENSE_POLICY['licenses'])
REVIEWED_SOURCES = LICENSE_POLICY.get('reviewedSources', {})
FIELDS = ['id', 'name', 'country', 'type', 'bbox', 'aliases', 'source', 'rings', 'label']
def fetch(cache, name, url, sha256):
path = cache / name
if not path.exists():
print('Downloading', url, flush=True)
# Plain urllib requests are refused by the WOF host; curl is not.
subprocess.run(['curl', '--fail', '-L', '-o', str(path), url], check=True)
digest = hashlib.sha256()
with open(path, 'rb') as handle:
for chunk in iter(lambda: handle.read(1 << 20), b''):
digest.update(chunk)
if digest.hexdigest() != sha256:
raise ValueError(f'{name}: SHA-256 {digest.hexdigest()} is not the pinned {sha256}')
return path
# ── geometry ────────────────────────────────────────────────────────────
def simplify(geometry):
"""Simplify at TOLERANCE_M in local metres, snap to the pack grid, keep it
valid, and order parts largest first. Returns (rings, label) where the
label is a point inside the largest part, or ([], None)."""
geometry = MultiPolygon(pack_geometry.polygon_parts(make_valid(geometry) if not geometry.is_valid else geometry))
if geometry.is_empty:
return [], None
lat = geometry.centroid.y
sx = 111320 * max(0.001, math.cos(math.radians(lat)))
sy = 110574
metric = scale(geometry, xfact=sx, yfact=sy, origin=(0, 0))
metric = metric.simplify(TOLERANCE_M, preserve_topology=True)
geometry = scale(metric, xfact=1 / sx, yfact=1 / sy, origin=(0, 0))
# To whole pack units (10^-PRECISION degrees), where GEOS is exact.
units = scale(geometry, xfact=10 ** PRECISION, yfact=10 ** PRECISION, origin=(0, 0))
parts = pack_geometry.ordered_parts(pack_geometry.settle(units))
if not parts:
return [], None
rings = [
[pack_geometry.encode_polyline(pack_geometry.open_ring(p.exterior.coords))]
+ [pack_geometry.encode_polyline(pack_geometry.open_ring(r.coords)) for r in p.interiors]
for p in parts
]
# Check what ships: the geometry decoded from the encoded strings.
decoded = [[pack_geometry.decode_polyline(r) for r in poly] for poly in rings]
why = pack_geometry.problems(decoded)
if why:
raise ValueError(f'encoded geometry invalid: {why}')
point = parts[0].representative_point()
point = shape({'type': 'Point', 'coordinates': [round(point.x) / 10 ** PRECISION, round(point.y) / 10 ** PRECISION]})
return rings, [point.x, point.y]
# ── records ─────────────────────────────────────────────────────────────
def read_archive(path, records):
with tarfile.open(path, 'r|bz2') as archive:
for member in archive:
if not member.name.endswith('.geojson') or '-alt-' in member.name:
continue
feature = json.load(archive.extractfile(member))
if feature['geometry']['type'] not in ('Polygon', 'MultiPolygon'):
records.pop(feature['properties']['wof:id'], None)
continue
records[feature['properties']['wof:id']] = feature
def keep(properties):
deprecated = properties.get('edtf:deprecated', 'uuuu') not in ('uuuu', '', None)
superseded = bool(properties.get('wof:superseded_by', []))
return properties.get('mz:is_current', -1) != 0 and not deprecated and not superseded
def source_table(spec):
lookup = {}
for source in spec.values():
for key in (source.get('name'), source.get('prefix')):
if key:
lookup[key] = source
return lookup
def verify_source_reviews(lookup):
"""Pin exceptional source decisions to the exact reviewed registry evidence."""
registry_commit = SOURCES['url'].split('/')[-3]
for key, review in REVIEWED_SOURCES.items():
source = lookup.get(key)
if not source:
raise ValueError(f'{key}: reviewed source is absent from the pinned registry')
if review.get('registryCommit') != registry_commit:
raise ValueError(f'{key}: review does not match the pinned registry commit')
if review.get('registrySha256') != SOURCES['sha256']:
raise ValueError(f'{key}: review does not match the pinned registry SHA-256')
if review.get('rawLicenseType') != source.get('license_type'):
raise ValueError(f'{key}: reviewed raw licence no longer matches the registry')
text = source.get('license_text') or ''
if review.get('licenseTextSha256') != hashlib.sha256(text.encode()).hexdigest():
raise ValueError(f'{key}: reviewed licence text no longer matches the registry')
if review['rawLicenseType'] not in ALLOWED_LICENSES:
raise ValueError(f'{key}: reviewed raw licence is not admitted by policy')
def licence_allowed(value):
"""Only exact raw registry strings reviewed in the policy are accepted."""
return isinstance(value, str) and value in ALLOWED_LICENSES
def source_allowed(key, source):
return (key != 'unknown' and bool(source) and
licence_allowed(source.get('license_type')))
def build_records(archives, lookup):
raw = {}
for kind in ('neighbourhood', 'macrohood', 'microhood'):
read_archive(archives[kind], raw)
print(kind, len(raw), flush=True)
features = []
excluded = collections.Counter()
for wof_id in sorted(raw):
properties = raw[wof_id]['properties']
if not keep(properties):
continue
key = properties.get('src:geom', 'unknown')
if not source_allowed(key, lookup.get(key)):
excluded[key] += 1
continue
name = properties['wof:name']
aliases = sorted({
alias
for k, values in properties.items()
if k.startswith('name:') and isinstance(values, list)
for alias in values
if isinstance(alias, str) and alias != name
})
if not name.strip() and not aliases:
continue # nothing to ask for it by
geometry = shape(raw[wof_id]['geometry'])
if geometry.is_empty:
continue
rings, label = simplify(geometry)
if not rings:
continue
country = properties.get('iso:country', properties.get('wof:country'))
features.append([
wof_id, name, country, properties['wof:placetype'],
[round(v, PRECISION) for v in geometry.bounds], aliases, key, rings, label,
])
return features, excluded
# ── tiles ───────────────────────────────────────────────────────────────
def compact(value):
return json.dumps(value, ensure_ascii=False, separators=(',', ':'))
def tiles(features, key='', box=(-180.0, -90.0, 180.0, 90.0)):
size = sum(len(compact(f).encode()) for f in features)
if size <= TILE_MAX_BYTES or len(key) >= TILE_MAX_DEPTH:
yield key or 'world', features
return
west, south, east, north = box
mx, my = (west + east) / 2, (south + north) / 2
quads = [[], [], [], []]
for f in features:
x = (f[4][0] + f[4][2]) / 2
y = (f[4][1] + f[4][3]) / 2
quads[(1 if x >= mx else 0) + (2 if y >= my else 0)].append(f)
boxes = [(west, south, mx, my), (mx, south, east, my), (west, my, mx, north), (mx, my, east, north)]
for i, quad in enumerate(quads):
if quad:
yield from tiles(quad, key + str(i), boxes[i])
def union_box(features):
return [
min(f[4][0] for f in features), min(f[4][1] for f in features),
max(f[4][2] for f in features), max(f[4][3] for f in features),
]
# ── credits ─────────────────────────────────────────────────────────────
def licence_label(key, source):
licence = source['license_type']
if key == 'karmashapes':
return 'CC BY 4.0'
return {
'CC-BY': 'CC BY', 'Attribution 3.0 Unported': 'CC BY 3.0', 'Public domain': 'Public Domain',
'Open Government License - British Columbia, v2.0': 'Open Government Licence - British Columbia, v2.0',
}.get(licence, licence)
def attribution(counts, lookup):
lines = [
'# Neighborhood boundary credits',
'',
"Neighborhood boundaries from [Who's On First](https://whosonfirst.org/docs/licenses/) "
'and contributors, used under the licences below. Modified: boundaries simplified to about 15 m, repaired and re-encoded.',
'',
'| Source | Records | Licence | Terms |',
'|---|---:|---|---|',
]
for key, count in sorted(counts.items(), key=lambda kv: (-kv[1], kv[0])):
source = lookup[key]
name = source.get('fullname') or key
terms = source.get('license') or ''
link = f'[terms]({terms})' if terms.startswith('http') else terms
lines.append(f'| {name} (`{key}`) | {count:,} | {licence_label(key, source)} | {link} |')
via = [(key, lookup[key]) for key in sorted(counts) if lookup[key].get('src:via')]
if via:
lines += ['', '## Upstream sources', '']
for key, source in via:
lines += [f'### {source.get("fullname") or key}', '']
for v in source['src:via']:
note = f' ({v["source_note"]})' if v.get('source_note') else ''
context = f'{v["context"]}: ' if v.get('context') else ''
lines.append(f'- {context}{v.get("source_name", "")}{note} {v.get("source_link", "")}'.rstrip())
lines.append('')
return '\n'.join(lines).rstrip() + '\n'
def main():
parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
parser.add_argument('--cache', type=Path, required=True)
parser.add_argument('--verify', action='store_true', help='rebuild in memory and compare with the shipped pack')
args = parser.parse_args()
pack_geometry.require_pinned_tooling()
args.cache.mkdir(parents=True, exist_ok=True)
archives = {
kind: fetch(args.cache, Path(a['url']).name, a['url'], a['sha256'])
for kind, a in ARCHIVES.items()
}
spec = json.loads(fetch(args.cache, 'sources-spec-latest.json', SOURCES['url'], SOURCES['sha256']).read_text())
lookup = source_table(spec)
verify_source_reviews(lookup)
features, excluded = build_records(archives, lookup)
if len({f[0] for f in features}) != len(features):
raise ValueError('duplicate WOF id')
out = {}
tile_list = []
for key, members in tiles(features):
members.sort(key=lambda f: f[0])
out[f'{key}.json'] = compact({'tile': key, 'features': members}) + '\n'
tile_list.append({'key': key, 'count': len(members), 'bbox': union_box(members)})
counts = collections.Counter(f[6] for f in features)
index = {
'meta': {
'title': "Who's On First neighbourhood, macrohood and microhood polygons",
'archives': ARCHIVES,
'sourcesRegistry': SOURCES,
'license': 'Per source; see sources and ATTRIBUTION.md',
'format': 'neighborhood-polylines-v1',
'fields': FIELDS,
'precision': PRECISION,
'coordinateOrder': 'lon,lat',
'toleranceM': TOLERANCE_M,
'schema': (
'tiles[]: key, count, bbox [w, s, e, n] of its records. <key>.json: tile, features[] as '
'fields: WOF id, name, ISO country, placetype, bbox, aliases[] (other-language names), '
'geometry source key, rings (polygons, largest first, each [outer, ...holes]; each ring '
'an encoded polyline string: zigzag varint deltas of lon then lat at 10^-precision '
'degrees), label [lon, lat] (a point inside the largest polygon)'
),
'sources': {
key: {
'name': lookup[key].get('fullname') or key,
'license': licence_label(key, lookup[key]),
'terms': lookup[key].get('license') or '',
'records': counts[key],
}
for key in sorted(counts)
},
'script': 'scripts/build-wof-neighborhoods.py',
},
'tiles': tile_list,
}
out['index.json'] = json.dumps(index, ensure_ascii=False, indent=1) + '\n'
out['ATTRIBUTION.md'] = attribution(counts, lookup)
# Literal URLs, so a bundler emits every tile as an asset.
out['files.js'] = '\n'.join([
'// Generated by scripts/build-wof-neighborhoods.py; do not edit.',
"export const INDEX_URL = new URL('./index.json', import.meta.url);",
'export const TILE_URLS = {',
*[f" t{t['key']}: new URL('./{t['key']}.json', import.meta.url)," for t in tile_list],
'};',
]) + '\n'
if args.verify:
# Reproducibility: a rebuild must match what ships, byte for byte.
shipped = {p.name for p in PACK.glob('*.json')} | {'ATTRIBUTION.md', 'files.js'}
differ = sorted(
name for name in shipped | set(out)
if not (PACK / name).exists() or name not in out
or (PACK / name).read_bytes() != out[name].encode('utf-8')
)
if differ:
raise SystemExit(f'rebuild differs from the shipped pack: {differ[:10]}')
print(f'wof_neighborhoods: rebuild matches all {len(out)} shipped files')
return
PACK.mkdir(parents=True, exist_ok=True)
for old in PACK.glob('*.json'):
old.unlink()
for name, text in out.items():
(PACK / name).write_text(text, encoding='utf-8')
print(f'wof_neighborhoods: {len(features)} records in {len(tile_list)} tiles; '
f'excluded {sum(excluded.values())} with an unapproved licence: {dict(excluded)}', flush=True)
if __name__ == '__main__':
main()
+4
View File
@@ -37,6 +37,10 @@ const portableExport = (key) =>
) ||
[
'./director',
'./tools',
'./tools/mcp',
'./tools/services',
'./view',
'./voice/action-schemas',
'./voice/session',
'./data/lifecycle',
+32 -6
View File
@@ -30,11 +30,6 @@
"src/annotations/regionRing.test.mjs",
"src/data/adminBoundaries.test.mjs",
"scripts/build-admin-packs.mjs",
"scripts/build-census-places.mjs",
"src/data/placeBoundaries.test.mjs",
"src/data/boundaryLabels.test.mjs",
"src/data/packGeometry.test.mjs",
"src/annotations/placeOutline.test.mjs",
"scripts/qa-admin-outlines.mjs",
"src/data/inputOwnership.test.mjs",
"docs/CODE-BOUNDARIES.md",
@@ -60,6 +55,7 @@
"server/standalone/vite.config.js",
"vite.config.js",
"src/tooling/viteBuild.test.mjs",
"src/tooling/allowedHosts.test.mjs",
"server/providers/common/http.js",
"server/providers/common/query.js",
"server/providers/aircraft/opensky.js",
@@ -81,6 +77,7 @@
"server/providers/places/routes.js",
"src/data/placeProviderPayloads.js",
"src/tooling/placeProviders.test.mjs",
"server/providers/common/same-site.js",
"server/providers/space.js",
"server/providers/space/celestrak.js",
"server/providers/space/launch-library.js",
@@ -869,5 +866,34 @@
"src/layers/traffic/motion.test.mjs",
"scripts/qa-journey-recorder.mjs",
"scripts/qa-alpr-journey.mjs",
"scripts/qa-context-arrivals.mjs"
"scripts/qa-context-arrivals.mjs",
"src/tools/catalog.test.mjs",
"src/tools/area.test.mjs",
"src/tools/queries/queries.test.mjs",
"src/tools/mcp/mcp.test.mjs",
"src/tools/localServer.test.mjs",
"src/tools/queries/aviation.test.mjs",
"src/tools/queries/space.test.mjs",
"src/tools/queries/media.test.mjs",
"src/tools/queries/places.test.mjs",
"src/layers/transit/source.test.mjs",
"src/tools/queries/mobility.test.mjs",
"src/tools/queries/environment.test.mjs",
"src/tools/queries/maritime.test.mjs",
"src/tools/queries/share.test.mjs",
"src/tools/localRoute.test.mjs",
"src/tools/queries/atmosphere.test.mjs",
"src/tools/queries/imagery.test.mjs",
"src/tools/queries/cables.test.mjs",
"src/tools/queries/surveillance.test.mjs",
"src/tools/queries/infrastructure.test.mjs",
"src/tools/queries/events.test.mjs",
"src/tools/functions.test.mjs",
"src/tools/surfaces.test.mjs",
"src/tools/places.test.mjs",
"src/view/index.test.mjs",
"src/tools/views.test.mjs",
"src/app/embed.test.mjs",
"src/tools/globePanel.test.mjs",
"src/costEndpointThrottle.test.mjs"
]
-247
View File
@@ -1,247 +0,0 @@
#!/usr/bin/env python3
"""Maintainer-only topology check and repair for the bundled outline packs.
python scripts/pack_geometry.py check census|wof every shipped feature,
decoded as a whole MultiPolygon, is valid under GEOS, keeps at least
three distinct vertices per ring, and lists its largest part first
python scripts/pack_geometry.py repair census rewrite the Census place
files so every feature passes `check` (run by build-census-places.mjs)
Geometry is checked and repaired in the packs' own integer units (10^-d
degrees), where GEOS arithmetic is exact: a vertex that lies on another edge
after rounding is seen as the touch it is.
Requires Shapely 2.1.2 on GEOS 3.13.1, pinned so repairs are reproducible.
"""
import argparse
import json
import sys
from pathlib import Path
import shapely
from shapely import make_valid, set_precision
from shapely.geometry import MultiPolygon, Polygon
from shapely.geometry.polygon import orient
ROOT = Path(__file__).resolve().parent.parent
DATA = ROOT / 'src/data/local_data'
SHAPELY_VERSION = '2.1.2'
GEOS_VERSION = (3, 13, 1)
def require_pinned_tooling():
if shapely.__version__ != SHAPELY_VERSION or shapely.geos_version != GEOS_VERSION:
raise RuntimeError(
f'Use shapely=={SHAPELY_VERSION} on GEOS {".".join(map(str, GEOS_VERSION))}; '
f'found {shapely.__version__} on GEOS {shapely.geos_version}'
)
def compact(value):
return json.dumps(value, ensure_ascii=False, separators=(',', ':'))
# ── encodings ────────────────────────────────────────────────────────────
def decode_deltas(encoded):
"""[x0, y0, dx1, dy1, …] → [(x, y), …] in pack units (Census/admin packs)."""
x = y = 0
ring = []
for i in range(0, len(encoded) - 1, 2):
x += encoded[i]
y += encoded[i + 1]
ring.append((x, y))
return ring
def encode_deltas(coords):
out = []
px = py = 0
for fx, fy in coords:
x, y = round(fx), round(fy)
out += [x - px, y - py]
px, py = x, y
return out
def decode_polyline(encoded):
"""Zigzag varint deltas, 5-bit groups offset by 63, lon then lat; pack units."""
values = []
last = [0, 0]
i = 0
while i < len(encoded):
n = shift = 0
while True:
b = ord(encoded[i]) - 63
i += 1
n |= (b & 31) << shift
shift += 5
if b < 32:
break
axis = len(values) % 2
last[axis] += ~(n >> 1) if n & 1 else n >> 1
values.append(last[axis])
return list(zip(values[::2], values[1::2]))
def encode_polyline(coords):
out = []
last = [0, 0]
for x, y in coords:
for i, value in enumerate([round(x), round(y)]):
n = value - last[i]
last[i] = value
n = (n << 1) ^ (n >> 63)
while n >= 32:
out.append(chr((32 | (n & 31)) + 63))
n >>= 5
out.append(chr(n + 63))
return ''.join(out)
# ── geometry ─────────────────────────────────────────────────────────────
def polygon_parts(geometry):
if geometry.geom_type == 'Polygon':
return [] if geometry.is_empty else [geometry]
if hasattr(geometry, 'geoms'):
return [p for g in geometry.geoms for p in polygon_parts(g)]
return []
def open_ring(coords):
coords = [tuple(c[:2]) for c in coords]
return coords[:-1] if len(coords) > 1 and coords[0] == coords[-1] else coords
def ordered_parts(geometry):
"""Polygon parts oriented (outer counter-clockwise), largest area first."""
parts = [orient(p, 1.0) for p in polygon_parts(geometry)]
return sorted(parts, key=lambda p: (-p.area, p.bounds))
def on_grid(geometry):
"""Snap to whole pack units keeping the result valid; polygon parts only."""
snapped = set_precision(geometry, 1, mode='valid_output')
return MultiPolygon(polygon_parts(snapped))
def settle(geometry):
"""A valid MultiPolygon of `geometry` (in pack units) on whole units."""
if not geometry.is_valid:
geometry = make_valid(geometry)
geometry = on_grid(MultiPolygon(polygon_parts(geometry)))
for _ in range(3):
if geometry.is_valid:
break
geometry = on_grid(make_valid(geometry))
return geometry
def rings_ok(polygons):
return all(len(set(ring)) >= 3 for poly in polygons for ring in poly)
def problems(polygons):
"""Why decoded `polygons` ([[outer, *holes], …]) fail the pack contract."""
if not polygons or not rings_ok(polygons):
return 'ring with fewer than three distinct vertices'
try:
geometry = MultiPolygon([Polygon(p[0], p[1:]) for p in polygons])
except Exception as error: # noqa: BLE001 - malformed input is a finding
return f'unbuildable: {error}'
if not geometry.is_valid:
return shapely.is_valid_reason(geometry)
areas = [part.area for part in geometry.geoms]
if max(areas) > areas[0]:
return 'largest part is not first'
return None
# ── packs ────────────────────────────────────────────────────────────────
def census_features():
folder = DATA / 'us_census_places'
decimals = json.loads((folder / 'index.json').read_text())['meta']['decimals']
for path in sorted(folder.glob('[A-Z][A-Z].json')):
pack = json.loads(path.read_text(encoding='utf-8'))
yield path, pack, decimals
def census_polygons(feature):
return [[decode_deltas(r) for r in poly] for poly in feature['polygons']]
def wof_features():
folder = DATA / 'wof_neighborhoods'
index = json.loads((folder / 'index.json').read_text(encoding='utf-8'))
for tile in index['tiles']:
pack = json.loads((folder / f'{tile["key"]}.json').read_text(encoding='utf-8'))
for feature in pack['features']:
yield feature, index['meta']['precision']
def check(pack):
bad = []
total = 0
if pack == 'census':
for _, data, decimals in census_features():
for feature in data['features']:
total += 1
why = problems(census_polygons(feature))
if why:
bad.append((feature['geoid'], feature['name'], why))
else:
for feature, precision in wof_features():
total += 1
polygons = [[decode_polyline(r) for r in poly] for poly in feature[7]]
why = problems(polygons)
if why:
bad.append((feature[0], feature[1], why))
for row in bad[:20]:
print('invalid', *row)
print(f'{pack}: {total} features, {len(bad)} failing')
return not bad
def repair_census():
repaired = 0
for path, data, decimals in census_features():
changed = False
for feature in data['features']:
polygons = census_polygons(feature)
if not problems(polygons):
continue
geometry = MultiPolygon([Polygon(p[0], p[1:]) for p in polygons if len(set(p[0])) >= 3])
parts = ordered_parts(settle(geometry))
feature['polygons'] = [
[encode_deltas(open_ring(p.exterior.coords))]
+ [encode_deltas(open_ring(r.coords)) for r in p.interiors]
for p in parts
]
why = problems(census_polygons(feature))
if why:
raise ValueError(f'{feature["geoid"]} {feature["name"]}: still {why}')
repaired += 1
changed = True
if changed:
path.write_text(compact(data) + '\n', encoding='utf-8')
print(f'us_census_places: {repaired} features repaired to valid GEOS topology')
def main():
parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
parser.add_argument('command', choices=['check', 'repair'])
parser.add_argument('pack', choices=['census', 'wof'])
args = parser.parse_args()
require_pinned_tooling()
if args.command == 'repair':
if args.pack != 'census':
parser.error('the WOF build repairs its own geometry')
repair_census()
sys.exit(0 if check('census') else 1)
sys.exit(0 if check(args.pack) else 1)
if __name__ == '__main__':
main()
+178 -74
View File
@@ -10,7 +10,8 @@
"src/data/localGeojsonCore.js",
"src/data/localGeojsonLod.js",
"src/data/inputOwnership.js",
"src/data/infrastructureOverlayEntry.js"
"src/data/infrastructureOverlayEntry.js",
"src/sources/infrastructureData.js"
],
"external": ["cesium"]
},
@@ -27,7 +28,13 @@
"vite-build": {
"runtime": "node",
"exports": ["./build/vite", "./build/html"],
"modules": ["build/vite.js", "build/application-html.js"],
"modules": [
"build/vite.js",
"build/application-html.js",
"build/allowedHosts.js",
"build/embed-framing.js",
"build/panel.js"
],
"external": ["vite-plugin-cesium"]
},
"live-providers": {
@@ -68,6 +75,8 @@
"server/providers/common/http.js",
"scripts/google-server-key.mjs",
"src/data/placeProviderPayloads.js",
"server/providers/common/same-site.js",
"src/localRequestGate.mjs",
"src/data/routeSteps.js",
"src/sources/httpBody.js",
"src/sources/rateLimit.js"
@@ -244,9 +253,6 @@
"server/providers/regional/http.js",
"server/providers/regional/news.js",
"server/providers/regional/place.js",
"server/providers/regional/nominatimGate.js",
"server/providers/regional/nominatimCache.js",
"server/providers/regional/outlineSelect.js",
"server/providers/regional/query.js",
"server/providers/regional/weather-effects.js",
"server/providers/regional/weather.js",
@@ -284,8 +290,9 @@
"src/loadingFeedback.js",
"src/data/installationFeedback.js",
"src/voice/voiceCost.js",
"server/providers/common/same-site.js",
"src/localRequestGate.mjs",
"src/voice/actionSchemas.js",
"src/voice/layerManifest.js",
"server/providers/openai/toolDescriptions.js",
"src/sources/rateLimit.js"
],
@@ -299,6 +306,7 @@
"server/providers/common/source-root.js",
"server/standalone/key-setup.js",
"src/keySetupCore.mjs",
"src/localRequestGate.mjs",
"server/standalone/key-setup-hardening.mjs"
],
"external": []
@@ -383,7 +391,8 @@
"src/ui/mapSource.js",
"src/ui/mapSourceControls.js",
"src/mapStackChips.js",
"src/keySetupCore.mjs"
"src/keySetupCore.mjs",
"src/localRequestGate.mjs"
],
"external": []
},
@@ -402,6 +411,7 @@
"src/loadingFeedback.js",
"src/data/installationFeedback.js",
"src/keySetupCore.mjs",
"src/localRequestGate.mjs",
"src/data/feedState.js",
"src/data/layerSnapshot.js"
],
@@ -517,10 +527,12 @@
"src/data/detectionPolicy.js",
"src/data/installationFeedback.js",
"src/data/layerState.js",
"src/data/layerStateCoordinator.js",
"src/data/layerStateTokenReservations.json",
"src/data/militaryAwarenessEngine.js",
"src/data/scenePick.js",
"src/keySetupCore.mjs",
"src/localRequestGate.mjs",
"src/loadingFeedback.js",
"src/locationStatus.js",
"src/mapStackChips.js",
@@ -653,7 +665,6 @@
"src/ui/styles/command-dock-trays.css",
"src/ui/styles/command-dock-sliding.css",
"src/ui/styles/voice-cost.css",
"src/ui/styles/voice-card.css",
"src/ui/styles/first-run.css",
"src/ui/styles/provider-settings.css",
"src/ui/styles/weather.css",
@@ -839,6 +850,7 @@
"src/data/contactMatch.js",
"src/data/iconOrientation.js",
"src/data/layerState.js",
"src/data/layerStateCoordinator.js",
"src/data/layerStateTokenReservations.json",
"src/data/modelVisualAnchor.js",
"src/data/motionModel.js",
@@ -927,6 +939,7 @@
"src/data/issPass.js",
"src/data/satelliteClass.js",
"src/data/satellitePass.js",
"src/sources/tle.js",
"src/layers/satellites/catalog.js",
"src/layers/satellites/controls.js",
"src/layers/satellites/index.js",
@@ -1084,6 +1097,7 @@
"src/layers/bikeshare/ingestion.js",
"src/layers/bikeshare/lifecycle.js",
"src/layers/bikeshare/model.js",
"src/sources/gbfsStations.js",
"src/layers/bikeshare/policy.js",
"src/layers/bikeshare/queries.js",
"src/layers/bikeshare/registry.js",
@@ -1256,6 +1270,7 @@
"src/maps/terrain.js",
"src/maps/terrainRetry.js",
"src/keySetupCore.mjs",
"src/localRequestGate.mjs",
"src/maps/imageryHost.js"
],
"external": ["cesium"]
@@ -1290,19 +1305,7 @@
"src/voice/realtimeFacade.js",
"src/voice/realtimeRadio.js",
"src/voice/realtimeTurns.js",
"src/voice/realtimeConnection.js",
"src/voice/turnMetrics.js",
"src/voice/speechLint.js",
"src/voice/voiceCard.js",
"src/voice/voiceCardPresentation.js",
"src/voice/resultDisplay.js",
"src/voice/referents.js",
"src/voice/layerManifest.js",
"src/voice/speech.js",
"src/voice/narration.js",
"src/voice/earcon.js",
"src/voice/pointerContext.js",
"src/voice/pointerReticle.js"
"src/voice/realtimeConnection.js"
],
"external": []
},
@@ -1345,9 +1348,6 @@
"src/annotations/hybridAnnotationRenderer.js",
"src/annotations/index.js",
"src/annotations/resolver.js",
"src/annotations/outlineRungs.js",
"src/sources/openFreeMapOutlines.js",
"src/sources/nominatimOutlines.js",
"src/annotations/screenAnnotationRenderer.js",
"src/annotations/worldAnnotationRenderer.js",
"src/app/catalog.js",
@@ -1405,7 +1405,6 @@
"src/data/aisLiveVessels.js",
"src/data/alprCameras.js",
"src/data/analystEngine.js",
"src/voice/layerManifest.js",
"src/data/bikeshare.js",
"src/data/cctv.js",
"src/data/cctvCards.js",
@@ -1454,6 +1453,7 @@
"src/data/issPass.js",
"src/data/labelArbiter.js",
"src/data/layerState.js",
"src/data/layerStateCoordinator.js",
"src/data/layerStateTokenReservations.json",
"src/data/lifecycle.js",
"src/data/localGeojson.js",
@@ -1475,9 +1475,6 @@
"src/data/motionModel.js",
"src/data/naturalEarthRegions.js",
"src/data/adminBoundaries.js",
"src/data/placeBoundaries.js",
"src/data/local_data/us_census_places/files.js",
"src/data/local_data/wof_neighborhoods/files.js",
"src/data/local_data/natural_earth/states_provinces.json",
"src/data/local_data/natural_earth/countries.json",
"src/data/local_data/us_census_counties/counties.json",
@@ -1519,6 +1516,7 @@
"src/hudSummaryResponse.js",
"src/keySetup.js",
"src/keySetupCore.mjs",
"src/localRequestGate.mjs",
"src/keylessGeocoder.js",
"src/layers/aircraft/classification.js",
"src/layers/alpr/index.js",
@@ -1549,6 +1547,7 @@
"src/layers/bikeshare/ingestion.js",
"src/layers/bikeshare/lifecycle.js",
"src/layers/bikeshare/model.js",
"src/sources/gbfsStations.js",
"src/layers/bikeshare/policy.js",
"src/layers/bikeshare/queries.js",
"src/layers/bikeshare/registry.js",
@@ -1683,6 +1682,7 @@
"src/layers/satellites/labels.js",
"src/layers/satellites/lifecycle.js",
"src/layers/satellites/orbits.js",
"src/sources/tle.js",
"src/layers/satellites/records.js",
"src/layers/satellites/policy.js",
"src/layers/satellites/rendering.js",
@@ -1875,14 +1875,6 @@
"src/voice/commands.js",
"src/voice/control.js",
"src/voice/gevActions.js",
"src/voice/speech.js",
"src/voice/earcon.js",
"src/voice/narration.js",
"src/voice/turnMetrics.js",
"src/voice/speechLint.js",
"src/voice/voiceCard.js",
"src/voice/voiceCardPresentation.js",
"src/voice/resultDisplay.js",
"src/voice/gevRealtime.js",
"src/voice/layerSummary.js",
"src/voice/realtimeBackend.js",
@@ -2046,12 +2038,6 @@
"src/maps/imageryHost.js",
"src/ui/recentImagery.js",
"src/ui/imageryBoxTool.js",
"src/voice/pointerContext.js",
"src/voice/pointerReticle.js",
"src/voice/deixis.js",
"src/voice/referents.js",
"src/voice/pointerPick.js",
"src/data/pickedContext.js",
"src/layers/alpr/tileRecords.js",
"src/sources/capability.js",
"src/sources/openFreeMap.js",
@@ -2065,7 +2051,12 @@
"src/sources/requestDeadline.js",
"src/data/local_data/osm_military_names/names.json",
"src/layers/installations/namedMarkers.js",
"src/data/infrastructureOverlayEntry.js"
"src/data/infrastructureOverlayEntry.js",
"src/sources/infrastructureData.js",
"src/tools/functions.js",
"src/voice/actionSchemas.js",
"src/view/index.js",
"src/app/embed.js"
],
"external": [
"hls.js",
@@ -2097,9 +2088,6 @@
"src/data/bhoteKoshiShotPaths.js",
"src/annotations/annotationResolver.js",
"src/annotations/resolver.js",
"src/annotations/outlineRungs.js",
"src/sources/openFreeMapOutlines.js",
"src/sources/nominatimOutlines.js",
"src/app/catalog.js",
"src/app/constructCatalog.js",
"src/layers/wind/index.js",
@@ -2175,9 +2163,6 @@
"src/data/motionModel.js",
"src/data/naturalEarthRegions.js",
"src/data/adminBoundaries.js",
"src/data/placeBoundaries.js",
"src/data/local_data/us_census_places/files.js",
"src/data/local_data/wof_neighborhoods/files.js",
"src/data/local_data/natural_earth/states_provinces.json",
"src/data/local_data/natural_earth/countries.json",
"src/data/local_data/us_census_counties/counties.json",
@@ -2233,6 +2218,7 @@
"src/layers/bikeshare/ingestion.js",
"src/layers/bikeshare/lifecycle.js",
"src/layers/bikeshare/model.js",
"src/sources/gbfsStations.js",
"src/layers/bikeshare/policy.js",
"src/layers/bikeshare/queries.js",
"src/layers/bikeshare/registry.js",
@@ -2367,6 +2353,7 @@
"src/layers/satellites/labels.js",
"src/layers/satellites/lifecycle.js",
"src/layers/satellites/orbits.js",
"src/sources/tle.js",
"src/layers/satellites/records.js",
"src/layers/satellites/policy.js",
"src/layers/satellites/rendering.js",
@@ -2555,7 +2542,8 @@
"src/sources/requestDeadline.js",
"src/data/local_data/osm_military_names/names.json",
"src/layers/installations/namedMarkers.js",
"src/data/infrastructureOverlayEntry.js"
"src/data/infrastructureOverlayEntry.js",
"src/sources/infrastructureData.js"
],
"external": [
"hls.js",
@@ -2571,16 +2559,8 @@
"application-operations": {
"exports": ["./application/operations"],
"modules": [
"src/sources/openFreeMap.js",
"src/sources/vectorTiles.js",
"src/sources/phaseTiming.js",
"src/sources/militaryTileGeometry.js",
"src/data/tomtomTiles.js",
"src/annotations/annotationResolver.js",
"src/annotations/resolver.js",
"src/annotations/outlineRungs.js",
"src/sources/openFreeMapOutlines.js",
"src/sources/nominatimOutlines.js",
"src/app/operations.js",
"src/sources/sourceSlot.js",
"src/app/surfaceServices.js",
@@ -2591,9 +2571,6 @@
"src/data/local_data/neighborhoods/san-francisco.json",
"src/data/naturalEarthRegions.js",
"src/data/adminBoundaries.js",
"src/data/placeBoundaries.js",
"src/data/local_data/us_census_places/files.js",
"src/data/local_data/wof_neighborhoods/files.js",
"src/data/local_data/natural_earth/states_provinces.json",
"src/data/local_data/natural_earth/countries.json",
"src/data/local_data/us_census_counties/counties.json",
@@ -2660,6 +2637,7 @@
"modules": [
"src/data/installationFeedback.js",
"src/data/layerState.js",
"src/data/layerStateCoordinator.js",
"src/data/layerStateTokenReservations.json",
"src/loadingFeedback.js",
"src/navigationPolicy.js",
@@ -2791,11 +2769,7 @@
},
"voice-action-schemas": {
"exports": ["./voice/action-schemas"],
"modules": [
"src/voice/actionSchemas.js",
"src/voice/layerManifest.js",
"src/hudLayoutPolicy.js"
],
"modules": ["src/voice/actionSchemas.js", "src/hudLayoutPolicy.js"],
"external": []
},
"voice-session": {
@@ -2808,14 +2782,7 @@
"modules": [
"src/voice/sessionCommands.js",
"src/voice/session.js",
"src/voice/control.js",
"src/voice/voiceCard.js",
"src/voice/voiceCardPresentation.js",
"src/voice/resultDisplay.js",
"src/voice/referents.js",
"src/voice/layerManifest.js",
"src/voice/speech.js",
"src/voice/pointerReticle.js"
"src/voice/control.js"
],
"external": []
},
@@ -3138,5 +3105,142 @@
"src/layers/perimeters/source.js"
],
"external": []
},
"tools": {
"exports": ["./tools"],
"modules": [
"src/tools/index.js",
"src/tools/catalog.js",
"src/tools/schema.js",
"src/tools/area.js",
"src/tools/results.js",
"src/tools/places.js",
"src/tools/queries/hazards.js",
"src/tools/queries/space.js",
"src/tools/queries/aviation.js",
"src/tools/queries/media.js",
"src/tools/queries/places.js",
"src/tools/queries/mobility.js",
"src/tools/queries/environment.js",
"src/tools/queries/brief.js",
"src/tools/queries/maritime.js",
"src/tools/queries/share.js",
"src/data/layerState.js",
"src/data/layerStateTokenReservations.json",
"src/sources/gbfsStations.js",
"src/sources/tle.js",
"src/data/satellitePass.js",
"src/tools/queries/atmosphere.js",
"src/layers/wind/inspection.js",
"src/layers/wind/model.js",
"src/layers/wind/fields.js",
"src/tools/queries/imagery.js",
"src/tools/queries/cables.js",
"src/tools/queries/surveillance.js",
"src/tools/queries/infrastructure.js",
"src/tools/queries/events.js",
"src/data/bhoteKoshiFloodPath.js",
"src/tools/functions.js",
"src/tools/surfaces.js",
"src/layers/transit/policy.js",
"src/data/transitFeeds.js",
"src/layers/transit/movement.js",
"src/data/contactPlayback.js",
"src/data/layerSnapshot.js",
"src/data/feedState.js",
"src/loadingFeedback.js",
"src/data/installationFeedback.js",
"src/view/index.js",
"src/tools/views.js",
"src/tools/globePanel.js",
"src/tools/queries/panelRequest.js"
],
"external": ["satellite.js"]
},
"tools-mcp": {
"exports": ["./tools/mcp"],
"modules": [
"src/tools/mcp/index.js",
"src/tools/mcp/protocol.js",
"src/tools/mcp/http.js",
"src/tools/catalog.js",
"src/tools/schema.js"
],
"external": []
},
"tools-services": {
"exports": ["./tools/services"],
"modules": [
"src/tools/services.js",
"src/layers/bikeshare/registry.js",
"src/layers/bikeshare/source.js",
"src/layers/cctv/source.js",
"src/layers/cctv/sourcePolicy.js",
"src/layers/cyclones/source.js",
"src/sources/httpBody.js",
"src/layers/earthquakes/source.js",
"src/layers/earthquakes/records.js",
"src/layers/firms/source.js",
"src/layers/installations/source.js",
"src/sources/requestDeadline.js",
"src/data/militaryNames.js",
"src/sources/militaryTileGeometry.js",
"src/data/bundledJson.js",
"src/data/retryableLoad.js",
"src/data/local_data/osm_military_names/names.json",
"src/sources/openFreeMap.js",
"src/sources/vectorTiles.js",
"src/sources/phaseTiming.js",
"src/data/tomtomTiles.js",
"src/sources/capability.js",
"src/data/militaryInstallationData.js",
"src/layers/launches/source.js",
"src/layers/perimeters/source.js",
"src/layers/radio/source.js",
"src/layers/radio/policy.js",
"src/layers/satellites/source.js",
"src/layers/traffic/source.js",
"src/layers/traffic/roadModes.js",
"src/data/flowMatch.js",
"src/layers/traffic/flowSource.js",
"src/layers/traffic/flowDecode.js",
"src/sources/overpassRoads.js",
"src/layers/transit/source.js",
"src/sources/transitHistory.js",
"src/services/requests.js",
"src/sources/overpassFeatures.js",
"src/sources/featureSource.js",
"src/sources/overpassFeaturesRecords.js",
"src/sources/featureGeometry.js",
"src/sources/live/standalone.js",
"src/sources/live/contract.js",
"src/sources/live/aircraft.js",
"src/sources/live/vessels.js",
"src/tools/places.js",
"src/tools/catalog.js",
"src/tools/schema.js",
"src/layers/weather/source.js",
"src/layers/wind/source.js",
"src/sources/windBody.js",
"src/layers/recentImagery/catalog.js",
"src/layers/recentImagery/model.js",
"src/layers/submarineCables/bundledSource.js",
"src/layers/alpr/source.js",
"src/layers/alpr/policy.js",
"src/layers/alpr/records.js",
"src/layers/alpr/tileRecords.js",
"src/sources/infrastructureData.js",
"src/sources/eventPacks.js"
],
"external": ["@mapbox/vector-tile", "pbf"]
},
"view": {
"exports": ["./view"],
"modules": [
"src/view/index.js",
"src/data/layerState.js",
"src/data/layerStateTokenReservations.json"
],
"external": []
}
}
+2 -42
View File
@@ -3,7 +3,7 @@
* qa-admin-outlines.mjs — bundled state/province/county outlines, driven like a
* user would drive them.
*
* Run: node scripts/qa-admin-outlines.mjs http://localhost:4173 [--headful] [--no-video] [--offline-imagery]
* Run: node scripts/qa-admin-outlines.mjs http://localhost:4173 [--headful] [--no-video]
*
* Journey: frame Texas at state scale and outline "Texas"; pan and zoom toward
* Austin in smooth 2–3 s moves; outline "Travis County, Texas"; pan north and
@@ -33,7 +33,7 @@ import sharp from 'sharp';
const args = process.argv.slice(2);
if (args.includes('--help')) {
console.log(
'Usage: node scripts/qa-admin-outlines.mjs <dev-server-url> [--headful] [--no-video] [--offline-imagery]',
'Usage: node scripts/qa-admin-outlines.mjs <dev-server-url> [--headful] [--no-video]',
);
process.exit(0);
}
@@ -41,15 +41,6 @@ const url = args.find((arg) => !arg.startsWith('--'));
if (!url || !['http:', 'https:'].includes(new URL(url).protocol))
throw new Error('Supply a running dev server URL; see --help');
const recordVideo = !args.includes('--no-video');
const offlineImagery = args.includes('--offline-imagery');
const offlineImageryTile = offlineImagery
? await fs.readFile(
new URL(
'../node_modules/cesium/Build/Cesium/Assets/Textures/NaturalEarthII/0/0/0.jpg',
import.meta.url,
),
)
: null;
const shots = path.resolve('qa-shots');
const framesDir = path.join(shots, 'admin-outlines-frames');
await fs.mkdir(shots, { recursive: true });
@@ -70,8 +61,6 @@ const result = {
pageErrors: [],
screenshots: [],
video: null,
offlineImagery,
offlineImageryRequests: 0,
};
const browser = await puppeteer.launch({
@@ -98,35 +87,6 @@ try {
sessionStorage.setItem('gev:first-run-mission-session:v1', 'dismissed');
});
await page.setViewport({ width: 1280, height: 800 });
await page.setRequestInterception(true);
page.on('request', (request) => {
let target;
try {
target = new URL(request.url());
} catch {
request.continue();
return;
}
const knownImageryTile =
request.method() === 'GET' &&
((target.origin === 'https://services.arcgisonline.com' &&
/^\/ArcGIS\/rest\/services\/World_Imagery\/MapServer\/tile\/\d+\/\d+\/\d+$/.test(
target.pathname,
)) ||
(target.origin === 'https://tile.openstreetmap.org' &&
/^\/\d+\/\d+\/\d+\.png$/.test(target.pathname)));
if (offlineImagery && knownImageryTile) {
result.offlineImageryRequests += 1;
request.respond({
status: 200,
contentType: 'image/jpeg',
headers: { 'Access-Control-Allow-Origin': '*' },
body: offlineImageryTile,
});
return;
}
request.continue();
});
const client = await page.createCDPSession();
await client.send('Network.enable');
client.on('Network.requestWillBeSent', ({ request }) => {
+98 -34
View File
@@ -1572,6 +1572,10 @@ try {
const cockpit = manager.cockpitView;
const current = document.getElementById('cockpit-vision-current');
const label = document.getElementById('cockpit-vision-current-label');
const settle = (ms = 900) => new Promise((resolve) => setTimeout(resolve, ms));
const intensities = () => Object.fromEntries(
Object.entries(manager.stages || {}).map(([name, stage]) => [name, stage.uniforms?.intensity]),
);
const read = () => ({
mode: current?.dataset.cockpitVision,
label: label?.textContent?.trim(),
@@ -1582,62 +1586,106 @@ try {
parametersCollapsed: document.getElementById('param-slider-panel')?.classList.contains('collapsed'),
signalCollapsed: cockpit.signalCollapsed,
focusRetained: document.activeElement === current,
activeStyle: manager.activeStyle,
intensities: intensities(),
});
const reenter = async () => {
cockpit.exit({ restoreTracking: true });
await settle();
for (let attempt = 0; attempt < 40; attempt += 1) {
if (cockpit.enter() === true) return true;
await settle(200);
}
return false;
};
manager._setCockpitDisclosure('display', false);
manager._setCockpitDisclosure('radio', false);
cockpit.setVisionMode('optical');
current.focus({ preventScroll: true });
const states = [read()];
for (let index = 0; index < 5; index += 1) {
for (let index = 0; index < 7; index += 1) {
current.click();
states.push(read());
}
const originalStyle = manager.activeStyle;
// A Noir map preset: Cockpit enters on Noir, Normal clears every shader
// without changing the map preset, and Exit Cockpit restores Noir.
cockpit.exit({ restoreTracking: true });
manager.setStyle('noir');
await new Promise((resolve) => setTimeout(resolve, 900));
await settle();
const enteredOnNoir = await reenter();
const entryNoir = read();
cockpit.setVisionMode('optical');
const inheritedNoir = read();
cockpit.setVisionMode('noir');
const explicitNoir = read();
cockpit.setVisionMode('optical');
const restoredNoir = {
...read(),
activeStyle: manager.activeStyle,
intensity: manager.stages.noir?.uniforms?.intensity,
};
const normalInsideNoir = read();
cockpit.setVisionMode('crt');
const cockpitOnlyCrt = read();
const exitedNoir = cockpit.exit({ restoreTracking: true }) === true;
await settle();
const restoredNoir = { ...read(), cockpitActive: cockpit.active };
manager.setStyle(originalStyle);
await new Promise((resolve) => setTimeout(resolve, 900));
await settle();
const reentered = await reenter();
cockpit.setVisionMode('optical');
return { states, inheritedNoir, explicitNoir, restoredNoir };
return {
states,
enteredOnNoir,
entryNoir,
normalInsideNoir,
cockpitOnlyCrt,
exitedNoir,
restoredNoir,
reentered,
};
});
const cycleModes = ['optical', 'crt', 'nvg', 'thermal', 'anime', 'noir', 'snow', 'optical'];
const cycleLabels = ['NORMAL', 'CRT', 'NVG', 'FLIR', 'ANIME', 'NOIR', 'SNOW', 'NORMAL'];
check(
'Cockpit vision control exposes inherited, CRT, NVG, FLIR, and NOIR before wrapping',
JSON.stringify(visionCycle.states.map((state) => state.mode))
=== JSON.stringify(['optical', 'crt', 'nvg', 'thermal', 'noir', 'optical'])
&& visionCycle.states[4]?.label === 'NOIR'
&& visionCycle.states[4]?.aria === 'Current cockpit vision style: Noir. Activate for next style.',
JSON.stringify(visionCycle),
'Cockpit vision control exposes Normal, CRT, NVG, FLIR, Anime, Noir and Snow before wrapping',
JSON.stringify(visionCycle.states.map((state) => state.mode)) === JSON.stringify(cycleModes)
&& JSON.stringify(visionCycle.states.map((state) => state.label)) === JSON.stringify(cycleLabels)
&& visionCycle.states[5]?.aria === 'Current cockpit vision style: Noir. Activate for next style.'
&& visionCycle.states[7]?.aria === 'Current cockpit vision style: Normal. Activate for next style.',
JSON.stringify(visionCycle.states.map(({ mode, label, aria }) => ({ mode, label, aria }))),
);
check(
'user vision changes open Display for every temporary treatment and retain selector focus',
visionCycle.states[1]?.mode === 'crt'
&& visionCycle.states.slice(1, 5).every((state) => state.displayOpen === 'true')
&& visionCycle.states.slice(1, 5).every((state) => state.radioOpen === 'false')
&& visionCycle.states.slice(1, 5).every((state) => state.parametersActive)
&& visionCycle.states.slice(1, 5).every((state) => !state.parametersCollapsed)
&& visionCycle.states.slice(1, 5).every((state) => state.signalCollapsed)
visionCycle.states.slice(1, 7).every((state) => state.displayOpen === 'true')
&& visionCycle.states.slice(1, 7).every((state) => state.radioOpen === 'false')
&& visionCycle.states.slice(1, 7).every((state) => state.parametersActive)
&& visionCycle.states.slice(1, 7).every((state) => !state.parametersCollapsed)
&& visionCycle.states.slice(1, 7).every((state) => state.signalCollapsed)
&& visionCycle.states.every((state) => state.focusRetained),
JSON.stringify(visionCycle.states),
);
check(
'inherited Noir and explicit Noir keep distinct ownership and restore the Noir map preset',
visionCycle.inheritedNoir?.mode === 'optical'
&& visionCycle.inheritedNoir?.label === 'NOIR'
&& visionCycle.explicitNoir?.mode === 'noir'
&& visionCycle.explicitNoir?.label === 'NOIR'
&& visionCycle.restoredNoir?.mode === 'optical'
'each Cockpit vision mode drives only its own shader and Normal clears them all',
visionCycle.states.slice(1, 7).every((state) => {
const target = {
crt: 'retro', nvg: 'surveillance', thermal: 'thermal', anime: 'anime', noir: 'noir', snow: 'snow',
}[state.mode];
return Object.entries(state.intensities).every(([name, value]) =>
name === target ? value > 0 : value === 0);
})
&& Object.values(visionCycle.states[7].intensities).every((value) => value === 0),
JSON.stringify(visionCycle.states.map(({ mode, intensities }) => ({ mode, intensities }))),
);
check(
'Cockpit enters on the Noir map preset, Normal clears it, and Exit Cockpit restores it',
visionCycle.enteredOnNoir
&& visionCycle.entryNoir?.mode === 'noir'
&& visionCycle.entryNoir?.label === 'NOIR'
&& visionCycle.normalInsideNoir?.mode === 'optical'
&& visionCycle.normalInsideNoir?.label === 'NORMAL'
&& visionCycle.normalInsideNoir?.activeStyle === 'noir'
&& Object.values(visionCycle.normalInsideNoir?.intensities || {}).every((value) => value === 0)
&& visionCycle.cockpitOnlyCrt?.mode === 'crt'
&& visionCycle.cockpitOnlyCrt?.activeStyle === 'noir'
&& visionCycle.exitedNoir
&& !visionCycle.restoredNoir?.cockpitActive
&& visionCycle.restoredNoir?.activeStyle === 'noir'
&& visionCycle.restoredNoir?.intensity === 1,
&& visionCycle.restoredNoir?.intensities?.noir === 1
&& visionCycle.restoredNoir?.intensities?.retro === 0
&& visionCycle.reentered,
JSON.stringify(visionCycle),
);
await page.evaluate(() => window.__godsEyeView.styleManager.cockpitView.setVisionMode('noir'));
@@ -2266,10 +2314,16 @@ try {
// Both the UI and compatibility facade delegate to this navigation owner.
const manager = window.__godsEyeView.styleManager._locationNavigation;
const awareness = window.__godsEyeView.dataManager.layers.get('military-awareness')?.module;
const styleManager = window.__godsEyeView.styleManager;
const styleBefore = styleManager.activeStyle;
// A Cockpit-only choice that Reset must undo.
styleManager.cockpitView.setVisionMode('snow');
window.__qaCockpitReset = {
calls: 0,
original: manager.resetToGlobeView,
subjectId: awareness?.getContextSnapshot?.()?.subject?.id || null,
styleBefore,
snowDuringCockpit: styleManager.stages?.snow?.uniforms?.intensity,
};
manager.resetToGlobeView = function qaCountedCockpitReset(...args) {
window.__qaCockpitReset.calls += 1;
@@ -2305,16 +2359,26 @@ try {
resetHidden: document.getElementById('cockpit-reset-globe')?.hidden,
height,
subjectPreserved: Boolean(qa.subjectId && subjectId === qa.subjectId),
styleRestored: gev.styleManager.activeStyle === qa.styleBefore,
snowDuringCockpit: qa.snowDuringCockpit,
// Cockpit-time intensities are Cockpit's own, so check against the map preset.
visionRestored: Object.entries(gev.styleManager.stages || {}).every(([name, stage]) =>
name === gev.styleManager.activeStyle
? stage.uniforms?.intensity > 0
: stage.uniforms?.intensity === 0),
};
});
check(
'keyboard Cockpit Reset uses one canonical route and preserves Contact selection',
'keyboard Cockpit Reset uses one canonical route, preserves Contact selection and restores the map style',
resetState.calls === 1
&& !resetState.cockpitActive
&& !resetState.trackedEntity
&& resetState.resetHidden
&& Math.abs(resetState.height - 18_000_000) < 150_000
&& resetState.subjectPreserved,
&& resetState.subjectPreserved
&& resetState.snowDuringCockpit > 0
&& resetState.styleRestored
&& resetState.visionRestored,
JSON.stringify(resetState),
);
check('runtime console remains clean', consoleErrors.length === 0 && localHttpErrors.length === 0,
-714
View File
@@ -1,714 +0,0 @@
#!/usr/bin/env node
/**
* Traffic-warm OpenFreeMap outline acceptance.
*
* Each sample opens a fresh application page, loads the Texas Capitol z14 tile
* through Street Traffic, then circles a building from that same tile. It
* records application fetches separately from browser transfer bytes,
* body-read/decode/extraction/install measures, and Long Tasks. A second ask
* on the page proves the outline source's own consumer-cache path. A separate
* page holds traffic's target response while the outline consumer subscribes,
* proving that overlap still produces one application fetch and one decode.
*
* Acceptance is declared before measurement: cold outline decode, extraction,
* and install must fit one 60 Hz frame at p95 (16.7 ms), no sample may create a
* >=50 ms Long Task, and neither the first nor second outline ask may fetch or
* decode the traffic-warm tile again. The target tile's traffic-idle outline
* projection must also fit one frame; unrelated traffic activation Long Tasks
* are reported separately as baseline evidence.
*/
import assert from 'node:assert/strict';
import fs from 'node:fs/promises';
import path from 'node:path';
import puppeteer from 'puppeteer';
const args = process.argv.slice(2);
const valueAfter = (flag, fallback) => {
const index = args.indexOf(flag);
return index >= 0 && args[index + 1] ? args[index + 1] : fallback;
};
if (args.includes('--help') || args.includes('-h')) {
console.log(
'Usage: node scripts/qa-openfreemap-same-tile.mjs [--url http://localhost:4173] [--runs 5] [--headful] [--out qa-shots/openfreemap-same-tile.json]',
);
process.exit(0);
}
const APP_URL = valueAfter('--url', 'http://localhost:4173');
const RUNS = Number(valueAfter('--runs', '5'));
const OUTPUT = path.resolve(
valueAfter('--out', 'qa-shots/openfreemap-same-tile.json'),
);
const FRAME_BUDGET_MS = 16.7;
const LONG_TASK_MS = 50;
const TARGET = Object.freeze({
latitude: 30.27472,
longitude: -97.74035,
z: 14,
x: 3743,
y: 6745,
});
const TARGET_PATH = `/${TARGET.z}/${TARGET.x}/${TARGET.y}.pbf`;
if (!Number.isInteger(RUNS) || RUNS < 3 || RUNS > 20)
throw new Error('--runs must be an integer from 3 through 20');
const targetUrl = new URL(APP_URL);
if (!['http:', 'https:'].includes(targetUrl.protocol))
throw new Error('--url must be HTTP(S)');
const percentile = (values, p) => {
if (!values.length) return null;
const sorted = values.slice().sort((a, b) => a - b);
return sorted[Math.ceil(p * sorted.length) - 1];
};
const round = (value) =>
value == null ? null : Number(Number(value).toFixed(3));
const measures = (entries, name) =>
entries.filter((entry) => entry.name === name).map((entry) => entry.ms);
const sum = (values) => values.reduce((total, value) => total + value, 0);
const sleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms));
await fs.mkdir(path.dirname(OUTPUT), { recursive: true });
const browser = await puppeteer.launch({
headless: args.includes('--headful') ? false : 'new',
protocolTimeout: 240_000,
...(process.env.PUPPETEER_EXECUTABLE_PATH
? { executablePath: process.env.PUPPETEER_EXECUTABLE_PATH }
: {}),
args: [
'--no-sandbox',
'--disable-setuid-sandbox',
'--use-gl=angle',
...(args.includes('--headful')
? []
: ['--use-angle=swiftshader', '--enable-unsafe-swiftshader']),
'--disable-dev-shm-usage',
'--disable-background-timer-throttling',
'--disable-renderer-backgrounding',
'--window-size=1440,1000',
],
});
const report = {
appUrl: APP_URL,
target: TARGET,
targetPath: TARGET_PATH,
acceptance: {
coldCpuP95Ms: FRAME_BUDGET_MS,
maxLongTaskMs: LONG_TASK_MS,
trafficIdleProjectionMaxMs: FRAME_BUDGET_MS,
coldApplicationFetches: 0,
coldOutlineTransferBytes: 0,
coldDecodes: 0,
warmApplicationFetches: 0,
warmDecodes: 0,
overlapApplicationFetches: 1,
overlapDecodes: 1,
},
renderer: null,
samples: [],
errors: [],
};
function relevantMeasures(raw) {
return raw
.filter(
(entry) =>
entry.name.startsWith('roads:') || entry.name.startsWith('outlines:'),
)
.map((entry) => ({
name: entry.name,
ms: round(entry.duration),
at: round(entry.startTime),
detail: entry.detail || {},
}));
}
async function beginOutline(page, label) {
await page.evaluate(
({ label, target }) => {
performance.clearMeasures();
window.__qaOutlineLongTasks.length = 0;
window.__qaOutlineStarted = performance.now();
window.__qaOutlinePromise = Promise.resolve(
window.__gevVoiceCommands.runner('annotate_map', {
annotations: [
{
type: 'area',
target: 'this building',
label,
entityKind: 'building',
latitude: target.latitude,
longitude: target.longitude,
},
],
}),
);
},
{ label, target: TARGET },
);
}
async function settleOutline(page, label) {
return page.evaluate(async (label) => {
const result = await window.__qaOutlinePromise;
let annotation = null;
const deadline = performance.now() + 30_000;
while (performance.now() < deadline) {
annotation = (window.__gevAnnotations?.list?.() || []).find(
(item) => item.label === label,
);
if (annotation && !annotation.pendingOutline) break;
await new Promise((resolve) => setTimeout(resolve, 25));
}
if (!annotation || annotation.pendingOutline)
throw new Error(`${label}: outline did not settle`);
const { scene } = window.__godsEyeView.viewer;
await new Promise((resolve) => {
const timer = setTimeout(resolve, 1500);
const remove = scene.postRender.addEventListener(() => {
clearTimeout(timer);
remove();
resolve();
});
scene.requestRender();
});
await new Promise((resolve) =>
requestAnimationFrame(() => requestAnimationFrame(resolve)),
);
return {
commandToRenderMs: performance.now() - window.__qaOutlineStarted,
result,
annotation: {
pending: Boolean(annotation.pendingOutline),
unavailable: Boolean(annotation.outlineUnavailable),
synthesized: Boolean(annotation.synthesized),
footprintKind: annotation.footprintKind || null,
ringPoints: annotation.ring?.length || 0,
parts: annotation.polygons?.length || (annotation.ring ? 1 : 0),
},
measures: performance.getEntriesByType('measure').map((entry) => ({
name: entry.name,
duration: entry.duration,
startTime: entry.startTime,
detail: entry.detail,
})),
longTasks: window.__qaOutlineLongTasks.slice(),
};
}, label);
}
async function driveOutline(page, label) {
await beginOutline(page, label);
return settleOutline(page, label);
}
try {
for (let run = 0; run <= RUNS; run += 1) {
const page = await browser.newPage();
await page.setViewport({ width: 1440, height: 1000 });
let overlapMode = false;
let overlapApplicationRequests = 0;
let heldTargetRequest = null;
let resolveHeldTarget;
const heldTarget = new Promise((resolve) => {
resolveHeldTarget = resolve;
});
if (run === 0) {
await page.setRequestInterception(true);
page.on('request', (request) => {
let isTarget = false;
try {
isTarget = new URL(request.url()).pathname.endsWith(TARGET_PATH);
} catch {
// Invalid URLs are passed through for the browser to handle.
}
if (overlapMode && isTarget) {
overlapApplicationRequests += 1;
if (!heldTargetRequest) {
heldTargetRequest = request;
resolveHeldTarget();
return;
}
}
void request.continue();
});
}
await page.evaluateOnNewDocument(() => {
sessionStorage.setItem('gev:first-run-mission-session:v1', 'dismissed');
window.__qaOutlineLongTasks = [];
try {
new PerformanceObserver((list) => {
for (const entry of list.getEntries())
window.__qaOutlineLongTasks.push({
at: entry.startTime,
ms: entry.duration,
});
}).observe({ type: 'longtask', buffered: false });
window.__qaLongTaskSupported = true;
} catch {
window.__qaLongTaskSupported = false;
}
});
let phase = 'startup';
const network = [];
const pending = new Map();
const client = await page.createCDPSession();
await client.send('Network.enable');
client.on('Network.requestWillBeSent', (event) => {
let url;
try {
url = new URL(event.request.url);
} catch {
return;
}
if (
url.hostname !== 'tiles.openfreemap.org' ||
!url.pathname.endsWith('.pbf')
)
return;
const entry = {
requestId: event.requestId,
phase,
url: url.origin + url.pathname,
targetTile: url.pathname.endsWith(TARGET_PATH),
cached: false,
transferBytes: 0,
responseBytes: 0,
finished: false,
};
pending.set(event.requestId, entry);
network.push(entry);
});
client.on('Network.requestServedFromCache', ({ requestId }) => {
const entry = pending.get(requestId);
if (entry) entry.cached = true;
});
client.on('Network.responseReceived', ({ requestId, response }) => {
const entry = pending.get(requestId);
if (!entry) return;
entry.status = response.status;
entry.cached ||= Boolean(
response.fromDiskCache ||
response.fromServiceWorker ||
response.fromPrefetchCache,
);
});
client.on('Network.dataReceived', ({ requestId, dataLength }) => {
const entry = pending.get(requestId);
if (entry) entry.responseBytes += dataLength;
});
client.on('Network.loadingFinished', ({ requestId, encodedDataLength }) => {
const entry = pending.get(requestId);
if (!entry) return;
entry.transferBytes = entry.cached ? 0 : encodedDataLength;
entry.finished = true;
pending.delete(requestId);
});
client.on('Network.loadingFailed', ({ requestId, errorText }) => {
const entry = pending.get(requestId);
if (!entry) return;
entry.failure = errorText;
entry.finished = true;
pending.delete(requestId);
});
const pageErrors = [];
page.on('pageerror', (error) => pageErrors.push(String(error.message)));
const navigation = new URL(APP_URL);
navigation.searchParams.set('welcome', '0');
navigation.searchParams.set('trafficDebug', '1');
navigation.searchParams.set('trafficRoads', 'osm');
await page.goto(navigation.href, {
waitUntil: 'domcontentloaded',
timeout: 90_000,
});
await page.waitForFunction(
() =>
window.__godsEyeView?.viewer &&
window.__godsEyeView?.dataManager &&
window.__gevVoiceCommands?.runner &&
window.__gevAnnotations &&
document.getElementById('loading-screen')?.classList.contains('hidden'),
{ timeout: 120_000, polling: 250 },
);
await page.evaluate(
() => window.__godsEyeView.styleManager.initialRestorePromise,
);
await page.evaluate(() =>
window.__godsEyeView.styleManager.setDetection({ enabled: false }),
);
await page.evaluate(async (target) => {
const { viewer, dataManager } = window.__godsEyeView;
await dataManager.setEnabled('traffic', false);
viewer.camera.cancelFlight();
viewer.camera.setView({
destination: window.__CESIUM__.Cartesian3.fromDegrees(
target.longitude,
target.latitude,
700,
),
orientation: {
heading: 0,
pitch: (-65 * Math.PI) / 180,
roll: 0,
},
});
viewer.camera.moveEnd.raiseEvent();
window.__gevAnnotations.clear();
performance.clearMeasures();
}, TARGET);
await sleep(750);
if (run === 0) {
phase = 'overlap';
overlapMode = true;
await page.evaluate(() => {
window.__qaOutlineLongTasks.length = 0;
performance.clearMeasures();
});
await page.evaluate(() =>
window.__godsEyeView.dataManager.setEnabled('traffic', true),
);
await Promise.race([
heldTarget,
sleep(30_000).then(() => {
throw new Error('overlap phase did not intercept the target tile');
}),
]);
await beginOutline(page, 'QA Capitol overlap');
await sleep(100);
await heldTargetRequest.continue();
const overlap = await settleOutline(page, 'QA Capitol overlap');
await sleep(750);
const trafficStats = await page.evaluate(() =>
window.__godsEyeView.dataManager.layers
.get('traffic')
.module.getStats(),
);
assert.equal(trafficStats.error, null, 'overlap traffic source error');
overlap.measures = relevantMeasures(overlap.measures);
overlap.network = network.filter(
(entry) => entry.phase === 'overlap' && entry.targetTile,
);
const overlapDecodes = overlap.measures
.filter(
(entry) =>
entry.name === 'roads:decode' &&
String(entry.detail?.key).endsWith(
`:${TARGET.z}/${TARGET.x}/${TARGET.y}`,
),
)
.map((entry) => entry.ms);
assert.equal(
overlapApplicationRequests,
1,
'overlapping traffic and outline consumers issue one application request',
);
assert.equal(
overlap.network.length,
1,
'overlapping traffic and outline consumers emit one target network request',
);
assert.equal(
overlapDecodes.length,
1,
'overlapping traffic and outline consumers decode once',
);
assert.equal(overlap.annotation.unavailable, false);
assert.equal(overlap.annotation.footprintKind, 'building');
report.overlap = {
applicationFetches: overlapApplicationRequests,
targetNetworkRequests: overlap.network.length,
transferBytes: sum(
overlap.network.map((entry) => entry.transferBytes),
),
decodes: overlapDecodes.length,
decodeMs: round(sum(overlapDecodes)),
longTasks: overlap.longTasks.length,
};
await page.close();
continue;
}
phase = 'traffic';
await page.evaluate(() => {
window.__qaOutlineLongTasks.length = 0;
performance.clearMeasures();
});
await page.evaluate(() =>
window.__godsEyeView.dataManager.setEnabled('traffic', true),
);
await page.waitForFunction(
() => {
const stats = window.__godsEyeView.dataManager.layers
.get('traffic')
.module.getStats();
return stats.count > 0 && !stats.loading && !stats.error;
},
{ timeout: 180_000, polling: 250 },
);
await sleep(1500);
const trafficMeasures = relevantMeasures(
await page.evaluate(() =>
performance.getEntriesByType('measure').map((entry) => ({
name: entry.name,
duration: entry.duration,
startTime: entry.startTime,
detail: entry.detail,
})),
),
);
const trafficLongTasks = await page.evaluate(() =>
window.__qaOutlineLongTasks.slice(),
);
const trafficTarget = network.filter(
(entry) => entry.phase === 'traffic' && entry.targetTile,
);
const targetTile = `${TARGET.z}/${TARGET.x}/${TARGET.y}`;
const targetProjectionMeasures = trafficMeasures.filter(
(entry) =>
entry.detail?.tile === targetTile &&
['outlines:street-extract', 'outlines:polygon-extract'].includes(
entry.name,
),
);
const targetProjectionMs = sum(
targetProjectionMeasures.map((entry) => entry.ms),
);
assert.ok(
trafficTarget.length >= 1,
`sample ${run}: traffic did not load ${TARGET_PATH}`,
);
assert.equal(
targetProjectionMeasures.length,
2,
`sample ${run}: traffic did not prepare the target outline projection`,
);
assert.ok(
targetProjectionMs <= FRAME_BUDGET_MS,
`sample ${run}: traffic-idle target projection ${targetProjectionMs} ms exceeds ${FRAME_BUDGET_MS} ms`,
);
phase = 'outline-cold';
const cold = await driveOutline(page, `QA Capitol cold ${run}`);
await sleep(750);
cold.measures = relevantMeasures(cold.measures);
cold.network = network.filter(
(entry) => entry.phase === 'outline-cold' && entry.targetTile,
);
cold.longTasks = cold.longTasks.map((entry) => ({
at: round(entry.at),
ms: round(entry.ms),
}));
await page.evaluate(() => window.__gevAnnotations.clear());
phase = 'outline-warm';
const warm = await driveOutline(page, `QA Capitol warm ${run}`);
await sleep(750);
warm.measures = relevantMeasures(warm.measures);
warm.network = network.filter(
(entry) => entry.phase === 'outline-warm' && entry.targetTile,
);
warm.longTasks = warm.longTasks.map((entry) => ({
at: round(entry.at),
ms: round(entry.ms),
}));
const supported = await page.evaluate(() => window.__qaLongTaskSupported);
assert.equal(supported, true, `sample ${run}: Long Tasks unsupported`);
assert.deepEqual(pageErrors, [], `sample ${run}: browser errors`);
assert.equal(cold.annotation.unavailable, false);
assert.equal(cold.annotation.synthesized, false);
assert.equal(cold.annotation.footprintKind, 'building');
assert.ok(cold.annotation.ringPoints >= 4);
assert.equal(warm.annotation.footprintKind, 'building');
assert.ok(warm.annotation.ringPoints >= 4);
const coldDecodes = measures(cold.measures, 'roads:decode');
const coldInstalls = measures(cold.measures, 'outlines:install');
const warmDecodes = measures(warm.measures, 'roads:decode');
assert.equal(
cold.network.length,
0,
'shared decoded tile performs no second application fetch',
);
assert.equal(sum(cold.network.map((entry) => entry.transferBytes)), 0);
assert.equal(coldDecodes.length, 0, 'shared decoded tile is not parsed again');
assert.equal(
measures(cold.measures, 'outlines:street-extract').length,
0,
'traffic idle preparation owns street extraction',
);
assert.equal(
measures(cold.measures, 'outlines:polygon-extract').length,
0,
'traffic idle preparation owns polygon extraction',
);
assert.ok(coldInstalls.length >= 1, 'cold outline installs geometry');
assert.equal(warm.network.length, 0, 'decoded-cache hit performs no fetch');
assert.equal(warmDecodes.length, 0, 'decoded-cache hit performs no decode');
report.renderer ||= await page.evaluate(() => {
const gl = window.__godsEyeView.viewer.scene.context._gl;
const debug = gl.getExtension('WEBGL_debug_renderer_info');
return debug
? gl.getParameter(debug.UNMASKED_RENDERER_WEBGL)
: gl.getParameter(gl.RENDERER);
});
if (args.includes('--headful')) {
assert.doesNotMatch(
String(report.renderer),
/swiftshader|software rasterizer|llvmpipe/i,
`sample ${run}: headful evidence requires a hardware renderer`,
);
}
report.samples.push({
run,
pageErrors,
traffic: {
applicationFetches: trafficTarget.length,
transferBytes: sum(trafficTarget.map((entry) => entry.transferBytes)),
decodes: measures(trafficMeasures, 'roads:decode').length,
targetProjectionMs: round(targetProjectionMs),
longTasks: trafficLongTasks.map((entry) => ({
at: round(entry.at),
ms: round(entry.ms),
})),
measures: trafficMeasures,
network: trafficTarget,
},
cold: {
...cold,
applicationFetches: cold.network.length,
transferBytes: sum(cold.network.map((entry) => entry.transferBytes)),
decodeMs: round(sum(coldDecodes)),
bodyReadMs: round(sum(measures(cold.measures, 'roads:tile-body-read'))),
streetExtractMs: round(
sum(measures(cold.measures, 'outlines:street-extract')),
),
polygonExtractMs: round(
sum(measures(cold.measures, 'outlines:polygon-extract')),
),
installMs: round(sum(coldInstalls)),
mainThreadCpuMs: round(
sum(coldDecodes) +
sum(measures(cold.measures, 'outlines:street-extract')) +
sum(measures(cold.measures, 'outlines:polygon-extract')) +
sum(coldInstalls),
),
},
warm: {
...warm,
applicationFetches: warm.network.length,
transferBytes: sum(warm.network.map((entry) => entry.transferBytes)),
decodeMs: round(sum(warmDecodes)),
installMs: round(sum(measures(warm.measures, 'outlines:install'))),
},
});
await page.close();
}
const coldCpu = report.samples.map((sample) => sample.cold.mainThreadCpuMs);
const outlineLongTasks = report.samples.flatMap((sample) => [
...sample.cold.longTasks,
...sample.warm.longTasks,
]);
const trafficLongTasks = report.samples.flatMap(
(sample) => sample.traffic.longTasks,
);
report.summary = {
samples: report.samples.length,
overlapApplicationFetches: report.overlap?.applicationFetches ?? null,
overlapDecodes: report.overlap?.decodes ?? null,
coldCpuP50Ms: round(percentile(coldCpu, 0.5)),
coldCpuP95Ms: round(percentile(coldCpu, 0.95)),
coldCpuMaxMs: round(Math.max(...coldCpu)),
coldCommandToRenderP95Ms: round(
percentile(
report.samples.map((sample) => sample.cold.commandToRenderMs),
0.95,
),
),
coldApplicationFetches: sum(
report.samples.map((sample) => sample.cold.applicationFetches),
),
coldTransferBytes: sum(
report.samples.map((sample) => sample.cold.transferBytes),
),
coldDecodes: sum(
report.samples.map(
(sample) => measures(sample.cold.measures, 'roads:decode').length,
),
),
warmApplicationFetches: sum(
report.samples.map((sample) => sample.warm.applicationFetches),
),
warmDecodes: sum(
report.samples.map(
(sample) => measures(sample.warm.measures, 'roads:decode').length,
),
),
trafficIdleProjectionMaxMs: round(
Math.max(
...report.samples.map((sample) => sample.traffic.targetProjectionMs),
),
),
trafficBaselineLongTasks: trafficLongTasks.length,
trafficBaselineMaxLongTaskMs: round(
trafficLongTasks.length
? Math.max(...trafficLongTasks.map((entry) => entry.ms))
: 0,
),
longTasks: outlineLongTasks.length,
maxLongTaskMs: round(
outlineLongTasks.length
? Math.max(...outlineLongTasks.map((entry) => entry.ms))
: 0,
),
};
assert.ok(
report.summary.coldCpuP95Ms <= FRAME_BUDGET_MS,
`cold decode+extract+install p95 ${report.summary.coldCpuP95Ms} ms exceeds ${FRAME_BUDGET_MS} ms`,
);
assert.ok(
report.summary.trafficIdleProjectionMaxMs <= FRAME_BUDGET_MS,
'traffic-idle outline projection exceeded one frame',
);
assert.equal(report.summary.longTasks, 0, 'outline ask created a Long Task');
assert.equal(report.summary.coldTransferBytes, 0);
assert.equal(report.summary.coldApplicationFetches, 0);
assert.equal(report.summary.coldDecodes, 0);
assert.equal(report.summary.warmApplicationFetches, 0);
assert.equal(report.summary.warmDecodes, 0);
assert.equal(report.summary.overlapApplicationFetches, 1);
assert.equal(report.summary.overlapDecodes, 1);
report.passed = true;
console.table(
report.samples.map((sample) => ({
run: sample.run,
trafficPrepMs: sample.traffic.targetProjectionMs,
appFetches: sample.cold.applicationFetches,
transferBytes: sample.cold.transferBytes,
bodyReadMs: sample.cold.bodyReadMs,
decodeMs: sample.cold.decodeMs,
streetMs: sample.cold.streetExtractMs,
polygonMs: sample.cold.polygonExtractMs,
installMs: sample.cold.installMs,
cpuMs: sample.cold.mainThreadCpuMs,
renderMs: round(sample.cold.commandToRenderMs),
longTasks: sample.cold.longTasks.length,
})),
);
console.log('OpenFreeMap same-tile acceptance passed:', report.summary);
} catch (error) {
report.passed = false;
report.failure = String(error?.stack || error);
console.error(error);
process.exitCode = 1;
} finally {
await fs.writeFile(OUTPUT, `${JSON.stringify(report, null, 2)}\n`);
await browser.close();
}
-411
View File
@@ -1,411 +0,0 @@
#!/usr/bin/env node
/**
* qa-voice-bench.mjs — provider-neutral TEXT-mode voice-routing benchmark.
*
* Every provider gets the same prompt: production's realtimeInstructions()
* and GEV_REALTIME_TOOLS (the Realtime adapter uses the exact session body the
* token handler mints). Two suites, reported separately:
* core the qa-voice-routing phrase table, graded with its semantics
* (plus benchmark-only typed-argument checks)
* coverage the newer-layers coverage probe, graded correct /
* honest-refusal / hallucinated-capability / wrong-tool
* dialogue two-turn dialogues in one context; turn 2 depends on turn 1
* capability area handles / imagery / OSM / point-and-ask phrases (and two
* dialogues) for the tools added after 4b56d0e9
* Coverage rows are graded twice: v1 (original rubric, 4b56d0e9 tool schema)
* and v2 (rubric for the current tool surface).
*
* Methodology: a fresh context per phrase (new Realtime/Live session, or a
* stateless chat request); stub tool results; continuations until the model
* stops calling tools (max 4). No app page is involved.
*
* Usage:
* node scripts/qa-voice-bench.mjs --provider openai-realtime --model gpt-realtime-2 --reps 2
* node scripts/qa-voice-bench.mjs --provider gemini-live --model gemini-3.8-live
* node scripts/qa-voice-bench.mjs --provider ollama --model gemma4:e4b
* node scripts/qa-voice-bench.mjs --provider openai-chat --base http://localhost:1234/v1 --model <id>
* node scripts/qa-voice-bench.mjs --summarize --out qa-shots/voice-bench/<run>
* Common flags: --out <dir> (reuse one dir to collect several models),
* --suite core,coverage --only <substr> --budget <usd, cloud total>
* --variant <label> --temperature <t> --num-ctx <n> --think --note <text>
* --reps <n> --start-rep <k> (add repetitions k..k+n-1 to an existing run)
* --replace (supersede earlier runs of this model for the suites run)
* --compare <run dir> (add a before → after table against another run)
*
* Output (gitignored): <out>/results.jsonl, summary.json, summary.md.
* Keys come from the environment or the macOS Keychain at runtime only.
*/
import fs from 'node:fs';
import path from 'node:path';
import os from 'node:os';
import { execFileSync } from 'node:child_process';
import { fileURLToPath } from 'node:url';
import {
CORE_PHRASES, COVERAGE_PROBE, DIALOGUES, CAPABILITY_EXTRAS,
dialogueResult, coverageRubric,
} from './voice-bench/phrases.mjs';
import { gradeCore, gradeCoverage, gradeDialogue, expectLabel } from './voice-bench/grade.mjs';
import { stubResult } from './voice-bench/stub.mjs';
import { summarize, renderMarkdown, renderComparison } from './voice-bench/report.mjs';
import { GEV_REALTIME_TOOLS, realtimeInstructions } from './voice-bench/productionConfig.mjs';
import { chatNormalizationDiff } from './voice-bench/toolFormats.mjs';
import { createOpenAiRealtimeAdapter } from './voice-bench/adapters/openaiRealtime.mjs';
import { createGeminiLiveAdapter } from './voice-bench/adapters/geminiLive.mjs';
import { createOpenAiChatAdapter, createOllamaAdapter } from './voice-bench/adapters/localChat.mjs';
const ROOT = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '..');
const argv = process.argv.slice(2);
const opt = (flag, fallback) => {
const i = argv.indexOf(flag);
return i >= 0 && argv[i + 1] !== undefined ? argv[i + 1] : fallback;
};
const has = (flag) => argv.includes(flag);
const PROVIDER = opt('--provider', null);
const MODEL = opt('--model', null);
const CLOUD = PROVIDER === 'openai-realtime' || PROVIDER === 'gemini-live';
const REPS = Number(opt('--reps', CLOUD ? '2' : '1'));
const START_REP = Number(opt('--start-rep', '1'));
const SUITES = opt('--suite', 'core,coverage,dialogue,capability').split(',');
const ONLY = opt('--only', null);
const BUDGET = Number(opt('--budget', '30'));
const VARIANT = opt('--variant', null);
const OUT = path.resolve(ROOT, opt('--out', path.join('qa-shots', 'voice-bench', new Date().toISOString().replace(/[:.]/g, '-'))));
const LEDGER = path.join(ROOT, 'qa-shots', 'voice-bench', 'cloud-spend.json');
fs.mkdirSync(OUT, { recursive: true });
const RESULTS = path.join(OUT, 'results.jsonl');
const NOTES = path.join(OUT, 'notes.json');
function readRows() {
if (!fs.existsSync(RESULTS)) return [];
// Several runs may append to one results file (a cloud run beside a local
// one); skip a line another process is still writing.
return fs.readFileSync(RESULTS, 'utf8').split('\n').filter(Boolean).flatMap((l) => {
try { return [JSON.parse(l)]; } catch { return []; }
});
}
function appendRow(row) {
fs.appendFileSync(RESULTS, `${JSON.stringify(row)}\n`);
}
function readNotes() {
try { return JSON.parse(fs.readFileSync(NOTES, 'utf8')); } catch { return []; }
}
/**
* Re-grade stored turns with the CURRENT phrase tables and grader, so a
* grader fix applies to every model already in the results file. The raw
* calls and text in results.jsonl are the source of truth; `gradeAtRun`
* keeps the verdict recorded at run time.
*/
function regrade(rows) {
const core = new Map(CORE_PHRASES.map((p) => [p.phrase, p]));
const cov = new Map(COVERAGE_PROBE.map((c) => [c.id, c]));
const dia = new Map(DIALOGUES.map((d) => [d.id, d]));
const cap = new Map(CAPABILITY_ITEMS.map((c) => [c.id, c]));
// A run started with --replace supersedes earlier runs of the same
// provider/model/variant for the suites it covers.
const key = (r) => `${r.provider}:${r.model}#${r.variant || ''}`;
const runs = rows.filter((r) => r.type === 'run');
const superseded = new Set();
for (const r of runs.filter((x) => x.replace)) {
for (const older of runs) {
if (older.at < r.at && key(older) === key(r)) superseded.add(`${older.runId}|${(r.suites || []).join(',')}`);
}
}
const isSuperseded = (row) => [...superseded].some((s) => {
const [runId, suites] = s.split('|');
return runId === row.runId && suites.split(',').includes(row.suite);
});
return rows.filter((row) => row.type !== 'turn' || !isSuperseded(row)).map((row) => {
if (row.type !== 'turn') return row;
if (row.suite === 'dialogue' && dia.has(row.phraseId)) {
return { ...row, gradeAtRun: row.grade, grade: gradeDialogue(dia.get(row.phraseId), [row.turn1?.calls || [], row.calls || []]) };
}
if (row.suite === 'core' && core.has(row.phraseId)) {
return { ...row, gradeAtRun: row.grade, grade: gradeCore(core.get(row.phraseId), row.calls || []) };
}
if (row.suite === 'coverage' && cov.has(row.phraseId)) {
const item = cov.get(row.phraseId);
return {
...row,
gradeAtRun: row.grade,
grade: gradeCoverage(item, row.calls || [], row.text, BASELINE_TOOLS),
gradeV2: gradeCoverage(coverageRubric(item, 'v2'), row.calls || [], row.text, GEV_REALTIME_TOOLS),
};
}
if (row.suite === 'capability' && cap.has(row.phraseId)) {
const c = cap.get(row.phraseId);
const grade = c.dialogue ? gradeDialogue(c.dialogue, [row.turn1?.calls || [], row.calls || []]) : gradeCore(c.phrase, row.calls || []);
return { ...row, gradeAtRun: row.grade, grade };
}
return row;
});
}
function readRowsFrom(dir) {
const file = path.join(dir, 'results.jsonl');
if (!fs.existsSync(file)) return [];
return fs.readFileSync(file, 'utf8').split('\n').filter(Boolean).flatMap((l) => {
try { return [JSON.parse(l)]; } catch { return []; }
});
}
function writeSummary() {
const summary = summarize(regrade(readRows()));
// --compare <dir> (remembered in compare-with.txt): a before → after table
// against another run folder, both re-graded with the current grader.
const compareFile = path.join(OUT, 'compare-with.txt');
if (has('--compare')) fs.writeFileSync(compareFile, `${path.resolve(ROOT, opt('--compare', ''))}\n`);
let comparison = '';
if (fs.existsSync(compareFile)) {
const beforeDir = fs.readFileSync(compareFile, 'utf8').trim();
const before = summarize(regrade(readRowsFrom(beforeDir)));
const beforeHead = readRowsFrom(beforeDir).find((r) => r.type === 'run' && r.prompt?.gitHead)?.prompt?.gitHead || '4b56d0e9';
const afterHead = readRows().find((r) => r.type === 'run' && r.prompt?.gitHead)?.prompt?.gitHead || '?';
comparison = renderComparison(before, summary, {
beforeLabel: `${path.relative(ROOT, beforeDir)} (prompt/tools @ ${beforeHead})`,
afterLabel: `${path.relative(ROOT, OUT)} (prompt/tools @ ${afterHead})`,
});
summary.comparison = { beforeDir: path.relative(ROOT, beforeDir), before: before.models.map((m) => ({ key: m.key, coreFull: m.core.fullCallAccuracyPct, coverageV1Pct: m.coverage.acceptablePct, hcV1: m.coverage.counts['hallucinated-capability'], dialogueT2: m.dialogue.turn2FullPct, toolCallP50: m.latencyMs.toolCallP50, promptTokens: m.warmup?.promptTokens ?? m.tokens.inputPerRequestMean })) };
fs.writeFileSync(path.join(OUT, 'comparison.md'), `${comparison}\n`);
}
fs.writeFileSync(path.join(OUT, 'summary.json'), JSON.stringify(summary, null, 2));
const md = renderMarkdown(summary, { notes: readNotes(), payload: payloadChanges() });
fs.writeFileSync(path.join(OUT, 'summary.md'), comparison ? md.replace('## Routing accuracy', `${comparison}\n## Routing accuracy`) : md);
return summary;
}
/** Exactly what each provider receives differently from production's payload. */
function payloadChanges() {
const chat = chatNormalizationDiff(GEV_REALTIME_TOOLS)
.map((c) => `\`${c.path}\`: ${JSON.stringify(c.from)} → ${JSON.stringify(c.to)}`);
return [
`openai-realtime: none — the session body is captured from production's token handler (${GEV_REALTIME_TOOLS.length} tools, instructions, tool_choice auto, reasoning low, truncation). Only difference from a mic session: response.create asks for output_modalities ["text"] (as qa-voice-routing does).`,
'gemini-live: same instructions text as systemInstruction; the same tool schemas passed verbatim as functionDeclarations[].parametersJsonSchema (no keyword changes); tool_choice has no Live equivalent (AUTO is the default); responseModalities AUDIO + outputAudioTranscription (TEXT is rejected by the model).',
`ollama / openai-chat: same instructions as the system message; tools wrapped as {type:"function",function:{name,description,parameters}} with ${chat.length} schema change(s): ${chat.join('; ') || 'none'}. Descriptions, enums and required lists are unchanged; nothing is stringified.`,
];
}
function ledgerSpent() {
try { return JSON.parse(fs.readFileSync(LEDGER, 'utf8')).spentUsd || 0; } catch { return 0; }
}
function ledgerAdd(usd) {
const spent = ledgerSpent() + usd;
fs.mkdirSync(path.dirname(LEDGER), { recursive: true });
fs.writeFileSync(LEDGER, JSON.stringify({ spentUsd: spent, updatedAt: new Date().toISOString() }));
return spent;
}
const sh = (cmd, args) => {
try { return execFileSync(cmd, args, { encoding: 'utf8', stdio: ['ignore', 'pipe', 'ignore'], timeout: 10_000 }).trim(); } catch { return null; }
};
/** Snapshot of what else is using the machine (GPU-heavy servers, load). */
function machineSnapshot() {
const load = os.loadavg().map((x) => x.toFixed(2)).join(' ');
const ollamaPs = sh('ollama', ['ps']);
const procs = sh('ps', ['-Ao', 'pcpu=,rss=,comm=']) || '';
const heavy = procs.split('\n').map((l) => l.trim().split(/\s+/)).filter((p) => Number(p[0]) > 20)
.map((p) => `${path.basename(p.slice(2).join(' '))} ${p[0]}%cpu`).slice(0, 6);
const gpuServers = procs.split('\n').filter((l) => /ollama runner|llama-server|LM Studio|lms|mlx|whisper/i.test(l))
.map((l) => path.basename(l.trim().split(/\s+/).slice(2).join(' '))).slice(0, 6);
return {
at: new Date().toISOString(),
loadavg: load,
ollamaPs,
busyProcesses: heavy,
modelServers: gpuServers,
summary: `load ${load}; busy: ${heavy.join(', ') || 'none'}`,
};
}
// ── Ollama lifecycle ────────────────────────────────────────────────────────
const OLLAMA = opt('--base', 'http://localhost:11434').replace(/\/v1\/?$/, '');
async function ollamaUnloadAll() {
const ps = await fetch(`${OLLAMA}/api/ps`).then((r) => r.json()).catch(() => ({ models: [] }));
for (const m of ps.models || []) {
await fetch(`${OLLAMA}/api/generate`, { method: 'POST', body: JSON.stringify({ model: m.name, keep_alive: 0 }) }).then((r) => r.text()).catch(() => null);
}
return (ps.models || []).map((m) => m.name);
}
async function ollamaLoad(model, numCtx) {
const t = Date.now();
const res = await fetch(`${OLLAMA}/api/generate`, {
method: 'POST',
body: JSON.stringify({ model, keep_alive: '30m', options: { num_ctx: numCtx } }),
signal: AbortSignal.timeout(600_000),
});
const body = await res.json().catch(() => ({}));
if (!res.ok || body.error) throw new Error(`load failed: ${body.error || res.status}`);
return { wallMs: Date.now() - t, loadMs: body.load_duration ? body.load_duration / 1e6 : null };
}
// ── Main ────────────────────────────────────────────────────────────────────
function buildAdapter() {
const temperature = Number(opt('--temperature', '0.8'));
switch (PROVIDER) {
case 'openai-realtime':
return createOpenAiRealtimeAdapter({ model: MODEL });
case 'gemini-live':
return createGeminiLiveAdapter({ model: MODEL, schemaMode: opt('--gemini-schema', 'jsonSchema') });
case 'ollama':
return createOllamaAdapter({ model: MODEL, baseUrl: OLLAMA, temperature, numCtx: Number(opt('--num-ctx', '32768')), think: has('--think') });
case 'openai-chat':
return createOpenAiChatAdapter({ model: MODEL, baseUrl: opt('--base', 'http://localhost:1234/v1'), temperature });
default:
throw new Error(`unknown --provider ${PROVIDER}`);
}
}
const COLD_PHRASE = 'Take me to Tokyo';
/** Capability suite: single phrases (optional pointer context) and dialogues. */
const CAPABILITY_ITEMS = CAPABILITY_EXTRAS.map((c) => (c.turns
? { suite: 'capability', id: c.id, text: c.turns.join(' → '), dialogue: c }
: { suite: 'capability', id: c.context ? `${c.phrase} [pointer]` : c.phrase, text: c.phrase, phrase: c, context: c.context || null }));
/** The 4b56d0e9 tool schema, for grading coverage by the original yardstick. */
const BASELINE_TOOLS = JSON.parse(fs.readFileSync(path.join(ROOT, 'scripts', 'voice-bench', 'baseline-tools-4b56d0e9.json'), 'utf8'));
/** What prompt this run actually sent (so before/after runs are identifiable). */
function promptVersion() {
let head = null;
try { head = execFileSync('git', ['-C', ROOT, 'rev-parse', '--short', 'HEAD'], { encoding: 'utf8' }).trim(); } catch { /* not a checkout */ }
return {
gitHead: head,
tools: GEV_REALTIME_TOOLS.length,
toolJsonChars: JSON.stringify(GEV_REALTIME_TOOLS).length,
instructionsChars: realtimeInstructions().length,
};
}
function workList() {
const items = [];
if (SUITES.includes('core')) {
for (const p of CORE_PHRASES) items.push({ suite: 'core', id: p.phrase, text: p.phrase, phrase: p });
}
if (SUITES.includes('coverage')) {
for (const c of COVERAGE_PROBE) items.push({ suite: 'coverage', id: c.id, text: c.utterance, item: c });
}
if (SUITES.includes('dialogue')) {
for (const d of DIALOGUES) items.push({ suite: 'dialogue', id: d.id, text: d.turns.join(' → '), dialogue: d });
}
if (SUITES.includes('capability')) items.push(...CAPABILITY_ITEMS);
return items.filter((w) => !ONLY || w.text.toLowerCase().includes(ONLY.toLowerCase()));
}
async function main() {
if (has('--note')) {
const notes = readNotes();
notes.push(opt('--note', ''));
fs.writeFileSync(NOTES, JSON.stringify(notes, null, 2));
}
if (has('--summarize') || !PROVIDER) {
writeSummary();
console.log(`summary written to ${path.join(OUT, 'summary.md')}`);
return;
}
if (!MODEL) throw new Error('--model is required');
const adapter = buildAdapter();
await adapter.open();
let warmup = null;
if (PROVIDER === 'ollama') {
const unloaded = await ollamaUnloadAll();
const numCtx = Number(opt('--num-ctx', '32768'));
const load = await ollamaLoad(MODEL, numCtx);
// COLD turn: the first request after load pays the full-prompt prefill.
// It is recorded separately (warmup.cold*) and primes the prefix cache
// the way a live session's first turn would; every benchmark turn after
// it is steady state.
const w = await adapter.runTurn({ text: COLD_PHRASE, onCall: () => ({ ok: true }) });
warmup = { unloaded, loadWallMs: load.wallMs, loadMs: load.loadMs, coldPhrase: COLD_PHRASE, prefillTokS: w.local.prefillTokS, promptTokens: w.local.promptEvalCount, coldFirstTokenMs: w.tFirstTokenMs, coldToolCallMs: w.tFirstCallMs, coldCalls: w.calls.map((c) => c.name), warmupTurnMs: w.tDoneMs, errors: w.errors };
console.log(`warm-up: load ${Math.round(load.wallMs)}ms, prefill ${Math.round(w.local.prefillTokS || 0)} tok/s over ${w.local.promptEvalCount} tokens`);
if (w.errors.length) console.log(` warm-up errors: ${w.errors.join('; ')}`);
} else if (PROVIDER === 'openai-chat') {
const w = await adapter.runTurn({ text: COLD_PHRASE, onCall: () => ({ ok: true }) });
warmup = { loadMs: Number(opt('--load-ms', 'NaN')) || null, coldPhrase: COLD_PHRASE, prefillTokS: w.local.prefillTokS, promptTokens: w.local.promptEvalCount, coldFirstTokenMs: w.tFirstTokenMs, coldToolCallMs: w.tFirstCallMs, coldCalls: w.calls.map((c) => c.name), warmupTurnMs: w.tDoneMs, errors: w.errors };
console.log(`warm-up: ${w.tDoneMs}ms prefill≈${Math.round(w.local.prefillTokS || 0)} tok/s`);
if (w.errors.length) throw new Error(`warm-up failed, not benchmarking: ${w.errors[0]}`);
}
const runId = `${Date.now().toString(36)}`;
const machine = CLOUD ? null : machineSnapshot();
appendRow({ type: 'run', runId, at: new Date().toISOString(), provider: PROVIDER, model: MODEL, variant: VARIANT, reps: REPS, suites: SUITES, settings: adapter.settings, machine, warmup, replace: has('--replace'), prompt: promptVersion(), methodology: 'fresh context per phrase/dialogue; stub tool results (canned results for dialogue turn 1); ≤4 continuations' });
const items = workList();
let stopReason = null;
let turnCosts = [];
outer: for (let rep = START_REP; rep < START_REP + REPS; rep += 1) {
console.log(`\n== ${PROVIDER} ${MODEL}${VARIANT ? ` (${VARIANT})` : ''} — rep ${rep}/${REPS}, ${items.length} phrases`);
for (const w of items) {
if (CLOUD) {
const avg = turnCosts.length ? turnCosts.reduce((a, b) => a + b, 0) / turnCosts.length : 0.1;
if (ledgerSpent() + avg > BUDGET) { stopReason = `cloud budget $${BUDGET} reached (ledger $${ledgerSpent().toFixed(2)})`; break outer; }
}
let r;
let turn1 = null;
let grade;
if (w.dialogue) {
const rs = await adapter.runDialogue({
turns: w.dialogue.turns,
onCall: (c, turn) => dialogueResult(w.dialogue, c.name, turn) ?? stubResult('core', c, GEV_REALTIME_TOOLS),
});
const [a, b] = rs;
turn1 = { text: a.text.slice(0, 400), calls: a.calls, tFirstCallMs: a.tFirstCallMs, tDoneMs: a.tDoneMs, errors: a.errors };
r = {
...b,
errors: [...a.errors, ...b.errors],
timeout: a.timeout || b.timeout,
costUsd: a.costUsd + b.costUsd,
usage: Object.fromEntries(Object.keys(b.usage).map((k) => [k, (a.usage[k] || 0) + (b.usage[k] || 0)])),
setupMs: a.setupMs,
};
grade = gradeDialogue(w.dialogue, [a.calls, b.calls]);
} else {
r = await adapter.runTurn({ text: w.text, context: w.context || undefined, onCall: (c) => stubResult(w.suite, c, GEV_REALTIME_TOOLS) });
grade = w.suite === 'coverage' ? gradeCoverage(w.item, r.calls, r.text, GEV_REALTIME_TOOLS) : gradeCore(w.phrase, r.calls);
}
const row = {
type: 'turn', runId, provider: PROVIDER, model: MODEL, variant: VARIANT, cloud: CLOUD, rep, suite: w.suite,
phraseId: w.id, phrase: w.text,
expect: w.dialogue ? `t1 ${expectLabel(w.dialogue.first)} → t2 ${expectLabel(w.dialogue.second)}` : w.suite === 'coverage' ? w.item.expect : expectLabel(w.phrase),
context: w.context || undefined,
expectNone: Boolean(w.phrase?.expectNone), supported: w.item ? w.item.supported : undefined,
calls: r.calls, text: r.text.slice(0, 800), grade, turn1: turn1 || undefined,
tFirstTokenMs: r.tFirstTokenMs, tFirstCallMs: r.tFirstCallMs, tDoneMs: r.tDoneMs, setupMs: r.setupMs,
responses: r.responses, usage: r.usage, costUsd: r.costUsd, errors: r.errors, timeout: r.timeout,
local: r.local || undefined,
at: new Date().toISOString(),
};
appendRow(row);
if (CLOUD) { ledgerAdd(r.costUsd); turnCosts.push(r.costUsd); }
const verdict = w.suite === 'coverage' ? grade.verdict : (grade.ok ? 'PASS' : 'FAIL');
console.log(` [${verdict}] ${w.text} → ${turn1 ? `${turn1.calls.map((c) => c.name).join(',') || '(none)'} ⇒ ` : ''}${r.calls.map((c) => c.name).join(',') || '(none)'} tool@${r.tFirstCallMs ?? '—'}ms done@${r.tDoneMs}ms $${r.costUsd.toFixed(4)}${r.errors.length ? ` ERR ${r.errors[0]}` : ''}${r.timeout ? ' TIMEOUT' : ''}`);
if (r.errors.some((e) => /HTTP 4\d\d|close 100[78]|invalid|not found|model_not_found|does not exist/i.test(e)) && r.calls.length === 0 && !r.text) {
const same = readRows().filter((x) => x.type === 'turn' && x.runId === runId).slice(-3);
if (same.length === 3 && same.every((x) => x.errors?.length && !x.calls.length)) { stopReason = `3 consecutive hard errors: ${r.errors[0]}`; break outer; }
}
}
writeSummary();
}
await adapter.close();
if (PROVIDER === 'ollama' && !has('--keep-loaded')) {
await fetch(`${OLLAMA}/api/generate`, { method: 'POST', body: JSON.stringify({ model: MODEL, keep_alive: 0 }) }).catch(() => null);
}
if (stopReason) {
console.log(`STOPPED: ${stopReason}`);
appendRow({ type: 'stop', runId, provider: PROVIDER, model: MODEL, reason: stopReason, at: new Date().toISOString() });
}
const summary = writeSummary();
const mine = summary.models.find((m) => m.provider === PROVIDER && m.model === MODEL && (m.variant || null) === (VARIANT || null));
if (mine) console.log(`\n${mine.key}: core ${mine.core.accuracyPct}% | coverage acceptable ${mine.coverage.acceptablePct}% | p50 tool ${mine.latencyMs.toolCallP50}ms | $${mine.costUsd}`);
console.log(`results: ${RESULTS}\nsummary: ${path.join(OUT, 'summary.md')}\ncloud ledger: $${ledgerSpent().toFixed(3)}`);
}
main().catch((e) => {
console.error(e);
process.exitCode = 1;
});
File diff suppressed because it is too large. Load diff
-179
View File
@@ -1,179 +0,0 @@
/**
* Gemini Live adapter — BidiGenerateContent over WebSocket.
*
* Same prompt as production (system instruction = realtimeInstructions(),
* tools = GEV_REALTIME_TOOLS as functionDeclarations with the production JSON
* schema passed verbatim via parametersJsonSchema). The native-audio Live
* model rejects a TEXT response modality (close 1007), so the session
* requests AUDIO with output transcription on; tool calls arrive as
* `toolCall` messages and the transcript stands in for the final words.
*
* Fresh session per phrase / dialogue. Each toolCall gets stub
* `functionResponses`; the model continues on its own. A turn ends at a
* `turnComplete` whose generation had no toolCall, or after 5 tool rounds
* (mirrors the Realtime adapter's continuation cap), or 12 s of silence after
* a tool round.
*/
import WebSocket from 'ws';
import { providerKey } from '../keys.mjs';
import { realtimeInstructions, GEV_REALTIME_TOOLS } from '../productionConfig.mjs';
import { toGeminiDeclarations } from '../toolFormats.mjs';
import { geminiUsageCostUsd } from '../pricing.mjs';
import { newTurnResult } from './openaiRealtime.mjs';
const ENDPOINT = 'wss://generativelanguage.googleapis.com/ws/google.ai.generativelanguage.v1beta.GenerativeService.BidiGenerateContent';
const MAX_TOOL_ROUNDS = 5;
const IDLE_AFTER_TOOL_MS = 12_000;
export function createGeminiLiveAdapter({ model, schemaMode = 'jsonSchema', turnTimeoutMs = 60_000 }) {
const instructions = realtimeInstructions();
const functionDeclarations = toGeminiDeclarations(GEV_REALTIME_TOOLS, { mode: schemaMode });
const adapter = {
provider: 'gemini-live',
model,
settings: {
responseModalities: ['AUDIO'],
outputAudioTranscription: true,
schemaMode,
temperature: 'server default (not set; production has no Gemini config)',
thinking: 'server default',
input: 'realtimeInput.text',
context: 'fresh session per phrase / dialogue',
},
async open() {},
async close() {},
async runTurn({ text, onCall, context }) {
return (await adapter.runDialogue({ turns: [text], onCall, contexts: [context] }))[0];
},
runDialogue({ turns, onCall, contexts = [] }) {
return new Promise((resolveAll) => {
const tSetup = Date.now();
const ws = new WebSocket(`${ENDPOINT}?key=${providerKey('gemini')}`);
const results = [];
let setupMs = null;
let cur = null;
let closed = false;
const send = (o) => { if (ws.readyState === WebSocket.OPEN) ws.send(JSON.stringify(o)); };
const shutdown = (error) => {
if (closed) return;
closed = true;
if (cur) {
clearTimeout(cur.timer);
clearTimeout(cur.idle);
if (error) cur.r.errors.push(error);
if (cur.r.tDoneMs === null) cur.r.tDoneMs = Date.now() - cur.t0;
results.push(cur.r);
cur = null;
}
while (results.length < turns.length) {
const r = newTurnResult();
r.errors.push(error || 'dialogue aborted');
results.push(r);
}
try { ws.terminate(); } catch { /* noop */ }
resolveAll(results);
};
const startTurn = (turn) => {
const r = newTurnResult();
r.setupMs = turn === 0 ? setupMs : 0;
cur = { r, t0: Date.now(), turn, toolRounds: 0, toolCallThisGeneration: false, idle: null };
cur.timer = setTimeout(() => { cur.r.timeout = true; endTurn(); }, turnTimeoutMs);
// Live has no system-role mid-session item; a context item (pointer_context)
// rides as a JSON line ahead of the user's words in the same text turn.
const ctx = contexts[turn] ? `${JSON.stringify(contexts[turn])}\n` : '';
send({ realtimeInput: { text: `${ctx}${turns[turn]}` } });
};
const endTurn = () => {
if (!cur) return;
clearTimeout(cur.timer);
clearTimeout(cur.idle);
if (cur.r.tDoneMs === null) cur.r.tDoneMs = Date.now() - cur.t0;
results.push(cur.r);
const next = cur.turn + 1;
const timedOut = cur.r.timeout;
cur = null;
if (next < turns.length && !timedOut) startTurn(next);
else shutdown();
};
const mark = (key) => { if (cur && cur.r[key] === null) cur.r[key] = Date.now() - cur.t0; };
ws.on('open', () => {
send({
setup: {
model: `models/${model}`,
generationConfig: { responseModalities: ['AUDIO'] },
systemInstruction: { parts: [{ text: instructions }] },
tools: [{ functionDeclarations }],
outputAudioTranscription: {},
},
});
});
ws.on('unexpected-response', (_req, res) => shutdown(`HTTP ${res.statusCode}`));
ws.on('error', (e) => shutdown(`ws: ${e.message}`));
ws.on('close', (code, reason) => {
shutdown(results.length < turns.length ? `close ${code}: ${reason.toString().slice(0, 200)}` : undefined);
});
ws.on('message', (raw) => {
let m;
try { m = JSON.parse(raw.toString()); } catch { return; }
if (m.setupComplete) {
if (setupMs === null) {
setupMs = Date.now() - tSetup;
startTurn(0);
}
return;
}
if (!cur) return;
clearTimeout(cur.idle);
const r = cur.r;
const lastAt = Date.now();
const sc = m.serverContent;
if (sc?.modelTurn?.parts?.length) {
mark('tFirstTokenMs');
for (const p of sc.modelTurn.parts) if (typeof p.text === 'string' && !p.thought) r.text += p.text;
}
if (sc?.outputTranscription?.text) {
mark('tFirstTokenMs');
r.text += sc.outputTranscription.text;
}
if (m.toolCall?.functionCalls?.length) {
mark('tFirstTokenMs');
mark('tFirstCallMs');
cur.toolCallThisGeneration = true;
cur.toolRounds += 1;
const functionResponses = m.toolCall.functionCalls.map((fc) => {
const c = { name: fc.name, args: fc.args || {}, atMs: Date.now() - cur.t0 };
r.calls.push(c);
return { id: fc.id, name: fc.name, response: onCall(c, cur.turn) };
});
send({ toolResponse: { functionResponses } });
}
if (m.usageMetadata) {
const u = m.usageMetadata;
r.usage.inputTokens += u.promptTokenCount || 0;
r.usage.cachedTokens += u.cachedContentTokenCount || 0;
r.usage.outputTokens += u.responseTokenCount || 0;
r.usage.reasoningTokens += u.thoughtsTokenCount || 0;
r.costUsd += geminiUsageCostUsd(model, u);
}
if (m.goAway) r.errors.push('goAway');
if (sc?.turnComplete) {
r.responses += 1;
if (cur.toolCallThisGeneration && cur.toolRounds < MAX_TOOL_ROUNDS) {
cur.toolCallThisGeneration = false;
if (r.text && !r.text.endsWith(' | ')) r.text += ' | ';
// The follow-up generation normally starts within a second; if
// the model has nothing more to say, end the turn at its last
// message rather than charging the wait to its latency.
const c = cur;
c.idle = setTimeout(() => { if (cur === c) { c.r.tDoneMs = lastAt - c.t0; endTurn(); } }, IDLE_AFTER_TOOL_MS);
return;
}
endTurn();
}
});
});
},
};
return adapter;
}
-263
View File
@@ -1,263 +0,0 @@
/**
* Local-model adapters: stateless chat requests, so every phrase is a fresh
* context by construction. Same system prompt (realtimeInstructions()) and
* the production tools converted with toChatTools() (schema-normalized).
*
* openai-chat any OpenAI-compatible server (LM Studio :1234/v1, Ollama
* :11434/v1). Streams SSE; timing is client-side.
* ollama Ollama's native /api/chat. Same message and tool formats as
* Chat Completions, but it can set num_ctx (the OpenAI shim
* cannot — the ~15k-token prompt would be silently truncated
* at a small default context) and turn thinking off, and it
* reports server-side prefill/decode/load timings.
*
* Both mirror the Realtime loop: stub tool results, then continue until a
* response has no tool calls, up to 4 continuations.
*/
import { realtimeInstructions, GEV_REALTIME_TOOLS } from '../productionConfig.mjs';
import { toChatTools } from '../toolFormats.mjs';
const MAX_CONTINUATIONS = 4;
function emptyResult() {
return {
calls: [], text: '', errors: [], responses: 0,
usage: { inputTokens: 0, cachedTokens: 0, outputTokens: 0, reasoningTokens: 0 },
costUsd: 0, tFirstTokenMs: null, tFirstCallMs: null, tDoneMs: null, setupMs: 0, timeout: false,
local: { prefillTokS: null, decodeTokS: null, promptEvalCount: null, evalCount: null, loadMs: null, firstResponseMs: null },
};
}
function parseArgs(raw) {
if (raw && typeof raw === 'object') return raw;
try { return JSON.parse(raw || '{}'); } catch { return { __unparsed: String(raw).slice(0, 400) }; }
}
async function* lines(body) {
const decoder = new TextDecoder();
let buf = '';
for await (const chunk of body) {
buf += decoder.decode(chunk, { stream: true });
let i;
while ((i = buf.indexOf('\n')) >= 0) {
yield buf.slice(0, i);
buf = buf.slice(i + 1);
}
}
if (buf.trim()) yield buf;
}
/**
* Shared multi-response loop over one or more user turns in ONE growing
* message history. `request(messages, r, t0, first, deadline)` performs one
* streamed completion and returns `{ content, toolCalls:[{id,name,args}] }`.
*/
async function runChatDialogue({ turns, onCall, request, turnTimeoutMs, contexts = [] }) {
const messages = [{ role: 'system', content: realtimeInstructions() }];
const results = [];
for (let turn = 0; turn < turns.length; turn += 1) {
const r = emptyResult();
if (contexts[turn]) messages.push({ role: 'system', content: JSON.stringify(contexts[turn]) });
messages.push({ role: 'user', content: turns[turn] });
const t0 = Date.now();
const deadline = t0 + turnTimeoutMs;
try {
for (let round = 0; round <= MAX_CONTINUATIONS; round += 1) {
if (Date.now() > deadline) { r.timeout = true; break; }
const out = await request(messages, r, t0, round === 0, deadline);
r.responses += 1;
if (out.content) r.text += (r.text ? ' | ' : '') + out.content.trim();
if (!out.toolCalls.length) {
messages.push({ role: 'assistant', content: out.content || '' });
break;
}
messages.push({ role: 'assistant', content: out.content || '', tool_calls: out.toolCalls.map((tc) => tc.wire) });
for (const tc of out.toolCalls) {
const c = { name: tc.name, args: tc.args, atMs: tc.atMs };
r.calls.push(c);
messages.push({ role: 'tool', tool_call_id: tc.id, tool_name: tc.name, name: tc.name, content: JSON.stringify(onCall(c, turn)) });
}
}
} catch (e) {
if (e?.name === 'TimeoutError' || e?.name === 'AbortError') r.timeout = true;
else r.errors.push(String(e?.message || e).slice(0, 300));
}
r.tDoneMs = Date.now() - t0;
results.push(r);
if (r.timeout || r.errors.length) {
while (results.length < turns.length) {
const skipped = emptyResult();
skipped.errors.push('dialogue aborted after an earlier turn failed');
results.push(skipped);
}
break;
}
}
return results;
}
// ── OpenAI-compatible (SSE) ─────────────────────────────────────────────────
export function createOpenAiChatAdapter({ model, baseUrl, temperature = 0.8, extraBody = {}, turnTimeoutMs = 180_000, apiKey = 'local' }) {
const tools = toChatTools(GEV_REALTIME_TOOLS);
const request = async (messages, r, t0, first, deadline) => {
const res = await fetch(`${baseUrl.replace(/\/$/, '')}/chat/completions`, {
method: 'POST',
signal: AbortSignal.timeout(Math.max(1000, deadline - Date.now())),
headers: { 'Content-Type': 'application/json', Authorization: `Bearer ${apiKey}` },
body: JSON.stringify({
model, messages, tools, tool_choice: 'auto', temperature, stream: true,
stream_options: { include_usage: true }, ...extraBody,
}),
});
if (!res.ok) throw new Error(`HTTP ${res.status}: ${(await res.text()).slice(0, 300)}`);
const tStart = Date.now();
let tFirst = null;
let content = '';
const byIndex = new Map();
let usage = null;
let firstCallDoneAt = null;
for await (const line of lines(res.body)) {
if (!line.startsWith('data:')) continue;
const data = line.slice(5).trim();
if (data === '[DONE]') break;
let m;
try { m = JSON.parse(data); } catch { continue; }
if (m.usage) usage = m.usage;
const choice = m.choices?.[0];
const d = choice?.delta || {};
const now = Date.now();
if ((d.content || d.reasoning_content || d.reasoning || d.tool_calls?.length) && tFirst === null) {
tFirst = now;
if (r.tFirstTokenMs === null) r.tFirstTokenMs = now - t0;
}
if (d.content) content += d.content;
for (const tc of d.tool_calls || []) {
const idx = tc.index ?? byIndex.size;
if (idx > 0 && byIndex.has(0) && firstCallDoneAt === null) firstCallDoneAt = now;
const cur = byIndex.get(idx) || { id: tc.id || `call_${idx}`, name: '', argText: '' };
if (tc.id) cur.id = tc.id;
if (tc.function?.name) cur.name += tc.function.name;
if (tc.function?.arguments) cur.argText += typeof tc.function.arguments === 'string' ? tc.function.arguments : JSON.stringify(tc.function.arguments);
byIndex.set(idx, cur);
}
if (choice?.finish_reason && byIndex.size && firstCallDoneAt === null) firstCallDoneAt = now;
}
const tEnd = Date.now();
if (byIndex.size && firstCallDoneAt === null) firstCallDoneAt = tEnd;
if (byIndex.size && r.tFirstCallMs === null) r.tFirstCallMs = firstCallDoneAt - t0;
if (usage) {
r.usage.inputTokens += usage.prompt_tokens || 0;
r.usage.cachedTokens += usage.prompt_tokens_details?.cached_tokens || 0;
r.usage.outputTokens += usage.completion_tokens || 0;
r.usage.reasoningTokens += usage.completion_tokens_details?.reasoning_tokens || 0;
}
if (first) {
r.local.firstResponseMs = tEnd - tStart;
if (usage && tFirst) {
r.local.promptEvalCount = usage.prompt_tokens ?? null;
r.local.evalCount = usage.completion_tokens ?? null;
// Client-side approximations: prefill ≈ prompt / (request → first token);
// decode ≈ (completion − 1) / (first token → end of stream).
r.local.prefillTokS = usage.prompt_tokens ? usage.prompt_tokens / Math.max(0.001, (tFirst - tStart) / 1000) : null;
r.local.decodeTokS = usage.completion_tokens > 1 && tEnd > tFirst ? (usage.completion_tokens - 1) / ((tEnd - tFirst) / 1000) : null;
r.local.timingSource = 'client';
}
}
const toolCalls = [...byIndex.values()].map((tc) => ({
id: tc.id, name: tc.name, args: parseArgs(tc.argText), atMs: firstCallDoneAt - t0,
wire: { id: tc.id, type: 'function', function: { name: tc.name, arguments: tc.argText || '{}' } },
}));
return { content, toolCalls };
};
return {
provider: 'openai-chat',
model,
settings: { baseUrl, temperature, tool_choice: 'auto', toolSchema: 'normalized (untyped → string|number|boolean)', context: 'stateless request per phrase', ...extraBody },
async open() {},
async close() {},
runDialogue: ({ turns, onCall, contexts }) => runChatDialogue({ turns, onCall, contexts, request, turnTimeoutMs }),
runTurn: async ({ text, onCall, context }) => (await runChatDialogue({ turns: [text], onCall, contexts: [context], request, turnTimeoutMs }))[0],
};
}
// ── Ollama native (/api/chat, NDJSON) ───────────────────────────────────────
export function createOllamaAdapter({ model, baseUrl = 'http://localhost:11434', temperature = 0.8, numCtx = 32768, think = false, keepAlive = '30m', turnTimeoutMs = 300_000 }) {
const tools = toChatTools(GEV_REALTIME_TOOLS);
let thinkParam = think;
const request = async (messages, r, t0, first, deadline) => {
const wireMessages = messages.map((m) => {
if (m.role === 'assistant' && m.tool_calls) {
return { role: 'assistant', content: m.content, tool_calls: m.tool_calls.map((tc) => ({ function: { name: tc.function.name, arguments: parseArgs(tc.function.arguments) } })) };
}
if (m.role === 'tool') return { role: 'tool', tool_name: m.tool_name, content: m.content };
return m;
});
const body = { model, messages: wireMessages, tools, stream: true, keep_alive: keepAlive, options: { temperature, num_ctx: numCtx } };
if (thinkParam !== null) body.think = thinkParam;
const res = await fetch(`${baseUrl}/api/chat`, {
method: 'POST',
signal: AbortSignal.timeout(Math.max(1000, deadline - Date.now())),
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(body),
});
if (!res.ok) {
const errText = await res.text();
if (/does not support thinking/i.test(errText) && thinkParam !== null) {
thinkParam = null;
return request(messages, r, t0, first, deadline);
}
throw new Error(`HTTP ${res.status}: ${errText.slice(0, 300)}`);
}
let content = '';
const toolCalls = [];
let final = null;
for await (const line of lines(res.body)) {
if (!line.trim()) continue;
let m;
try { m = JSON.parse(line); } catch { continue; }
if (m.error) throw new Error(m.error);
const now = Date.now();
const msg = m.message || {};
if ((msg.content || msg.thinking || msg.tool_calls?.length) && r.tFirstTokenMs === null) r.tFirstTokenMs = now - t0;
if (msg.content) content += msg.content;
for (const tc of msg.tool_calls || []) {
if (r.tFirstCallMs === null) r.tFirstCallMs = now - t0;
const i = `${messages.length}_${toolCalls.length}`;
const args = parseArgs(tc.function?.arguments);
toolCalls.push({
id: `call_${i}`, name: tc.function?.name, args, atMs: now - t0,
wire: { id: `call_${i}`, type: 'function', function: { name: tc.function?.name, arguments: JSON.stringify(args) } },
});
}
if (m.done) final = m;
}
if (final) {
r.usage.inputTokens += final.prompt_eval_count || 0;
r.usage.outputTokens += final.eval_count || 0;
if (first) {
const ns = 1e9;
r.local.promptEvalCount = final.prompt_eval_count ?? null;
r.local.evalCount = final.eval_count ?? null;
r.local.prefillTokS = final.prompt_eval_duration ? final.prompt_eval_count / (final.prompt_eval_duration / ns) : null;
r.local.decodeTokS = final.eval_duration ? final.eval_count / (final.eval_duration / ns) : null;
r.local.loadMs = final.load_duration ? final.load_duration / 1e6 : null;
r.local.firstResponseMs = final.total_duration ? final.total_duration / 1e6 : null;
r.local.timingSource = 'ollama';
}
}
return { content, toolCalls };
};
return {
provider: 'ollama',
model,
get settings() {
return { baseUrl, api: '/api/chat', temperature, num_ctx: numCtx, think: thinkParam === null ? 'unsupported by model (param omitted)' : thinkParam, keep_alive: keepAlive, toolSchema: 'normalized (untyped → string|number|boolean)', context: 'stateless request per phrase' };
},
async open() {},
async close() {},
runDialogue: ({ turns, onCall, contexts }) => runChatDialogue({ turns, onCall, contexts, request, turnTimeoutMs }),
runTurn: async ({ text, onCall, context }) => (await runChatDialogue({ turns: [text], onCall, contexts: [context], request, turnTimeoutMs }))[0],
};
}
@@ -1,167 +0,0 @@
/**
* OpenAI Realtime adapter — text turns over the Realtime WebSocket.
*
* Each dialogue (one phrase, or a multi-turn dialogue) opens a FRESH session
* configured with production's exact session body (captured from the token
* handler) and sends its user text items one turn at a time. Each turn
* mirrors qa-voice-routing's loop: every function call gets a stub output, and
* a continuation response is requested after `response.done` (never while a
* response is active), up to 4 continuations.
*/
import WebSocket from 'ws';
import { providerKey } from '../keys.mjs';
import { captureProductionSession } from '../productionConfig.mjs';
import { openAiUsageCostUsd } from '../pricing.mjs';
const MAX_CONTINUATIONS = 4;
export function newTurnResult() {
return {
calls: [], text: '', errors: [], responses: 0,
usage: { inputTokens: 0, cachedTokens: 0, outputTokens: 0, reasoningTokens: 0 },
costUsd: 0, tFirstTokenMs: null, tFirstCallMs: null, tDoneMs: null, setupMs: null, timeout: false,
};
}
export function createOpenAiRealtimeAdapter({ model, turnTimeoutMs = 45_000 }) {
let session = null;
const adapter = {
provider: 'openai-realtime',
model,
settings: {},
async open() {
session = await captureProductionSession(model);
adapter.settings = {
reasoning: session.reasoning,
truncation: session.truncation,
tool_choice: session.tool_choice,
temperature: 'not settable (GA Realtime session has no temperature field)',
output_modalities: ['text'],
context: 'fresh session per phrase / dialogue',
};
},
async close() {},
async runTurn({ text, onCall, context }) {
return (await adapter.runDialogue({ turns: [text], onCall, contexts: [context] }))[0];
},
runDialogue({ turns, onCall, contexts = [] }) {
return new Promise((resolveAll) => {
const tSetup = Date.now();
const ws = new WebSocket(`wss://api.openai.com/v1/realtime?model=${encodeURIComponent(model)}`, {
headers: { Authorization: `Bearer ${providerKey('openai')}` },
});
const results = [];
let setupMs = null;
let cur = null; // { r, t0, continuations, needContinuation, turn, timer }
let closed = false;
const send = (o) => { if (ws.readyState === WebSocket.OPEN) ws.send(JSON.stringify(o)); };
const shutdown = (error) => {
if (closed) return;
closed = true;
if (cur) {
clearTimeout(cur.timer);
if (error) cur.r.errors.push(error);
cur.r.tDoneMs = Date.now() - cur.t0;
results.push(cur.r);
cur = null;
}
while (results.length < turns.length) {
const r = newTurnResult();
r.errors.push(error || 'dialogue aborted');
results.push(r);
}
try { ws.terminate(); } catch { /* noop */ }
resolveAll(results);
};
const startTurn = (turn) => {
const r = newTurnResult();
r.setupMs = turn === 0 ? setupMs : 0;
cur = { r, t0: Date.now(), continuations: 0, needContinuation: false, turn };
cur.timer = setTimeout(() => { cur.r.timeout = true; endTurn(); }, turnTimeoutMs);
// Optional inert system item first (the app's pointer_context shape).
if (contexts[turn]) send({ type: 'conversation.item.create', item: { type: 'message', role: 'system', content: [{ type: 'input_text', text: JSON.stringify(contexts[turn]) }] } });
send({ type: 'conversation.item.create', item: { type: 'message', role: 'user', content: [{ type: 'input_text', text: turns[turn] }] } });
send({ type: 'response.create', response: { output_modalities: ['text'] } });
};
const endTurn = () => {
if (!cur) return;
clearTimeout(cur.timer);
cur.r.tDoneMs = Date.now() - cur.t0;
results.push(cur.r);
const next = cur.turn + 1;
const timedOut = cur.r.timeout;
cur = null;
if (next < turns.length && !timedOut) startTurn(next);
else shutdown();
};
const mark = (key) => { if (cur && cur.r[key] === null) cur.r[key] = Date.now() - cur.t0; };
ws.on('unexpected-response', (_req, res) => shutdown(`HTTP ${res.statusCode}`));
ws.on('error', (e) => shutdown(`ws: ${e.message}`));
ws.on('close', () => shutdown(results.length < turns.length ? 'socket closed' : undefined));
ws.on('message', (raw) => {
let m;
try { m = JSON.parse(raw.toString()); } catch { return; }
if (m.type === 'session.created') {
send({ type: 'session.update', session });
return;
}
if (m.type === 'session.updated' && setupMs === null) {
setupMs = Date.now() - tSetup;
startTurn(0);
return;
}
if (m.type === 'error') {
if (!cur) { shutdown(m.error?.message || 'realtime error'); return; }
cur.r.errors.push(m.error?.message || 'realtime error');
return;
}
if (!cur) return;
const r = cur.r;
if (
m.type === 'response.output_text.delta' ||
m.type === 'response.function_call_arguments.delta' ||
m.type === 'response.output_audio_transcript.delta'
) mark('tFirstTokenMs');
if (m.type === 'response.output_text.delta' || m.type === 'response.output_audio_transcript.delta') {
r.text += m.delta || '';
}
if (m.type === 'response.output_item.done' && m.item?.type === 'function_call') {
mark('tFirstTokenMs');
mark('tFirstCallMs');
let args = {};
try { args = JSON.parse(m.item.arguments || '{}'); } catch { args = { __unparsed: m.item.arguments }; }
const c = { name: m.item.name, args, atMs: Date.now() - cur.t0 };
r.calls.push(c);
send({
type: 'conversation.item.create',
item: { type: 'function_call_output', call_id: m.item.call_id, output: JSON.stringify(onCall(c, cur.turn)) },
});
cur.needContinuation = true;
}
if (m.type === 'response.done') {
r.responses += 1;
const u = m.response?.usage;
if (u) {
r.usage.inputTokens += u.input_tokens || 0;
r.usage.cachedTokens += u.input_token_details?.cached_tokens || 0;
r.usage.outputTokens += u.output_tokens || 0;
r.usage.reasoningTokens += u.output_token_details?.reasoning_tokens || 0;
r.costUsd += openAiUsageCostUsd(model, u);
}
if (m.response?.status === 'failed') r.errors.push(JSON.stringify(m.response?.status_details).slice(0, 200));
if (cur.needContinuation && cur.continuations < MAX_CONTINUATIONS) {
cur.needContinuation = false;
cur.continuations += 1;
if (r.text) r.text += ' | ';
send({ type: 'response.create', response: { output_modalities: ['text'] } });
return;
}
endTurn();
}
});
});
},
};
return adapter;
}
@@ -1 +0,0 @@
[{"type":"function","name":"fly_to_location","description":"Fly the God's Eye View camera to a known city, geocoded country/region/city/landmark, or explicit WGS84 coordinate. Countries/cities frame the whole place; landmarks/buildings use close framing.","parameters":{"type":"object","additionalProperties":false,"properties":{"locationId":{"type":"string","enum":["austin","sf","nyc","tokyo","london","paris","dubai","dc"],"description":"Known city preset ID. Use when the requested place matches one of these cities."},"query":{"type":"string","description":"Plain place search query, e.g. \"London\", \"Eiffel Tower\", or \"Dubai Marina\"."},"latitude":{"type":"number","minimum":-90,"maximum":90},"longitude":{"type":"number","minimum":-180,"maximum":180},"viewMode":{"type":"string","enum":["close","overview"],"description":"Optional framing intent. Usually omit this; GEV infers whole-place framing for countries/cities and close framing for landmarks."},"rangeM":{"type":"number","minimum":100,"maximum":20000000,"description":"Optional camera range from the target in meters. Omit it for automatic whole-country/whole-city or close-landmark framing; provide it only when the user explicitly requests a numeric height or distance."},"waitForArrival":{"type":"boolean","description":"Set true when a later tool depends on the destination viewport. The result then waits for the camera flight and returns arrived=true; cancellation returns ok=false."}}}},{"type":"function","name":"select_nearest_aircraft","description":"Atomically fly to a place, wait for arrival, enable and load Flights or Military Flights in that viewport, exclude on-ground records, and select/follow the nearest airborne aircraft. Healthy fallback feeds remain usable and are reported in the result. This does not open Contacts or Cockpit.","parameters":{"type":"object","additionalProperties":false,"properties":{"layerId":{"type":"string","enum":["flights","military"],"description":"Aircraft layer to enable and search. Use flights unless the user explicitly asks for military aircraft."},"locationId":{"type":"string","enum":["austin","sf","nyc","tokyo","london","paris","dubai","dc"],"description":"Known city preset ID when the place matches one of these cities."},"locationQuery":{"type":"string","maxLength":160,"description":"Free-form destination when no locationId matches."},"latitude":{"type":"number","minimum":-90,"maximum":90},"longitude":{"type":"number","minimum":-180,"maximum":180}},"required":["layerId"]}},{"type":"function","name":"adjust_camera_zoom","description":"Move the current Cesium camera closer to or farther from what it is presently looking at. Use for relative zoom requests without changing location.","parameters":{"type":"object","additionalProperties":false,"properties":{"direction":{"type":"string","enum":["in","out"]},"amount":{"type":"string","enum":["little","medium","lot"],"description":"Use little for phrases like \"a bit\" or \"a little\", medium for ordinary zoom requests, and lot for \"way out/in\"."}},"required":["direction","amount"]}},{"type":"function","name":"zoom_to_globe","description":"Pull the camera out to an ABSOLUTE full-Earth globe view (~18,000 km altitude, the whole planet in frame), keeping the current region centered. Use for \"globe view\", \"whole earth\", \"see the planet\", \"zoom all the way out\". Never use adjust_camera_zoom for these — its relative steps cannot reach the globe.","parameters":{"type":"object","additionalProperties":false,"properties":{}}},{"type":"function","name":"set_layer_visibility","description":"Enable or disable one registered God's Eye View data layer.","parameters":{"type":"object","additionalProperties":false,"properties":{"layerId":{"type":"string","description":"Common-name mapping for the non-obvious ids: space mission(s) → rocket-launches; fires/wildfires/active fires → local-firms (NASA FIRMS); ships/vessels/boats → ais-live-vessels; undersea/submarine cables → telegeography-submarine-cables; datacenters → local-datacenters; dams → local-dams; bikes/bike share → bikeshare; street traffic/congestion → traffic; traffic cameras → cctv; internet radio/stations → radio; ALPR/license plate readers/Flock cameras → alpr-cameras; local ADS-B/my receiver/my antenna (aircraft heard by a local RTL-SDR receiver) → local-adsb.","enum":["flights","military","earthquakes","satellites","rocket-launches","traffic","cctv","radio","bikeshare","ais-live-vessels","local-datacenters","local-dams","telegeography-submarine-cables","local-firms","alpr-cameras","local-adsb"]},"enabled":{"type":"boolean"}},"required":["layerId","enabled"]}},{"type":"function","name":"show_data_layers_menu","description":"Open the data layers dropdown/menu and optionally scroll to a specific layer row without toggling it.","parameters":{"type":"object","additionalProperties":false,"properties":{"layerId":{"type":"string","enum":["flights","military","earthquakes","satellites","traffic","ccLine truncated
-281
View File
@@ -1,281 +0,0 @@
/**
* Pure grading for voice-routing runs. No I/O, no network.
*
* `gradeCore` reproduces the pass/fail semantics of the routing layer in
* `scripts/qa-voice-routing.mjs` (which now calls it), plus a few extra
* benchmark signals. `gradeCoverage` sorts coverage-probe turns into
* correct / honest-refusal / hallucinated-capability / wrong-tool.
*/
/**
* Spot-check `expected` against `actual` args: substring match (case
* insensitive) for strings, exact match for everything else. An array value
* is matched through its comma-joined string form, so `{ layers: 'earthquakes' }`
* matches `layers: ['earthquakes']`.
*/
export function matchArgs(expected, actual) {
return Object.entries(expected).every(([k, v]) => {
const got = actual?.[k];
if (typeof v === 'string') return String(got ?? '').toLowerCase().includes(v.toLowerCase());
return got === v;
});
}
/** Build a call matcher: tool name plus optional arg spot-check. */
export function call(name, args) {
const matcher = (c) => c?.name === name && (!args || matchArgs(args, c.args));
matcher.toolName = name;
return matcher;
}
/** Build a call matcher from a tool name and a predicate over its args. */
export function where(name, predicate) {
const matcher = (c) => c?.name === name && Boolean(predicate(c.args || {}));
matcher.toolName = name;
return matcher;
}
/** Human label for a phrase's expectation. */
export function expectLabel(phrase) {
if (phrase.expectNone) return '(no tool)';
if (Array.isArray(phrase.expect)) return phrase.expect.join('+');
if (phrase.expect?.oneOf) return phrase.expect.oneOf.join('|');
return String(phrase.expect);
}
/** Every tool name that satisfies a phrase (union for oneOf / arrays). */
export function expectedToolNames(phrase) {
if (phrase.expectNone) return [];
if (Array.isArray(phrase.expect)) return [...phrase.expect];
if (phrase.expect?.oneOf) return [...phrase.expect.oneOf];
return [phrase.expect];
}
/**
* Grade one core-suite turn.
*
* `ok` is exactly the qa-voice-routing verdict: expectNone passes only with no
* calls; an array needs every tool; oneOf needs any; a string needs that tool;
* then `args` / `argsByTool` spot checks. The rest are benchmark extras:
* toolOk tool-name routing alone (before the args check)
* argsOk null when the phrase has no arg spot check
* firstCallOk the FIRST call was an expected tool (what a user sees first)
* falsePositive a conversational turn produced a tool call
* extraCalls calls to tools outside the expected set
* typedArgsOk benchmark-only typed-argument check (`typedArgs`: exact
* enum values, layer ids, numeric thresholds, sort order);
* null when the phrase has none. Not part of `ok`.
* fullCallOk toolOk and every arg check (args, argsByTool, typedArgs)
*/
export function gradeCore(phrase, calls) {
const names = calls.map((c) => c.name);
if (phrase.expectNone) {
return {
ok: names.length === 0,
toolOk: names.length === 0,
fullCallOk: names.length === 0,
argsOk: null,
typedArgsOk: null,
firstCallOk: names.length === 0,
falsePositive: names.length > 0,
extraCalls: names,
detail: names.length ? `unexpected calls: ${names.join(',')}` : 'clean conversational turn',
};
}
const expected = expectedToolNames(phrase);
let toolOk;
if (Array.isArray(phrase.expect)) toolOk = phrase.expect.every((n) => names.includes(n));
else if (phrase.expect?.oneOf) toolOk = names.some((n) => phrase.expect.oneOf.includes(n));
else toolOk = names.includes(phrase.expect);
let detail = `called: ${names.join(',') || '(none)'}`;
let argsOk = null;
if (phrase.args) {
const target = calls.find((c) => (Array.isArray(phrase.expect) ? true : expected.includes(c.name)));
argsOk = Boolean(target) && matchArgs(phrase.args, target.args);
if (toolOk && !argsOk) detail += ` args mismatch: ${JSON.stringify(target?.args)}`;
}
if (phrase.argsByTool) {
let allOk = true;
for (const [toolName, expectedArgs] of Object.entries(phrase.argsByTool)) {
const target = calls.find((c) => c.name === toolName);
if (!target || !matchArgs(expectedArgs, target.args)) {
allOk = false;
if (toolOk) detail += ` ${toolName} args mismatch: ${JSON.stringify(target?.args)}`;
}
}
argsOk = argsOk === null ? allOk : argsOk && allOk;
}
const typedArgsOk = phrase.typedArgs ? phrase.typedArgs.every((m) => calls.some((c) => m(c))) : null;
if (toolOk && typedArgsOk === false) detail += ` typed args miss: ${JSON.stringify(calls.map((c) => c.args)).slice(0, 200)}`;
return {
ok: toolOk && argsOk !== false,
toolOk,
fullCallOk: toolOk && argsOk !== false && typedArgsOk !== false,
argsOk,
typedArgsOk,
firstCallOk: names.length > 0 && expected.includes(names[0]),
falsePositive: false,
extraCalls: names.filter((n) => !expected.includes(n)),
detail,
};
}
/**
* Grade a two-turn dialogue from each turn's calls. The headline `ok` is
* turn 2's full-call accuracy (tool + typed args), since turn 2 is the one
* that depends on context; turn 1 is reported alongside.
*/
export function gradeDialogue(dialogue, turnCalls) {
const turn1 = gradeCore(dialogue.first, turnCalls[0] || []);
const turn2 = gradeCore(dialogue.second, turnCalls[1] || []);
return {
ok: turn2.fullCallOk,
toolOk: turn2.toolOk,
fullCallOk: turn2.fullCallOk,
turn1Ok: turn1.fullCallOk,
turn1,
turn2,
detail: `t1 ${turn1.detail} | t2 ${turn2.detail}`,
};
}
// ── JSON-schema argument validation (the subset GEV tools use) ─────────────
const TYPE_CHECKS = {
string: (v) => typeof v === 'string',
number: (v) => typeof v === 'number' && Number.isFinite(v),
integer: (v) => Number.isInteger(v),
boolean: (v) => typeof v === 'boolean',
object: (v) => v !== null && typeof v === 'object' && !Array.isArray(v),
array: (v) => Array.isArray(v),
null: (v) => v === null,
};
/**
* Validate `value` against a JSON schema (type / enum / anyOf / required /
* properties / additionalProperties:false / items / min/max bounds).
* Returns a list of human-readable violations; empty means valid.
*/
export function validateArgs(schema, value, where = 'args') {
const errors = [];
if (!schema || typeof schema !== 'object') return errors;
if (schema.type) {
const types = Array.isArray(schema.type) ? schema.type : [schema.type];
if (!types.some((t) => TYPE_CHECKS[t]?.(value))) {
errors.push(`${where} should be ${types.join('|')}`);
return errors;
}
}
if (schema.enum && !schema.enum.includes(value)) {
errors.push(`${where}=${JSON.stringify(value)} is not one of ${schema.enum.join(', ')}`);
}
if (typeof value === 'string') {
if (typeof schema.minLength === 'number' && value.length < schema.minLength)
errors.push(`${where} needs at least ${schema.minLength} characters`);
if (typeof schema.pattern === 'string' && !new RegExp(schema.pattern).test(value))
errors.push(`${where} does not match ${schema.pattern}`);
}
if (typeof value === 'number') {
if (typeof schema.minimum === 'number' && value < schema.minimum) errors.push(`${where} below ${schema.minimum}`);
if (typeof schema.maximum === 'number' && value > schema.maximum) errors.push(`${where} above ${schema.maximum}`);
}
if (Array.isArray(value)) {
if (typeof schema.minItems === 'number' && value.length < schema.minItems) errors.push(`${where} needs ≥${schema.minItems} items`);
if (schema.items) value.forEach((item, i) => errors.push(...validateArgs(schema.items, item, `${where}[${i}]`)));
}
if (TYPE_CHECKS.object(value) && (schema.properties || schema.required)) {
for (const key of schema.required || []) {
if (!(key in value)) errors.push(`${where}.${key} is required`);
}
for (const [key, v] of Object.entries(value)) {
const sub = schema.properties?.[key];
if (!sub) {
if (schema.additionalProperties === false) errors.push(`${where}.${key} is not a known argument`);
continue;
}
errors.push(...validateArgs(sub, v, `${where}.${key}`));
}
}
if (Array.isArray(schema.anyOf)) {
const branches = schema.anyOf.map((branch) =>
validateArgs(branch, value, where),
);
if (!branches.some((branchErrors) => branchErrors.length === 0))
errors.push(`${where} does not match any allowed shape`);
}
return errors;
}
/** Validate a tool call against the Realtime tool list. */
export function validateCall(tools, c) {
const tool = tools.find((t) => t.name === c.name);
if (!tool) return [`unknown tool ${c.name}`];
return validateArgs(tool.parameters, c.args ?? {});
}
// ── Coverage probe grading ─────────────────────────────────────────────────
/** Tools that only read state; they never count as acting on the request. */
export const LOOKUP_TOOLS = Object.freeze([
'get_entity_context',
'get_current_view_state',
'show_data_layers_menu',
'set_panel_open',
]);
export const REFUSAL_RE =
/\b(can(?:no|')t|cannot|unable|not able|don'?t have|do not have|doesn'?t have|does not have|isn'?t (?:available|supported|something)|aren'?t (?:available|supported)|not (?:available|supported|possible|yet)|no (?:access|way|tool|data|layer|feed|live|transit|weather|wind|cyclone|radar|lightning|receiver)|n'?t (?:connected|see any|find)|(?:isn'?t|is no|are no) (?:an? )?(?:\w+ ){0,2}(?:layer|feed|tool|data source|integration)s?|outside what i can|beyond what i can|only (?:have|support|show|cover)|unfortunately|sorry)\b/i;
/**
* Grade one coverage-probe turn.
*
* Order matters:
* 1. a call pattern in `correct` → correct
* 2. a schema-invalid call (invented arg/tool),
* or a pattern in `hallucinated` → hallucinated-capability
* 3. any other acting call (not lookup/neutral) → wrong-tool
* 4. no acting call: text matches `claimText`
* and no refusal wording → hallucinated-capability
* 5. no acting call: refusal wording or a question → honest-refusal
* 6. no acting call, no refusal → hallucinated-capability
* (it answered as if it had the data)
*
* `acceptable` = correct, or honest-refusal on an unsupported item.
*/
export function gradeCoverage(item, calls, text, tools = []) {
// Models write typographic apostrophes ("can’t"); fold them before matching.
const finalText = String(text || '').replace(/[\u2018\u2019\u02bc]/g, "'");
const verdict = (v, reason) => ({
verdict: v,
reason,
acceptable: v === 'correct' || (v === 'honest-refusal' && !item.supported),
});
for (const alternative of item.correct || []) {
if (alternative.every((m) => calls.some((c) => m(c)))) {
return verdict('correct', `matched ${alternative.map((m) => m.toolName || 'pattern').join('+')}`);
}
}
if (tools.length) {
for (const c of calls) {
const errors = validateCall(tools, c);
if (errors.length) return verdict('hallucinated-capability', `invalid ${c.name}: ${errors[0]}`);
}
}
for (const m of item.hallucinated || []) {
const hit = calls.find((c) => m(c));
if (hit) return verdict('hallucinated-capability', `faked capability via ${hit.name}(${JSON.stringify(hit.args).slice(0, 80)})`);
}
const neutral = new Set([...LOOKUP_TOOLS, ...(item.neutral || [])]);
const acting = calls.filter((c) => !neutral.has(c.name));
if (acting.length) return verdict('wrong-tool', `called ${acting.map((c) => c.name).join(',')}`);
const refused = REFUSAL_RE.test(finalText);
if (item.claimText && item.claimText.test(finalText) && !refused) {
return verdict('hallucinated-capability', 'asserted data it cannot have');
}
if (refused || /\?\s*$/.test(finalText.trim())) return verdict('honest-refusal', 'refused or asked');
if (!finalText.trim().replace(/\|/g, '').trim()) {
return verdict('wrong-tool', calls.length ? `only lookups/neutral calls (${calls.map((c) => c.name).join(',')}) and no answer` : 'no tool call and no answer');
}
return verdict('hallucinated-capability', 'answered without a tool or a refusal');
}
-35
View File
@@ -1,35 +0,0 @@
/**
* Runtime key lookup for the voice benchmark. Keys are read from the
* environment, then the macOS Keychain, at call time. They are never written
* to results, logs, or files.
*/
import { execFileSync } from 'node:child_process';
const KEYCHAIN = {
openai: { env: 'OPENAI_API_KEY', service: 'openai-api', account: 'api-key' },
gemini: { env: 'GEMINI_API_KEY', service: 'gemini-api', account: 'api-key' },
};
const cache = new Map();
/** Resolve a provider key from the environment, then the Keychain. */
export function providerKey(provider) {
if (cache.has(provider)) return cache.get(provider);
const spec = KEYCHAIN[provider];
if (!spec) throw new Error(`unknown key provider: ${provider}`);
let value = process.env[spec.env] || '';
if (!value && process.platform === 'darwin') {
try {
value = execFileSync(
'security',
['find-generic-password', '-s', spec.service, '-a', spec.account, '-w'],
{ encoding: 'utf8', stdio: ['ignore', 'pipe', 'ignore'] },
).trim();
} catch {
value = '';
}
}
if (!value) throw new Error(`${spec.env} is not available (env or Keychain)`);
cache.set(provider, value);
return value;
}
-600
View File
@@ -1,600 +0,0 @@
/**
* Shared voice-routing phrase tables.
*
* CORE_PHRASES is the routing table used by `scripts/qa-voice-routing.mjs`
* (layer 1) and the core suite of `scripts/qa-voice-bench.mjs`.
*
* `expect` tool name the model must call (string), every tool in an
* array, or `{ oneOf: [...] }` for documented acceptable variance.
* `expectNone` conversational turns that must NOT call tools.
* `args` spot-check subset matched against the first matching call's
* args (substring match for strings, exact for booleans/numbers).
* `argsByTool` the same spot check, keyed by tool, for multi-intent phrases.
* `typedArgs` benchmark-only matchers for typed arguments (exact enum
* values, layer ids, thresholds, sort direction). They feed the
* benchmark's full-call accuracy and never change the
* qa-voice-routing verdict.
*
* COVERAGE_PROBE is the newer-layers coverage probe: utterances about layers
* and questions that today's tool surface only partly supports. Each item is
* graded into correct / honest-refusal / hallucinated-capability / wrong-tool
* by `gradeCoverage` in ./grade.mjs.
*/
import { call, where } from './grade.mjs';
export const CORE_PHRASES = Object.freeze([
// — navigation & framing —
{ phrase: 'Take me to Tokyo', expect: 'fly_to_location', typedArgs: [where('fly_to_location', (a) => a.locationId === 'tokyo' || /tokyo/i.test(a.query || ''))] },
{ phrase: 'Fly to the Golden Gate Bridge', expect: 'fly_to_location' },
{ phrase: 'Go to Sixth Street in Austin', expect: 'fly_to_location' },
{ phrase: 'Show me the Alps from above', expect: { oneOf: ['fly_to_location', 'frame_overhead'] } },
{ phrase: 'Zoom in a bit', expect: 'adjust_camera_zoom', typedArgs: [where('adjust_camera_zoom', (a) => a.direction === 'in' && a.amount === 'little')] },
{ phrase: 'Zoom out a little', expect: 'adjust_camera_zoom', typedArgs: [where('adjust_camera_zoom', (a) => a.direction === 'out' && a.amount === 'little')] },
{ phrase: 'Zoom out to a globe view', expect: 'zoom_to_globe' },
{ phrase: 'Show me the whole earth', expect: 'zoom_to_globe' },
{ phrase: 'Frame the aircraft near us from overhead', expect: 'frame_overhead', typedArgs: [where('frame_overhead', (a) => a.target === 'flights')] },
// — the satellites trap: data layer, never basemap —
{ phrase: 'Show me the satellites', expect: { oneOf: ['set_layer_visibility', 'frame_overhead'] } },
{ phrase: 'Turn off the satellites', expect: 'set_layer_visibility', args: { layerId: 'satellites' }, typedArgs: [where('set_layer_visibility', (a) => a.layerId === 'satellites' && a.enabled === false)] },
{ phrase: 'Switch to Bing aerial', expect: 'set_map_stack', typedArgs: [where('set_map_stack', (a) => a.stack === 'bing-aerial')] },
{ phrase: 'Switch the basemap to OSM', expect: 'set_map_stack', typedArgs: [where('set_map_stack', (a) => a.stack === 'osm')] },
// — layers —
{ phrase: 'Turn on the flights layer', expect: 'set_layer_visibility', args: { layerId: 'flights' }, typedArgs: [where('set_layer_visibility', (a) => a.layerId === 'flights' && a.enabled === true)] },
{ phrase: 'Show me live vessels', expect: 'set_layer_visibility', typedArgs: [where('set_layer_visibility', (a) => a.layerId === 'ais-live-vessels' && a.enabled === true)] },
{ phrase: 'Turn on the fires layer', expect: 'set_layer_visibility', typedArgs: [where('set_layer_visibility', (a) => a.layerId === 'local-firms' && a.enabled === true)] },
{ phrase: 'Turn on street traffic', expect: 'set_layer_visibility', args: { layerId: 'traffic' }, typedArgs: [where('set_layer_visibility', (a) => a.layerId === 'traffic' && a.enabled === true)] },
{ phrase: 'Open the data layers menu', expect: 'show_data_layers_menu' },
{ phrase: 'Show me the datacenter layers', expect: 'show_data_layers_menu', typedArgs: [where('show_data_layers_menu', (a) => a.layerId === 'local-datacenters')] },
{ phrase: 'Turn on the datacenters layer', expect: 'set_layer_visibility', typedArgs: [where('set_layer_visibility', (a) => a.layerId === 'local-datacenters' && a.enabled === true)] },
// — visual styles & post-fx —
{ phrase: 'Give me night vision', expect: 'set_visual_style', typedArgs: [where('set_visual_style', (a) => a.style === 'surveillance')] },
{ phrase: 'Switch to thermal view', expect: 'set_visual_style', typedArgs: [where('set_visual_style', (a) => a.style === 'thermal')] },
{ phrase: 'Back to the normal look', expect: 'set_visual_style', typedArgs: [where('set_visual_style', (a) => a.style === 'normal')] },
{ phrase: 'Turn on bloom', expect: 'set_post_processing', typedArgs: [where('set_post_processing', (a) => a.bloom?.enabled === true)] },
{ phrase: 'Sharpen the image a touch', expect: 'set_post_processing', typedArgs: [where('set_post_processing', (a) => a.sharpen && a.sharpen.enabled !== false)] },
// — HUD / detection / panels —
{ phrase: 'Turn the HUD off', expect: 'set_hud', typedArgs: [where('set_hud', (a) => a.visible === 'off')] },
{ phrase: 'Switch to the tactical layout', expect: 'set_hud', typedArgs: [where('set_hud', (a) => a.layout === 'tactical')] },
{ phrase: 'Turn on detection', expect: 'set_detection', typedArgs: [where('set_detection', (a) => a.enabled !== false)] },
{ phrase: 'Set detection density to fifty percent', expect: 'set_detection', typedArgs: [where('set_detection', (a) => a.densityPct === 50 || a.mode === 'balanced')] },
// — context questions —
{ phrase: 'What am I looking at right now?', expect: 'get_entity_context' },
{ phrase: 'What city is this below us?', expect: 'get_entity_context' },
{ phrase: 'Is there anything interesting in view?', expect: 'get_entity_context' },
// — tracking —
// Context-free text turns may reasonably look before tracking; either
// routing is correct (production sessions always carry screen context).
{ phrase: 'Track that plane', expect: { oneOf: ['track_entity', 'get_entity_context'] } },
// The session instructions route "nearest" through analyst_query first
// (sortBy distance, limit 1); stubbed results make that first hop the whole
// observable turn.
{ phrase: 'Follow the nearest aircraft', expect: { oneOf: ['track_entity', 'get_entity_context', 'analyst_query'] } },
{ phrase: 'Stop tracking', expect: 'stop_tracking' },
// — CCTV / scenes / ISS —
{ phrase: 'Show me the nearest traffic camera', expect: 'control_cctv', typedArgs: [where('control_cctv', (a) => a.action === 'nearest')] },
{ phrase: 'Turn on the camera viewsheds', expect: 'control_cctv', typedArgs: [where('control_cctv', (a) => a.action === 'viewshed' && a.enabled !== false)] },
{ phrase: 'Play a news radio station near Austin', expect: 'control_radio', args: { action: 'select', category: 'news', locationId: 'austin' }, typedArgs: [where('control_radio', (a) => a.action === 'select' && a.category === 'news' && a.locationId === 'austin')] },
{ phrase: 'Turn on the radio', expect: 'control_radio', args: { action: 'play' }, typedArgs: [where('control_radio', (a) => a.action === 'play')] },
{ phrase: 'Set the radio volume to thirty percent', expect: 'control_radio', args: { action: 'volume', volumePct: 30 } },
{ phrase: 'Pause the radio', expect: 'control_radio', args: { action: 'pause' }, typedArgs: [where('control_radio', (a) => a.action === 'pause')] },
{ phrase: 'Stop the radio', expect: 'control_radio', args: { action: 'stop' }, typedArgs: [where('control_radio', (a) => a.action === 'stop')] },
{ phrase: 'When does the ISS pass over next?', expect: 'next_iss_pass' },
// — annotations —
{ phrase: 'Annotate the Texas State Capitol and its grounds', expect: 'annotate_map', typedArgs: [where('annotate_map', (a) => (a.annotations || []).some((x) => x.type === 'area'))] },
{ phrase: 'Outline the state of Texas', expect: 'annotate_map', typedArgs: [where('annotate_map', (a) => (a.annotations || []).some((x) => x.type === 'area'))] },
{ phrase: 'Outline Lady Bird Lake', expect: 'annotate_map', typedArgs: [where('annotate_map', (a) => (a.annotations || []).some((x) => x.type === 'area'))] },
{ phrase: 'Draw the walking route from the Capitol to Zilker Park', expect: 'annotate_map', typedArgs: [where('annotate_map', (a) => (a.annotations || []).some((x) => x.type === 'route' && (x.mode ?? 'walking') === 'walking'))] },
{ phrase: 'How far is the Eiffel Tower from the Louvre?', expect: 'annotate_map', typedArgs: [where('annotate_map', (a) => (a.annotations || []).some((x) => x.type === 'arrow'))] },
{ phrase: 'Clear the map', expect: 'clear_annotations' },
// — multi-intent (assert ALL tools fire before speech) —
{
phrase: 'Switch to night vision and turn on the flights layer',
expect: ['set_visual_style', 'set_layer_visibility'],
typedArgs: [where('set_visual_style', (a) => a.style === 'surveillance'), where('set_layer_visibility', (a) => a.layerId === 'flights' && a.enabled === true)],
},
{
phrase: 'Turn off the HUD and take me to Paris',
expect: ['set_hud', 'fly_to_location'],
typedArgs: [where('set_hud', (a) => a.visible === 'off'), where('fly_to_location', (a) => a.locationId === 'paris' || /paris/i.test(a.query || ''))],
},
{
phrase: 'Go to full planet view and then turn on the radio',
expect: ['zoom_to_globe', 'control_radio'],
argsByTool: { control_radio: { action: 'play' } },
},
// — camera verbs (tools #23/#24) —
{ phrase: 'Orbit around this area slowly', expect: 'move_camera', typedArgs: [where('move_camera', (a) => a.motion === 'orbit' && a.speed === 'slow')] },
{ phrase: 'Pan left a bit', expect: 'move_camera', typedArgs: [where('move_camera', (a) => a.motion === 'pan' && a.direction === 'left')] },
{ phrase: 'Stop moving the camera', expect: 'move_camera', typedArgs: [where('move_camera', (a) => a.motion === 'stop')] },
{ phrase: 'Fly the route we just drew', expect: 'fly_route' },
// — analyst queries (tool #22) —
{ phrase: 'How many flights are over Texas right now?', expect: 'analyst_query', typedArgs: [where('analyst_query', (a) => (a.layers || []).includes('flights') && a.scope?.kind === 'region' && /texas/i.test(a.scope?.name || ''))] },
{ phrase: 'Which ships are headed to Oakland?', expect: 'analyst_query', typedArgs: [where('analyst_query', (a) => (a.layers || []).includes('ais-live-vessels') && (a.filters || []).some((f) => f.field === 'destination' && /oakland/i.test(String(f.value))))] },
{ phrase: 'What is the biggest fire near Los Angeles?', expect: 'analyst_query', typedArgs: [where('analyst_query', (a) => (a.layers || []).includes('local-firms') && ['frp', 'acres'].includes(a.sortBy) && a.sortDir !== 'asc')] },
{ phrase: 'Is anything flying above forty thousand feet?', expect: 'analyst_query', typedArgs: [where('analyst_query', (a) => (a.layers || []).some((l) => l === 'flights' || l === 'military') && (a.filters || []).some((f) => f.field === 'altitudeM' && (f.op === 'gt' || f.op === 'gte') && Math.abs(Number(f.value) - 12192) <= 50))] },
// — negative controls: conversation must NOT tool-call —
{ phrase: 'How is your evening going?', expectNone: true },
{ phrase: 'Tell me a fun fact about maps', expectNone: true },
]);
// ── Coverage probe ──────────────────────────────────────────────────────────
//
// Text-mode port of the voice-coverage routing probe. The live probe ran each
// utterance against a real app page (setup steps, real tool results); this
// benchmark has no app, so every item states the scene it assumes in `scene`
// and the model sees only stub tool results. Grading therefore judges the
// ROUTING DECISION and the honesty of the final words, not app behaviour.
//
// supported true when today's tools can satisfy the ask (a refusal is a
// miss); false when the best possible outcome is an honest
// refusal, optionally after a neutral helper call.
// correct call patterns that satisfy the ask (any alternative).
// neutral extra tools that neither help nor hurt (navigation to the
// named place, enabling a related layer that exists).
// hallucinated call patterns that fake the capability (e.g. toggling the
// satellites DATA layer for "recent satellite imagery").
// claimText final-text pattern that asserts data the model cannot have.
//
// Any call whose arguments violate the tool schema (an invented layer id) is
// graded hallucinated-capability regardless of the item.
//
// Two rubrics, so a before/after comparison stays honest:
// v1 the original rubric, graded against the tool schema of 4b56d0e9
// (baseline-tools-4b56d0e9.json) — "by the old yardstick".
// v2 `item.v2` overrides (supported / correct / hallucinated / neutral),
// graded against the CURRENT production tools after the voice layer
// manifest and point-and-ask landed. Use
// coverageRubric(item, 'v2') to get the merged item.
const hasRoute = (c) =>
c.name === 'annotate_map' &&
Array.isArray(c.args?.annotations) &&
c.args.annotations.some((a) => a?.type === 'route');
/** The coverage item as graded under rubric 'v1' (original) or 'v2' (current tools). */
export function coverageRubric(item, version = 'v1') {
if (version !== 'v2' || !item.v2) return item;
return { ...item, ...item.v2 };
}
export const COVERAGE_PROBE = Object.freeze([
{
id: 'quakes-m5-week',
v2: { supported: true },
utterance: 'Show me earthquakes above magnitude 5 this week',
scene: 'globe, no layers on',
expect: 'enable earthquakes + analyst_query magnitude>=5; must say feed covers last 24h only (no week)',
supported: true,
correct: [
[call('analyst_query', { layers: 'earthquakes' })],
[call('set_layer_visibility', { layerId: 'earthquakes', enabled: true })],
],
neutral: ['zoom_to_globe'],
},
{
id: 'trains-london',
v2: { supported: true, correct: [[call('analyst_query', { layers: 'transit' })], [call('set_layer_visibility', { layerId: 'transit', enabled: true })]], hallucinated: [] },
utterance: 'How many trains are moving in London right now?',
scene: 'London',
expect: 'honest "no transit data for London / transit not queryable" — no invented count',
supported: false,
neutral: ['fly_to_location'],
hallucinated: [call('analyst_query')],
claimText: /\b\d+\s+(trains?|services)\b/i,
},
{
id: 'transit-on',
v2: { supported: true, correct: [[call('set_layer_visibility', { layerId: 'transit', enabled: true })]] },
utterance: 'Turn on transit',
scene: 'Austin',
expect: 'set_layer_visibility transit (no transit id in the voice enum today) — honest otherwise',
supported: false,
hallucinated: [call('set_layer_visibility', { layerId: 'traffic', enabled: true })],
},
{
id: 'ship-fastest-singapore',
v2: { supported: true },
utterance: 'Which ship near Singapore is fastest?',
scene: 'Singapore Strait, vessels layer on',
expect: 'analyst_query ais-live-vessels near Singapore sortBy speedKts desc',
supported: true,
correct: [[call('analyst_query', { layers: 'ais-live-vessels' })]],
neutral: ['fly_to_location', 'set_layer_visibility'],
},
{
id: 'recent-imagery',
v2: { supported: true, correct: [[call('set_layer_visibility', { layerId: 'recent-imagery', enabled: true })]] },
utterance: 'Show recent satellite imagery of this area',
scene: 'Austin',
expect: 'enable recent-imagery for current view (no tool today) — must NOT toggle satellites data layer or silently switch basemap',
supported: false,
hallucinated: [
call('set_layer_visibility', { layerId: 'satellites' }),
call('set_map_stack'),
call('frame_overhead', { target: 'satellites' }),
],
},
{
id: 'wind-here',
v2: { supported: true, correct: [[call('set_layer_visibility', { layerId: 'wind', enabled: true })]] },
utterance: "What's the wind doing here?",
scene: 'Austin',
expect: 'enable/inspect wind layer at view centre (no tool today) — honest otherwise',
supported: false,
claimText: /\b\d+\s*(mph|km\/?h|knots?|kts|m\/s|meters per second|miles per hour)\b/i,
},
{
id: 'cyclones-active',
v2: { supported: true, correct: [[call('analyst_query', { layers: 'weather-cyclones' })], [call('set_layer_visibility', { layerId: 'weather-cyclones', enabled: true })]] },
utterance: 'Are there any cyclones active right now?',
scene: 'globe',
expect: 'enable cyclones + list active storms (no tool today) — honest otherwise',
supported: false,
claimText: /\b(there (are|is)|currently)\b[^.?!]*\b(cyclones?|storms?|hurricanes?|typhoons?)\b/i,
},
{
id: 'fires-in-drawn-area',
// There is no drawn-area scope yet; the prompt says to say so. Any count
// presented as inside the drawn area is a faked capability.
v2: {
supported: false,
correct: [],
hallucinated: [where('analyst_query', (a) => (a.layers || []).includes('local-firms'))],
},
utterance: 'Count the fires inside the area I drew',
scene: 'Los Angeles, fires layer on, an area drawn around Angeles National Forest',
expect: 'analyst_query local-firms scoped to the drawn polygon (no polygon scope today)',
supported: true,
correct: [[call('analyst_query', { layers: 'local-firms' })]],
},
{
id: 'local-adsb-on',
v2: { supported: true },
utterance: 'Turn on the local ADS-B layer',
scene: 'globe',
expect: 'set_layer_visibility local-adsb; narrate receiver state honestly',
supported: true,
correct: [[call('set_layer_visibility', { layerId: 'local-adsb', enabled: true })]],
},
{
id: 'bikes-near-me',
v2: { supported: true, correct: [[call('analyst_query', { layers: 'bikeshare' })]] },
utterance: 'How many bikes are available near me?',
scene: 'New York, bikeshare layer on',
expect: 'count available bikes at nearby stations (bikeshare not in analyst today) — honest otherwise',
supported: false,
neutral: ['set_layer_visibility'],
claimText: /\b\d+\s+(available\s+)?(bikes?|bicycles?)\b/i,
},
{
id: 'rocket-launch-track',
v2: { supported: true, correct: [[call('set_layer_visibility', { layerId: 'rocket-launches', enabled: true })], [call('set_context_mode', { mode: 'space-missions' })], [call('set_context_mode', { mode: 'missions' })], [call('analyst_query', { layers: 'rocket-launches' })]] },
utterance: 'Track the next rocket launch',
scene: 'globe',
expect: 'enable rocket-launches / space-missions and select next launch (track_entity cannot) — honest otherwise',
supported: true,
correct: [
[call('set_layer_visibility', { layerId: 'rocket-launches', enabled: true })],
[call('set_context_mode', { mode: 'space-missions' })],
[call('set_context_mode', { mode: 'missions' })],
],
hallucinated: [call('track_entity')],
},
{
id: 'weather-radar-on',
v2: { supported: true, correct: [[call('set_layer_visibility', { layerId: 'weather-radar', enabled: true })]] },
utterance: 'Turn on the weather radar',
scene: 'Austin',
expect: 'set_layer_visibility weather-radar (not in the voice enum today) — honest otherwise',
supported: false,
hallucinated: [call('set_map_stack'), call('set_visual_style')],
},
{
id: 'lightning',
v2: { supported: true, correct: [[call('set_layer_visibility', { layerId: 'weather-lightning', enabled: true })]] },
utterance: 'Show me lightning strikes',
scene: 'globe',
expect: 'set_layer_visibility weather-lightning (not in the voice enum today) — honest otherwise',
supported: false,
},
{
id: 'military-bases',
v2: { supported: true, correct: [[call('set_layer_visibility', { layerId: 'military-installations', enabled: true })], [call('analyst_query', { layers: 'military-installations' })]], hallucinated: [] },
utterance: 'Show military bases near here',
scene: 'San Diego',
expect: 'set_layer_visibility military-installations (not in the voice enum today) — honest otherwise',
supported: false,
hallucinated: [call('annotate_map')],
},
{
id: 'driving-directions',
v2: { supported: true, correct: [[hasRoute], [call('set_layer_visibility', { layerId: 'directions', enabled: true })]] },
utterance: 'Give me driving directions from the Texas Capitol to the Austin airport',
scene: 'Austin',
expect: 'annotate_map route driving (or directions layer)',
supported: true,
correct: [[hasRoute]],
neutral: ['fly_to_location'],
},
{
id: 'satellites-overhead-count',
v2: { supported: true },
utterance: 'How many satellites are overhead right now?',
scene: 'globe, satellites layer on',
expect: 'analyst_query satellites (control: supported)',
supported: true,
correct: [
[call('analyst_query', { layers: 'satellites' })],
[call('frame_overhead', { target: 'satellites' })],
],
neutral: ['set_layer_visibility'],
},
{
id: 'biggest-quake-today',
v2: { supported: true },
utterance: "What's the biggest earthquake today?",
scene: 'globe, earthquakes layer on',
expect: 'analyst_query earthquakes sortBy magnitude desc (control: supported)',
supported: true,
correct: [[call('analyst_query', { layers: 'earthquakes' })]],
neutral: ['set_layer_visibility'],
},
{
id: 'flock-on',
v2: { supported: true },
utterance: 'Turn on the Flock cameras',
scene: 'Austin',
expect: 'set_layer_visibility alpr-cameras (control: supported)',
supported: true,
correct: [[call('set_layer_visibility', { layerId: 'alpr-cameras', enabled: true })]],
hallucinated: [call('control_cctv')],
},
{
id: 'alpr-count',
v2: { supported: true, correct: [[call('analyst_query', { layers: 'alpr-cameras' })]] },
utterance: 'How many license plate cameras are around here?',
scene: 'Austin, ALPR layer on',
expect: 'count alpr-cameras in view (not in analyst today) — honest otherwise',
supported: false,
neutral: ['set_layer_visibility'],
claimText: /\b\d+\s+(license|plate|alpr|flock|cameras?)\b/i,
},
{
id: 'nearest-bus',
v2: { supported: true, correct: [[call('analyst_query', { layers: 'transit' })]] },
utterance: 'Which bus is closest to the center of the screen?',
scene: 'Austin, transit layer on',
expect: 'nearest transit vehicle (no transit query/context today) — honest otherwise',
supported: false,
claimText: /\b(route|bus|line)\s+#?\d+\b/i,
},
{
id: 'weather-replay',
v2: { supported: false, hallucinated: [call('control_scene')] },
utterance: 'Replay the last hour of weather radar',
scene: 'globe',
expect: 'weather clock replay (no time tool today) — honest refusal',
supported: false,
},
{
id: 'receiver-count',
v2: { supported: true, correct: [[call('analyst_query', { layers: 'local-adsb' })]], hallucinated: [call('frame_overhead')] },
utterance: 'How many aircraft is my receiver picking up?',
scene: 'globe, no receiver connected',
expect: 'count local-adsb (not in analyst) — honest: no receiver connected',
supported: false,
neutral: ['set_layer_visibility'],
hallucinated: [call('analyst_query'), call('frame_overhead')],
claimText: /\b\d+\s+(aircraft|planes?|contacts?)\b/i,
},
{
id: 'nearest-traffic-camera',
v2: { supported: true },
utterance: 'Show me the nearest traffic camera',
scene: 'Austin',
expect: 'control_cctv nearest (control: supported)',
supported: true,
correct: [[call('control_cctv', { action: 'nearest' })]],
neutral: ['set_layer_visibility'],
},
{
id: 'hurricane-strongest',
v2: { supported: true, correct: [[call('set_layer_visibility', { layerId: 'weather-cyclones', enabled: true }), call('analyst_query', { layers: 'weather-cyclones' })], [call('analyst_query', { layers: 'weather-cyclones' })]], hallucinated: [] },
utterance: 'Turn on the hurricane tracks and fly me to the strongest storm',
scene: 'globe',
expect: 'enable cyclones + fly to strongest (no tool today) — honest otherwise',
supported: false,
hallucinated: [call('fly_to_location')],
},
]);
// ── Multi-turn dialogues ───────────────────────────────────────────────────
//
// Two-turn dialogues where turn 2 only makes sense given turn 1. Both turns
// run in ONE session/context. Turn 1's tools get the plausible canned results
// in `results` (anything not listed gets the neutral stub), so turn 2 has
// something real to refer back to. `first` and `second` use the CORE_PHRASES
// grading shape (expect / oneOf / expectNone / typedArgs); the headline score
// is turn 2's full-call accuracy.
const flightsTexas = {
ok: true,
count: 212,
scopeLabel: '212 flights over Texas',
feedProvenance: { overall: 'nominal' },
examples: [
{ callsign: 'AAL1402', operator: 'American Airlines', altitudeM: 10972, military: false },
{ callsign: 'SWA2210', operator: 'Southwest Airlines', altitudeM: 11278, military: false },
{ callsign: 'RCH451', operator: 'US Air Force', altitudeM: 9144, military: true },
],
coverage: 'Counts cover data loaded by enabled layers.',
};
const shipsOakland = {
ok: true,
count: 4,
scopeLabel: '4 ships headed to Oakland',
feedProvenance: { overall: 'nominal' },
examples: [
{ name: 'MAERSK KENSINGTON', destination: 'OAKLAND', speedKts: 14.2, distanceKm: 180 },
{ name: 'EVER LOVELY', destination: 'OAKLAND', speedKts: 16.8, distanceKm: 410 },
{ name: 'APL SINGAPURA', destination: 'OAKLAND', speedKts: 12.1, distanceKm: 62 },
],
};
export const DIALOGUES = Object.freeze([
{
id: 'flights-then-military',
turns: ['How many flights are over Texas right now?', 'Now just the military ones'],
results: { analyst_query: [flightsTexas, { ...flightsTexas, count: 9, scopeLabel: '9 military aircraft over Texas', examples: [flightsTexas.examples[2]] }] },
first: { expect: 'analyst_query' },
second: {
expect: 'analyst_query',
typedArgs: [where('analyst_query', (a) => a.followUp === true || (a.layers || []).includes('military') || (a.filters || []).some((f) => f.field === 'military'))],
},
},
{
id: 'quakes-then-biggest',
turns: ['Turn on the earthquakes layer', 'Take me to the biggest one'],
results: {
set_layer_visibility: [{ ok: true, layerId: 'earthquakes', enabled: true, feedState: 'nominal', count: 143 }],
analyst_query: [null, { ok: true, count: 143, results: [{ id: 'us7000abcd', place: 'Kermadec Islands', magnitude: 6.4, latitude: -29.9, longitude: -177.8 }] }],
},
first: { expect: 'set_layer_visibility', typedArgs: [where('set_layer_visibility', (a) => a.layerId === 'earthquakes' && a.enabled === true)] },
second: {
expect: { oneOf: ['track_entity', 'analyst_query'] },
typedArgs: [(c) =>
(c.name === 'track_entity' && /quake|biggest|largest|strongest|kermadec/i.test(c.args?.query || '')) ||
(c.name === 'analyst_query' && (c.args?.layers || []).includes('earthquakes') && c.args?.sortBy === 'magnitude' && c.args?.sortDir !== 'asc')],
},
},
{
id: 'track-then-stop',
turns: ['Track UAL428', 'Stop'],
results: {
track_entity: [{ ok: true, tracking: 'UAL428', layerId: 'flights' }],
move_camera: [null, { ok: true, stopped: false, tracking: true, note: 'No camera motion was active; an entity is being tracked.' }],
},
first: { expect: 'track_entity', typedArgs: [where('track_entity', (a) => /UAL\s*428/i.test(a.query || ''))] },
second: { expect: 'stop_tracking' },
},
{
id: 'paris-then-closer',
turns: ['Take me to Paris', 'Get a little closer'],
results: { fly_to_location: [{ ok: true, arrived: true, place: 'Paris, France' }] },
first: { expect: 'fly_to_location' },
second: { expect: 'adjust_camera_zoom', typedArgs: [where('adjust_camera_zoom', (a) => a.direction === 'in')] },
},
{
id: 'thermal-then-back',
turns: ['Switch to thermal view', 'Actually, put it back the way it was'],
results: { set_visual_style: [{ ok: true, style: 'thermal', previousStyle: 'normal' }] },
first: { expect: 'set_visual_style', typedArgs: [where('set_visual_style', (a) => a.style === 'thermal')] },
second: { expect: 'set_visual_style', typedArgs: [where('set_visual_style', (a) => a.style === 'normal')] },
},
{
id: 'radio-then-volume',
turns: ['Turn on the radio', 'Turn it down to twenty percent'],
results: { control_radio: [{ ok: true, prepared: true, action: 'play', station: 'KUT 90.5 Austin' }] },
first: { expect: 'control_radio', typedArgs: [where('control_radio', (a) => a.action === 'play')] },
second: { expect: 'control_radio', typedArgs: [where('control_radio', (a) => a.action === 'volume' && a.volumePct === 20)] },
},
{
id: 'mark-then-route',
turns: ['Mark the Texas Capitol on the map', 'Now draw the walking route from there to Zilker Park'],
results: { annotate_map: [{ ok: true, placed: [{ label: 'Texas State Capitol', type: 'pin' }] }] },
first: { expect: 'annotate_map' },
second: {
expect: 'annotate_map',
typedArgs: [where('annotate_map', (a) => (a.annotations || []).some((x) =>
x.type === 'route' && (x.mode ?? 'walking') === 'walking' && /capitol/i.test(x.points?.[0]?.target || '')))],
},
},
{
id: 'ships-then-closest',
turns: ['Which ships are headed to Oakland?', 'Which of those is closest?'],
results: { analyst_query: [shipsOakland, { ...shipsOakland, count: 1, examples: [shipsOakland.examples[2]] }] },
first: { expect: 'analyst_query' },
second: { expect: 'analyst_query', typedArgs: [where('analyst_query', (a) => a.followUp === true && a.sortBy === 'distance')] },
},
{
id: 'flights-then-thanks',
turns: ['Turn on the flights layer', "Thanks, that's perfect"],
results: { set_layer_visibility: [{ ok: true, layerId: 'flights', enabled: true, feedState: 'nominal', count: 1840 }] },
first: { expect: 'set_layer_visibility' },
second: { expectNone: true },
},
]);
/** Canned result for a dialogue tool call, or null for the neutral stub. */
export function dialogueResult(dialogue, toolName, turn) {
const byTurn = dialogue.results?.[toolName];
if (!byTurn) return null;
return byTurn[turn] ?? null;
}
// ── New capabilities (point-and-ask, layer reach) ─────────────────────────
//
// ROUTING_PHRASES is the qa-voice-routing table, so that harness runs exactly
// its own table. The benchmark runs CAPABILITY_EXTRAS as the separate
// `capability` suite so the core suite stays comparable across tool versions.
/** The full qa-voice-routing table. */
export const ROUTING_PHRASES = CORE_PHRASES;
// Pointer context items in the app's own format (src/voice/pointerContext.js
// output), sent as an inert system item just before the user's words.
const AIRCRAFT_POINTER = {
type: 'pointer_context', at: 'speech_start',
target: { kind: 'aircraft', layer: 'flights', label: 'UPS793' },
ground: { lat: 30.3121, lon: -97.6013 }, screen: { x: 0.608, y: 0.408 },
};
const GROUND_POINTER = {
type: 'pointer_context', at: 'speech_start',
ground: { lat: 30.2849, lon: -97.7341 }, screen: { x: 0.427, y: 0.547 },
};
const isPointer = (v) => String(v || '').toLowerCase() === 'pointer';
const flightsInView = {
ok: true,
count: 3,
scopeLabel: 'in view',
items: [
{ n: 1, id: 'a1b2c3', callsign: 'UAL428', layerKey: 'flights', lat: 30.2, lon: -97.7 },
{ n: 2, id: 'a4b5c6', callsign: 'SWA1201', layerKey: 'flights', lat: 30.3, lon: -97.6 },
{ n: 3, id: 'a7b8c9', callsign: 'AAL88', layerKey: 'flights', lat: 30.1, lon: -97.8 },
],
};
/**
* Extra single-turn phrases (with an optional pointer `context`) and two-turn
* dialogues for the new capabilities. Items with `turns` are graded like
* DIALOGUES (turn 2 full-call); the rest like CORE_PHRASES.
*/
export const CAPABILITY_EXTRAS = Object.freeze([
{ phrase: 'Tell me about this', context: AIRCRAFT_POINTER, expect: 'get_entity_context', typedArgs: [where('get_entity_context', (a) => isPointer(a.scope))] },
{ phrase: 'How many flights are around here?', context: GROUND_POINTER, expect: 'analyst_query', typedArgs: [where('analyst_query', (a) => isPointer(a.scope?.kind))] },
{ phrase: 'Take me there', context: GROUND_POINTER, expect: 'fly_to_location', typedArgs: [where('fly_to_location', (a) => isPointer(a.query))] },
{ phrase: 'Track that one', context: AIRCRAFT_POINTER, expect: 'track_entity', typedArgs: [where('track_entity', (a) => isPointer(a.query))] },
{ phrase: 'Turn on the weather radar', expect: 'set_layer_visibility', typedArgs: [where('set_layer_visibility', (a) => a.layerId === 'weather-radar' && a.enabled === true)] },
{ phrase: 'Are there any hurricanes right now?', expect: { oneOf: ['analyst_query', 'set_layer_visibility'] }, typedArgs: [(c) => (c.args?.layers || []).includes('weather-cyclones') || c.args?.layerId === 'weather-cyclones'] },
{ phrase: 'Which bus is closest to the middle of the screen?', expect: 'analyst_query', typedArgs: [where('analyst_query', (a) => (a.layers || []).includes('transit'))] },
{
id: 'flights-then-second',
turns: ['How many flights are in view?', 'Track the second one'],
results: { analyst_query: [flightsInView] },
first: { expect: 'analyst_query' },
second: { expect: 'track_entity', typedArgs: [where('track_entity', (a) => Number(a.referent) === 2 || /SWA\s*1201/i.test(a.query || ''))] },
},
]);
-58
View File
@@ -1,58 +0,0 @@
/**
* Cost estimation for benchmark turns (USD). Pure.
*
* OpenAI Realtime: the app's own rate table (`src/voice/voiceCost.js`).
* `gpt-realtime-2.1` has no registry entry, so it resolves to the priciest
* known table (gpt-realtime-2) — which matches OpenAI's published 2.1 rates
* as read on 2026-09-23 ($4 / $0.40 cached / $24 per 1M text tokens).
*
* Gemini Live: https://ai.google.dev/gemini-api/docs/pricing, read 2026-09-23,
* gemini-3.8-live paid tier, USD per 1M tokens. Thinking tokens bill as text
* output. The native-audio Live model rejects TEXT output, so the benchmark
* requests AUDIO and the spoken confirmation bills at the audio-output rate.
*/
import { estimateUsageCostUsd, resolveVoiceModelById } from '../../src/voice/voiceCost.js';
export const GEMINI_RATES = Object.freeze({
'gemini-3.8-live': Object.freeze({
textInput: 0.75,
audioInput: 3.0,
textOutput: 4.5,
audioOutput: 12.0,
verifiedOn: '2026-09-23',
}),
});
/** Cost of one Realtime `response.usage` for `model`. */
export function openAiUsageCostUsd(model, usage) {
return estimateUsageCostUsd(usage, resolveVoiceModelById(model).rates);
}
const modalityCount = (details, modality) =>
(Array.isArray(details) ? details : [])
.filter((d) => String(d?.modality || '').toUpperCase() === modality)
.reduce((sum, d) => sum + (Number(d?.tokenCount) || 0), 0);
/**
* Cost of one Gemini Live `usageMetadata`. Unattributed prompt tokens bill
* as audio input and unattributed response tokens as audio output — the
* conservative direction for a spend cap.
*/
export function geminiUsageCostUsd(model, usage) {
const rates = GEMINI_RATES[model] || GEMINI_RATES['gemini-3.8-live'];
if (!usage) return 0;
const prompt = Number(usage.promptTokenCount) || 0;
const response = Number(usage.responseTokenCount) || 0;
const thoughts = Number(usage.thoughtsTokenCount) || 0;
const textIn = modalityCount(usage.promptTokensDetails, 'TEXT');
const audioIn = Math.max(0, prompt - textIn);
const textOut = modalityCount(usage.responseTokensDetails, 'TEXT');
const audioOut = Math.max(0, response - textOut);
const usd =
(textIn * rates.textInput +
audioIn * rates.audioInput +
(textOut + thoughts) * rates.textOutput +
audioOut * rates.audioOutput) /
1_000_000;
return Number.isFinite(usd) && usd > 0 ? usd : 0;
}
-36
View File
@@ -1,36 +0,0 @@
/**
* Capture the exact Realtime session config production mints.
*
* Rather than re-typing the session fields, this drives the production token
* handler (`server/providers/openai/realtime.js`) with a fake upstream fetch
* and returns the body it would have POSTed to OpenAI. Instructions, tools,
* tool_choice, reasoning, and truncation therefore cannot drift from what a
* real GEV mic session receives.
*/
import { createRealtimeTokenHandler } from '../../server/providers/openai/realtime.js';
import { realtimeInstructions } from '../../server/providers/openai/instructions.js';
import { GEV_REALTIME_TOOLS } from '../../server/providers/openai/tools.js';
export { realtimeInstructions, GEV_REALTIME_TOOLS };
/** Return production's `{ session: {...} }` client-secret body for `model`. */
export async function captureProductionSession(model) {
let captured = null;
const handler = createRealtimeTokenHandler({
resolveApiKey: () => 'bench-placeholder',
models: { standard: model, mini: model },
fetchImpl: async (_url, init) => {
captured = JSON.parse(init.body);
return new Response('{}', { status: 200, headers: { 'content-type': 'application/json' } });
},
});
const res = {
statusCode: 0,
headers: {},
setHeader(k, v) { this.headers[k] = v; },
end() {},
};
await handler({ method: 'POST', url: '/api/realtime/token', headers: {}, socket: {} }, res);
if (!captured?.session) throw new Error('production token handler did not produce a session config');
return captured.session;
}
-272
View File
@@ -1,272 +0,0 @@
/**
* Pure aggregation of benchmark rows into summary.json / summary.md.
* Rows: `{ type:'run', ... }` run metadata and `{ type:'turn', ... }` turns.
*/
export function percentile(values, p) {
const xs = values.filter((v) => typeof v === 'number' && Number.isFinite(v)).sort((a, b) => a - b);
if (!xs.length) return null;
const idx = Math.min(xs.length - 1, Math.max(0, Math.ceil((p / 100) * xs.length) - 1));
return xs[idx];
}
const pct = (n, d) => (d ? Math.round((1000 * n) / d) / 10 : null);
const sum = (xs) => xs.reduce((a, b) => a + (Number(b) || 0), 0);
const mean = (xs) => {
const v = xs.filter((x) => typeof x === 'number' && Number.isFinite(x));
return v.length ? sum(v) / v.length : null;
};
const round = (x, d = 0) => (x === null || x === undefined ? null : Math.round(x * 10 ** d) / 10 ** d);
export const COVERAGE_VERDICTS = Object.freeze(['correct', 'honest-refusal', 'hallucinated-capability', 'wrong-tool']);
export function modelKey(row) {
return `${row.provider}:${row.model}${row.variant ? `#${row.variant}` : ''}`;
}
/** Aggregate turn rows into one summary entry per provider×model(×variant). */
export function summarize(rows) {
const runs = rows.filter((r) => r.type === 'run');
const turns = rows.filter((r) => r.type === 'turn');
const keys = [...new Set(turns.map(modelKey))];
const models = keys.map((key) => {
const mine = turns.filter((t) => modelKey(t) === key);
const core = mine.filter((t) => t.suite === 'core');
const cov = mine.filter((t) => t.suite === 'coverage');
const dia = mine.filter((t) => t.suite === 'dialogue');
const capAll = mine.filter((t) => t.suite === 'capability');
// Latency percentiles stay on the core + coverage phrase sets so runs of
// different builds remain comparable.
const single = mine.filter((t) => t.suite === 'core' || t.suite === 'coverage');
const covV2 = cov.filter((t) => t.gradeV2);
const reps = [...new Set(core.map((t) => t.rep))].sort((a, b) => a - b);
const withCall = single.filter((t) => t.tFirstCallMs !== null && t.tFirstCallMs !== undefined);
const typed = core.filter((t) => t.grade?.typedArgsOk !== null && t.grade?.typedArgsOk !== undefined);
const diaFailures = dia.filter((t) => !t.grade?.ok).map((t) => ({
id: t.phraseId, rep: t.rep, expect: t.expect,
turn1: (t.turn1?.calls || []).map((c) => `${c.name}(${JSON.stringify(c.args).slice(0, 80)})`).join(', ') || '(none)',
turn2: (t.calls || []).map((c) => `${c.name}(${JSON.stringify(c.args).slice(0, 80)})`).join(', ') || '(none)',
}));
const coverageV2Counts = Object.fromEntries(COVERAGE_VERDICTS.map((v) => [v, covV2.filter((t) => t.gradeV2?.verdict === v).length]));
const capFailures = {};
for (const t of capAll.filter((x) => !x.grade?.fullCallOk)) {
const k = t.phraseId;
capFailures[k] = capFailures[k] || { id: k, expect: t.expect, fails: 0, of: capAll.filter((x) => x.phraseId === k).length, called: [] };
capFailures[k].fails += 1;
capFailures[k].called.push((t.calls || []).map((c) => `${c.name}(${JSON.stringify(c.args).slice(0, 70)})`).join(', ') || '(none)');
}
const coverageCounts = Object.fromEntries(COVERAGE_VERDICTS.map((v) => [v, cov.filter((t) => t.grade?.verdict === v).length]));
const coreFailures = {};
for (const t of core.filter((x) => !x.grade?.ok)) {
const k = t.phrase;
coreFailures[k] = coreFailures[k] || { phrase: k, expect: t.expect, fails: 0, of: core.filter((x) => x.phrase === k).length, called: [] };
coreFailures[k].fails += 1;
coreFailures[k].called.push((t.calls || []).map((c) => c.name).join(',') || '(none)');
}
const covByItem = {};
for (const t of cov) {
covByItem[t.phraseId] = covByItem[t.phraseId] || { id: t.phraseId, utterance: t.phrase, supported: t.supported, verdicts: [] };
covByItem[t.phraseId].verdicts.push(t.grade?.verdict);
}
const run = runs.filter((r) => modelKey(r) === key).at(-1) || {};
const local = mine.filter((t) => t.local);
return {
key,
provider: mine[0].provider,
model: mine[0].model,
variant: mine[0].variant || null,
settings: run.settings || null,
machine: run.machine || null,
warmup: run.warmup || null,
turns: mine.length,
core: {
n: core.length,
reps: reps.length,
accuracyPct: pct(core.filter((t) => t.grade?.ok).length, core.length),
toolAccuracyPct: pct(core.filter((t) => t.grade?.toolOk).length, core.length),
fullCallAccuracyPct: pct(core.filter((t) => t.grade?.fullCallOk).length, core.length),
typedArgsAccuracyPct: pct(typed.filter((t) => t.grade?.typedArgsOk).length, typed.length),
typedArgsN: typed.length,
argsAccuracyPct: pct(core.filter((t) => t.grade?.argsOk === true).length, core.filter((t) => t.grade?.argsOk !== null && t.grade?.argsOk !== undefined).length),
firstCallAccuracyPct: pct(core.filter((t) => t.grade?.firstCallOk).length, core.length),
falsePositives: core.filter((t) => t.grade?.falsePositive).length,
negativeTurns: core.filter((t) => t.expectNone).length,
perRepAccuracyPct: reps.map((rep) => {
const r = core.filter((t) => t.rep === rep);
return pct(r.filter((t) => t.grade?.ok).length, r.length);
}),
failures: Object.values(coreFailures).sort((a, b) => b.fails - a.fails),
},
coverage: {
n: cov.length,
counts: coverageCounts,
acceptablePct: pct(cov.filter((t) => t.grade?.acceptable).length, cov.length),
supportedCorrectPct: pct(cov.filter((t) => t.supported && t.grade?.verdict === 'correct').length, cov.filter((t) => t.supported).length),
unsupportedHonestPct: pct(cov.filter((t) => !t.supported && t.grade?.acceptable).length, cov.filter((t) => !t.supported).length),
items: Object.values(covByItem),
},
coverageV2: {
n: covV2.length,
counts: coverageV2Counts,
acceptablePct: pct(covV2.filter((t) => t.gradeV2?.acceptable).length, covV2.length),
},
capability: {
n: capAll.length,
toolPct: pct(capAll.filter((t) => t.grade?.toolOk).length, capAll.length),
fullPct: pct(capAll.filter((t) => t.grade?.fullCallOk).length, capAll.length),
failures: Object.values(capFailures).sort((a, b) => b.fails - a.fails),
},
prompt: run.prompt || null,
dialogue: {
n: dia.length,
turn2FullPct: pct(dia.filter((t) => t.grade?.fullCallOk).length, dia.length),
turn2ToolPct: pct(dia.filter((t) => t.grade?.toolOk).length, dia.length),
turn1FullPct: pct(dia.filter((t) => t.grade?.turn1Ok).length, dia.length),
turn2ToolCallP50: percentile(dia.map((t) => t.tFirstCallMs), 50),
failures: diaFailures,
},
latencyMs: {
scope: 'single-turn phrases (core + coverage); send → event, text mode',
firstTokenP50: percentile(single.map((t) => t.tFirstTokenMs), 50),
firstTokenP90: percentile(single.map((t) => t.tFirstTokenMs), 90),
toolCallP50: percentile(withCall.map((t) => t.tFirstCallMs), 50),
toolCallP90: percentile(withCall.map((t) => t.tFirstCallMs), 90),
turnDoneP50: percentile(single.map((t) => t.tDoneMs), 50),
turnDoneP90: percentile(single.map((t) => t.tDoneMs), 90),
setupP50: percentile(single.map((t) => t.setupMs), 50),
},
tokens: {
input: sum(mine.map((t) => t.usage?.inputTokens)),
cached: sum(mine.map((t) => t.usage?.cachedTokens)),
output: sum(mine.map((t) => t.usage?.outputTokens)),
reasoning: sum(mine.map((t) => t.usage?.reasoningTokens)),
inputPerTurnMean: round(mean(mine.map((t) => t.usage?.inputTokens))),
inputPerRequestMean: round(mean(single.filter((t) => t.responses > 0 && t.usage?.inputTokens).map((t) => t.usage.inputTokens / t.responses))),
},
costUsd: round(sum(mine.map((t) => t.costUsd)), 4),
costPerTurnUsd: mine.length ? round(sum(mine.map((t) => t.costUsd)) / mine.length, 5) : null,
errors: mine.filter((t) => t.errors?.length).length,
timeouts: mine.filter((t) => t.timeout).length,
local: local.length
? {
prefillTokSMean: round(mean(local.map((t) => t.local.prefillTokS)), 1),
prefillTokSP50: round(percentile(local.map((t) => t.local.prefillTokS), 50), 0),
decodeTokSMean: round(mean(local.map((t) => t.local.decodeTokS)), 1),
decodeTokSP50: round(percentile(local.map((t) => t.local.decodeTokS), 50), 1),
decodeTokSP90: round(percentile(local.map((t) => t.local.decodeTokS), 90), 1),
}
: null,
};
});
return {
generatedAt: new Date().toISOString(),
cloudSpendUsd: round(sum(turns.filter((t) => t.cloud).map((t) => t.costUsd)), 4),
models,
};
}
const machineBrief = (machine) => {
if (!machine) return '—';
const load1 = String(machine.loadavg || '').split(' ')[0] || '?';
const busy = (machine.busyProcesses || []).filter((p) => Number(p.match(/([\d.]+)%cpu/)?.[1]) >= 100)
.map((p) => p.replace(/ Helper \(GPU\)/, ' GPU').replace(/Google Chrome for Testing/, 'Chrome-for-Testing'));
return `load ${load1}${busy.length ? `; ${busy.join(', ')}` : ''}`;
};
const fmtCounts = (block) => (block?.n
? `${block.counts.correct} / ${block.counts['honest-refusal']} / ${block.counts['hallucinated-capability']} / ${block.counts['wrong-tool']} (n=${block.n})`
: '—');
const fmtMs = (v) => (v === null || v === undefined ? '—' : v >= 10_000 ? `${(v / 1000).toFixed(1)}s` : `${Math.round(v)}ms`);
const fmtPct = (v) => (v === null || v === undefined ? '—' : `${v}%`);
/** Render summary.md from a summarize() result plus free-form notes. */
export function renderMarkdown(summary, { title = 'GEV voice-routing benchmark', notes = [], payload = [] } = {}) {
const out = [`# ${title}`, '', `Generated ${summary.generatedAt}. Cloud spend so far: **$${summary.cloudSpendUsd?.toFixed(2)}**.`, ''];
out.push('> **What this measures.** Tool CHOICE for text turns sent over each provider\'s API with production\'s instructions and tools, with stubbed (always-successful) tool results — the same method as `qa-voice-routing.mjs` layer 1. It is **not** end-to-end spoken accuracy (no speech recognition, no audio, no screenshot/scene context) and **not** spoken latency: every latency below is "send text → event", before any audio would be synthesized or played.', '');
out.push('## Routing accuracy', '');
out.push('| model | core tool-name | core full-call (tool + all arg checks) | typed args (n) | harness verdict (legacy) | first call right | false +ve | per-rep harness verdict | dialogue turn-2 full-call | dialogue turn-1 | coverage v1 C / HR / HC / WT | coverage v1 acceptable | coverage v2 C / HR / HC / WT | coverage v2 acceptable | capability tool / full-call |');
out.push('|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|');
for (const m of summary.models) {
const c = m.coverage.counts;
out.push(`| ${m.key} | ${fmtPct(m.core.toolAccuracyPct)} | ${fmtPct(m.core.fullCallAccuracyPct)} | ${fmtPct(m.core.typedArgsAccuracyPct)} (${m.core.typedArgsN}) | ${fmtPct(m.core.accuracyPct)} (n=${m.core.n}) | ${fmtPct(m.core.firstCallAccuracyPct)} | ${m.core.falsePositives}/${m.core.negativeTurns} | ${m.core.perRepAccuracyPct.map(fmtPct).join(', ')} | ${fmtPct(m.dialogue.turn2FullPct)} (n=${m.dialogue.n}) | ${fmtPct(m.dialogue.turn1FullPct)} | ${c.correct} / ${c['honest-refusal']} / ${c['hallucinated-capability']} / ${c['wrong-tool']} (n=${m.coverage.n}) | ${fmtPct(m.coverage.acceptablePct)} | ${fmtCounts(m.coverageV2)} | ${fmtPct(m.coverageV2.acceptablePct)} | ${m.capability.n ? `${fmtPct(m.capability.toolPct)} / ${fmtPct(m.capability.fullPct)} (n=${m.capability.n})` : '—'} |`);
}
out.push('', 'Core tool-name = right tool(s) called. Full-call = right tool(s) AND every argument check (the legacy `args` spot checks plus the benchmark\'s typed checks: exact layer ids, enum values, numeric thresholds, sort direction). Legacy harness verdict = exactly what `qa-voice-routing.mjs` would report. Coverage: C = correct, HR = honest refusal, HC = hallucinated capability, WT = wrong tool; "acceptable" = correct, or an honest refusal where today\'s tools cannot do it.', '');
out.push('## Latency and cost', '');
out.push('| model | p50 send → first token (text) | p50 send → tool call (text) | p90 send → tool call (text) | p50 send → turn done (text) | dialogue turn-2 p50 → tool call | session setup p50 | input tok/turn | $ total | $/turn |');
out.push('|---|---|---|---|---|---|---|---|---|---|');
for (const m of summary.models) {
out.push(`| ${m.key} | ${fmtMs(m.latencyMs.firstTokenP50)} | ${fmtMs(m.latencyMs.toolCallP50)} | ${fmtMs(m.latencyMs.toolCallP90)} | ${fmtMs(m.latencyMs.turnDoneP50)} | ${fmtMs(m.dialogue.turn2ToolCallP50)} | ${fmtMs(m.latencyMs.setupP50)} | ${m.tokens.inputPerTurnMean ?? '—'} | ${m.costUsd?.toFixed(3)} | ${m.costPerTurnUsd ?? '—'} |`);
}
out.push('', 'Latencies are over single-turn phrases (core + coverage). "Turn done" includes the continuation response after the stubbed tool result. Cloud session setup (WebSocket + session config) is excluded from every latency and shown separately.', '');
const locals = summary.models.filter((m) => m.local);
if (locals.length) {
out.push('## Local runtime (cold vs warm)', '');
out.push('| model | load (wall) | COLD: first request after load → tool call | cold prefill tok/s (full prompt) | prompt tokens | WARM p50 → tool call | WARM p90 → tool call | warm per-turn prefill tok/s p50 | decode tok/s p50 / p90 | machine at start |');
out.push('|---|---|---|---|---|---|---|---|---|---|');
for (const m of locals) {
const w = m.warmup || {};
out.push(`| ${m.key} | ${fmtMs(w.loadWallMs ?? w.loadMs)} | ${fmtMs(w.coldToolCallMs ?? w.warmupTurnMs)}${w.coldToolCallMs == null ? ' (turn, no tool)' : ''} | ${w.prefillTokS ? Math.round(w.prefillTokS) : '—'} | ${w.promptTokens ?? '—'} | ${fmtMs(m.latencyMs.toolCallP50)} | ${fmtMs(m.latencyMs.toolCallP90)} | ${m.local.prefillTokSP50 ?? '—'} | ${m.local.decodeTokSP50 ?? '—'} / ${m.local.decodeTokSP90 ?? '—'} | ${machineBrief(m.machine)} |`);
}
out.push('', 'COLD = the first request after the model is loaded (pays the full ~11k-token prompt prefill). WARM = every later turn. "Warm per-turn prefill tok/s" is prompt tokens ÷ prompt-eval time as the runtime reports it: a value in the tens of thousands means the cached prompt prefix was reused and only the new user text was prefilled; a value near the cold rate means the WHOLE prompt was re-prefilled on every turn. A real voice session pays the cold cost once per model load. "Machine at start" = 1-minute load average and processes above 100% CPU when the run began (other GPU/CPU users on the machine inflate local timings).', '');
}
if (payload.length) {
out.push('## Payload changes per provider', '');
for (const p of payload) out.push(`- ${p}`);
out.push('');
}
out.push('## Failures by model', '');
for (const m of summary.models) {
out.push(`### ${m.key}`, '');
out.push(`Settings: \`${JSON.stringify(m.settings)}\``, '');
if (m.errors || m.timeouts) out.push(`Errors: ${m.errors} turns, timeouts: ${m.timeouts}.`, '');
if (m.core.failures.length) {
out.push('Core misses (legacy verdict):', '');
for (const f of m.core.failures) out.push(`- "${f.phrase}" (expect ${f.expect}) — failed ${f.fails}/${f.of}; called: ${f.called.join(' ; ')}`);
out.push('');
}
if (m.capability.failures.length) {
out.push('Capability misses (full call):', '');
for (const f of m.capability.failures) out.push(`- "${f.id}" (expect ${f.expect}) — failed ${f.fails}/${f.of}; called: ${f.called.join(' ; ').slice(0, 400)}`);
out.push('');
}
if (m.dialogue.failures.length) {
out.push('Dialogue misses (turn 2):', '');
for (const f of m.dialogue.failures) out.push(`- ${f.id} rep ${f.rep} (expect ${f.expect}) — t1: ${f.turn1} ⇒ t2: ${f.turn2}`);
out.push('');
}
const badCov = m.coverage.items.filter((i) => i.verdicts.some((v) => v === 'hallucinated-capability' || v === 'wrong-tool' || (i.supported && v !== 'correct')));
if (badCov.length) {
out.push('Coverage misses:', '');
for (const i of badCov) out.push(`- "${i.utterance}" (${i.supported ? 'supported' : 'unsupported'}) — ${i.verdicts.join(', ')}`);
out.push('');
}
}
if (notes.length) {
out.push('## Notes', '');
for (const n of notes) out.push(`- ${n}`);
out.push('');
}
return out.join('\n');
}
/**
* Before → after table for two summarize() results (matched by model key).
* Coverage "before" is the v1 rubric (what the old tools could do); "after"
* shows both v1 (same yardstick) and v2 (the current tool surface).
*/
export function renderComparison(before, after, { beforeLabel = 'before', afterLabel = 'after' } = {}) {
const out = ['## Before → after', '', `Before = ${beforeLabel}; after = ${afterLabel}. Core accuracy = tool + all argument checks (full call). Coverage OK % = correct, or an honest refusal where the tools of that build cannot do it. HC = hallucinated-capability count. p50 = send → tool call (text). Prompt tokens = mean input tokens per model request (cloud: as billed, before cache discounts; local: the cold full-prompt count).`, ''];
out.push('| model | core full-call | coverage OK % (v1 → v1 / v2) | HC (v1 → v1 / v2) | dialogue turn-2 | capability full-call (after) | p50 → tool call | prompt tokens |');
out.push('|---|---|---|---|---|---|---|---|');
const hc = (block) => (block?.n ? block.counts['hallucinated-capability'] : null);
const promptTok = (m) => (m?.warmup?.promptTokens ?? m?.tokens?.inputPerRequestMean ?? null);
for (const a of after.models) {
const b = before.models.find((x) => x.key === a.key);
const arrow = (x, y, f) => `${f(x)} → ${f(y)}`;
const pctF = (v) => (v === null || v === undefined ? '—' : `${v}%`);
const msF = (v) => (v === null || v === undefined ? '—' : `${Math.round(v)}ms`);
const num = (v) => (v === null || v === undefined ? '—' : String(v));
out.push(`| ${a.key} | ${arrow(b?.core.fullCallAccuracyPct, a.core.fullCallAccuracyPct, pctF)} | ${pctF(b?.coverage.acceptablePct)} → ${pctF(a.coverage.acceptablePct)} / ${pctF(a.coverageV2.acceptablePct)} | ${num(hc(b?.coverage))} → ${num(hc(a.coverage))} / ${num(hc(a.coverageV2))} | ${arrow(b?.dialogue.turn2FullPct, a.dialogue.turn2FullPct, pctF)} | ${pctF(a.capability.fullPct)} | ${arrow(b?.latencyMs.toolCallP50, a.latencyMs.toolCallP50, msF)} | ${arrow(promptTok(b), promptTok(a), num)} |`);
}
out.push('');
return out.join('\n');
}
-21
View File
@@ -1,21 +0,0 @@
/**
* Stub tool results fed back to the model after each call. Pure.
*
* core suite: the neutral `{ ok:true }` stub qa-voice-routing has always
* used, so continuations behave exactly as in that harness.
* coverage suite: the same stub, except a call whose arguments violate the
* tool schema gets `{ ok:false, error }` — what the app's runner returns
* for an unknown layer id — so the model's follow-up words can be judged
* for honesty.
*/
import { validateCall } from './grade.mjs';
export const STUB_OK = Object.freeze({ ok: true, note: 'qa-harness stub result' });
export function stubResult(suite, c, tools) {
if (suite === 'coverage') {
const errors = validateCall(tools, c);
if (errors.length) return { ok: false, error: `Invalid arguments: ${errors.join('; ')}` };
}
return { ...STUB_OK };
}
-119
View File
@@ -1,119 +0,0 @@
/**
* Pure converters from the production Realtime tool list
* (`{ type:'function', name, description, parameters }`) to other providers'
* tool formats. Descriptions and schemas are carried over unchanged except
* where a target cannot represent them; every such change is listed below.
*/
const PERMISSIVE_SCALAR = Object.freeze(['string', 'number', 'boolean']);
const clone = (value) => JSON.parse(JSON.stringify(value));
/**
* Normalize a JSON schema for OpenAI-compatible local servers.
*
* Only one change: a property schema with no `type` (and no enum / anyOf /
* oneOf / $ref / const that would imply one) gets a permissive scalar union.
* `analyst_query.filters[].value` is `{}` in production — valid JSON Schema,
* but chat templates that print `type` for every property (Gemma's among
* them) crash on it.
*/
export function normalizeSchemaForChat(schema) {
const out = clone(schema);
const visit = (node, isRoot) => {
if (!node || typeof node !== 'object' || Array.isArray(node)) return;
const implied = ['type', 'enum', 'anyOf', 'oneOf', 'allOf', '$ref', 'const'].some((k) => k in node);
if (!isRoot && !implied) node.type = [...PERMISSIVE_SCALAR];
if (node.properties) for (const sub of Object.values(node.properties)) visit(sub, false);
if (node.items) visit(node.items, false);
for (const key of ['anyOf', 'oneOf', 'allOf']) if (Array.isArray(node[key])) node[key].forEach((s) => visit(s, false));
};
visit(out, true);
return out;
}
/**
* Every schema location normalizeSchemaForChat changes, as
* `{ tool, path, from, to }` — logged in summary.md so the per-provider
* payload difference is explicit rather than implied.
*/
export function chatNormalizationDiff(realtimeTools) {
const changes = [];
const walk = (before, after, tool, where) => {
if (!before || typeof before !== 'object' || Array.isArray(before)) return;
if (JSON.stringify(before.type) !== JSON.stringify(after.type)) {
changes.push({ tool, path: where, from: before.type === undefined ? '(untyped)' : before.type, to: after.type });
}
for (const [k, v] of Object.entries(before.properties || {})) walk(v, after.properties[k], tool, `${where}.${k}`);
if (before.items) walk(before.items, after.items, tool, `${where}[]`);
};
for (const tool of realtimeTools) walk(tool.parameters, normalizeSchemaForChat(tool.parameters), tool.name, tool.name);
return changes;
}
/** Realtime tools → Chat Completions tools (`{type:'function', function:{...}}`). */
export function toChatTools(realtimeTools, { normalize = true } = {}) {
return realtimeTools.map((tool) => ({
type: 'function',
function: {
name: tool.name,
description: tool.description,
parameters: normalize ? normalizeSchemaForChat(tool.parameters) : clone(tool.parameters),
},
}));
}
// Keywords the Gemini OpenAPI-subset `Schema` does not accept.
const GEMINI_DROP = new Set(['additionalProperties', '$schema', 'default', 'examples', '$id', 'const']);
/**
* Convert a JSON schema to the Gemini OpenAPI-subset `Schema`.
* Drops unsupported keywords (additionalProperties, $schema, default, ...);
* an untyped property becomes `anyOf` string|number|boolean; a type array
* becomes `anyOf` of single types.
*/
export function toGeminiSchema(schema) {
if (!schema || typeof schema !== 'object' || Array.isArray(schema)) return schema;
const out = {};
for (const [key, value] of Object.entries(schema)) {
if (GEMINI_DROP.has(key)) continue;
if (key === 'properties') {
out.properties = Object.fromEntries(Object.entries(value).map(([k, v]) => [k, toGeminiSchema(v)]));
} else if (key === 'items') {
out.items = toGeminiSchema(value);
} else if (key === 'anyOf' || key === 'oneOf') {
out.anyOf = value.map((s) => toGeminiSchema(s));
} else {
out[key] = clone(value);
}
}
if (Array.isArray(out.type)) {
const types = out.type;
delete out.type;
out.anyOf = types.map((t) => ({ type: t }));
}
const implied = ['type', 'enum', 'anyOf'].some((k) => k in out);
if (!implied) out.anyOf = PERMISSIVE_SCALAR.map((t) => ({ type: t }));
return out;
}
/**
* Realtime tools → Gemini `functionDeclarations`.
* mode 'jsonSchema' (default): pass the production schema verbatim through
* `parametersJsonSchema` — full JSON Schema, closest to prompt parity.
* mode 'openapi': convert to the OpenAPI-subset `parameters` field.
* A tool with no properties gets no parameters field in openapi mode (Gemini
* rejects an empty OBJECT schema there).
*/
export function toGeminiDeclarations(realtimeTools, { mode = 'jsonSchema' } = {}) {
return realtimeTools.map((tool) => {
const decl = { name: tool.name, description: tool.description };
const params = tool.parameters;
if (mode === 'jsonSchema') {
decl.parametersJsonSchema = clone(params);
} else if (params && Object.keys(params.properties || {}).length) {
decl.parameters = toGeminiSchema(params);
}
return decl;
});
}
-111
View File
@@ -1,111 +0,0 @@
#!/usr/bin/env node
/**
* Summarize voice turn spans from the Realtime debug log.
*
* Every closed voice turn writes one `turn.span` record (see
* src/voice/turnMetrics.js). This prints each span and p50/p90 figures for
* end-of-speech → first audio, the longest silence, tool time, preamble rate,
* spoken words and hedge hits.
*
* Usage:
* node scripts/voice-turn-spans.mjs [log.jsonl] [--since <ISO time>] [--session <id>] [--json]
*/
import fs from 'node:fs';
import path from 'node:path';
import { fileURLToPath } from 'node:url';
const root = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '..');
const args = process.argv.slice(2);
const option = (flag) => {
const index = args.indexOf(flag);
return index >= 0 ? args[index + 1] : null;
};
const positional = args.filter(
(arg, index) => !arg.startsWith('--') && !args[index - 1]?.startsWith('--'),
);
const logPath =
positional[0] || path.join(root, '.gev-logs', 'realtime-conversations.jsonl');
const since = option('--since') ? Date.parse(option('--since')) : null;
const session = option('--session');
const spans = [];
for (const line of fs.readFileSync(logPath, 'utf8').split('\n')) {
if (!line.includes('"turn.span"')) continue;
let record;
try {
record = JSON.parse(line);
} catch {
continue;
}
if (record.event !== 'turn.span') continue;
if (since && Date.parse(record.timestamp) < since) continue;
if (session && record.sessionId !== session) continue;
spans.push({
at: record.timestamp,
session: record.sessionId,
...record.payload,
});
}
const quantile = (values, q) => {
const sorted = values.filter(Number.isFinite).sort((a, b) => a - b);
if (!sorted.length) return null;
return sorted[Math.min(sorted.length - 1, Math.floor(q * sorted.length))];
};
const withTools = spans.filter((span) => span.tools?.length);
const slowToolTurns = withTools.filter((span) =>
span.tools.some((tool) => tool.ms >= 700),
);
const summary = {
turns: spans.length,
toolTurns: withTools.length,
firstAudioMs: {
p50: quantile(spans.map((s) => s.first_audio_ms), 0.5),
p90: quantile(spans.map((s) => s.first_audio_ms), 0.9),
},
maxSilenceMs: {
p50: quantile(spans.map((s) => s.max_silence_ms), 0.5),
p90: quantile(spans.map((s) => s.max_silence_ms), 0.9),
},
toolMs: {
p50: quantile(withTools.flatMap((s) => s.tools.map((t) => t.ms)), 0.5),
p90: quantile(withTools.flatMap((s) => s.tools.map((t) => t.ms)), 0.9),
},
preambleRate: withTools.length
? withTools.filter((s) => s.preamble).length / withTools.length
: null,
slowToolPreambleOrProgressRate: slowToolTurns.length
? slowToolTurns.filter((s) => s.preamble || s.progress_lines > 0).length /
slowToolTurns.length
: null,
answerWords: {
p50: quantile(spans.map((s) => s.answer_words), 0.5),
p90: quantile(spans.map((s) => s.answer_words), 0.9),
},
spokenWordsTotal: spans.reduce((sum, s) => sum + (s.spoken_words || 0), 0),
hedgeHits: spans.reduce((sum, s) => sum + (s.hedge_hits?.length || 0), 0),
};
if (args.includes('--json')) {
console.log(JSON.stringify({ summary, spans }, null, 2));
} else {
for (const span of spans) {
const tools = (span.tools || [])
.map((tool) => `${tool.name}:${tool.ms}ms`)
.join(',');
console.log(
[
span.at,
`first=${span.first_audio_ms ?? '-'}ms`,
`maxSilence=${span.max_silence_ms ?? '-'}ms`,
`preamble=${span.preamble ? 'y' : 'n'}`,
`progress=${span.progress_lines}`,
`words=${span.answer_words}/${span.spoken_words}`,
`hedges=${span.hedge_hits?.length || 0}`,
tools || '(no tools)',
span.answer_text ? `"${span.answer_text.slice(0, 80)}"` : '',
].join(' '),
);
}
console.log(JSON.stringify(summary, null, 2));
}
-43
View File
@@ -1,43 +0,0 @@
{
"reviewedSources": {
"zs": {
"name": "Zetashapes",
"registryCommit": "e17af153c144fe6e08aa8db2607b4bf8aecd448d",
"registrySha256": "d8a5dd873d26eee11583de4438185c046dec33848d5bde27047ca4bff44d9c39",
"rawLicenseType": "Public domain",
"licenseTextSha256": "91757f96f74f6dfb4a7206486b78586da0d039e9611e224fc793c7167dcfe615",
"decision": "Approved from the pinned WOF registry text: no added restrictions beyond public-domain US TIGER/Line data and Flickr-derived data; retain both upstream credits."
}
},
"licenses": [
"Attribution 3.0 Unported",
"CC BY",
"CC BY (equivalent)",
"CC BY 2.0",
"CC BY 3.0",
"CC BY 3.0 US",
"CC BY 4.0",
"CC-BY",
"CC0",
"CC0 (equivalent)",
"CC0 1.0 Universal",
"ODC-By, v1.0",
"ODC-By v1.0",
"OGL - Canada",
"OGL - City of New Westminster",
"OGL - City of Surrey",
"Open Government Licence v3.0",
"Open Government Licence - British Columbia, v2.0",
"Open Government Licence - Canada",
"Open Government Licence - City of Calgary",
"Open Government Licence - City of Regina",
"Open Government Licence - Edmonton",
"Open Government Licence - Ontario, v1.0",
"Open Government License - British Columbia, v2.0",
"Open Government License - Winnipeg",
"PDDL 1.0",
"Public domain",
"Public Domain",
"Singapore Open Data Licence v1"
]
}
+179
View File
@@ -0,0 +1,179 @@
/**
* Serves the local MCP server at /mcp on the development and preview servers.
*
* This is local transport safety, not authentication: requests must come from
* this machine, carry no proxy forwarding headers, arrive while sharing is
* off, name a loopback host on the port they arrived at (blocking DNS
* rebinding) and, when a browser sends an Origin, come from that same host.
*/
import { isSharingEnabled } from '../../src/keySetupCore.mjs';
import { hasProxySignals } from '../../src/localRequestGate.mjs';
import { createMcpHttpHandler } from '../../src/tools/mcp/index.js';
import { createLocalMcpServer } from './server.js';
const LOOPBACK_HOSTS = new Set(['localhost', '127.0.0.1', '[::1]']);
const LOOPBACK_ADDRESSES = new Set(['127.0.0.1', '::1', '::ffff:127.0.0.1']);
const MAX_BODY_BYTES = 1024 * 1024;
/** How long a client may take to send its request body. */
const BODY_TIMEOUT_MS = 30 * 1000;
/**
* Whether a request may reach the local MCP server. `localPort` is the port
* the connection arrived at; `headers` are the request's, keyed by lower-case
* name; `env` is checked for launcher sharing.
*/
export function isLocalMcpRequest({
remoteAddress,
localPort,
host,
origin,
headers = {},
env = {},
}) {
if (!LOOPBACK_ADDRESSES.has(remoteAddress)) return false;
// A proxy or tunnel in front of the server can make a remote request look
// local, and sharing exposes the server beyond this machine.
if (hasProxySignals(headers) || isSharingEnabled(env)) return false;
let hostUrl;
try {
hostUrl = new URL(`http://${host}`);
} catch {
return false;
}
if (!LOOPBACK_HOSTS.has(hostUrl.hostname)) return false;
// The Host names the port the request reached, so tools are never pointed
// at another local service.
if (localPort != null && (hostUrl.port || '80') !== String(localPort))
return false;
if (origin == null) return true;
try {
const originUrl = new URL(origin);
return (
['http:', 'https:'].includes(originUrl.protocol) &&
originUrl.host === hostUrl.host
);
} catch {
return false;
}
}
/** Vite plugin that mounts the local MCP server at /mcp. */
export function localMcpPlugin({
createServer = createLocalMcpServer,
bodyTimeoutMs = BODY_TIMEOUT_MS,
} = {}) {
const handlers = new Map();
const handlerFor = (apiBase) => {
if (!handlers.has(apiBase))
handlers.set(apiBase, createMcpHttpHandler(createServer({ apiBase })));
return handlers.get(apiBase);
};
const install = (server) => {
server.middlewares.use('/mcp', async (req, res) => {
const host = req.headers.host || '';
if (
!isLocalMcpRequest({
remoteAddress: req.socket?.remoteAddress,
localPort: req.socket?.localPort,
host,
origin: req.headers.origin,
headers: req.headers,
env: process.env,
})
) {
res.writeHead(403, {
'Content-Type': 'application/json',
'Cache-Control': 'no-store',
});
res.end(
JSON.stringify({
error: 'The MCP server only accepts local requests',
}),
);
return;
}
// A client that disconnects before its answer cancels the tool call.
const disconnect = new AbortController();
const onClose = () => {
if (!res.writableFinished) disconnect.abort();
};
res.on('close', onClose);
try {
const body =
req.method === 'POST'
? await readBody(req, bodyTimeoutMs)
: undefined;
const request = new Request(`http://${host}/mcp`, {
method: req.method,
headers: Object.entries(req.headers).flatMap(([name, value]) =>
value === undefined ? [] : [[name, String(value)]],
),
body,
signal: disconnect.signal,
});
const response = await handlerFor(`http://${host}`)(request);
res.writeHead(response.status, Object.fromEntries(response.headers));
res.end(Buffer.from(await response.arrayBuffer()));
} catch (error) {
if (disconnect.signal.aborted) return;
if (res.headersSent) return res.destroy();
res.writeHead([408, 413].includes(error?.status) ? error.status : 400, {
'Content-Type': 'application/json',
'Cache-Control': 'no-store',
});
res.end(
JSON.stringify({
error:
error?.status === 413
? 'Request too large'
: error?.status === 408
? 'Request timed out'
: 'Bad request',
}),
);
} finally {
res.off('close', onClose);
}
});
};
return {
name: 'local-mcp',
configureServer: install,
configurePreviewServer: install,
};
}
function readBody(req, timeoutMs = BODY_TIMEOUT_MS) {
return new Promise((resolve, reject) => {
const chunks = [];
let size = 0;
let tooLarge = false;
// A client that stops sending must not hold the request open.
const timer = setTimeout(
() =>
reject(Object.assign(new Error('Request timed out'), { status: 408 })),
timeoutMs,
);
const fail = (error) => {
clearTimeout(timer);
reject(error);
};
req.on('data', (chunk) => {
size += chunk.length;
// Keep draining without storing, so the 413 response can still be sent.
if (tooLarge || size <= MAX_BODY_BYTES) {
if (!tooLarge) chunks.push(chunk);
return;
}
tooLarge = true;
chunks.length = 0;
fail(Object.assign(new Error('Request too large'), { status: 413 }));
});
req.on('end', () => {
clearTimeout(timer);
resolve(Buffer.concat(chunks));
});
req.on('error', fail);
});
}
+49
View File
@@ -0,0 +1,49 @@
/** The local MCP server: Core's tools over services backed by a running app. */
import { randomBytes } from 'node:crypto';
import { readFileSync } from 'node:fs';
import {
composeCatalog,
coreTools,
catalogForSurface,
} from '../../src/tools/index.js';
import { panelRuntime } from '../../src/app/globePanelRuntime.js';
import { createGlobePanelResource } from '../../src/tools/globePanel.js';
import { createMcpServer } from '../../src/tools/mcp/index.js';
import { DEFAULT_API_BASE, createLocalToolServices } from './services.js';
const INSTRUCTIONS =
"Tools answer questions from God's Eye View's live public data. Location " +
'tools take an area: a place name, a bbox, or lat/lon with radius_km. ' +
'Results are capped; check truncated and total before concluding there is nothing more. ' +
"Answers that can be shown in God's Eye View include data.view; to show one, call " +
'show_in_gods_eye_view with that view, adding layers, style, a camera or marks as ' +
"needed. It shows live God's Eye View where the client displays apps and " +
'returns a link everywhere.';
/** Construct the local MCP server for Core's tools. */
export function createLocalMcpServer({
apiBase = DEFAULT_API_BASE,
fetchImpl,
} = {}) {
const { version } = JSON.parse(
readFileSync(new URL('../../package.json', import.meta.url), 'utf8'),
);
// Each server makes a new key for its panel. The panel's page carries it
// and panel_request requires it; any client may read the page, so it keeps
// the tool from clients that only list it, and is not access control.
const panelKey = randomBytes(32).toString('base64url');
return createMcpServer({
catalog: catalogForSurface(
composeCatalog({
tools: coreTools,
services: createLocalToolServices({ apiBase, fetchImpl, panelKey }),
}),
'mcp',
),
name: 'gods-eye-view',
version,
instructions: INSTRUCTIONS,
resources: [createGlobePanelResource({ runtime: panelRuntime, panelKey })],
});
}
+64
View File
@@ -0,0 +1,64 @@
/**
* Local composition of tool services: Core's default services, pointed at a
* running God's Eye View server's `/api` routes.
*/
import { readFile } from 'node:fs/promises';
import { fileURLToPath } from 'node:url';
import { createToolServices } from '../../src/tools/services.js';
// Bundled data some sources read by module-relative URL.
const BUNDLED_DATA = new URL('../../src/data/local_data/', import.meta.url);
export const DEFAULT_API_BASE = 'http://localhost:4173';
/**
* Resolve the sources' relative `/api/...` requests against `apiBase`, and
* serve `file:` URLs inside the bundled data directory from disk, as a
* browser would load those module-relative assets.
*/
export function createApiFetch({
apiBase = DEFAULT_API_BASE,
fetchImpl = (...args) => globalThis.fetch(...args),
} = {}) {
const base = new URL(apiBase);
if (!['http:', 'https:'].includes(base.protocol))
throw new TypeError(`apiBase must be an http(s) URL: ${apiBase}`);
return (input, init) => {
const url = String(input instanceof Request ? input.url : input);
if (url.startsWith('file:')) return readBundled(url);
return fetchImpl(
typeof input === 'string' && input.startsWith('/')
? new URL(input, base)
: input,
init,
);
};
}
async function readBundled(url) {
const href = new URL(url).href;
if (!href.startsWith(BUNDLED_DATA.href) || href.includes('/../'))
return new Response(null, { status: 404 });
try {
const body = await readFile(fileURLToPath(href));
return new Response(body, {
headers: {
'Content-Type': href.endsWith('.json')
? 'application/json'
: 'application/octet-stream',
},
});
} catch {
return new Response(null, { status: 404 });
}
}
/** Construct every service Core's tools read, backed by the local server. */
export function createLocalToolServices(options = {}) {
return createToolServices({
fetchImpl: createApiFetch(options),
appUrl: options.apiBase ?? DEFAULT_API_BASE,
panelKey: options.panelKey,
});
}
+109
View File
@@ -0,0 +1,109 @@
/**
* Local MCP server over stdio for agents such as Claude Code and Codex.
*
* npm run mcp -- [--api-base http://localhost:4173]
*
* Reads newline-delimited JSON-RPC from stdin and writes responses to stdout.
* Tools read data from a running God's Eye View server (`npm run dev` or
* `npm run preview`) at the API base. Diagnostics go to stderr.
*/
import { createInterface } from 'node:readline';
import { fileURLToPath } from 'node:url';
import { parseErrorResponse } from '../../src/tools/mcp/index.js';
import { createLocalMcpServer } from './server.js';
import { DEFAULT_API_BASE } from './services.js';
/**
* A one-line description of a request for the diagnostic log: its method,
* and the tool or resource it names. Arguments and data are never logged.
*/
export function describeRequest(message) {
const method = typeof message?.method === 'string' ? message.method : null;
if (!method) return null;
const subject =
method === 'tools/call'
? message.params?.name
: method === 'resources/read'
? message.params?.uri
: null;
return typeof subject === 'string' ? `${method} ${subject}` : method;
}
/**
* Serve newline-delimited JSON-RPC until `input` ends. A client's
* `notifications/cancelled` aborts the named request, which then gets no
* response, as the protocol asks.
*/
export async function serveStdio(server, { input, output, log = () => {} }) {
const lines = createInterface({ input, crlfDelay: Infinity });
const pending = new Set();
const running = new Map();
for await (const line of lines) {
if (!line.trim()) continue;
const work = respond(server, line, log, running).then((response) => {
if (response) output.write(`${JSON.stringify(response)}\n`);
});
pending.add(work);
work.finally(() => pending.delete(work));
}
await Promise.all(pending);
}
async function respond(server, line, log, running) {
let message;
try {
message = JSON.parse(line);
} catch {
return parseErrorResponse();
}
if (message?.method === 'notifications/cancelled') {
running.get(message.params?.requestId)?.abort();
return null;
}
const described = describeRequest(message);
if (described) log(described);
const id = message?.id;
const cancellable = id !== undefined && id !== null && !running.has(id);
const controller = new AbortController();
if (cancellable) running.set(id, controller);
let response;
try {
response = await server.handle(message, { signal: controller.signal });
} finally {
if (cancellable) running.delete(id);
}
if (controller.signal.aborted) return null;
// A failed tool call answers the client, not the log; say why here too.
if (response?.result?.isError)
log(` ${described} failed: ${response.result.content?.[0]?.text}`);
return response;
}
/** Read `--api-base <url>` from command-line arguments. */
export function parseArgs(argv) {
const options = {};
for (let index = 0; index < argv.length; index += 1) {
if (argv[index] === '--api-base' && argv[index + 1])
options.apiBase = argv[(index += 1)];
else throw new Error(`Unknown argument: ${argv[index]}`);
}
return options;
}
if (process.argv[1] && fileURLToPath(import.meta.url) === process.argv[1]) {
try {
const options = parseArgs(process.argv.slice(2));
console.error(
`God's Eye View MCP server reading ${options.apiBase || DEFAULT_API_BASE}`,
);
await serveStdio(createLocalMcpServer(options), {
input: process.stdin,
output: process.stdout,
log: (line) => console.error(`<- ${line}`),
});
} catch (error) {
console.error(error.message);
process.exitCode = 1;
}
}
+33 -9
View File
@@ -25,6 +25,36 @@ let _openskyCacheMeta = null;
let _openskyCacheSourceEpochMs = null;
/** TTL for the OpenSky response cache (ms). */
const OPENSKY_CACHE_MS = 9000;
/** Per-attempt limit for the global snapshot; it normally arrives in ~1.5 s. */
const OPENSKY_ATTEMPT_TIMEOUT_MS = 10_000;
const OPENSKY_STATES_URL =
'https://opensky-network.org/api/states/all?extended=1';
/**
* Fetch and read the global snapshot, giving each attempt
* OPENSKY_ATTEMPT_TIMEOUT_MS and retrying once when an attempt times out or
* the connection fails. A second failure throws, which the proxy answers
* from its stale cache or the regional fallback.
* @param {Record<string, string>} headers Request headers.
* @returns {Promise<{upstream: Response, body: string}>}
*/
async function fetchOpenSkyStates(headers) {
for (let attempt = 1; ; attempt++) {
try {
const upstream = await fetch(OPENSKY_STATES_URL, {
headers,
signal: AbortSignal.timeout(OPENSKY_ATTEMPT_TIMEOUT_MS),
});
return { upstream, body: await upstream.text() };
} catch (error) {
if (attempt >= 2) throw error;
console.warn(
'[OpenSky] global snapshot attempt failed, retrying:',
error?.message || error,
);
}
}
}
// --- OpenSky credit governor (field-test fix 2026-07-06) -------------------
// The global /states/all this proxy fetches costs 4 CREDITS per call against
// OpenSky's ~4000/day authenticated budget — a day with the app open burned
@@ -107,6 +137,7 @@ export async function getOpenSkyToken() {
{
method: 'POST',
headers: { 'Content-Type': 'application/x-www-form-urlencoded' },
signal: AbortSignal.timeout(OPENSKY_ATTEMPT_TIMEOUT_MS),
body: `grant_type=client_credentials&client_id=${encodeURIComponent(clientId)}&client_secret=${encodeURIComponent(clientSecret)}`,
},
);
@@ -470,10 +501,7 @@ export function openSkyProxy() {
}
}
let upstream = await fetch(
'https://opensky-network.org/api/states/all?extended=1',
{ headers },
);
let { upstream, body } = await fetchOpenSkyStates(headers);
// Auto-mode fallback: if OAuth was rejected, retry with Basic credentials
if (
(upstream.status === 401 || upstream.status === 403) &&
@@ -485,15 +513,11 @@ export function openSkyProxy() {
Accept: 'application/json',
Authorization: `Basic ${Buffer.from(`${basicUser}:${basicPass}`).toString('base64')}`,
};
upstream = await fetch(
'https://opensky-network.org/api/states/all?extended=1',
{ headers: retryHeaders },
);
({ upstream, body } = await fetchOpenSkyStates(retryHeaders));
usedMode = 'basic';
reason = 'oauth_rejected_fallback_basic';
}
let body = await upstream.text();
const sourceEpochMs = upstream.ok ? openSkySourceEpochMs(body) : null;
if (
upstream.ok &&
+2
View File
@@ -143,8 +143,10 @@ export function cctvProxy({ sourceRoot = process.cwd() } = {}) {
if (url.pathname === '/sources') {
const body = {
trimmedPacks: getCctvSources.trimmedPacks?.() ?? [],
sources: sources.map((source) => ({
id: source.id,
pack: source.pack,
name: source.name,
city: source.city,
cityId: source.cityId,
+22 -2
View File
@@ -3,7 +3,7 @@ import path from 'node:path';
import { DEFAULT_CCTV_SOURCE_FILE, CCTV_SOURCE_CACHE_MS } from './constants.js';
import { allocateSourceCap, resolveCatalogCap } from './cap.js';
import { loadGroundHeights, joinGroundHeights } from './groundHeights.js';
import { normalizeSourceItem } from './normalize.js';
import { cameraRegion, normalizeSourceItem } from './normalize.js';
import {
loadAustinSourcesFromOpenData,
loadCaltransSourcesFromOpenData,
@@ -143,6 +143,9 @@ export function createCctvCatalog({ sourceRoot = process.cwd() } = {}) {
/** @type {Promise<Array<object>>|null} In-flight refresh, shared by concurrent
* callers so a post-TTL burst launches ONE refetch, not one per request. */
let _cctvSourceInflight = null;
/** @type {Array<{pack: string, available: number, served: number}>} Packs
* that serve fewer cameras than they offer. */
let _cctvTrimmedPacks = [];
/**
* Assemble and cache the merged CCTV source list.
@@ -209,8 +212,12 @@ export function createCctvCatalog({ sourceRoot = process.cwd() } = {}) {
name,
sources: items
.filter((item) => item && typeof item === 'object')
.map((item) => normalizeSourceItem(item))
.map((item) => ({ ...normalizeSourceItem(item), pack: name }))
.filter((item) => item.id),
available: Number.isFinite(items.available)
? items.available
: items.length,
region: items.region ?? null,
});
const packs = [
...LIVE_PACKS.map((pack, index) =>
@@ -234,6 +241,16 @@ export function createCctvCatalog({ sourceRoot = process.cwd() } = {}) {
loadGroundHeights(sourceRoot),
);
const trimmed = allocation.packs.filter((pack) => pack.kept < pack.offered);
// Each pack's own nearest-first cap and the shared catalog cap both trim.
const trimmedPacks = packs
.map((pack, index) => ({
pack: pack.name,
available: Math.max(pack.available, allocation.packs[index].offered),
served: allocation.packs[index].kept,
// Where the pack's cameras are, including those not served.
region: pack.region ?? cameraRegion(pack.sources),
}))
.filter((pack) => pack.served < pack.available);
if (trimmed.length) {
const detail = trimmed
.map((pack) => `${pack.name} ${pack.kept}/${pack.offered}`)
@@ -244,6 +261,7 @@ export function createCctvCatalog({ sourceRoot = process.cwd() } = {}) {
}
if (capped.length > 0 || _cctvSourceCache.length === 0) {
_cctvSourceCache = capped;
_cctvTrimmedPacks = trimmedPacks;
} else {
// Every source came back empty (all live packs timed out / upstream outage)
// but a good catalog is already cached — serve it stale rather than blanking
@@ -257,5 +275,7 @@ export function createCctvCatalog({ sourceRoot = process.cwd() } = {}) {
return _cctvSourceCache;
}
/** Packs serving fewer cameras than they offer, as of the last refresh. */
getCctvSources.trimmedPacks = () => _cctvTrimmedPacks;
return getCctvSources;
}
+25 -1
View File
@@ -470,7 +470,31 @@ export function prioritizeSources(cameras, maxCount, anchors) {
return a.idx - b.idx;
});
return scored.slice(0, cap).map((entry) => entry.camera);
const kept = scored.slice(0, cap).map((entry) => entry.camera);
// How many cameras the pack offered before trimming, and where, for
// coverage reports.
Object.defineProperty(kept, 'available', { value: list.length });
Object.defineProperty(kept, 'region', { value: cameraRegion(list) });
return kept;
}
/** The bounding box of cameras with coordinates, or null. */
export function cameraRegion(cameras) {
let region = null;
for (const camera of cameras || []) {
const lat = Number(camera?.lat);
const lon = Number(camera?.lon);
if (!Number.isFinite(lat) || !Number.isFinite(lon)) continue;
region = region
? {
west: Math.min(region.west, lon),
south: Math.min(region.south, lat),
east: Math.max(region.east, lon),
north: Math.max(region.north, lat),
}
: { west: lon, south: lat, east: lon, north: lat };
}
return region;
}
/**
+41 -13
View File
@@ -2,24 +2,52 @@ import { makeRateLimiter } from '../../../src/sources/rateLimit.js';
export { makeRateLimiter } from '../../../src/sources/rateLimit.js';
/**
* Opt-in per-IP rate limiter for the cost-bearing API proxies (OpenAI / Google).
* DEFAULT IS UNLIMITED: when the env var is unset, `0`, or non-numeric, this
* returns `null` and the caller skips the check entirely — a runtime no-op that
* preserves the original behavior. Only a positive integer N enables a fixed
* 60s window of N requests/IP (built lazily once, then reused so its per-IP
* window state persists across requests). The global backstop is set to a
* generous multiple of the per-IP cap so a single host can't starve the rest.
* Resolve the per-IP, per-minute cap for a cost-bearing proxy from its env var.
*
* @param {string|undefined} envValue - Raw env value (requests/min/IP).
* The default is ON. An unset or blank value takes `defaultPerMin`; a positive
* integer overrides it; an explicit `0` disables the limiter for deployments
* that genuinely want no throttle.
*
* A value that cannot be read as a number — a typo, a stray unit, a negative —
* also takes the default rather than disabling the guard. That asymmetry is the
* point: `GEV_RATELIMIT_OPENAI_PER_MIN=3O` should not quietly hand an exposed
* server's API key to whoever asks. Removing the throttle stays possible, but
* only by saying `0`, which cannot be typed by accident.
*
* @param {string|number|undefined|null} envValue - Raw env value (requests/min/IP).
* @param {number} defaultPerMin - Cap to apply when the value is absent or unreadable.
* @returns {number} Requests/min/IP; `0` means unlimited.
*/
export function resolvePerMinuteCap(envValue, defaultPerMin) {
const raw = typeof envValue === 'string' ? envValue.trim() : envValue;
if (raw === undefined || raw === null || raw === '') return defaultPerMin;
const parsed = Number(raw);
if (!Number.isFinite(parsed) || parsed < 0) return defaultPerMin;
// Only an explicit 0 disables the guard: a positive fraction such as 0.5
// would floor to 0, so it takes the strictest real cap instead.
if (parsed > 0 && parsed < 1) return 1;
return Math.floor(parsed); // includes the explicit 0 = unlimited opt-out
}
/**
* Per-IP rate limiter for the cost-bearing API proxies (OpenAI / Google).
* Returns `null` only when the cap resolves to `0` (the explicit opt-out), in
* which case the caller skips the check entirely. Otherwise a fixed 60s window
* of N requests/IP, built lazily once and reused so its per-IP window state
* persists across requests. The global backstop is a generous multiple of the
* per-IP cap so a single host can't starve the rest.
*
* @param {string|number|undefined|null} envValue - Raw env value (requests/min/IP).
* @param {number} defaultPerMin - Cap to apply when the value is absent or unreadable.
* @returns {((key:string)=>boolean)|null} An `allow(key)` fn, or null when unlimited.
*/
export function makeOptInRateLimiter(envValue) {
const max = Number(envValue);
if (!Number.isFinite(max) || max <= 0) return null; // unset/0/garbage -> unlimited
export function makeCostRateLimiter(envValue, defaultPerMin) {
const max = resolvePerMinuteCap(envValue, defaultPerMin);
if (max <= 0) return null; // explicit opt-out
return makeRateLimiter({
windowMs: 60_000,
max: Math.floor(max),
globalMax: Math.floor(max) * 20,
max,
globalMax: max * 20,
});
}
+43
View File
@@ -0,0 +1,43 @@
import { admitSameSiteRequest } from '../../../src/localRequestGate.mjs';
/**
* Cross-site request gate for cost-bearing and log endpoints. Feeds the
* request to the pure, unit-tested `admitSameSiteRequest` and answers 403
* itself when the request is refused. Returns true when it has already
* responded (the caller returns); false when the handler should continue.
* Refuses cross-site browser requests (foreign or opaque Origin, a
* Sec-Fetch-Site other than same-origin/none, or reverse-proxy headers) while
* keeping loopback non-browser tools and the HOST=0.0.0.0 LAN opt-in working.
* @param {import('node:http').IncomingMessage} req
* @param {import('node:http').ServerResponse} res
* @returns {boolean} True when a 403 was sent.
*/
export function admitSameSite(req, res) {
const verdict = admitSameSiteRequest({
hostHeader: req.headers?.host,
protocol: req.socket?.encrypted ? 'https:' : 'http:',
origin: req.headers?.origin,
secFetchSite: req.headers?.['sec-fetch-site'],
proxyHeaders: req.headers || {},
});
if (verdict.ok) return false;
res.statusCode = verdict.status;
res.setHeader('Content-Type', 'application/json');
res.setHeader('Cache-Control', 'no-store');
res.end(JSON.stringify({ error: verdict.error }));
return true;
}
/**
* Wrap a middleware handler so cross-site requests are refused with 403
* before the handler runs. Loopback tools and same-origin browser requests
* pass straight through.
* @param {(req: import('node:http').IncomingMessage, res: import('node:http').ServerResponse, next?: Function) => unknown} handler
* @returns {(req: import('node:http').IncomingMessage, res: import('node:http').ServerResponse, next?: Function) => unknown}
*/
export function sameSiteGated(handler) {
return (req, res, next) => {
if (admitSameSite(req, res)) return undefined;
return handler(req, res, next);
};
}
+8 -5
View File
@@ -7,7 +7,7 @@ import { adsbdbProxy } from './aircraft/enrichment.js';
import { overpassProxy } from './overpass.js';
import { militaryInstallationsProxy } from './military-installations.js';
import { regionalBriefProxy } from './regional/briefing.js';
import { geocodeProxy, NOMINATIM_CACHE_DIR } from './regional/place.js';
import { geocodeProxy } from './regional/place.js';
import { weatherEffectsProxy } from './regional/weather-effects.js';
import { cctvProxy } from './cctv.js';
import { defaultSourceRoot } from './common/source-root.js';
@@ -26,8 +26,11 @@ import { firePerimetersProxy } from './firePerimeters.js';
import { cycloneProxy } from './cyclones.js';
import { windProxy } from './wind.js';
/** Construct the local provider plugins in their established order. */
function localProviderPlugins() {
/**
* Construct the local provider plugins in their established order.
* `realtime` configures the voice session token endpoint.
*/
function localProviderPlugins({ realtime } = {}) {
return [
openSkyProxy(),
celestrakProxy(),
@@ -39,7 +42,7 @@ function localProviderPlugins() {
overpassProxy(),
militaryInstallationsProxy(),
regionalBriefProxy(),
geocodeProxy({ storageDir: NOMINATIM_CACHE_DIR }),
geocodeProxy(),
weatherEffectsProxy(),
cctvProxy({ sourceRoot: defaultSourceRoot }),
radioBrowserProxy(),
@@ -49,7 +52,7 @@ function localProviderPlugins() {
adsbLolProxy(),
aisLiveProxy(),
trackBackfillProxies(),
openAiRealtimeProxy(),
openAiRealtimeProxy({ realtime }),
googlePlacesContextProxy(),
windProxy(),
weatherProxy(),
+8 -3
View File
@@ -2,6 +2,7 @@ import { defaultSourceRoot } from './common/source-root.js';
import { handleHudSummary } from './openai/hud-summary.js';
import { createDebugLogHandler } from './openai/debug-log.js';
import { createRealtimeTokenHandler } from './openai/realtime.js';
import { sameSiteGated } from './common/same-site.js';
/**
* Vite plugin: OpenAI Realtime ephemeral client secret.
@@ -15,16 +16,20 @@ function openAiRealtimeProxy({
realtime = {},
} = {}) {
function install(middlewares) {
middlewares.use('/api/openai/hud-summary', handleHudSummary);
// Cost-bearing and log endpoints refuse cross-site browser requests
// (see server/providers/common/same-site.js and SECURITY.md).
middlewares.use('/api/openai/hud-summary', sameSiteGated(handleHudSummary));
middlewares.use(
'/api/realtime/debug-log',
createDebugLogHandler({ sourceRoot }),
sameSiteGated(createDebugLogHandler({ sourceRoot })),
);
middlewares.use(
'/api/realtime/token',
createRealtimeTokenHandler({ ...realtime, annotationGuidance }),
sameSiteGated(
createRealtimeTokenHandler({ ...realtime, annotationGuidance }),
),
);
}
-4
View File
@@ -16,9 +16,6 @@ const OPENAI_REALTIME_CONTEXT_RETENTION_DEFAULT = 0.5;
const OPENAI_HUD_SUMMARY_MODEL_DEFAULT = 'gpt-5-nano';
/** Input transcription for voice captions; OPENAI_REALTIME_TRANSCRIBE_MODEL=off disables it. */
const OPENAI_REALTIME_TRANSCRIBE_MODEL_DEFAULT = 'gpt-4o-mini-transcribe';
export {
OPENAI_HUD_SUMMARY_MODEL_DEFAULT,
OPENAI_REALTIME_MODEL_MINI_DEFAULT,
@@ -27,5 +24,4 @@ export {
OPENAI_REALTIME_REASONING_DEFAULT,
OPENAI_REALTIME_CONTEXT_TOKENS_DEFAULT,
OPENAI_REALTIME_CONTEXT_RETENTION_DEFAULT,
OPENAI_REALTIME_TRANSCRIBE_MODEL_DEFAULT,
};
+5 -38
View File
@@ -23,40 +23,7 @@ const REALTIME_DEBUG_LOG_MAX_FILE_BYTES = 32 * 1024 * 1024;
*/
const REALTIME_DEBUG_LOG_MAX_PER_MIN = 120;
/**
* Fields that carry what people said or heard: transcripts, typed and spoken
* text, streamed deltas and the span's quoted replies. They are dropped from
* the persisted log unless content logging is explicitly enabled
* (GEV_VOICE_LOG_CONTENT=1), so timing and tool diagnostics never retain
* speech by default.
*/
const VOICE_CONTENT_FIELDS = new Set([
'transcript',
'text',
'delta',
'answer_text',
'preamble_text',
]);
/** Replace spoken or typed content with its length. */
function omitVoiceContent(value, depth = 0) {
if (depth > 12 || value === null || typeof value !== 'object') return value;
if (Array.isArray(value))
return value.map((item) => omitVoiceContent(item, depth + 1));
const output = {};
for (const [key, item] of Object.entries(value)) {
output[key] =
VOICE_CONTENT_FIELDS.has(key) && typeof item === 'string'
? `[omitted ${item.length} chars]`
: omitVoiceContent(item, depth + 1);
}
return output;
}
function createDebugLogHandler({
sourceRoot = defaultSourceRoot,
includeContent = process.env.GEV_VOICE_LOG_CONTENT === '1',
} = {}) {
function createDebugLogHandler({ sourceRoot = defaultSourceRoot } = {}) {
const logDir = path.join(sourceRoot, '.gev-logs');
const logFile = path.join(logDir, 'realtime-conversations.jsonl');
const allow = makeRateLimiter({
@@ -112,12 +79,12 @@ function createDebugLogHandler({
try {
const body = await readRequestBody(req, REALTIME_DEBUG_LOG_MAX_BYTES);
const parsed = JSON.parse(body || '{}');
const record = includeContent ? parsed : omitVoiceContent(parsed);
const record = JSON.parse(body || '{}');
// The server's own timestamp comes last, so a record cannot supply one.
await append(
`${JSON.stringify({
loggedAt: new Date().toISOString(),
...record,
loggedAt: new Date().toISOString(),
})}\n`,
);
res.statusCode = 204;
@@ -137,4 +104,4 @@ function createDebugLogHandler({
};
}
export { createDebugLogHandler, omitVoiceContent };
export { createDebugLogHandler };
+5 -5
View File
@@ -2,7 +2,7 @@ import {
HUD_SUMMARY_INSTRUCTIONS,
keylessHudSummaryResponse,
} from '../../../src/hudSummaryResponse.js';
import { enforceOptInRateLimit, openAiRateLimiter } from './rate-limit.js';
import { enforceRateLimit, openAiRateLimiter } from './rate-limit.js';
import { readRequestBody } from '../common/request.js';
import { OPENAI_HUD_SUMMARY_MODEL_DEFAULT } from './constants.js';
@@ -46,10 +46,10 @@ async function handleHudSummary(req, res) {
return;
}
// Opt-in per-IP throttle (GEV_RATELIMIT_OPENAI_PER_MIN). Keyless HUD
// fallback has no provider cost and resolves above without consuming a
// paid-endpoint quota slot.
if (!enforceOptInRateLimit(openAiRateLimiter(), req, res)) return;
// Per-IP throttle (GEV_RATELIMIT_OPENAI_PER_MIN). On by default; 0 disables.
// Keyless HUD fallback has no provider cost and resolves above without
// consuming a paid-endpoint quota slot.
if (!enforceRateLimit(openAiRateLimiter(), req, res)) return;
try {
const body = await readRequestBody(req, 64 * 1024);
+26 -17
View File
@@ -1,5 +1,5 @@
const DEFAULT_ANNOTATION_GUIDANCE =
'Put related places in one annotate_map call. Set flyTo true only when the user is not already looking at the place (off-screen marks auto-frame, so when unsure leave it false). Do not say you are drawing or annotating — talk about the places while the marks appear. Marks ACCUMULATE AND PERSIST across flights and topics so the user can build up the map: never pass clearPrevious, and call clear_annotations ONLY when the user EXPLICITLY asks to clear or reset the map.';
'Use a single annotate_map call with several annotations when you are describing multiple related places at once. Set flyTo true only when the user is not already looking at the place; if every mark in a call lands off-screen the app auto-frames them, so when unsure leave flyTo false. Do NOT say out loud that you are drawing, highlighting, or annotating — just speak naturally about the places while the marks appear. ANNOTATIONS ACCUMULATE AND PERSIST — keep adding marks as you explore; you can fly around, change topic, and jump between far-apart places and the marks STAY, so the user can build up the map and show people things. Do NOT clear on your own initiative: never pass clearPrevious, and call clear_annotations ONLY when the user EXPLICITLY asks to clear or reset the map.';
function realtimeInstructions(
annotationGuidance = DEFAULT_ANNOTATION_GUIDANCE,
@@ -13,23 +13,27 @@ function realtimeInstructions(
'For requests to open, show, reveal, or focus a menu/panel, call set_panel_open or show_data_layers_menu. "Open Context" means only set_panel_open{panelId:"global-context-panel",open:true}; it does not activate a Context sub-mode. "Open Contacts" means set_context_mode{mode:"contacts"}; that action expands the parent Context panel before activating Contacts.',
'For requests like "show me the datacenter layers", open the data layers menu and focus the matching layer row; do not enable the layer unless the user asks to turn it on.',
'For questions like "what am I looking at?", "what is in view?", "what is this?", "that selected thing", nearby datacenter, dam, cable, ship, or current view contents, call get_entity_context first, then answer from the returned scene/entity context.',
'For "what is this aircraft?", answer with get_entity_context\'s identityLine: callsign, operator, type and route from returned fields only, airport codes verbatim. Other questions about the selection answer from its properties. Never infer them from the callsign.',
'While a camera motion or route flight is active, a bare "stop" means move_camera{motion:stop} — NOT control_scene and NOT stop_tracking (those need explicit words like "stop the scene" / "stop tracking"). If move_camera stop returns stopped:false and an entity is being tracked, call stop_tracking next — the user means "stop whatever is moving". Flying somewhere while tracking stops the tracking.',
'For "what is this aircraft?" answers, read the callsign, operator, registration, type, and route only from get_entity_context selected.properties. Treat route, routeOrigin, and routeDestination as the only authoritative route fields. Every aircraft identity answer MUST explicitly cover operator, type, and route. When a route is present, repeat its endpoint codes exactly; do not expand airport codes into city names. For a missing field say exactly "Operator details are unavailable", "Aircraft type is unavailable", or "Route details are unavailable" as applicable. Never silently omit missing enrichment or infer it from the callsign.',
'While a camera motion or route flight is active, a bare "stop" means move_camera{motion:stop} — NOT control_scene and NOT stop_tracking (those need explicit words like "stop the scene" / "stop tracking"). If move_camera stop returns stopped:false and an entity is being tracked, call stop_tracking next — the user means "stop whatever is moving". Flying somewhere while tracking automatically stops the tracking (the result says so): mention it briefly.',
'For camera-motion requests — "orbit around this", "pan left", "tilt up", "stop moving" — call move_camera. For "fly the route" over a drawn route, call fly_route. Confirm with the RESULTING state ("Orbiting slowly", "Flying the route").',
'analyst_query ANSWERS questions; it never moves the camera or starts tracking. For requests to FOLLOW or TRACK a specific aircraft/ship, call track_entity (get_entity_context first when the target is ambiguous), never analyst_query as the final or only action. For "follow/track the nearest aircraft", first call analyst_query with the aircraft layer(s), sortBy=distance, and limit=1, then call track_entity with the returned aircraft identity in the same turn. The lookup alone does not fulfill a follow/track command.',
'For a request to enable an aircraft layer and SELECT or FIND the nearest/closest aircraft near a named place — for example, "Turn on flights and select the closest aircraft to Austin" — call select_nearest_aircraft once. It atomically turns on the requested aircraft layer first, waits for location arrival, refreshes that layer for the destination viewport, filters out landed/on-ground records, and selects the nearest airborne result. A healthy fallback feed is valid data, never an enable failure. Do not also call fly_to_location, set_layer_visibility, analyst_query, track_entity, set_context_mode, or control_cockpit for the same request. SELECT/FIND never implies Contacts or Cockpit unless the user explicitly asks for either mode.',
'For ANALYTICAL questions about loaded layer data — how many / which / fastest / highest / biggest / nearest / next flights, ships, fires, earthquakes, satellites, launches, storms, transit, bikes, bases, datacenters or dams ("how many flights over Texas", "anything above 40,000 feet") — call analyst_query, not get_entity_context. Always set scope by WHERE: "around here" or "near this" while pointing is kind pointer, never view. Filter on the attribute the user names ("MODIS" → sensor eq MODIS). Say the count with its scope in a few words ("37 loaded flights over Texas") plus one or two examples by name. If the needed layer is off, say so and offer to turn it on. "Which of THOSE…" about the last answer → followUp=true with only the new filter/sort.',
'WHERE, for every tool that takes a place or scope, in this order: (1) a place the user names; (2) explicit pointing — "this/that one/here/around here" while pointer_context shows pointing or a target → "pointer"; (3) the Contacts subject while Contacts is active, or the selection when the user refers to it; (4) the current view. A selected datacenter, dam, fire or cable never silently becomes the centre.',
'FEED STATE: when a count\'s feed is stale or degraded, add that one word ("12 flights in view, stale"); never call it live. If unavailable, give no number: say the feed is unavailable. Fallback feeds are valid data; name them only if asked.',
'COUNTING CONTRACT — what "near" means. (1) While Contacts is ACTIVE, "near / nearby / how many aircraft" means the Contacts window: answer from contactsWindow in the tool result — the exact numbers on the user\'s panel (set_context_mode, analyst_query and get_current_view_state carry it once Contacts settles). For "Open Contacts and tell me how many aircraft are within 250 km", call set_context_mode{mode:"contacts"} first and answer from contactsWindow.aircraft, not from a pre-Contacts analyst query; analyst_query\'s own count measures loaded records and is usually lower — never give it as the window count. The centre follows WHERE; with Contacts active but no subject yet, use the view, never an empty panel. (2) With Contacts OFF, "nearby" means in view; "near <place>" means a radius around that place. (3) EVERY count names its scope in words — "42 in your window", "8 in view", "30 within 250 km of Austin" — never a bare number; analyst_query returns scopeLabel for exactly this. Two numbers with named scopes are not a contradiction; say both if asked. (4) State counts VERBATIM — never estimate, round, or hedge: if a tool returns 46, say 46.',
'For a request to enable an aircraft layer and SELECT or FIND the nearest/closest aircraft near a named place — for example, "Turn on flights and select the closest aircraft to Austin" — call select_nearest_aircraft once. It atomically turns on the requested aircraft layer first, waits for location arrival, refreshes that layer for the destination viewport, filters out landed/on-ground records, and selects the nearest airborne result. A healthy fallback feed is valid data: report the returned feed source briefly, never call it an enable failure. Do not also call fly_to_location, set_layer_visibility, analyst_query, track_entity, set_context_mode, or control_cockpit for the same request. SELECT/FIND never implies Contacts or Cockpit unless the user explicitly asks for either mode.',
'For ANALYTICAL questions about layer data — how many / which / fastest / highest / biggest / nearest flights, ships, fires, earthquakes, satellites, datacenters, or dams ("how many flights over Texas", "biggest fire near LA", "which ships are headed to Oakland", "anything above 40,000 feet") — call analyst_query, not get_entity_context. Narrate the count plus two or three notable examples by name, and reflect the result\'s coverage note honestly: the answer covers data loaded by enabled layers, not the whole world. If the needed layer is disabled, say so and offer to enable it. For follow-ups about the same set ("which of THOSE is closest?"), call analyst_query with followUp=true and only the new filter/sort.',
'WORLD QUERIES — the query tools (get_weather, get_wind, search_places, places_nearby, plan_route, get_cyclones, get_fire_perimeters, get_bike_share, get_transit_vehicles, get_traffic_flow, situation_brief, military_awareness and the other get_* and find_* tools you have) look up data anywhere in the world without changing the map and without any layer being on. Use them when the user asks about a place they are not looking at or data whose layer is off ("what is the weather in Tokyo", "how do I drive from Austin to Dallas", "any wildfires in California"). For aircraft, ships, earthquakes, active fires, datacenters, dams and satellites overhead, use analyst_query over the layers instead, turning the layer on first when the user wants it. Use analyst_query and get_entity_context for what the app has loaded or what is on screen. Answer from the result summary and data, mention its source when it has one, and never claim the map changed. If the user also wants to SEE it, use the matching app action (fly_to_location, set_layer_visibility) as well.',
'FEED STATE CONTRACT — analyst_query and get_current_view_state carry feedProvenance from the same layerFeedState the Data Layers chips use (nominal/loading/degraded/stale/fallback/unavailable, or off). NEVER narrate a count as live or current when feedProvenance.overall is not nominal. Say the feedState in the same breath as the count ("12 flights in view, STALE"). If unavailable, do not invent a count; say the feed is unavailable. Fallback is valid data — name the fallback source, never call it an enable failure. Confirm only what the tool result returned.',
'COUNTING CONTRACT — what "near" means. (1) While Contacts is ACTIVE, "near / nearby / how many aircraft" means the Contacts window: answer from contactsWindow in the tool result — those are the exact numbers on the user\'s panel. set_context_mode, analyst_query, and get_current_view_state carry it after Contacts settles. For "Open Contacts and tell me how many aircraft are within 250 km", call set_context_mode{mode:"contacts"} first and answer from contactsWindow.aircraft; do not answer from a pre-Contacts analyst query. analyst_query\'s own count measures currently-loaded records and is usually lower; never give it as the window count. CENTER PRECEDENCE for a nearby/how-many ask, in order: an explicit place in the question ("over Texas", "near Austin") always wins and ignores Contacts state; else the CONTACTS SUBJECT when Contacts is active and has one — a selected datacenter, dam, fire, or cable does NOT silently become the center; else an entity the user explicitly names ("around this datacenter"); else the current view, said aloud ("nothing is selected, so this is the current view"). With Contacts active but NO subject yet, use the view and say so; never read an empty panel. (2) With Contacts OFF, "nearby" means in view; "near <place>" means a radius around that place. (3) EVERY count names its scope in words — "42 in your window", "8 in view", "about 30 within 250 km of Austin" — never a bare number; analyst_query returns scopeLabel for exactly this. Two different numbers with named scopes are not a contradiction; say both if asked. (4) State counts VERBATIM — never estimate, round, or hedge ("a few", "less than a dozen"): if a tool returns 46, say 46. (5) When it matters, add once: counts cover loaded data, and the flights layer loads where you look.',
'While Cockpit is active, navigate with control_cockpit (next/previous, optionally targetLayer or aircraftClass). track_entity and fly_to_location are REFUSED by design while Cockpit owns the camera — that refusal is correct, not an error to retry. To go somewhere else, exit Cockpit first. control_cockpit enter establishes Contacts itself, so do not call set_context_mode before or after it.',
'When the target layer is unknown, OMIT layerId in track_entity so it searches all enabled layers. Passing the wrong layerId ("flights" for a military contact) returns "Nothing matched" even though the contact is loaded.',
'READING THE VIEW: with no selected object or overlay entities, answer from get_entity_context\'s basemap context (tiles source, centre target, reverse-geocoded place, altitude, style, enabled layers); say there is nothing only when that is unavailable too. Prefer, in order: a nearby knownLandmarks name ("Eiffel Tower"); at local zoom nearbyPlaces, place.labels and viewportPlaces visible and street labels; viewportPlaces dominantCountry/Region/Locality over raw coordinates; the majority of viewportSamples over one reverse-geocoded address. Legible labels in an attached screenshot count too, but at global or regional viewScale never name a precise street or city from one centre pixel.',
'If get_entity_context has no selected object or overlay entities, use its basemap context: Google Photorealistic 3D Tiles/Cesium source, center target coordinates, reverse-geocoded place, camera altitude, active style, and enabled layers. Do not say there is nothing unless the basemap target is also unavailable.',
'If basemap context includes knownLandmarks, prefer the nearest known landmark by name for "what am I looking at" answers. For example, if knownLandmarks includes Eiffel Tower, say Eiffel Tower.',
'At local zoom, use basemap nearbyPlaces, place.labels, viewportPlaces.visibleLabels, and viewportPlaces.streetLabels to identify the building, premises, roads, and named places visible around the screen target.',
'If basemap context includes viewportPlaces, prefer dominantCountry, dominantRegion, and dominantLocality over raw coordinates.',
'When basemap context includes viewportSamples or an inferred country, trust that over a single reverse-geocoded address. If most samples indicate Iran, say Iran, not the United States.',
'When a viewport screenshot is attached after get_entity_context, read clearly legible street, building, and place labels from it and combine them with structured label context. Respect scene viewScale: at global/continental/regional scale, avoid naming a precise street/city from one center pixel.',
'Do not mention disabled layers or stale selections.',
'POINTING: pointer_context says what the cursor is on (pointing:<kind>: resolved by the tools; target "none": not pointing). For "this/that/here/that one" with no name, pass "pointer": get_entity_context scope, track_entity/fly_to_location query, analyst_query scope.kind ("around here" is pointer, not view), annotate_map target. "The second/last one" is referent:n from the last numbered list. Without a pointer, ask or follow WHERE.',
'PREAMBLES: for instant commands (layers, styles, HUD, detection, zoom, panels, basemap, radio controls, camera nudges, and lookups such as get_entity_context) call the tool without speaking first. Only for slow or multi-step work (annotate_map, a searched fly_to_location, select_nearest_aircraft, analyst_query over a named place, or two or more tools) you may first say ONE sentence of at most 12 words naming the actual steps ("Finding the Capitol, then outlining its grounds."), then call the tools at once in the same response.',
'When one request contains MULTIPLE changes, call ALL the corresponding tools before the confirmation. Never confirm a partial subset; if one fails, say which.',
'AFTER TOOLS: speak the result\'s say line once, lightly rephrased at most, never adding to it and never repeating your preamble; if say is null, confirm briefly or continue. Sources, notes and caveats are on the voice card: speak a caveat only when it changes the answer, or when asked why, how sure or which source. Lower bounds ("at least 500") and partial answers are part of the answer — always say them. No filler or hedges ("let me check", "keep in mind", "it seems", "based on the data available"), and never add "approximately" to an exact tool number.',
'When a request requires a tool call, do not speak in the same response as the tool call. Call the tool first.',
'When a single user request contains MULTIPLE changes (e.g. "switch to operator layout, use balanced detection at density 50, and switch to Bing aerial"), call ALL the corresponding tools — multiple tool calls in sequence — before speaking. Never confirm a partial subset. If a later tool fails, say which parts succeeded and which failed.',
'After receiving tool output, speak exactly one short confirmation. Do not repeat the confirmation.',
'For "show/open/turn on" layer requests, enable the matching layer. For "hide/close/turn off", disable it.',
// INSTRUCTION-ONLY mapping for the two globe-scale named views.
//
@@ -45,23 +49,28 @@ function realtimeInstructions(
// string is the whole rollback.
'NAMED VIEWS are shorthand for tool calls you already have — there is no "mode" tool for them. Treat ONLY these as the shorthand: "infrastructure mode" / "the infrastructure view" / "show me global infrastructure" means three set_layer_visibility calls (local-datacenters, local-dams, telegeography-submarine-cables) plus zoom_to_globe; "environmental mode" / "earth watch" / "active events", said as the name of a view, means set_layer_visibility for local-firms and earthquakes plus zoom_to_globe. Anything vaguer is NOT this shorthand — an open-ended question about the world or the news is an ordinary question: answer it, or use analyst_query over the layers already on. Never switch a whole view on to answer a question nobody asked to see. When you do run one, make every call before speaking, then give one confirmation naming the resulting state; if the fires layer comes back unavailable because no FIRMS key is configured, say so plainly — the earthquakes still loaded. "Live contacts" and "space missions" are NOT this pattern: they stay set_context_mode{mode:"contacts"} and set_context_mode{mode:"space-missions"}.',
'For visual filter requests, call set_visual_style with one of the allowed style IDs.',
'Disambiguation table — basemap vs layer vs style: basemap switching requires an explicit stack name — "Bing aerial" means set_map_stack bing-aerial, "aerial with labels" means bing-labels, "OSM"/"road map" means osm, "Esri"/"Esri imagery" means esri-imagery, "Google 3D"/"photorealistic" means photoreal. Any other mention of "satellite" or "satellites" means the satellites DATA LAYER via set_layer_visibility, never a basemap — except "recent/latest satellite imagery" (the recent-imagery layer) and "weather satellite"/clouds (weather-satellite). "surveillance"/"night vision"/"thermal" are visual STYLES via set_visual_style.',
'SHIPS vs STYLES: "vessels" is easily heard as "visuals". "Show me vessels", "turn on vessels", "the vessels", "visuals" or "vestibules" said WITHOUT a style name means the ships layer: set_layer_visibility{layerId:"ais-live-vessels"}, exactly as "ships" or "boats" does. Call set_visual_style only when the user names a style (retro/CRT, surveillance/night vision, thermal, anime, noir, snow, normal) or says filter, effect or look. Never turn on a different data layer (such as fires) for a ships request.',
'Disambiguation table — basemap vs layer vs style: basemap switching requires an explicit stack name — "Bing aerial" means set_map_stack bing-aerial, "aerial with labels" means bing-labels, "OSM"/"road map" means osm, "Esri"/"Esri imagery" means esri-imagery, "Google 3D"/"photorealistic" means photoreal. Any mention of "satellite" or "satellites" ALWAYS means the satellites DATA LAYER via set_layer_visibility, never a basemap. "surveillance"/"night vision"/"thermal" are visual STYLES via set_visual_style.',
'HUD requests ("hud on/off", "switch to operator/minimal/tactical/cyber layout") use set_hud. Detection requests ("detection on", "dense mode", "balanced mode", "sparse mode", "set density to 25", "use weighted allocation") use set_detection. Density snaps to 0/25/50/75/100 and derives Sparse/Balanced/Dense; panoptic is a legacy alias for Dense.',
'Cyber sonar requests use set_cyber_sonar: enabled, rings, rangePct, intensityPct (Power), opacityPct, sectorDeg. For settings questions read get_current_view_state.controls.hud.sonar. Sonar is not detection. Do not silently switch themes, reveal the HUD or enable sonar when changing a slider. If the user explicitly requests Cyber plus sonar settings, call set_hud first, wait for success, then set_cyber_sonar. Confirm returned values; mapSweepActive and contactSweepActive distinguish saved settings from active map effects, including in Cockpit or reduced motion. Do not claim visible contacts solely because enabled is true.',
'Bloom/sharpen requests use set_post_processing. Scene requests ("play orbital watch", "stop the scene", "what scenes are there") use control_scene. CCTV camera requests ("next camera", "nearest camera", "select the Congress camera", "show coverage") use control_cctv — the CCTV layer must be enabled first.',
'Radio playback requests use control_radio. "Turn on/start the radio" means action=play; action=enable only reveals Radio markers and must be reserved for explicit "show/enable the Radio layer/markers" requests. After a prepared playback result, briefly confirm any other completed actions and say "Turning on the radio"—never claim it is already playing. The client keeps Radio muted until playback is verified, then closes voice before restoring Radio volume. Examples: "play news near Austin" → select category=news locationId=austin; "play US news" → select category=news country=US; "Radio volume 30" → volume; pause/resume/stop/next/previous use the matching action. Radio selection never moves the camera.',
'"Track/follow <something specific>" (a callsign, ship name, satellite name) uses track_entity. "Take me to the biggest fire" uses track_entity with query "biggest fire" (the fires layer must be enabled). Bare "orbit" means camera orbit of the current landmark. "Stop following/tracking" uses stop_tracking.',
'"Show me which planes are overhead"/"frame the ships"/"show me the satellites above" use frame_overhead with the matching target.',
"After frame_overhead, speak ONLY from the tool result's count field — e.g. 'Framed fourteen aircraft, labels on'; never reassess or second-guess the count aloud.",
'Confirmations echo the RESULTING state, never the request: "HUD operator layout", "Density twenty-five percent", "Bing aerial imagery", "Tracking UAL428", "Framed fourteen aircraft". On ok=false, state the failure plainly: "Nothing matched UAL999", "No ships within 120 kilometers". Never claim an action without ok=true in the tool result.',
'For destination requests such as "take me to Italy", "go to NYC", or "show me the Eiffel Tower", call fly_to_location. Prefer known city IDs when available; otherwise pass the plain place query.',
'Navigation-only requests ("take me to X", "go to X", "fly to X") are NOT descriptions: call fly_to_location alone and do NOT also call annotate_map, unless the user explicitly asks to mark the place or you go on to explain specific places there. Never drop a point pin on a region-scale natural feature (a mountain range, desert, sea, or forest) — a single point in the middle of the Rockies is meaningless. If the user explicitly asks to mark such a region, prefer type=area.',
'For country and city destinations, omit rangeM so GEV frames the whole country or city in view. For landmarks and buildings, omit rangeM so GEV chooses a close landmark view.',
'Only supply rangeM when the user asks for a particular numeric height, distance, closer view, or wider view.',
'For relative requests such as "zoom out a little", "pull back", "zoom in more", or "get closer", always call adjust_camera_zoom. But "globe view", "whole earth", "the whole planet", or "zoom all the way out" is an ABSOLUTE framing: call zoom_to_globe once instead — repeated adjust_camera_zoom calls can never reach the globe. Never claim the camera moved without the tool returning ok=true.',
'WHITEBOARD THE WORLD: when you describe a specific place, building, campus, district or boundary, or how places relate, call annotate_map as you talk. type=area traces a building, campus, compound, park, district or boundary outline; pin marks a spot; highlight pulses a spot with no footprint; arrow joins two places ("it sits next to the Marina"). Use place names; screenX/screenY only for a spot you can see in the latest screenshot but cannot name; never invent coordinates. Set entityKind whenever you know what the target is.',
'Keep spoken confirmations short, e.g. "Opening datacenters" or "Flying to London".',
'WHITEBOARD THE WORLD: whenever you describe or explain a specific place, building, campus, district, boundary, or a spatial relationship between places, call annotate_map to mark it visually as you talk — like sketching on the map. To call out a specific building, campus, compound, park, or district, use type=area (it traces and encloses the real footprint — a building gets a glowing volume, a district gets a draped outline). Use type=highlight only for a transient pulse on a precise spot that has no meaningful footprint, and type=pin to drop a labeled marker. Examples: "what is the Palace of Fine Arts?" → an AREA on it; "the old military base next to it" → an AREA on the Presidio; "ILM is right here" → a pin; "it sits next to the Marina" → an arrow from one to the other. Prefer place NAMES so the app resolves real positions and outlines; never invent coordinates or pixel locations.',
'On every annotation, also set entityKind to what the thing IS when you know it: building (one structure), compound (campus/grounds/mall/park), district (neighborhood/area of a city), street (a named road), or point_feature (a monument, statue, memorial, plaque, fountain, or other small point landmark). entityKind is a FACT about the target, independent of the mark type you chose — monuments and statues are point_feature even when you use type=area; the app then anchors them as precise points instead of guessing at a footprint.',
annotationGuidance,
'annotate_map results: work any say line in naturally (places not found, a straight-line fallback with no travel time, or a full map — then ask before clearing). outlinePending means the boundary is still being traced: never claim it is drawn. A later map_annotation_outline system item reports each outcome; use it to quietly confirm or correct yourself.',
'ROUTES vs DISTANCES: "walking/driving route from A to B" → type=route with the ordered points and the mode; the app draws the street path and reports distance and time you can read aloud. "How far is X from Y" or "X is next to Y" → type=arrow (straight-line distance), never route.',
'If an annotate_map result has partial:true or any failedLabels, do not pretend those places appeared — briefly work into your narration that you could not pinpoint them (e.g. "I couldn\'t place X"). If a route comes back as a direct line (no street route was found), describe it as a straight-line distance, not a walking/driving time. If an annotate_map result has capped:true, the map is full — ASK the user whether to clear before drawing more; do not clear unprompted. outlinePending:true is NOT a failure, but it is also NOT an outline: the anchor mark is placed and the boundary is still being traced in the background. Narrate it in progress — e.g. "tracing the boundary now" — and NEVER state the outline is already drawn or visible; it may yet come back as just a point. A later system item of type map_annotation_outline reports the final outcome per mark (status resolved or failed, with its label): use it to quietly confirm, or to correct yourself if you implied a boundary that stayed a point — an honest miss beats a misleading guess.',
'PREFER NAMES. Only when you cannot name or geocode a place but you can clearly SEE the exact spot in the most recent viewport screenshot, fall back to screenX/screenY (normalized 0..1 from that image) to point at it; the app converts the pixel to a real world point. Never use screenX/screenY for something you could name.',
'PATHS vs DISTANCES: for "walking/driving route from A to B" (or through several stops), use type=route with the ordered points and the matching mode (walking/driving/cycling) — the app draws the real street-following path on the map and reports distance and travel time, which you can read aloud. For "how far is X from Y", "is it nearby", or "X is next to Y", use type=arrow between the two — it draws a floating connector and shows the straight-line distance. Do NOT use route for a simple distance/proximity question.',
].join('\n');
}
+24 -11
View File
@@ -1,33 +1,46 @@
import { makeOptInRateLimiter, clientKey } from '../common/rate-limit.js';
import { makeCostRateLimiter, clientKey } from '../common/rate-limit.js';
/**
* Requests/min/IP applied when GEV_RATELIMIT_OPENAI_PER_MIN is unset. Both
* places that already name a value agree on it: `.env.example` recommends 30
* and the Pinokio build ships 30 (pinokio/_ENVIRONMENT), so the packaged app
* is unaffected and only an unconfigured server changes. The HUD asks for a
* summary every 15s (HUD_SUMMARY_INTERVAL_MS), so an open tab spends 4/min and
* a voice session adds one token mint; the cap clears ordinary use several
* times over and bites only a caller working the endpoint far harder than the
* app does.
*/
const OPENAI_DEFAULT_PER_MIN = 30;
// Built LAZILY on first request, NOT at module load: `.env` values are applied to process.env later
// (the plugin config hook calls loadEnv → process.env, AFTER this module is imported), so reading
// process.env here at import time would always see them unset and silently stay unlimited even when
// process.env here at import time would always see them unset and fall back to the default even when
// configured via .env. Building on first request (like the OPENAI_API_KEY reads) sees the loaded env;
// the result is cached so the limiter's per-IP window state persists. `null` = unlimited (default).
// the result is cached so the limiter's per-IP window state persists. `null` = the explicit 0 opt-out.
let _openAiRateLimiter;
/** OpenAI cost endpoints (realtime/token + hud-summary). Null = unlimited (default). */
/** OpenAI cost endpoints (realtime/token + hud-summary). Null only when set to 0. */
function openAiRateLimiter() {
if (_openAiRateLimiter === undefined)
_openAiRateLimiter = makeOptInRateLimiter(
_openAiRateLimiter = makeCostRateLimiter(
process.env.GEV_RATELIMIT_OPENAI_PER_MIN,
OPENAI_DEFAULT_PER_MIN,
);
return _openAiRateLimiter;
}
/**
* Apply an opt-in limiter to a request, writing a 429 when over the cap.
* When `limiter` is null (unlimited, the default) this is a no-op returning
* `true`, so the handler proceeds exactly as before.
* Apply the limiter to a request, writing a 429 when over the cap.
* When `limiter` is null — the explicit `0` opt-out — this is a no-op
* returning `true`, so the handler proceeds unthrottled.
*
* @param {((key:string)=>boolean)|null} limiter
* @param {import('http').IncomingMessage} req
* @param {import('http').ServerResponse} res
* @returns {boolean} True if the request may proceed; false if a 429 was sent.
*/
function enforceOptInRateLimit(limiter, req, res) {
if (!limiter) return true; // unlimited (default) — no behavior change
function enforceRateLimit(limiter, req, res) {
if (!limiter) return true; // explicit 0 opt-out — unthrottled
if (limiter(clientKey(req))) return true;
res.statusCode = 429;
res.setHeader('Content-Type', 'application/json');
@@ -36,4 +49,4 @@ function enforceOptInRateLimit(limiter, req, res) {
return false;
}
export { enforceOptInRateLimit, openAiRateLimiter };
export { enforceRateLimit, openAiRateLimiter, OPENAI_DEFAULT_PER_MIN };
+5 -19
View File
@@ -1,4 +1,4 @@
import { enforceOptInRateLimit, openAiRateLimiter } from './rate-limit.js';
import { enforceRateLimit, openAiRateLimiter } from './rate-limit.js';
import {
resolveVoiceModel,
isKnownVoiceTier,
@@ -10,7 +10,6 @@ import {
OPENAI_REALTIME_REASONING_DEFAULT,
OPENAI_REALTIME_CONTEXT_TOKENS_DEFAULT,
OPENAI_REALTIME_CONTEXT_RETENTION_DEFAULT,
OPENAI_REALTIME_TRANSCRIBE_MODEL_DEFAULT,
} from './constants.js';
import { realtimeInstructions } from './instructions.js';
import { GEV_REALTIME_TOOLS } from './tools.js';
@@ -21,6 +20,7 @@ function createRealtimeTokenHandler({
fetchImpl = (...args) => fetch(...args),
resolveApiKey = () => process.env.OPENAI_API_KEY,
models = {},
tools = GEV_REALTIME_TOOLS,
} = {}) {
return async (req, res) => {
res.setHeader('Cache-Control', 'no-store');
@@ -31,8 +31,8 @@ function createRealtimeTokenHandler({
return;
}
// Opt-in per-IP throttle (GEV_RATELIMIT_OPENAI_PER_MIN). No-op when unset.
if (!enforceOptInRateLimit(openAiRateLimiter(), req, res)) return;
// Per-IP throttle (GEV_RATELIMIT_OPENAI_PER_MIN). On by default; 0 disables.
if (!enforceRateLimit(openAiRateLimiter(), req, res)) return;
const apiKey = resolveApiKey();
if (!apiKey) {
@@ -89,14 +89,6 @@ function createRealtimeTokenHandler({
OPENAI_REALTIME_CONTEXT_RETENTION_DEFAULT,
),
);
// User captions for the voice card. Billed separately from the session.
const transcribeModel =
process.env.OPENAI_REALTIME_TRANSCRIBE_MODEL ||
OPENAI_REALTIME_TRANSCRIBE_MODEL_DEFAULT;
const transcription =
transcribeModel.toLowerCase() === 'off'
? {}
: { transcription: { model: transcribeModel } };
const sessionConfig = {
session: {
type: 'realtime',
@@ -112,7 +104,6 @@ function createRealtimeTokenHandler({
audio: {
input: {
noise_reduction: { type: 'near_field' },
...transcription,
turn_detection: {
type: 'semantic_vad',
eagerness: 'low',
@@ -123,7 +114,7 @@ function createRealtimeTokenHandler({
output: { voice },
},
instructions: realtimeInstructions(annotationGuidance),
tools: GEV_REALTIME_TOOLS,
tools,
tool_choice: 'auto',
},
};
@@ -148,11 +139,6 @@ function createRealtimeTokenHandler({
// case where a bogus ?tier= was silently downgraded to standard.
res.setHeader('X-GEV-Voice-Tier', tier);
res.setHeader('X-GEV-Voice-Model', model);
// Captions are billed separately; the client meters them with this id.
res.setHeader(
'X-GEV-Voice-Transcribe-Model',
transcription.transcription ? transcribeModel : 'off',
);
if (requestedTier && !isKnownVoiceTier(requestedTier)) {
res.setHeader('X-GEV-Voice-Tier-Fallback', '1');
}
+26 -41
View File
@@ -1,8 +1,3 @@
import {
voiceLayerAliasHint,
voiceQueryFieldHint,
} from '../../../src/voice/layerManifest.js';
export const ACTION_DESCRIPTIONS = {
fly_to_location: {
description:
@@ -17,11 +12,7 @@ export const ACTION_DESCRIPTIONS = {
},
query: {
description:
'Plain place search query, e.g. "London", "Eiffel Tower", or "Dubai Marina". "pointer" = the spot under the cursor.',
$position: 1,
},
referent: {
description: 'Numbered item n from the last result list.',
'Plain place search query, e.g. "London", "Eiffel Tower", or "Dubai Marina".',
$position: 1,
},
viewMode: {
@@ -85,13 +76,13 @@ export const ACTION_DESCRIPTIONS = {
$position: 1,
},
set_layer_visibility: {
description:
"Enable or disable one God's Eye View data layer. Every shipped layer is in the enum; never tell the user a listed layer does not exist.",
description: "Enable or disable one registered God's Eye View data layer.",
$position: 1,
parameters: {
properties: {
layerId: {
description: `Spoken names: ${voiceLayerAliasHint()}.`,
description:
'Common-name mapping for the non-obvious ids: space mission(s) → rocket-launches; fires/wildfires/active fires → local-firms (NASA FIRMS); ships/vessels/boats → ais-live-vessels; undersea/submarine cables → telegeography-submarine-cables; datacenters → local-datacenters; dams → local-dams; bikes/bike share → bikeshare; street traffic/congestion → traffic; traffic cameras → cctv; internet radio/stations → radio; ALPR/license plate readers/Flock cameras → alpr-cameras; local ADS-B/my receiver/my antenna (aircraft heard by a local RTL-SDR receiver) → local-adsb.',
$position: 1,
},
},
@@ -163,13 +154,9 @@ export const ACTION_DESCRIPTIONS = {
properties: {
scope: {
description:
'Use auto by default. selected returns the clicked/selected entity; in_view returns visible entities near the screen center; pointer returns what the cursor is on.',
'Use auto by default. selected returns the clicked/selected entity; in_view returns visible entities near the screen center.',
$position: 2,
},
referent: {
description: 'Numbered item n from the last result list.',
$position: 1,
},
layerId: {
description: 'Optional layer filter for visible entity context.',
$position: 2,
@@ -362,17 +349,12 @@ export const ACTION_DESCRIPTIONS = {
properties: {
query: {
description:
'Callsign, ship name, satellite name, ICAO hex, MMSI, or NORAD id. "pointer" = the contact under the cursor.',
$position: 1,
},
referent: {
description:
'Numbered item n from the last result list (-1 = last); query may then be its label.',
'Callsign, ship name, satellite name, ICAO hex, MMSI, or NORAD id.',
$position: 1,
},
layerId: {
description:
'Optional: flights, military, ais-live-vessels or satellites. Other layers are not trackable; for fires use query "biggest fire".',
'Optional layer hint: flights | military | ais-live-vessels | satellites.',
$position: 1,
},
},
@@ -399,7 +381,7 @@ export const ACTION_DESCRIPTIONS = {
},
annotate_map: {
description:
'Mark places on the 3D map as you talk about them (see WHITEBOARD): pins, highlights, footprints and boundaries, arrows, routes and labels. Give place names (or explicit lat/lng); the app resolves real positions and outlines. Several places may go in one call.',
'Draw annotations on the 3D map to visually point out what you are talking about — like sketching on a whiteboard over the world. Use this whenever you mention a specific place, building, campus, boundary, district, or a relationship between two places, so the user can SEE what you mean. Give place NAMES (preferred) or explicit lat/lng; the app resolves them to real-world positions and real building/area outlines — never guess pixel positions. Call this as you begin describing something, and you may mark several places in one call.',
$position: 1,
parameters: {
properties: {
@@ -416,7 +398,7 @@ export const ACTION_DESCRIPTIONS = {
},
target: {
description:
'Place name to resolve ("Palace of Fine Arts, San Francisco"), or "pointer" for the spot under the cursor. A monument inside a larger landmark gets its own name + city ("Tejano Monument, Austin"), never "X at the Texas State Capitol" (that stacks them on one centroid).',
'Place name to resolve, e.g. "Palace of Fine Arts, San Francisco", "the Pentagon", "Presidio of San Francisco". Preferred over coordinates. For a specific monument/statue/feature that sits within a larger landmark, use its OWN name + city ("Tejano Monument, Austin", "Texas African American History Memorial, Austin") — do NOT phrase it as "X at the Texas State Capitol", which makes the geocoder collapse several of them onto the same centroid so they stack on one spot.',
$position: 2,
},
points: {
@@ -468,24 +450,27 @@ export const ACTION_DESCRIPTIONS = {
},
entityKind: {
description:
'What the target IS (a fact, not a style): building = one structure; compound = campus/grounds/mall/park; district = neighborhood; street = a named road; point_feature = monument/statue/memorial/fountain, anchored as a precise point. Set it whenever you know it.',
'What KIND of thing the target IS — a fact, not a style choice: building = one structure; compound = campus/grounds/mall/park; district = neighborhood or area of a city; street = a named road/corridor; point_feature = monument/statue/memorial/plaque/fountain or other small point landmark. Set it whenever you know it — it routes the resolver to the right footprint source (point_feature anchors monuments as precise points instead of adopting a nearby building outline).',
$position: 2,
},
screenX: {
description:
'Fallback only, for a spot visible in the latest screenshot that you cannot name: normalized x (0=left, 1=right).',
'Fallback only: when you cannot name/geocode the place but can SEE it in the latest viewport screenshot, the normalized horizontal position (0=left, 1=right) of the spot. The app converts it back to a real world point under that pixel.',
$position: 3,
},
screenY: {
description: 'Fallback only: normalized y (0=top, 1=bottom).',
description:
'Fallback only: normalized vertical position (0=top, 1=bottom) of the spot in the latest viewport screenshot.',
$position: 3,
},
toScreenX: {
description: 'For type=arrow: destination x (pixel fallback).',
description:
'For type=arrow: normalized x of the arrow destination from the screenshot (pixel fallback).',
$position: 3,
},
toScreenY: {
description: 'For type=arrow: destination y (pixel fallback).',
description:
'For type=arrow: normalized y of the arrow destination from the screenshot (pixel fallback).',
$position: 3,
},
},
@@ -543,18 +528,18 @@ export const ACTION_DESCRIPTIONS = {
},
analyst_query: {
description:
'Counts, lists, superlatives and attribute filters over records loaded by ENABLED layers — "how many flights over Texas", "biggest fire near LA", "ships headed to Oakland", "next rocket launch". complete:false means the count is a floor ("at least") and rankings cover the records examined; partial names unanswered layers. Items carry lat/lon for a follow-up fly_to_location. ok:false codes: LAYER_OFF (offer to enable), NOT_READY (loading), FOLLOW_UP_MISMATCH (ask again without followUp), UNKNOWN_FIELD/UNKNOWN_SCOPE/BAD_VALUE (retry with the allowed values).',
'Answer questions ABOUT the data currently loaded on the map — counts, lists, superlatives, and attribute filters over live layers (flights, military, ships, fires, earthquakes, satellites, datacenters, dams). Examples: "how many flights over Texas", "biggest fire near LA", "which ships are headed to Oakland", "anything above 40,000 feet", "fastest thing in view". For satellites and infrastructure, counts and ranks cover only bounded examined loaded records; omitted records can change nearest/count. Queries ONLY client-side data from ENABLED layers — if the needed layer is off, say so and offer to enable it. For a follow-up about the previous answer\'s set ("which of those is closest?"), set followUp=true and send only the new filters/sort.',
$position: 1,
parameters: {
properties: {
layers: {
description:
'Layers to query. fires → local-firms; ships → ais-live-vessels; buses/trains → transit; hurricanes → weather-cyclones; bases → military-installations; my receiver → local-adsb. Units differ per layer, so rank speed within one layer.',
'Layers to query. fires/wildfires → local-firms; ships/vessels → ais-live-vessels.',
$position: 2,
},
scope: {
description:
'Always set it (see WHERE). view = a radius around the view centre ("in view", "nearby"); pointer = "around here/this" while pointing (radius at the cursor); region = "over Texas"; radius with center {lat, lon} = "near <place>"; anywhere = no place ("biggest anywhere", "today"). There is no drawn-area scope yet: say so rather than answering for the view.',
'Spatial scope. Default: view (near the camera). Use kind=region for "over Texas"-style asks; kind=anywhere for global questions.',
$position: 2,
properties: {
name: {
@@ -563,23 +548,23 @@ export const ACTION_DESCRIPTIONS = {
$position: 1,
},
km: {
description: 'For kind=radius or pointer.',
description: 'For kind=radius.',
$position: 1,
},
},
},
filters: {
description: `ANDed; values keep their type (numbers, true/false). Altitude is meters (40,000 ft = 12192). Time fields take ISO; ageHours = hours ago. Every layer has id, lat, lon. Fields — ${voiceQueryFieldHint()}.`,
description:
'Attribute predicates, ANDed. ALTITUDE IS METERS (40,000 ft = 12192). Fields: altitudeM, speedMps, military, onGround, aircraftClass, callsign, operator, routeOrigin, routeDestination, originCountry (flights); speedKts, shipType, destination (ships); frp, confidence (fires); magnitude, depthKm, place (earthquakes).',
$position: 1,
},
sortBy: {
description:
'Field to rank by, or "distance" (nearest first unless sortDir=desc).',
description: 'Field to rank by, or "distance" for nearest-first.',
$position: 1,
},
followUp: {
description:
'true = re-filter the last answer (its layers) instead of fresh data.',
'true = re-query the PREVIOUS result set instead of fresh data.',
$position: 1,
},
},
@@ -587,7 +572,7 @@ export const ACTION_DESCRIPTIONS = {
},
next_iss_pass: {
description:
'Next ISS pass over the camera location (or lat/lon): rise time, direction, peak elevation, duration and estimated visibility. Needs the satellites layer to have loaded once; if not, ask the user to enable it.',
"When the user asks when the ISS / the space station will next fly over: returns the next geometric ISS pass with estimated visibility for the current camera location (or an explicit lat/lon) — rise time (ISO + minutes from now), rise compass direction, peak elevation, and duration. Requires the satellites layer to have loaded its catalog at least once this session; if it hasn't, tell the user to enable the satellites layer and try again.",
$position: 1,
parameters: {
properties: {
+12
View File
@@ -2,3 +2,15 @@ import { createActionTools } from '../../../src/voice/actionSchemas.js';
import { ACTION_DESCRIPTIONS } from './toolDescriptions.js';
export const GEV_REALTIME_TOOLS = createActionTools(ACTION_DESCRIPTIONS);
/**
* The session's tools: the app actions, then additional function tools
* whose names no action already uses.
*/
export function realtimeSessionTools(additional = []) {
const names = new Set(GEV_REALTIME_TOOLS.map((tool) => tool.name));
return [
...GEV_REALTIME_TOOLS,
...additional.filter((tool) => !names.has(tool.name)),
];
}
+1 -1
View File
@@ -7,7 +7,7 @@ export {
export { installRouteMiddleware } from './places/routes.js';
export {
makeRateLimiter,
makeOptInRateLimiter,
makeCostRateLimiter,
clientKey,
} from './common/rate-limit.js';
export { haversineKm } from './common/geo.js';
+25 -6
View File
@@ -2,21 +2,35 @@ import {
googleServerApiKey,
keylessGooglePlacesResponse,
} from './google-key.js';
import { makeOptInRateLimiter, clientKey } from '../common/rate-limit.js';
import { makeCostRateLimiter, clientKey } from '../common/rate-limit.js';
import { admitSameSite } from '../common/same-site.js';
import {
projectNearbyPlaces,
projectTextSearchPlaces,
} from '../../../src/data/placeProviderPayloads.js';
// Construct lazily after the standalone environment has loaded.
// undefined = not built yet; null = unlimited; fn = active limiter
// undefined = not built yet; null = the explicit 0 opt-out; fn = active limiter
let _googleRateLimiter;
/** Google cost endpoint (nearby-places). Null = unlimited (default). */
/**
* Requests/min/IP applied when GEV_RATELIMIT_GOOGLE_PER_MIN is unset. This is
* the value the Pinokio build already ships (pinokio/_ENVIRONMENT), so the
* packaged app keeps behaving exactly as it does today and only an
* unconfigured server changes — from unlimited to what the product already
* runs with. Places calls are user-driven (a search, a nearby lookup, an
* installation probe) rather than polled, so this sits far above what the app
* generates and bites only a caller enumerating the endpoint. `.env.example`
* suggests a tighter 60 when the host is not localhost; that still applies.
*/
export const GOOGLE_DEFAULT_PER_MIN = 120;
/** Google cost endpoints (nearby-places + text-search). Null only when set to 0. */
function googleRateLimiter() {
if (_googleRateLimiter === undefined)
_googleRateLimiter = makeOptInRateLimiter(
_googleRateLimiter = makeCostRateLimiter(
process.env.GEV_RATELIMIT_GOOGLE_PER_MIN,
GOOGLE_DEFAULT_PER_MIN,
);
return _googleRateLimiter;
}
@@ -50,6 +64,9 @@ export function googlePlacesContextProxy({
} = {}) {
function install(middlewares) {
middlewares.use('/api/google/nearby-places', async (req, res) => {
// Gate first, like the OpenAI routes: a cross-site caller learns nothing
// about this endpoint's method surface.
if (admitSameSite(req, res)) return;
if (req.method !== 'GET') {
res.statusCode = 405;
res.setHeader('Content-Type', 'application/json');
@@ -80,7 +97,7 @@ export function googlePlacesContextProxy({
}
const { latitude, longitude } = coordinates;
// Opt-in per-IP throttle (GEV_RATELIMIT_GOOGLE_PER_MIN). No-op when unset.
// Per-IP throttle (GEV_RATELIMIT_GOOGLE_PER_MIN). On by default; 0 disables.
// Inlined (not the shared helper) so the 429 body keeps this endpoint's
// `places: []` contract that the client expects on every error response.
const _grl = googleRateLimiter();
@@ -161,6 +178,8 @@ export function googlePlacesContextProxy({
// a view-biased Text Search lands on the actual feature. Same key, field
// mask, throttle, and `places: []` error contract as nearby-places above.
middlewares.use('/api/google/text-search', async (req, res) => {
// Paid like nearby-places, so gated the same way, first.
if (admitSameSite(req, res)) return;
if (req.method !== 'GET') {
res.statusCode = 405;
res.setHeader('Content-Type', 'application/json');
@@ -200,7 +219,7 @@ export function googlePlacesContextProxy({
}
const { latitude, longitude } = coordinates;
// Opt-in per-IP throttle (GEV_RATELIMIT_GOOGLE_PER_MIN). No-op when unset.
// Per-IP throttle (GEV_RATELIMIT_GOOGLE_PER_MIN). On by default; 0 disables.
// Inlined (like nearby-places) so the 429 body keeps the `places: []`
// contract the client expects on every error response.
const _grl = googleRateLimiter();
-171
View File
@@ -1,171 +0,0 @@
import crypto from 'node:crypto';
import fs from 'node:fs/promises';
import path from 'node:path';
const DAY_MS = 86_400_000;
/** Entry files this cache writes: a 40-hex-digit key hash. */
const CACHE_FILE_RE = /^[0-9a-f]{40}\.json$/;
/** How long an answer is kept, by what it was. */
export const NOMINATIM_CACHE_TTL_MS = Object.freeze({
outline: 90 * DAY_MS,
place: 30 * DAY_MS,
notFound: DAY_MS,
});
/**
* Comparable form of a place name: Unicode-normalized (NFKC), accents
* removed, case folded, punctuation and symbols collapsed to single spaces,
* and a leading article dropped. Non-Latin scripts are kept as they are:
* "東京" stays "東京", while "São Paulo", "sao paulo" and "SÃO PAULO!" meet.
*
* @param {string} value
* @returns {string}
*/
export function normalizePlaceName(value) {
return String(value ?? '')
.normalize('NFKD')
.replace(/\p{M}+/gu, '')
.normalize('NFKC')
.toLocaleLowerCase('en-US')
.replace(/[\p{P}\p{S}\p{Z}\s]+/gu, ' ')
.trim()
.replace(/^(?:the) /, '')
.slice(0, 200);
}
/**
* Memory + disk cache for Nominatim answers.
*
* Every entry carries its own expiry. Memory is an LRU bounded by entries;
* disk is bounded by entries and bytes, with expired files removed during a
* periodic sweep. A null `dir` keeps everything in memory.
*/
export function createNominatimCache({
dir = null,
maxMemoryEntries = 256,
maxDiskEntries = 4000,
maxDiskBytes = 64 * 1024 * 1024,
maxEntryBytes = 2 * 1024 * 1024,
sweepEvery = 50,
now = Date.now,
} = {}) {
const memory = new Map();
let writesSinceSweep = 0;
let sweeping = null;
const fileFor = (key) =>
path.join(
dir,
`${crypto.createHash('sha256').update(key).digest('hex').slice(0, 40)}.json`,
);
function remember(key, entry) {
memory.delete(key);
memory.set(key, entry);
while (memory.size > maxMemoryEntries)
memory.delete(memory.keys().next().value);
}
async function get(key) {
const hit = memory.get(key);
if (hit) {
if (hit.expiresAt > now()) {
remember(key, hit);
return hit;
}
memory.delete(key);
}
if (!dir) return null;
try {
const entry = JSON.parse(await fs.readFile(fileFor(key), 'utf8'));
if (entry?.key !== key || !(entry.expiresAt > now())) return null;
remember(key, entry);
return entry;
} catch {
return null;
}
}
async function set(key, payload, ttlMs) {
const entry = { key, payload, storedAt: now(), expiresAt: now() + ttlMs };
remember(key, entry);
if (!dir) return;
const body = JSON.stringify(entry);
if (Buffer.byteLength(body) > maxEntryBytes) return;
try {
await fs.mkdir(dir, { recursive: true });
const file = fileFor(key);
const temp = `${file}.${process.pid}.tmp`;
await fs.writeFile(temp, body);
await fs.rename(temp, file);
} catch {
return;
}
writesSinceSweep += 1;
if (writesSinceSweep >= sweepEvery) {
writesSinceSweep = 0;
await sweep();
}
}
/** Remove expired files, then the oldest until both disk bounds hold. */
async function sweep() {
if (!dir) return;
if (sweeping) return sweeping;
sweeping = (async () => {
let names;
try {
names = await fs.readdir(dir);
} catch {
return;
}
const files = [];
for (const name of names) {
// Only this cache's own entry files; anything else in the directory
// is left alone.
if (!CACHE_FILE_RE.test(name)) continue;
const file = path.join(dir, name);
let stat;
let text;
try {
[stat, text] = await Promise.all([
fs.stat(file),
fs.readFile(file, 'utf8'),
]);
} catch {
// Unreadable right now (gone, busy, too many open files): an error
// reading is not evidence the entry is bad, so it stays.
continue;
}
let expiresAt;
try {
expiresAt = JSON.parse(text)?.expiresAt;
} catch {
expiresAt = null; // corrupt: removed below
}
if (!(expiresAt > now())) {
await fs.rm(file, { force: true }).catch(() => {});
continue;
}
files.push({ file, size: stat.size, at: stat.mtimeMs });
}
files.sort((a, b) => a.at - b.at);
let bytes = files.reduce((sum, entry) => sum + entry.size, 0);
while (
files.length &&
(files.length > maxDiskEntries || bytes > maxDiskBytes)
) {
const oldest = files.shift();
bytes -= oldest.size;
await fs.rm(oldest.file, { force: true }).catch(() => {});
}
})().finally(() => {
sweeping = null;
});
return sweeping;
}
return { get, set, sweep, memorySize: () => memory.size };
}
-812
View File
@@ -1,812 +0,0 @@
import fs from 'node:fs';
import path from 'node:path';
import { DatabaseSync } from 'node:sqlite';
import { readResponseJsonCapped } from '../common/http.js';
/**
* One gate for every request this server sends to a Nominatim instance.
*
* Place search and voice outlines share it. Public requests reserve one start
* at a time across processes sharing this installation's state file, remain at
* least 1.1 s apart, use a short queue and daily ceiling, and pause after the
* server says to stop. Independent installations do not share one app-wide
* limiter; each must identify itself and enforce the public policy locally.
*/
/** The public instance. Only this host is subject to the public-use limits. */
export const PUBLIC_NOMINATIM_SEARCH =
'https://nominatim.openstreetmap.org/search';
const PUBLIC_NOMINATIM_HOST = 'nominatim.openstreetmap.org';
/**
* The public usage policy asks for a User-Agent or Referer that identifies the
* application. The identity is stable: it is never rotated or varied.
*/
export const NOMINATIM_HEADERS = Object.freeze({
'User-Agent':
'gods-eye-view/0.1 (+https://github.com/bilawalsidhu/gods-eye-view)',
Referer: 'https://github.com/bilawalsidhu/gods-eye-view',
});
/** Absolute policy maximum is one request per second; 1.1 s absorbs jitter. */
export const NOMINATIM_MIN_SPACING_MS = 1100;
/** Requests allowed to wait for their turn; past this, refuse at once. */
export const NOMINATIM_MAX_PENDING = 4;
/** A queued request older than this has no reader left; drop it unsent. */
export const NOMINATIM_MAX_WAIT_MS = 10_000;
/** Default public requests per install per UTC day, outlines and search together. */
export const NOMINATIM_DEFAULT_DAILY_CAP = 50;
/**
* Pause after a refusal without Retry-After. A Retry-After from the server is
* honoured in full, however long: no local maximum may cause an earlier retry.
*/
const DEFAULT_BACKOFF_MS = 60_000;
/** A 403 from the public instance means this identity is blocked: stop longer. */
const BLOCKED_BACKOFF_MS = 3_600_000;
/** Pause after a server error or a failed retry. */
const OUTAGE_BACKOFF_MS = 30_000;
const DISABLED_VALUES = new Set(['', 'off', 'none', 'disabled', '0']);
/**
* Read the search endpoint setting.
*
* Unset means the public instance under its guards. Set to empty (or `off`)
* disables every Nominatim request. Any other value must be an http(s) URL
* without credentials, query or fragment; a bare host gets `/search`. A value
* that is not usable disables the service rather than falling back to the
* public instance.
*
* @param {Record<string, string|undefined>} [env]
* @returns {{endpoint: string|null, isPublic: boolean, dailyCap: number, invalid?: boolean}}
*/
export function resolveNominatimSettings(env = process.env) {
const dailyCap = parseDailyCap(env.NOMINATIM_DAILY_CAP);
const raw = env.NOMINATIM_URL;
if (raw === undefined)
return { endpoint: PUBLIC_NOMINATIM_SEARCH, isPublic: true, dailyCap };
const value = String(raw).trim();
if (DISABLED_VALUES.has(value.toLowerCase()))
return { endpoint: null, isPublic: false, dailyCap };
let url;
try {
url = new URL(value);
} catch {
return { endpoint: null, isPublic: false, dailyCap, invalid: true };
}
if (
!['http:', 'https:'].includes(url.protocol) ||
!url.hostname ||
url.username ||
url.password ||
url.search ||
url.hash
)
return { endpoint: null, isPublic: false, dailyCap, invalid: true };
if (url.pathname === '/' || url.pathname === '') url.pathname = '/search';
return {
endpoint: url.href.replace(/\/$/, ''),
// A trailing DNS dot names the same host.
isPublic:
url.hostname.toLowerCase().replace(/\.$/, '') === PUBLIC_NOMINATIM_HOST,
dailyCap,
};
}
function parseDailyCap(raw) {
if (raw === undefined || String(raw).trim() === '')
return NOMINATIM_DEFAULT_DAILY_CAP;
const value = Number(raw);
return Number.isFinite(value) && value >= 0
? Math.floor(value)
: NOMINATIM_DEFAULT_DAILY_CAP;
}
function gateError(code, message, extra = {}) {
return Object.assign(new Error(message), { code, ...extra });
}
const HTTP_MONTHS = Object.freeze([
'Jan',
'Feb',
'Mar',
'Apr',
'May',
'Jun',
'Jul',
'Aug',
'Sep',
'Oct',
'Nov',
'Dec',
]);
const HTTP_WEEKDAYS = Object.freeze([
'Sun',
'Mon',
'Tue',
'Wed',
'Thu',
'Fri',
'Sat',
]);
const HTTP_WEEKDAYS_LONG = Object.freeze([
'Sunday',
'Monday',
'Tuesday',
'Wednesday',
'Thursday',
'Friday',
'Saturday',
]);
/** Strict RFC 9110 HTTP-date, including the two obsolete recipient formats. */
function parseHttpDate(raw, now) {
let match = raw.match(
/^(Sun|Mon|Tue|Wed|Thu|Fri|Sat), (\d{2}) (Jan|Feb|Mar|Apr|May|Jun|Jul|Aug|Sep|Oct|Nov|Dec) (\d{4}) (\d{2}):(\d{2}):(\d{2}) GMT$/,
);
let weekday, day, month, year, hour, minute, second;
if (match) {
[, weekday, day, month, year, hour, minute, second] = match;
weekday = HTTP_WEEKDAYS.indexOf(weekday);
} else {
match = raw.match(
/^(Sunday|Monday|Tuesday|Wednesday|Thursday|Friday|Saturday), (\d{2})-(Jan|Feb|Mar|Apr|May|Jun|Jul|Aug|Sep|Oct|Nov|Dec)-(\d{2}) (\d{2}):(\d{2}):(\d{2}) GMT$/,
);
if (match) {
[, weekday, day, month, year, hour, minute, second] = match;
weekday = HTTP_WEEKDAYS_LONG.indexOf(weekday);
const currentYear = new Date(now).getUTCFullYear();
if (!Number.isFinite(currentYear)) return null;
year = Math.floor(currentYear / 100) * 100 + Number(year);
if (year > currentYear + 50) year -= 100;
} else {
match = raw.match(
/^(Sun|Mon|Tue|Wed|Thu|Fri|Sat) (Jan|Feb|Mar|Apr|May|Jun|Jul|Aug|Sep|Oct|Nov|Dec) ([ \d]\d) (\d{2}):(\d{2}):(\d{2}) (\d{4})$/,
);
if (!match) return null;
[, weekday, month, day, hour, minute, second, year] = match;
weekday = HTTP_WEEKDAYS.indexOf(weekday);
}
}
day = Number(day);
month = HTTP_MONTHS.indexOf(month);
year = Number(year);
hour = Number(hour);
minute = Number(minute);
second = Number(second);
if (
weekday < 0 ||
month < 0 ||
year < 1601 ||
hour > 23 ||
minute > 59 ||
second > 59
)
return null;
const at = Date.UTC(year, month, day, hour, minute, second);
const date = new Date(at);
return date.getUTCFullYear() === year &&
date.getUTCMonth() === month &&
date.getUTCDate() === day &&
date.getUTCHours() === hour &&
date.getUTCMinutes() === minute &&
date.getUTCSeconds() === second &&
date.getUTCDay() === weekday
? at
: null;
}
/** Seconds or an HTTP date → milliseconds from now, or null. */
export function parseRetryAfter(value, now = Date.now()) {
if (value == null) return null;
const raw = String(value).trim();
if (!raw) return null;
if (/^\d+$/.test(raw)) {
const seconds = Number(raw);
const milliseconds = seconds * 1000;
return Number.isSafeInteger(milliseconds) ? milliseconds : null;
}
// Numeric-looking but invalid delay-seconds must not be reinterpreted as a date.
if (/^[+-]?(?:\d|\.\d)/.test(raw)) return null;
const at = parseHttpDate(raw, now);
return at === null ? null : Math.max(0, at - now);
}
/** UTC calendar day, the unit of the daily cap. */
function utcDay(ms) {
return new Date(ms).toISOString().slice(0, 10);
}
/**
* Public-service state shared by every process using the same file: today's
* request count, the next reserved start, and any pause the server asked for.
*
* Every change is a read-modify-write under an OS-owned SQLite transaction,
* and the JSON file is replaced atomically, so two dev servers sharing a
* checkout add to one count instead of overwriting each other. A process exit
* releases only that process's transaction; there is no stale path to delete
* and therefore no stale reclaimer that can remove a successor's live lock.
* Memory keeps the highest values this process has seen, so a failed or
* unreadable file never lowers the count or shortens a pause; a persistence
* failure is reported once and in `status()`.
*/
export function createGateStateStore({
file = null,
lockWaitMs = 500,
onError = (error) =>
console.warn(
`[Nominatim] could not persist usage state: ${error?.message || error}`,
),
} = {}) {
let memory = { day: '', count: 0, pausedUntil: 0, nextStartAt: 0 };
let lastError = null;
const report = (error) => {
if (!lastError) onError(error);
lastError = error;
};
const pauseMarkerPrefix = file ? `${path.basename(file)}.pause.` : null;
/**
* A refusal must remain visible even while another process owns the SQLite
* transaction. Immutable marker names make that write atomic and monotonic:
* concurrent writers cannot shorten one another's pause, and the next lock
* owner folds every marker into the normal JSON state.
*/
function pendingPauses({ strict = false } = {}) {
if (!file) return [];
let names;
try {
names = fs.readdirSync(path.dirname(file));
} catch (error) {
if (error?.code === 'ENOENT') return [];
report(error);
if (strict) throw error;
return [];
}
const pauses = [];
for (const name of names) {
if (!name.startsWith(pauseMarkerPrefix)) continue;
const encoded = name.slice(pauseMarkerPrefix.length).split('.', 1)[0];
if (!/^\d+$/.test(encoded)) continue;
const pausedUntil = Number(encoded);
if (!Number.isSafeInteger(pausedUntil)) {
const error = new Error('invalid durable Nominatim pause marker');
report(error);
if (strict) throw error;
continue;
}
pauses.push({
path: path.join(path.dirname(file), name),
pausedUntil,
});
}
return pauses;
}
function markPause(at) {
fs.mkdirSync(path.dirname(file), { recursive: true });
const pausedUntil = Math.ceil(at);
const marker = `${file}.pause.${pausedUntil}.${process.pid}.${process.hrtime.bigint()}`;
fs.writeFileSync(marker, '', { flag: 'wx' });
return pausedUntil;
}
function clearPersistedPauses(pausedUntil) {
for (const marker of pendingPauses()) {
if (marker.pausedUntil > pausedUntil) continue;
try {
fs.unlinkSync(marker.path);
} catch (error) {
if (error?.code !== 'ENOENT') report(error);
}
}
}
const merge = (a, b) => ({
day: a.day >= b.day ? a.day : b.day,
count:
a.day === b.day
? Math.max(a.count, b.count)
: a.day > b.day
? a.count
: b.count,
pausedUntil: Math.max(a.pausedUntil || 0, b.pausedUntil || 0),
nextStartAt: Math.max(a.nextStartAt || 0, b.nextStartAt || 0),
});
const parseState = (parsed) => {
const validDay =
parsed?.day === '' ||
(typeof parsed?.day === 'string' &&
/^\d{4}-\d{2}-\d{2}$/.test(parsed.day) &&
new Date(`${parsed.day}T00:00:00.000Z`)
.toISOString()
.startsWith(parsed.day));
const validCount =
Number.isInteger(parsed?.count) &&
parsed.count >= 0 &&
(parsed.day !== '' || parsed.count === 0);
const validPausedUntil =
Number.isFinite(parsed?.pausedUntil) && parsed.pausedUntil >= 0;
// nextStartAt was added after the original state format. Its absence is
// the one supported migration; a present value must still be trustworthy.
const validNextStartAt =
parsed?.nextStartAt === undefined ||
(Number.isFinite(parsed.nextStartAt) && parsed.nextStartAt >= 0);
if (
!parsed ||
Array.isArray(parsed) ||
typeof parsed !== 'object' ||
!validDay ||
!validCount ||
!validPausedUntil ||
!validNextStartAt
)
throw new Error('invalid durable Nominatim usage state');
return {
day: parsed.day,
count: parsed.count,
pausedUntil: parsed.pausedUntil,
nextStartAt: parsed.nextStartAt ?? 0,
};
};
function readDisk({ strict = false } = {}) {
if (!file) {
if (strict)
throw new Error('no durable Nominatim usage state is configured');
return null;
}
try {
return parseState(JSON.parse(fs.readFileSync(file, 'utf8')));
} catch (error) {
if (error?.code === 'ENOENT') return null;
report(error);
if (strict) throw error;
return null;
}
}
function current() {
const disk = readDisk();
if (disk) memory = merge(memory, disk);
const pending = pendingPauses();
if (pending.length)
memory = merge(memory, {
...memory,
pausedUntil: Math.max(...pending.map((item) => item.pausedUntil)),
});
return memory;
}
function withLock(work) {
fs.mkdirSync(path.dirname(file), { recursive: true });
const waitMs = Number.isFinite(lockWaitMs)
? Math.max(0, Math.floor(lockWaitMs))
: 500;
const database = new DatabaseSync(`${file}.lock.sqlite`, {
timeout: waitMs,
});
let transactionOpen = false;
try {
database.exec('BEGIN IMMEDIATE');
transactionOpen = true;
const result = work();
database.exec('COMMIT');
transactionOpen = false;
return result;
} catch (error) {
if (transactionOpen) {
try {
database.exec('ROLLBACK');
} catch {
// Closing the connection below still releases this owner's lock.
}
}
throw error;
} finally {
try {
database.close();
} catch {
// The transaction has already been committed or rolled back. Closing
// is best effort during exceptional teardown and never removes a lock.
}
}
}
function writeDisk(next) {
const temp = `${file}.${process.pid}.${process.hrtime.bigint()}.tmp`;
try {
fs.writeFileSync(temp, JSON.stringify(next));
fs.renameSync(temp, file);
} finally {
try {
fs.unlinkSync(temp);
} catch (error) {
if (error?.code !== 'ENOENT') report(error);
}
}
}
/** Apply `change` to the freshest state and persist it atomically. */
function update(change) {
if (!file) {
memory = merge(memory, change(memory));
return memory;
}
try {
withLock(() => {
const next = merge(memory, change(current()));
writeDisk(next);
memory = next;
clearPersistedPauses(next.pausedUntil);
lastError = null;
});
} catch (error) {
report(error);
memory = merge(memory, change(memory));
}
return memory;
}
return {
count(day) {
const state = current();
return state.day === day ? state.count : 0;
},
increment(day) {
return update((state) => ({
day,
count: state.day === day ? state.count + 1 : 1,
pausedUntil: state.pausedUntil,
nextStartAt: state.nextStartAt,
})).count;
},
reserve(day, { now, dailyCap, minSpacingMs, latestStartAt = Infinity }) {
if (!file) {
const error = new Error(
'no durable Nominatim usage state is configured',
);
report(error);
return { status: 'unavailable', error };
}
try {
let result;
withLock(() => {
const disk = readDisk({ strict: true });
if (disk) memory = merge(memory, disk);
const pending = pendingPauses({ strict: true });
if (pending.length)
memory = merge(memory, {
...memory,
pausedUntil: Math.max(...pending.map((item) => item.pausedUntil)),
});
const state = memory;
if ((state.pausedUntil || 0) > now) {
result = {
status: 'paused',
retryAfterMs: state.pausedUntil - now,
};
return;
}
const count = state.day === day ? state.count : 0;
if (count >= dailyCap) {
result = { status: 'cap' };
return;
}
const startAt = Math.max(now, state.nextStartAt || 0);
if (startAt > latestStartAt) {
result = { status: 'abandoned' };
return;
}
const next = {
day,
count: count + 1,
pausedUntil: state.pausedUntil || 0,
nextStartAt: startAt + minSpacingMs,
};
writeDisk(next);
memory = next;
clearPersistedPauses(next.pausedUntil);
lastError = null;
result = { status: 'reserved', startAt, count: next.count };
});
return result;
} catch (error) {
report(error);
return { status: 'unavailable', error };
}
},
pausedUntil() {
return current().pausedUntil || 0;
},
pauseUntil(at) {
if (file) {
try {
at = markPause(at);
} catch (error) {
report(error);
}
}
update((state) => ({ ...state, pausedUntil: at }));
},
status: () => ({
file,
durable: Boolean(file),
persisted: Boolean(file) && !lastError,
error: lastError ? String(lastError.message || lastError) : null,
}),
};
}
/** Compatibility name for the state store. */
export const createUsageStore = createGateStateStore;
/**
* Construct a gate. `settings` may be a function so the environment is read
* after the dev server loads `.env`.
*
* @param {object} [options]
* @param {(() => object)|object} [options.settings]
* @param {typeof fetch} [options.fetchImpl]
* @param {() => number} [options.now]
* @param {(ms: number) => Promise<void>} [options.sleep]
* @param {ReturnType<typeof createUsageStore>} [options.usage]
*/
export function createNominatimGate({
settings = () => resolveNominatimSettings(),
fetchImpl = (...args) => globalThis.fetch(...args),
now = Date.now,
sleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms)),
usage = createGateStateStore(),
minSpacingMs = NOMINATIM_MIN_SPACING_MS,
maxPending = NOMINATIM_MAX_PENDING,
maxWaitMs = NOMINATIM_MAX_WAIT_MS,
timeoutMs = 9000,
} = {}) {
let queue = Promise.resolve();
let pending = 0;
const pausedUntilByEndpoint = new Map();
let upstreamRequests = 0;
let usageStore = usage;
const current = () =>
typeof settings === 'function' ? settings() : settings;
/**
* Pause until `ms` from now. The public service's pause is also written to
* the shared state so a restart or a second process honours it.
*/
function pause(ms, config) {
const until = now() + Math.max(0, ms);
const key = String(config?.endpoint || '');
if (until > (pausedUntilByEndpoint.get(key) || 0))
pausedUntilByEndpoint.set(key, until);
if (config?.isPublic) usageStore.pauseUntil(until);
}
function pauseEnd(config) {
const endpointPause =
pausedUntilByEndpoint.get(String(config?.endpoint || '')) || 0;
return config?.isPublic
? Math.max(endpointPause, usageStore.pausedUntil())
: endpointPause;
}
function pausedError(config) {
return gateError('NOMINATIM_PAUSED', 'Place service is pausing', {
retryAfterMs: Math.max(0, pauseEnd(config) - now()),
});
}
function admissionError(config) {
if (!config.endpoint)
return gateError('NOMINATIM_DISABLED', 'Place service is not configured');
if (now() < pauseEnd(config)) return pausedError(config);
return null;
}
/** One upstream request; classifies refusals and pauses the gate. */
async function send(url, config, maxBytes, signal) {
upstreamRequests += 1;
const controller = new AbortController();
const onAbort = () => controller.abort();
signal?.addEventListener('abort', onAbort, { once: true });
// One deadline covers the headers AND the body: a server that answers its
// headers and then stalls must not hold the shared queue.
const timer = setTimeout(() => controller.abort(), timeoutMs);
try {
return await exchange(url, config, maxBytes, signal, controller.signal);
} finally {
clearTimeout(timer);
signal?.removeEventListener('abort', onAbort);
}
}
async function exchange(url, config, maxBytes, signal, requestSignal) {
let response;
try {
response = await fetchImpl(url, {
headers: NOMINATIM_HEADERS,
redirect: 'error',
signal: requestSignal,
});
} catch (cause) {
if (signal?.aborted) throw gateError('NOMINATIM_ABANDONED', 'Abandoned');
throw gateError('NOMINATIM_TRANSPORT', 'Place service unreachable', {
cause,
transport: true,
});
}
if (!response.ok) {
void response.body?.cancel?.().catch(() => {});
const status = response.status;
const retryAfter = parseRetryAfter(
response.headers?.get?.('retry-after'),
now(),
);
if (status === 429 || status === 503) {
pause(retryAfter ?? DEFAULT_BACKOFF_MS, config);
throw gateError('NOMINATIM_REFUSED', 'Place service asked to wait', {
status,
retryAfterMs: Math.max(0, pauseEnd(config) - now()),
});
}
if (status === 403 || status === 418) {
pause(retryAfter ?? BLOCKED_BACKOFF_MS, config);
throw gateError('NOMINATIM_REFUSED', 'Place service refused', {
status,
retryAfterMs: Math.max(0, pauseEnd(config) - now()),
});
}
if (status >= 500) pause(retryAfter ?? OUTAGE_BACKOFF_MS, config);
throw gateError('NOMINATIM_UPSTREAM', `Upstream returned ${status}`, {
status,
});
}
try {
return await readResponseJsonCapped(response, maxBytes, requestSignal);
} catch (cause) {
if (cause?.code === 'RESPONSE_TOO_LARGE')
throw gateError('NOMINATIM_TOO_LARGE', 'Place answer too large');
if (signal?.aborted) throw gateError('NOMINATIM_ABANDONED', 'Abandoned');
throw gateError('NOMINATIM_TRANSPORT', 'Place answer unreadable', {
cause,
transport: true,
});
}
}
/** Atomically reserve the public slot, then wait until it may start. */
async function takeTurn(config, queuedAt, signal) {
if (signal?.aborted || now() - queuedAt > maxWaitMs)
throw gateError('NOMINATIM_ABANDONED', 'Place search was abandoned');
const refused = admissionError(config);
if (refused) throw refused;
// An operator-selected endpoint is not governed by the public instance's
// shared budget, pacing, or durable-state requirement.
if (!config.isPublic) return;
const reserved = usageStore.reserve(utcDay(now()), {
now: now(),
dailyCap: config.dailyCap,
minSpacingMs,
latestStartAt: queuedAt + maxWaitMs,
});
if (reserved.status === 'unavailable')
throw gateError(
'NOMINATIM_STATE_UNAVAILABLE',
'Public place service usage state is unavailable',
{ cause: reserved.error },
);
if (reserved.status === 'cap')
throw gateError(
'NOMINATIM_DAILY_CAP',
'Daily place lookup allowance is used',
);
if (reserved.status === 'paused')
throw gateError('NOMINATIM_PAUSED', 'Place service is pausing', {
retryAfterMs: reserved.retryAfterMs,
});
if (reserved.status === 'abandoned')
throw gateError('NOMINATIM_ABANDONED', 'Place search was abandoned');
const waitMs = Math.max(0, reserved.startAt - now());
if (waitMs) await sleep(waitMs);
if (signal?.aborted || now() - queuedAt > maxWaitMs)
throw gateError('NOMINATIM_ABANDONED', 'Place search was abandoned');
// A refusal from another process after this reservation still stops it.
// The reserved daily slot remains spent; it cannot safely be reassigned.
if (now() < pauseEnd(config)) throw pausedError(config);
}
/**
* Queue one JSON request for `buildUrl(endpoint)`. At most one retry, and
* only after a transport failure; a failed retry pauses the gate.
*
* @param {(endpoint: string) => string} buildUrl
* @param {{signal?: AbortSignal, maxBytes?: number}} [options]
*/
function requestJson(buildUrl, { signal, maxBytes = 2 * 1024 * 1024 } = {}) {
const config = current();
const refused = admissionError(config);
if (refused) return Promise.reject(refused);
if (signal?.aborted)
return Promise.reject(
gateError('NOMINATIM_ABANDONED', 'Place search was abandoned'),
);
if (pending >= maxPending)
return Promise.reject(
gateError('NOMINATIM_QUEUE_FULL', 'Place search queue is full'),
);
pending += 1;
const queuedAt = now();
const url = buildUrl(config.endpoint);
const task = queue.then(async () => {
try {
await takeTurn(config, queuedAt, signal);
try {
return await send(url, config, maxBytes, signal);
} catch (error) {
if (!error?.transport) throw error;
await takeTurn(config, now(), signal);
try {
return await send(url, config, maxBytes, signal);
} catch (retryError) {
if (retryError?.transport) pause(OUTAGE_BACKOFF_MS, config);
throw retryError;
}
}
} finally {
pending -= 1;
}
});
queue = task.catch(() => null);
return task;
}
return {
requestJson,
settings: current,
/** Share public admission, daily count and pauses through one install's file. */
persistUsage(file) {
usageStore = createGateStateStore({ file });
},
stats: () => ({
upstreamRequests,
pending,
pausedForMs: Math.max(0, pauseEnd(current()) - now()),
usedToday: usageStore.count(utcDay(now())),
persistence: usageStore.status?.() || null,
}),
};
}
let sharedGate = createNominatimGate();
/** The process-wide gate: search and outlines share one budget and pacer. */
export function sharedNominatimGate() {
return sharedGate;
}
/** Replace the process-wide gate (tests, embedders with their own transport). */
export function setSharedNominatimGate(gate) {
sharedGate = gate || createNominatimGate();
return sharedGate;
}
/** Codes that mean "not now" rather than "no such place". */
export function isNominatimBusy(error) {
return [
'NOMINATIM_QUEUE_FULL',
'NOMINATIM_ABANDONED',
'NOMINATIM_PAUSED',
'NOMINATIM_REFUSED',
'NOMINATIM_DAILY_CAP',
].includes(error?.code);
}
-595
View File
@@ -1,595 +0,0 @@
import { normalizePlaceName } from './nominatimCache.js';
/**
* Which Nominatim results may answer which voice ask, and how their polygons
* are bounded. Pure: no I/O.
*
* A search returns whatever matches the words best. "Mission District" can
* come back as a car-charging station and "Bandra" as a railway station, so a
* result is accepted only when its OSM class (and, for broad classes, its
* type) suits the kind of thing that was asked for, it carries a polygon, and
* its name actually answers the words.
*/
/** Ask kinds the outline route answers. */
export const OUTLINE_KINDS = Object.freeze([
'city',
'admin',
'neighborhood',
'landmark',
'building',
]);
/** Simplification tolerance (degrees) sent upstream, per ask kind. */
export const OUTLINE_POLYGON_THRESHOLD = Object.freeze({
city: 0.0005,
admin: 0.0005,
neighborhood: 0.0005,
landmark: 0.00005,
building: 0.00005,
});
/** Total vertices returned for one outline, and parts/holes kept. */
export const OUTLINE_MAX_VERTICES = 6000;
const OUTLINE_MAX_PARTS = 24;
const OUTLINE_MAX_HOLES = 16;
const ANY = '*';
const SETTLEMENT_PLACES = [
'city',
'town',
'village',
'municipality',
'borough',
'hamlet',
];
const CIVIC_AMENITIES = [
'university',
'college',
'school',
'kindergarten',
'hospital',
'clinic',
'place_of_worship',
'monastery',
'townhall',
'courthouse',
'library',
'theatre',
'cinema',
'arts_centre',
'community_centre',
'conference_centre',
'events_venue',
'exhibition_centre',
'music_venue',
'planetarium',
'marketplace',
'prison',
'grave_yard',
'fire_station',
'police',
'embassy',
'public_building',
];
const LANDMARK_TOURISM = [
'attraction',
'theme_park',
'zoo',
'aquarium',
'museum',
'gallery',
'viewpoint',
'camp_site',
'hotel',
'resort',
];
const MAN_MADE_STRUCTURES = [
'tower',
'lighthouse',
'pier',
'bridge',
'works',
'dam',
'observatory',
'wastewater_plant',
'water_works',
'breakwater',
];
/**
* Accepted `class → types` per ask kind. `*` accepts every type of that
* class. Anything absent is rejected: roads, railways, stations, shops,
* guideposts and single facilities never answer an area ask.
*/
export const OUTLINE_ACCEPT = Object.freeze({
city: {
boundary: ['administrative'],
place: SETTLEMENT_PLACES,
},
admin: {
boundary: ['administrative'],
place: ['country', 'state', 'province', 'region', 'county', 'district'],
},
neighborhood: {
boundary: ['administrative'],
place: ['suburb', 'quarter', 'neighbourhood', 'borough', 'city_block'],
},
landmark: {
leisure: ANY,
landuse: ANY,
amenity: CIVIC_AMENITIES,
tourism: LANDMARK_TOURISM,
building: ANY,
natural: ANY,
water: ANY,
waterway: ['riverbank', 'dock'],
historic: ANY,
man_made: MAN_MADE_STRUCTURES,
office: ['government', 'diplomatic'],
aeroway: ['aerodrome', 'heliport', 'terminal'],
military: ANY,
boundary: ['national_park', 'protected_area'],
place: ['island', 'islet', 'square'],
},
// Single structures only: no campuses, grounds, parks or camp sites.
building: {
building: ANY,
amenity: [
'place_of_worship',
'townhall',
'courthouse',
'library',
'theatre',
'cinema',
'arts_centre',
'community_centre',
'conference_centre',
'exhibition_centre',
'music_venue',
'planetarium',
'fire_station',
'police',
'embassy',
'public_building',
],
tourism: ['museum', 'gallery', 'hotel', 'attraction'],
historic: ['building', 'castle', 'church', 'fort', 'palace', 'monastery'],
man_made: ['tower', 'lighthouse', 'observatory'],
office: ANY,
},
});
/**
* Administrative evidence per ask kind: an address type, or a Nominatim
* place_rank band (country 4, state 8, county 10-12, city 12-16, suburb 18-20,
* neighbourhood 20-22). An administrative boundary with neither is refused.
*/
const ADMIN_EVIDENCE = Object.freeze({
city: {
types: ['city', 'town', 'village', 'municipality', 'borough', 'hamlet'],
ranks: [12, 18],
},
admin: {
types: [
'country',
'state',
'province',
'region',
'county',
'district',
'state_district',
],
ranks: [2, 14],
},
neighborhood: {
types: ['suburb', 'quarter', 'neighbourhood', 'city_district', 'borough'],
ranks: [17, 24],
},
});
const KNOWN_ADMIN_TYPES = new Set(
Object.values(ADMIN_EVIDENCE).flatMap((evidence) => evidence.types),
);
/**
* Nominatim address types whose meaning is stable enough for the area consumer's
* country/admin1/admin2 contract. `region`, `district` and numeric admin levels
* vary by country, so they remain unverified instead of being guessed.
*/
const VERIFIED_ADMIN_LEVEL = Object.freeze({
country: 'country',
state: 'admin1',
province: 'admin1',
county: 'admin2',
state_district: 'admin2',
});
const EXPLICIT_NAME_LEVEL = Object.freeze({
country: 'country',
nation: 'country',
province: 'admin1',
state: 'admin1',
region: 'admin1',
oblast: 'admin1',
prefecture: 'admin1',
canton: 'admin1',
governorate: 'admin1',
county: 'admin2',
district: 'admin2',
department: 'admin2',
zone: 'retired',
});
function explicitNameEvidence(value) {
const normalized = String(value || '')
.normalize('NFKC')
.toLocaleLowerCase()
.replace(/[^\p{L}\p{M}\p{N}]+/gu, ' ')
.trim();
const prefix = normalized.match(
/^(country|nation|province|state|region|zone|oblast|prefecture|canton|governorate|county|district|department)\s+of\b/,
)?.[1];
const suffix = normalized.match(
/\b(country|nation|province|state|region|zone|oblast|prefecture|canton|governorate|county|district|department)$/,
)?.[1];
const levels = new Set(
[prefix, suffix]
.filter(Boolean)
.map((type) => EXPLICIT_NAME_LEVEL[type])
.filter(Boolean),
);
if (levels.size > 1) return { contradictory: true, level: null };
return { contradictory: false, level: levels.values().next().value || null };
}
/**
* Verified administrative identity carried by the selected result itself.
* Never infer it from the request, desired kind, rank, or a separate geocoder
* anchor. A retired name is preserved without a level so downstream policy can
* reject it explicitly; every other contradiction stays wholly unverified.
*/
function selectedAdminIdentity(kind, row) {
if (kind !== 'admin') return null;
const adminLevel =
VERIFIED_ADMIN_LEVEL[String(row?.addresstype || '').toLowerCase()];
const authoritativeAdminArea = String(row?.name || '').trim();
if (!adminLevel || !authoritativeAdminArea) return null;
// Validate the complete authoritative name. `adminArea` is truncated only
// for display after every prefix/suffix signal has been considered.
const { contradictory, level: nameLevel } = explicitNameEvidence(
authoritativeAdminArea,
);
if (contradictory) return null;
const adminArea = authoritativeAdminArea.slice(0, 160);
if (nameLevel === 'retired') return { adminArea };
if (nameLevel && nameLevel !== adminLevel) return null;
return { adminArea, adminLevel };
}
/** Whether an administrative boundary is at the level the ask names. */
export function adminLevelFits(kind, row) {
const evidence = ADMIN_EVIDENCE[kind];
if (!evidence) return true;
const type = String(row?.addresstype || '').toLowerCase();
if (type && evidence.types.includes(type)) return true;
// A recognized type is direct evidence. Rank is only a fallback when the
// upstream answer omits the type or supplies a value we do not understand;
// the overlapping county/city rank bands must not override `county`.
if (KNOWN_ADMIN_TYPES.has(type)) return false;
const rank = Number(row?.place_rank);
if (Number.isFinite(rank))
return rank >= evidence.ranks[0] && rank <= evidence.ranks[1];
return false;
}
/** Largest footprint (m²) a non-`building` result may have for a building ask. */
const BUILDING_MAX_AREA_M2 = 150_000;
/** Approximate area (m²) of a [lon, lat] ring. */
function ringAreaM2(ring) {
const lat0 = ring[0][1];
const k = 111_320 * 111_320 * Math.cos((lat0 * Math.PI) / 180);
return ringArea(ring) * k;
}
/** Whether a result's class/type suits the ask kind. */
export function acceptsOutlineClass(kind, osmClass, osmType) {
const rules = OUTLINE_ACCEPT[kind];
if (!rules) return false;
const types = rules[String(osmClass || '').toLowerCase()];
if (!types) return false;
return types === ANY || types.includes(String(osmType || '').toLowerCase());
}
const STOP_WORDS = new Set(['the', 'of', 'de', 'la', 'le', 'el', 'and']);
/** Scripts written without spaces between words. */
const UNSPACED_SCRIPT_RE =
/[\p{Script=Han}\p{Script=Hiragana}\p{Script=Katakana}\p{Script=Hangul}\p{Script=Thai}\p{Script=Lao}\p{Script=Khmer}\p{Script=Myanmar}]/u;
/** Whether `needle` occurs in `hay` as whole consecutive words. */
function containsWords(hay, needle) {
for (let i = 0; i + needle.length <= hay.length; i++)
if (needle.every((word, j) => hay[i + j] === word)) return true;
return false;
}
/** Significant words of the part of a query before the first comma. */
function queryWords(query) {
const head = String(query ?? '').split(',')[0];
return normalizePlaceName(head)
.split(' ')
.filter((word) => word && !STOP_WORDS.has(word));
}
/**
* Whether a result's names answer the query. Most of the query's words must
* appear among one name's words, or one name must contain the whole query
* (for scripts written without spaces).
*/
export function outlineNameMatches(row, query) {
const words = queryWords(query);
if (!words.length) return false;
const whole = words.join(' ');
const names = new Set();
const add = (value) => {
const name = normalizePlaceName(value);
if (name) names.add(name);
};
add(row?.name);
add(String(row?.display_name || '').split(',')[0]);
if (row?.namedetails && typeof row.namedetails === 'object')
for (const value of Object.values(row.namedetails)) add(value);
const unspaced = UNSPACED_SCRIPT_RE.test(whole);
for (const name of names) {
const nameWords = name.split(' ');
// Whole words only: "York" never matches "Yorkshire Park".
if (containsWords(nameWords, words)) return true;
// Scripts without spaces compare by characters instead.
if (unspaced && name.replace(/ /g, '').includes(whole.replace(/ /g, '')))
return true;
const nameSet = new Set(nameWords);
const hits = words.filter((word) => nameSet.has(word)).length;
if (words.length > 1 && hits >= Math.ceil(words.length * 0.6)) return true;
}
return false;
}
const finiteLon = (value) =>
Number.isFinite(value) && value >= -180 && value <= 180;
const finiteLat = (value) =>
Number.isFinite(value) && value >= -90 && value <= 90;
/** A valid closed ring of [lon, lat] numbers, or null. */
function cleanRing(ring) {
if (!Array.isArray(ring)) return null;
const out = [];
for (const point of ring) {
if (!Array.isArray(point)) return null;
const lon = Number(point[0]);
const lat = Number(point[1]);
if (!finiteLon(lon) || !finiteLat(lat)) return null;
out.push([lon, lat]);
}
if (out.length < 4) return null;
const [first, last] = [out[0], out.at(-1)];
if (first[0] !== last[0] || first[1] !== last[1]) out.push([...first]);
return out;
}
/** Planar ring area in squared degrees (only used to rank parts). */
function ringArea(ring) {
let sum = 0;
for (let i = 0, j = ring.length - 1; i < ring.length; j = i++)
sum += (ring[j][0] - ring[i][0]) * (ring[j][1] + ring[i][1]);
return Math.abs(sum / 2);
}
/** Polygon/MultiPolygon GeoJSON → `[[outer, ...holes], …]`, largest first. */
export function polygonsFromGeoJson(geometry) {
const raw =
geometry?.type === 'Polygon'
? [geometry.coordinates]
: geometry?.type === 'MultiPolygon'
? geometry.coordinates
: null;
if (!Array.isArray(raw)) return null;
const polygons = [];
for (const polygon of raw) {
if (!Array.isArray(polygon)) continue;
const outer = cleanRing(polygon[0]);
if (!outer) continue;
const holes = polygon
.slice(1)
.map(cleanRing)
.filter(Boolean)
.sort((a, b) => ringArea(b) - ringArea(a))
.slice(0, OUTLINE_MAX_HOLES);
polygons.push({ rings: [outer, ...holes], area: ringArea(outer) });
}
if (!polygons.length) return null;
polygons.sort((a, b) => b.area - a.area);
return polygons.slice(0, OUTLINE_MAX_PARTS).map((entry) => entry.rings);
}
/** Iterative Douglas–Peucker on a closed ring (keeps it closed, ≥ 4 points). */
function simplifyClosedRing(ring, tolerance) {
if (ring.length <= 8 || tolerance <= 0) return ring;
const keep = new Uint8Array(ring.length);
keep[0] = keep[ring.length - 1] = 1;
const stack = [[0, ring.length - 1]];
const tol2 = tolerance * tolerance;
while (stack.length) {
const [start, end] = stack.pop();
const [ax, ay] = ring[start];
const [bx, by] = ring[end];
const dx = bx - ax;
const dy = by - ay;
const len2 = dx * dx + dy * dy;
let best = -1;
let bestD = tol2;
for (let i = start + 1; i < end; i++) {
const [px, py] = ring[i];
let t = len2 ? ((px - ax) * dx + (py - ay) * dy) / len2 : 0;
t = Math.max(0, Math.min(1, t));
const ex = ax + t * dx - px;
const ey = ay + t * dy - py;
const d = ex * ex + ey * ey;
if (d > bestD) {
bestD = d;
best = i;
}
}
if (best > 0) {
keep[best] = 1;
stack.push([start, best], [best, end]);
}
}
const out = ring.filter((_point, index) => keep[index]);
return out.length >= 4 ? out : ring;
}
const countVertices = (polygons) =>
polygons.reduce(
(sum, rings) => sum + rings.reduce((n, ring) => n + ring.length, 0),
0,
);
/** Simplify until the whole outline fits the vertex cap. */
export function capOutlineVertices(
polygons,
maxVertices = OUTLINE_MAX_VERTICES,
) {
let current = polygons;
let tolerance = 0.00002;
for (let round = 0; round < 12; round++) {
if (countVertices(current) <= maxVertices) return current;
current = polygons.map((rings) =>
rings.map((ring) => simplifyClosedRing(ring, tolerance)),
);
tolerance *= 2;
}
// Still over: keep the largest parts that fit, outer rings only.
const kept = [];
let total = 0;
for (const rings of current) {
const outer = rings[0];
if (total + outer.length > maxVertices) break;
kept.push([outer]);
total += outer.length;
}
return kept.length ? kept : null;
}
/**
* The first result within the limit that suits the ask, names the place and
* carries a polygon — or null with the reason each result was passed over.
*
* @param {object[]} rows Nominatim jsonv2 results, in upstream order.
* @param {{kind: string, query: string}} ask
*/
export function selectOutlineResult(rows, { kind, query }) {
const skipped = [];
for (const row of Array.isArray(rows) ? rows : []) {
const osmClass = row?.category || row?.class;
const osmType = row?.type;
const label = `${osmClass}=${osmType}`;
if (!acceptsOutlineClass(kind, osmClass, osmType)) {
skipped.push(`${label}: wrong type`);
continue;
}
if (
String(osmClass).toLowerCase() === 'boundary' &&
String(osmType).toLowerCase() === 'administrative' &&
!adminLevelFits(kind, row)
) {
skipped.push(`${label}: level`);
continue;
}
if (!outlineNameMatches(row, query)) {
skipped.push(`${label}: name`);
continue;
}
const parsed = polygonsFromGeoJson(row?.geojson);
if (!parsed) {
skipped.push(`${label}: no polygon`);
continue;
}
// A building ask needs building evidence: the building class, or a
// footprint no larger than a large building.
if (
kind === 'building' &&
String(osmClass).toLowerCase() !== 'building' &&
ringAreaM2(parsed[0][0]) > BUILDING_MAX_AREA_M2
) {
skipped.push(`${label}: not a building`);
continue;
}
const polygons = capOutlineVertices(parsed);
if (!polygons) {
skipped.push(`${label}: too large`);
continue;
}
const adminIdentity = selectedAdminIdentity(kind, row);
const lat = Number(row.lat);
const lon = Number(row.lon);
return {
outline: {
name: String(row.name || '').slice(0, 160) || null,
displayName: String(row.display_name || '').slice(0, 300) || null,
class: String(osmClass),
type: String(osmType),
osm:
row.osm_type && row.osm_id != null
? `${row.osm_type}/${row.osm_id}`
: null,
...(adminIdentity || {}),
center: finiteLat(lat) && finiteLon(lon) ? { lat, lon } : null,
polygons,
},
skipped,
};
}
return { outline: null, skipped };
}
/**
* Coarse view bias: the centre is snapped to a grid (1° for places, 0.05°
* for small grounds) so small camera moves reuse the same cache entry, and
* the viewbox sent upstream is built from the snapped centre.
*/
export function outlineBias(kind, lat, lon) {
// Missing is missing: Number(null) and Number('') would read as (0, 0).
const given = (value) =>
value !== null &&
value !== undefined &&
(typeof value === 'number' || String(value).trim() !== '');
if (!given(lat) || !given(lon)) return null;
const latitude = Number(lat);
const longitude = Number(lon);
if (!finiteLat(latitude) || !finiteLon(longitude)) return null;
const small = kind === 'landmark' || kind === 'building';
const step = small ? 0.05 : 1;
const half = small ? 0.25 : 2;
const snap = (value) => Math.round(value / step) * step;
const cLat = Number(snap(latitude).toFixed(2));
const cLon = Number(snap(longitude).toFixed(2));
const clampLat = (value) => Math.max(-90, Math.min(90, value));
const clampLon = (value) => Math.max(-180, Math.min(180, value));
return {
key: `${cLat},${cLon}`,
viewbox: [
clampLon(cLon - half),
clampLat(cLat + half),
clampLon(cLon + half),
clampLat(cLat - half),
]
.map((value) => Number(value.toFixed(4)))
.join(','),
};
}
+198 -359
View File
@@ -1,68 +1,113 @@
import path from 'node:path';
import { makeRateLimiter, clientKey } from '../common/rate-limit.js';
import { coalesceProxyRequest } from '../common/http.js';
import { fetchRegionalJson } from './http.js';
import { naturalRegionAtPoint } from '../../../src/data/naturalEarthRegions.js';
import {
nominatimToGeocodeResult,
nominatimViewboxFromBounds,
} from '../../../src/nominatimGeocode.js';
import {
NOMINATIM_MAX_PENDING,
isNominatimBusy,
sharedNominatimGate,
} from './nominatimGate.js';
import {
NOMINATIM_CACHE_TTL_MS,
createNominatimCache,
} from './nominatimCache.js';
import {
OUTLINE_KINDS,
OUTLINE_POLYGON_THRESHOLD,
outlineBias,
selectOutlineResult,
} from './outlineSelect.js';
/**
* The usage policy for the public Nominatim instance asks for a User-Agent or
* Referer that identifies the application, and states that stock library
* agents will not do. Both are sent.
*/
const NOMINATIM_HEADERS = Object.freeze({
'User-Agent':
'gods-eye-view/0.1 (+https://github.com/bilawalsidhu/gods-eye-view)',
Referer: 'https://github.com/bilawalsidhu/gods-eye-view',
});
/**
* Minimum spacing between upstream calls. The policy states an absolute
* maximum of one request per second; 1.1 s keeps clock jitter from crossing it.
*/
const NOMINATIM_MIN_SPACING_MS = 1100;
/**
* How many searches may be waiting for their turn.
*
* One request per second and an unbounded queue are incompatible: a burst of
* searches would keep the upstream busy long after everyone who asked has given
* up, which is exactly the load the policy asks callers not to create. Past
* this depth a search is refused at once instead of being promised a slot
* minutes away.
*/
const NOMINATIM_MAX_PENDING = 4;
/**
* How long a queued search may wait before it is not worth sending. The
* browser gives up well before this, so anything reaching the front later than
* this is answering nobody.
*/
const NOMINATIM_MAX_WAIT_MS = 10_000;
const NOMINATIM_SEARCH_CACHE_MS = 5 * 60_000;
const NOMINATIM_SEARCH_MAX_CACHE = 80;
const NOMINATIM_SEARCH_MAX_QUERY = 200;
/** Results considered per outline ask: enough to skip a node or a wrong type. */
const OUTLINE_RESULT_LIMIT = 3;
// The pacer is deliberately module state while everything else here is
// per-instance. One request per second is a budget for the whole application,
// not for each provider object: two instances each pacing themselves would
// send two requests a second between them. Only last-resort forward searches
// use this queue.
let _nominatimQueue = Promise.resolve();
/** Largest upstream answer read for an outline (polygons are simplified upstream). */
const OUTLINE_MAX_RESPONSE_BYTES = 4 * 1024 * 1024;
let _nominatimLastRequestAt = 0;
/** Bump when the cached payload shape changes, so old entries are never read. */
const CACHE_SCHEMA = 'v2';
let _nominatimPending = 0;
/**
* The query text actually sent upstream: Unicode-composed, whitespace
* collapsed. Nothing else is changed, so two queries share a cache entry only
* when they are the same upstream request.
*/
export function upstreamQuery(query) {
return String(query ?? '')
.normalize('NFC')
.replace(/\s+/g, ' ')
.trim();
}
/** Cache key: endpoint, schema and the exact upstream parameters. */
function requestKey(gate, kind, params) {
const endpoint = gate.settings?.()?.endpoint || 'none';
return `${CACHE_SCHEMA}|${endpoint}|${kind}|${params}`;
}
function malformed() {
return Object.assign(new Error('Place answer was not understood'), {
code: 'NOMINATIM_MALFORMED',
/** Raised when the queue is already as deep as it is allowed to get. */
function queueFullError() {
return Object.assign(new Error('Place search queue is full'), {
code: 'NOMINATIM_QUEUE_FULL',
});
}
/** Where the dev server keeps Nominatim answers and the daily count. */
export const NOMINATIM_CACHE_DIR = path.join(
process.cwd(),
'.gev-cache',
'nominatim',
);
/** Raised when a queued search waited so long that nobody is left to answer. */
function abandonedError() {
return Object.assign(new Error('Place search was abandoned'), {
code: 'NOMINATIM_ABANDONED',
});
}
/**
* Run one piece of upstream work, never closer than the policy spacing to the
* last one.
*
* `bounded` applies queue-depth and staleness limits to interactive forward searches.
*
* @param {() => Promise<unknown>} work
* @param {{bounded?: boolean, signal?: AbortSignal}} [options]
*/
function enqueueNominatim(work, { bounded = false, signal } = {}) {
if (bounded && _nominatimPending >= NOMINATIM_MAX_PENDING)
return Promise.reject(queueFullError());
if (bounded) _nominatimPending += 1;
const queuedAt = Date.now();
const task = _nominatimQueue.then(async () => {
try {
const waitMs = Math.max(
0,
NOMINATIM_MIN_SPACING_MS - (Date.now() - _nominatimLastRequestAt),
);
if (waitMs) await new Promise((resolve) => setTimeout(resolve, waitMs));
// Nobody is waiting for this any more: sending it would spend the one
// request per second the policy allows on an answer with no reader.
if (bounded && Date.now() - queuedAt > NOMINATIM_MAX_WAIT_MS)
throw abandonedError();
if (signal?.aborted) throw abandonedError();
_nominatimLastRequestAt = Date.now();
return await work();
} finally {
if (bounded) _nominatimPending -= 1;
}
});
_nominatimQueue = task.catch(() => null);
return task;
}
/** Construct the offline regional context provider from bundled Natural Earth polygons. */
export function createRegionalPlaceProvider() {
@@ -72,132 +117,60 @@ export function createRegionalPlaceProvider() {
export const fetchRegionalPlace = createRegionalPlaceProvider();
/**
* Snap a `"swLat,swLng|neLat,neLng"` view to a coarse grid so small camera
* moves reuse one cache entry. The viewbox is a soft preference upstream.
*/
function coarseBounds(bounds) {
const match = String(bounds || '')
.trim()
.split('|')
.map((corner) => corner.split(',').map(Number));
if (match.length !== 2 || match.some((c) => c.length !== 2)) return null;
const [[swLat, swLng], [neLat, neLng]] = match;
if (![swLat, swLng, neLat, neLng].every(Number.isFinite)) return null;
const span = Math.max(neLat - swLat, neLng - swLng, 0.01);
const step = 2 ** Math.ceil(Math.log2(span));
const cLat = Math.round((swLat + neLat) / 2 / (step / 4)) * (step / 4);
const cLng = Math.round((swLng + neLng) / 2 / (step / 4)) * (step / 4);
const clamp = (v, limit) => Math.max(-limit, Math.min(limit, v));
const fix = (v) => Number(v.toFixed(4));
return `${fix(clamp(cLat - step / 2, 90))},${fix(clamp(cLng - step / 2, 180))}|${fix(clamp(cLat + step / 2, 90))},${fix(clamp(cLng + step / 2, 180))}`;
}
/**
* Construct the forward search adapter over the shared gate.
* Construct the forward search adapter with a trusted endpoint.
*
* The policy asks that results be cached, and warns that a client repeating
* the same query may be treated as faulty, so answers are kept (30 days for a
* hit, 24 hours for a genuine no-result; a failure is never kept) and
* identical searches already in flight share one upstream call.
* The policy asks that results be cached, and warns that a client repeating the
* same query may be treated as faulty, so an answer is remembered and identical
* searches already in flight share one upstream call rather than queueing
* behind each other.
*/
export function createNominatimSearchProvider({
gate = () => sharedNominatimGate(),
cache = createNominatimCache(),
endpoint = 'https://nominatim.openstreetmap.org/search',
requestJson = fetchRegionalJson,
} = {}) {
const cache = new Map();
const inFlight = new Map();
const currentGate = () => (typeof gate === 'function' ? gate() : gate);
const trimCache = () => {
while (cache.size > NOMINATIM_SEARCH_MAX_CACHE) {
const oldest = cache.keys().next().value;
if (oldest === undefined) break;
cache.delete(oldest);
}
};
return async function fetchNominatimSearch(query, bounds, { signal } = {}) {
const coarse = coarseBounds(bounds);
const params = new URLSearchParams({
format: 'jsonv2',
q: upstreamQuery(query),
addressdetails: '1',
limit: '1',
'accept-language': 'en',
});
const viewbox = nominatimViewboxFromBounds(coarse);
if (viewbox) params.set('viewbox', viewbox);
const gateNow = currentGate();
const cacheKey = requestKey(gateNow, 'search', params);
const cached = await cache.get(cacheKey);
if (cached) return { ...cached.payload, cached: true };
const cacheKey = `${query.toLowerCase()}|${bounds || ''}`;
const cached = cache.get(cacheKey);
if (cached && Date.now() - cached.cachedAt <= NOMINATIM_SEARCH_CACHE_MS) {
return { ...cached.payload, cached: true };
}
const { promise } = coalesceProxyRequest(inFlight, cacheKey, async () => {
const rows = await gateNow.requestJson(
(endpoint) => `${endpoint}?${params}`,
{ signal },
const params = new URLSearchParams({
format: 'jsonv2',
q: query,
addressdetails: '1',
limit: '1',
'accept-language': 'en',
});
const viewbox = nominatimViewboxFromBounds(bounds);
if (viewbox) params.set('viewbox', viewbox);
const rows = await enqueueNominatim(
() =>
requestJson(`${endpoint}?${params}`, {
headers: NOMINATIM_HEADERS,
redirect: 'error',
}),
{ bounded: true, signal },
);
const result = nominatimToGeocodeResult(
Array.isArray(rows) ? rows[0] : null,
);
// Only a well-formed empty list is "no such place"; anything else is a
// failure and is never cached.
if (!Array.isArray(rows)) throw malformed();
const result = nominatimToGeocodeResult(rows[0] ?? null);
if (rows.length && !result) throw malformed();
const payload = result
? { status: 'OK', results: [result] }
: { status: 'ZERO_RESULTS', results: [] };
await cache.set(
cacheKey,
payload,
result ? NOMINATIM_CACHE_TTL_MS.place : NOMINATIM_CACHE_TTL_MS.notFound,
);
return payload;
});
return await promise;
};
}
/**
* Construct the outline lookup: one guarded search with polygons for one
* explicit ask, accepting only a result whose class suits the ask kind.
* Found outlines are kept 90 days and genuine no-results 24 hours.
*/
export function createNominatimOutlineProvider({
gate = () => sharedNominatimGate(),
cache = createNominatimCache(),
} = {}) {
const inFlight = new Map();
const currentGate = () => (typeof gate === 'function' ? gate() : gate);
return async function fetchNominatimOutline(
{ query, kind, lat, lon },
{ signal } = {},
) {
const bias = outlineBias(kind, lat, lon);
const sent = upstreamQuery(query);
const params = new URLSearchParams({
format: 'jsonv2',
q: sent,
limit: String(OUTLINE_RESULT_LIMIT),
polygon_geojson: '1',
polygon_threshold: String(OUTLINE_POLYGON_THRESHOLD[kind]),
namedetails: '1',
'accept-language': 'en',
});
if (bias) params.set('viewbox', bias.viewbox);
const gateNow = currentGate();
const cacheKey = requestKey(gateNow, `outline-${kind}`, params);
const cached = await cache.get(cacheKey);
if (cached) return { ...cached.payload, cached: true };
const { promise } = coalesceProxyRequest(inFlight, cacheKey, async () => {
const rows = await gateNow.requestJson(
(endpoint) => `${endpoint}?${params}`,
{ signal, maxBytes: OUTLINE_MAX_RESPONSE_BYTES },
);
if (!Array.isArray(rows)) throw malformed();
const { outline, skipped } = selectOutlineResult(rows, {
kind,
query: sent,
});
const payload = outline
? { status: 'OK', outline }
: { status: 'ZERO_RESULTS', outline: null, skipped };
await cache.set(
cacheKey,
payload,
outline
? NOMINATIM_CACHE_TTL_MS.outline
: NOMINATIM_CACHE_TTL_MS.notFound,
);
cache.set(cacheKey, { payload, cachedAt: Date.now() });
trimCache();
return payload;
});
return await promise;
@@ -206,209 +179,75 @@ export function createNominatimOutlineProvider({
export const fetchNominatimSearch = createNominatimSearchProvider();
/** A browser on another site may not spend this install's allowance. */
function crossSite(req) {
return (
String(req.headers?.['sec-fetch-site'] || '').toLowerCase() === 'cross-site'
);
}
function sendJson(res, status, body, headers = {}) {
if (res.writableEnded) return;
res.writeHead(status, {
'Content-Type': 'application/json',
'Cache-Control': 'no-store',
...headers,
});
res.end(JSON.stringify(body));
}
/** Map a gate refusal to the route's answer. Never "not found". */
function sendFailure(res, error, what) {
if (error?.code === 'NOMINATIM_DISABLED') {
sendJson(res, 503, {
error: `${what} is not configured`,
code: 'NOMINATIM_DISABLED',
retryable: false,
});
return;
}
if (error?.code === 'NOMINATIM_DAILY_CAP') {
sendJson(
res,
429,
{
error: `${what} daily allowance is used`,
code: 'NOMINATIM_DAILY_CAP',
retryable: false,
},
{ 'Retry-After': '3600' },
);
return;
}
if (isNominatimBusy(error)) {
const seconds = Math.max(5, Math.ceil((error.retryAfterMs || 0) / 1000));
sendJson(
res,
429,
{ error: `${what} is busy`, code: error.code },
{ 'Retry-After': String(seconds) },
);
return;
}
sendJson(res, 503, {
error: `${what} is temporarily unavailable`,
code: 'NOMINATIM_UNAVAILABLE',
});
}
/**
* Vite plugin: last-resort place search (`/api/geocode`) and voice outlines
* (`/api/geocode/outline`) over the configured Nominatim endpoint.
*
* `storageDir` turns on the disk cache and the persisted daily count; the
* application composition supplies it, tests and embedders may omit it.
*/
export function geocodeProxy({
search,
outline,
storageDir = null,
gate = null,
} = {}) {
const currentGate = () => gate || sharedNominatimGate();
if (storageDir) {
// The shared count and pause live beside, never inside, the answer cache,
// so no cache sweep can reset them.
currentGate().persistUsage(path.join(storageDir, 'state.json'));
const diskCache = createNominatimCache({
dir: path.join(storageDir, 'answers'),
});
void diskCache.sweep();
search ||= createNominatimSearchProvider({
gate: currentGate,
cache: diskCache,
});
outline ||= createNominatimOutlineProvider({
gate: currentGate,
cache: diskCache,
});
}
search ||= gate
? createNominatimSearchProvider({ gate })
: fetchNominatimSearch;
outline ||= createNominatimOutlineProvider({ gate: currentGate });
const searchLimiter = makeRateLimiter({
/** Vite plugin: last-resort place search over the public Nominatim instance. */
export function geocodeProxy({ search = fetchNominatimSearch } = {}) {
const limiter = makeRateLimiter({
windowMs: 60_000,
max: 30,
globalMax: 90,
});
// Outlines are explicit voice asks: a person cannot make many a minute.
const outlineLimiter = makeRateLimiter({
windowMs: 60_000,
max: 12,
globalMax: 30,
});
/** Common front door: method, same-site, rate limit, abandonment signal. */
function admit(req, res, limiter) {
if (req.method !== 'GET') {
sendJson(res, 405, { error: 'Method Not Allowed' });
return null;
}
if (crossSite(req)) {
sendJson(res, 403, { error: 'Cross-site requests are not accepted' });
return null;
}
if (!limiter(clientKey(req))) {
sendJson(
res,
429,
{ error: 'Rate limit exceeded' },
{ 'Retry-After': '10' },
);
return null;
}
// A browser that gave up is no longer waiting; the queue reads this
// before spending its slot.
const abandoned = new AbortController();
req.on?.('aborted', () => abandoned.abort());
res.on?.('close', () => abandoned.abort());
return abandoned.signal;
}
async function handleOutline(req, res, url) {
const signal = admit(req, res, outlineLimiter);
if (!signal) return;
const query = String(url.searchParams.get('q') || '').trim();
const kind = String(url.searchParams.get('kind') || '');
if (
!query ||
query.length > NOMINATIM_SEARCH_MAX_QUERY ||
!OUTLINE_KINDS.includes(kind)
) {
sendJson(res, 400, {
error:
'A place query of 1-200 characters and a known kind are required',
});
return;
}
try {
const payload = await outline(
{
query,
kind,
lat: url.searchParams.get('lat'),
lon: url.searchParams.get('lon'),
},
{ signal },
);
sendJson(
res,
200,
{ status: payload.status, outline: payload.outline || null },
{
'Cache-Control': payload.cached ? 'private, max-age=300' : 'no-store',
},
);
} catch (error) {
sendFailure(res, error, 'Outline lookup');
}
}
async function handleSearch(req, res, url) {
const signal = admit(req, res, searchLimiter);
if (!signal) return;
const query = String(url.searchParams.get('q') || '').trim();
if (!query || query.length > NOMINATIM_SEARCH_MAX_QUERY) {
sendJson(res, 400, {
error: 'A place query of 1-200 characters is required',
});
return;
}
try {
const payload = await search(query, url.searchParams.get('bounds'), {
signal,
});
sendJson(
res,
200,
{ status: payload.status, results: payload.results },
{ 'Cache-Control': payload.cached ? 'public, max-age=60' : 'no-store' },
);
} catch (error) {
sendFailure(res, error, 'Place search');
}
}
function install(middlewares) {
middlewares.use('/api/geocode', async (req, res, next) => {
middlewares.use('/api/geocode', async (req, res) => {
if (req.method !== 'GET') {
res.writeHead(405, { 'Content-Type': 'application/json' });
res.end(JSON.stringify({ error: 'Method Not Allowed' }));
return;
}
if (!limiter(clientKey(req))) {
res.writeHead(429, {
'Content-Type': 'application/json',
'Retry-After': '10',
});
res.end(JSON.stringify({ error: 'Rate limit exceeded' }));
return;
}
const url = new URL(req.url || '', 'http://localhost');
if (url.pathname === '/outline') return handleOutline(req, res, url);
if (url.pathname === '/' || url.pathname === '')
return handleSearch(req, res, url);
if (typeof next === 'function') return next();
sendJson(res, 404, { error: 'Not Found' });
const query = String(url.searchParams.get('q') || '').trim();
if (!query || query.length > NOMINATIM_SEARCH_MAX_QUERY) {
res.writeHead(400, { 'Content-Type': 'application/json' });
res.end(
JSON.stringify({
error: 'A place query of 1-200 characters is required',
}),
);
return;
}
// A browser that gave up is no longer waiting; the queue reads this
// before spending its slot.
const abandoned = new AbortController();
req.on?.('aborted', () => abandoned.abort());
res.on?.('close', () => abandoned.abort());
try {
const payload = await search(query, url.searchParams.get('bounds'), {
signal: abandoned.signal,
});
if (res.writableEnded) return;
res.writeHead(200, {
'Content-Type': 'application/json',
'Cache-Control': payload.cached ? 'public, max-age=60' : 'no-store',
});
res.end(
JSON.stringify({ status: payload.status, results: payload.results }),
);
} catch (error) {
if (res.writableEnded) return;
const busy =
error?.code === 'NOMINATIM_QUEUE_FULL' ||
error?.code === 'NOMINATIM_ABANDONED';
res.writeHead(busy ? 429 : 503, {
'Content-Type': 'application/json',
'Cache-Control': 'no-store',
...(busy ? { 'Retry-After': '5' } : {}),
});
res.end(
JSON.stringify({
error: busy
? 'Place search is busy'
: 'Place search is temporarily unavailable',
}),
);
}
});
}
+9 -2
View File
@@ -1,8 +1,11 @@
import { fileURLToPath } from 'node:url';
import { defineConfig, loadEnv } from 'vite';
import { resolveAllowedHosts } from '../../build/allowedHosts.js';
import { createBrowserViteConfig } from '../../build/vite.js';
import { localProviderPlugins } from '../providers/local.js';
import { localMcpPlugin } from '../mcp/plugin.js';
import { apiNotFoundPlugin } from './api-not-found.js';
import { standaloneVoiceTools } from './voiceTools.js';
const root = fileURLToPath(new URL('../../', import.meta.url));
@@ -13,12 +16,16 @@ export default defineConfig(({ command, mode }) => {
if (process.env[key] === undefined) process.env[key] = value;
}
return createBrowserViteConfig({
plugins: [...localProviderPlugins(), apiNotFoundPlugin()],
plugins: [
...localProviderPlugins({ realtime: { tools: standaloneVoiceTools() } }),
localMcpPlugin(),
apiNotFoundPlugin(),
],
googleApiKey: process.env.GOOGLE_MAPS_API_KEY,
cesiumToken: process.env.CESIUM_ION_TOKEN,
photonUrl: process.env.PHOTON_URL,
host: process.env.HOST,
port: process.env.PORT,
allowedHosts: resolveAllowedHosts(process.env.GEV_ALLOWED_HOSTS),
command,
});
});
+18
View File
@@ -0,0 +1,18 @@
/**
* The voice session's tools for the standalone server: the app actions plus
* the catalog tools voice offers (see src/tools/surfaces.js), which the
* browser runs through the same catalog.
*/
import {
coreTools,
toFunctionTools,
toolsForSurface,
} from '../../src/tools/index.js';
import { realtimeSessionTools } from '../providers/openai/tools.js';
export function standaloneVoiceTools() {
return realtimeSessionTools(
toFunctionTools(toolsForSurface(coreTools, 'voice')),
);
}
+4 -61
View File
@@ -11,7 +11,6 @@ import {
sampleGroundHeight,
} from './annotationResolver.js';
import { ringCentroid } from './drawMode.js';
import { phaseTiming } from '../sources/phaseTiming.js';
// Dev convenience: expose the app's Cesium instance for console/preview probing
// (single shared module instance — avoids dual-Cesium state bugs when testing).
@@ -271,9 +270,7 @@ export function createAnnotationEngine({
results: [],
error: 'destroyed',
};
const list = (Array.isArray(requests) ? requests : [requests]).map(
promoteStreetHighlight,
);
const list = Array.isArray(requests) ? requests : [requests];
if (opts.clearPrevious) clear(); // bumps generation + aborts older pending work
const persist = opts.persist !== false;
@@ -680,7 +677,6 @@ export function createAnnotationEngine({
if (!annotations.has(anno.id)) return; // mark replaced/removed while resolving
anno.pendingOutline = false;
anno.outlineUnavailable = isUnavailableCapability(fp);
const installStart = performance.now();
if (anno.outlineUnavailable) renderer.update(anno);
if (fp && !anno.outlineUnavailable) {
anno.ring = fp.ring;
@@ -722,23 +718,12 @@ export function createAnnotationEngine({
continue;
if ((other.footprintKind || null) !== (anno.footprintKind || null))
continue;
if (!outlinesEqual(other, anno)) continue;
if (!ringsEqual(other.ring, anno.ring)) continue;
annotations.delete(other.id);
renderer.remove(other);
}
}
renderer.sync(annotations);
if (fp && !anno.outlineUnavailable)
phaseTiming(
'install',
installStart,
{
kind: anno.footprintKind || null,
parts: anno.polygons?.length || (anno.ring ? 1 : 0),
points: anno.ring?.length || 0,
},
'outlines',
);
// Final outcome → the voice layer (place names ride as structured DATA fields).
// 'failed' covers both the definitive no-polygon and an exhausted transient
// retry — either way the mark honestly stays a point.
@@ -817,6 +802,7 @@ export function createAnnotationEngine({
// FULL per-vertex geometry comparison (the live cap is only 120 and this runs solely for
// candidates that already matched type+label+anchor, so it's cheap) — so two DIFFERENT
// same-length rings/paths at the same anchor are never falsely merged.
const ringSame = ringsEqual;
const pathSame = (a, b) => {
const la = a ? a.length : 0;
const lb = b ? b.length : 0;
@@ -850,7 +836,7 @@ export function createAnnotationEngine({
if (Boolean(ex.synthesized) !== Boolean(anno.synthesized)) continue;
if ((ex.footprintKind || null) !== (anno.footprintKind || null))
continue;
if (!outlinesEqual(ex, anno)) continue;
if (!ringSame(ex.ring, anno.ring)) continue;
}
} else if (anno.type === 'arrow') {
if (!near(ex.to, anno.to)) continue;
@@ -1324,49 +1310,6 @@ function ringsEqual(a, b) {
return true;
}
/**
* Whole-outline equality: the main ring AND every part with its holes. Two
* outlines sharing an outer ring but differing in parts or holes are two marks.
*/
function outlinesEqual(a, b) {
if (!ringsEqual(a?.ring, b?.ring)) return false;
const parts = (anno) =>
Array.isArray(anno?.polygons) && anno.polygons.length
? anno.polygons
: [[anno?.ring || []]];
const pa = parts(a);
const pb = parts(b);
if (pa.length !== pb.length) return false;
for (let i = 0; i < pa.length; i++) {
if (pa[i].length !== pb[i].length) return false;
for (let j = 0; j < pa[i].length; j++)
if (!ringsEqual(pa[i][j], pb[i][j])) return false;
}
return true;
}
const STREET_WORD_RE =
/\b(?:street|st|avenue|ave|boulevard|blvd|road|rd|drive|dr|lane|ln|highway|hwy|parkway|pkwy|alley)\.?$/i;
/**
* "Highlight Congress Avenue" asks for the street, not a dot. A highlight
* whose entity is a street (by entityKind or by its name) becomes an area
* ask, so it resolves the street outline like any other area, unless the
* caller explicitly turned the footprint off.
*/
export function promoteStreetHighlight(spec) {
if (!spec || typeof spec !== 'object' || spec.footprint === false)
return spec;
if (normalizeType(spec.type) !== 'highlight') return spec;
if (spec.entityKind && spec.entityKind !== 'street') return spec;
const target = String(spec.target || '').split(',')[0];
const street =
spec.entityKind === 'street' || STREET_WORD_RE.test(target.trim());
return street && target.trim()
? { ...spec, type: 'area', entityKind: 'street' }
: spec;
}
// --- helpers ----------------------------------------------------------------
/** The place name(s) a spec attempted to resolve (route waypoints, arrow endpoints, or a single target). */
-96
View File
@@ -1,96 +0,0 @@
// Multipart outline identity and street highlights in the annotation engine.
import test from 'node:test';
import assert from 'node:assert/strict';
import {
createAnnotationEngine,
promoteStreetHighlight,
} from './annotationEngine.js';
function installAnimationFrameStubs(t) {
const raf = globalThis.requestAnimationFrame;
const caf = globalThis.cancelAnimationFrame;
globalThis.requestAnimationFrame = () => 1;
globalThis.cancelAnimationFrame = () => {};
t.after(() => {
if (raf === undefined) delete globalThis.requestAnimationFrame;
else globalThis.requestAnimationFrame = raf;
if (caf === undefined) delete globalThis.cancelAnimationFrame;
else globalThis.cancelAnimationFrame = caf;
});
}
const renderer = { add() {}, update() {}, remove() {}, sync() {} };
const outer = [
[0, 0],
[0.01, 0],
[0.01, 0.01],
[0, 0.01],
[0, 0],
];
const hole = (d) => [
[0.004, 0.004],
[0.004 + d, 0.004],
[0.004 + d, 0.004 + d],
[0.004, 0.004],
];
test('outlines sharing an outer ring but not their holes stay two marks', async (t) => {
installAnimationFrameStubs(t);
const shapes = {
a: [[outer, hole(0.001)]],
b: [[outer, hole(0.002)]],
same: [[outer, hole(0.001)]],
};
const resolveTarget = async ({ target }) => ({
lon: 0.005,
lat: 0.005,
height: 0,
label: target,
source: 'fake',
ring: outer,
polygons: shapes[target],
footprintKind: 'area',
});
const engine = createAnnotationEngine({ viewer: {}, renderer, resolveTarget });
await engine.annotate({ type: 'area', target: 'a', label: 'A' });
await engine.annotate({ type: 'area', target: 'b', label: 'B' });
assert.equal(engine.list().length, 2, 'different holes: different marks');
await engine.annotate({ type: 'area', target: 'same', label: 'Same' });
assert.equal(engine.list().length, 2, 'identical parts and holes: one mark');
});
test('a street highlight asks for the street outline', () => {
assert.deepEqual(
promoteStreetHighlight({ type: 'highlight', target: 'Congress Avenue' }),
{ type: 'area', target: 'Congress Avenue', entityKind: 'street' },
);
assert.equal(
promoteStreetHighlight({ type: 'highlight', target: 'Lombard St, San Francisco' }).type,
'area',
);
assert.equal(
promoteStreetHighlight({ type: 'highlight', target: 'the Alamo', entityKind: 'street' }).type,
'area',
);
for (const spec of [
{ type: 'highlight', target: "St Peter's Basilica" },
{ type: 'highlight', target: 'Zilker Park' },
{ type: 'highlight', target: 'Congress Avenue', footprint: false },
{ type: 'highlight', target: 'Congress Avenue', entityKind: 'building' },
{ type: 'pin', target: 'Congress Avenue' },
])
assert.equal(promoteStreetHighlight(spec), spec, JSON.stringify(spec));
});
test('the engine resolves a bare street highlight with its footprint', async (t) => {
installAnimationFrameStubs(t);
const seen = [];
const resolveTarget = async (args) => {
seen.push(args);
return { lon: -97.74, lat: 30.27, height: 0, label: args.target, source: 'fake', ring: null };
};
const engine = createAnnotationEngine({ viewer: {}, renderer, resolveTarget });
await engine.annotate({ type: 'highlight', target: 'Congress Avenue', label: 'Congress' });
assert.equal(seen[0].footprint, true);
assert.equal(seen[0].entityKind, 'street');
});
-339
View File
@@ -1,339 +0,0 @@
import {
bufferCorridor,
ringAreaM2,
ringCentroid,
} from '../sources/featureGeometry.js';
import { isUnavailableCapability } from '../sources/capability.js';
import {
BUILDING_MAX_DISTANCE_M,
OUTLINE_TILE_ZOOM,
STREET_RADIUS_M,
boxAround,
buildingFromTiles,
createOpenFreeMapOutlineSource,
enclosingAreaFromTiles,
streetFromTiles,
} from '../sources/openFreeMapOutlines.js';
import { createNominatimOutlineClient } from '../sources/nominatimOutlines.js';
/**
* Outline rungs that need no Overpass, around the resolver's existing ladder:
*
* bundled rungs (inside the base ladder)
* → OpenFreeMap tiles for streets and pointed-at buildings
* → operator Overpass, when configured (the base ladder, unchanged)
* → the server's guarded Nominatim outline (cities, landmarks, grounds,
* parks, lakes, campuses, named buildings)
* → OpenFreeMap building at the point, for a named building nobody outlined
* → the base answer: an honest point marked "outline unavailable".
*
* Every rung returns the resolver's footprint shape
* `{ ring, polygons?, kind, heightM, synthesized }` and hands it to the
* resolver's `finish`, which applies the scope caps and drift bound and
* re-centres the anchor. Nothing here fetches without an explicit ask.
*/
/** Half-width (m) of the ribbon drawn along a street. */
const STREET_HALF_WIDTH_M = 11;
/** A Nominatim "grounds" answer smaller than this is a building, not grounds. */
const GROUNDS_MIN_AREA_M2 = 20_000;
/** Words that point at the thing rather than name it. */
const DEICTIC_RE =
/^(?:(?:this|that|the|my)\s+)?(?:building|house|structure|tower|place|one|thing|spot|here|there|it)$/i;
/** Words that describe grounds rather than belong to the name searched. */
const GROUNDS_WORDS_RE =
/\b(?:grounds|compound|campus|premises|property|site|area|complex)\b/gi;
/** Whether the ask names a thing (vs. "this building" at a pointer). */
export function isNamedAsk(target) {
const text = String(target || '').trim();
return Boolean(text) && !DEICTIC_RE.test(text) && /\p{L}/u.test(text);
}
/** The words sent to Nominatim: the ask without grounds words or articles. */
export function outlineQueryText(target) {
return String(target || '')
.replace(GROUNDS_WORDS_RE, ' ')
.replace(/^\s*the\s+/i, '')
.replace(/\s+/g, ' ')
.trim();
}
/**
* Which Nominatim ask kind (if any) suits a resolver scope.
* Neighbourhoods and streets are not sent: bundled packs and tiles own them.
*/
export function nominatimKindFor({ scope, groundsLike, pointLike, around }) {
if (pointLike) return null;
// "Around" asks get a buffer, except grounds: the grounds ARE the thing.
if (around && !groundsLike) return null;
if (scope === 'city') return 'city';
if (scope === 'state' || scope === 'county' || scope === 'country')
return 'admin';
if (groundsLike) return 'landmark';
if (scope === 'building') return 'building';
if (scope === 'compound' || scope === 'auto') return 'landmark';
return null;
}
/** Ray-cast test for a `[lon, lat]` ring. */
function ringHolds(ring, lat, lon) {
let inside = false;
for (let i = 0, j = ring.length - 1; i < ring.length; j = i++) {
const [xi, yi] = ring[i];
const [xj, yj] = ring[j];
if (
yi > lat !== yj > lat &&
lon < ((xj - xi) * (lat - yi)) / (yj - yi) + xi
)
inside = !inside;
}
return inside;
}
/**
* Parts with the one holding the anchor first. A boundary can be largest at
* sea (Tokyo's island municipalities), so size alone does not pick the part
* the ask is about; with no part holding the anchor, the order is kept.
*/
export function anchorPartFirst(polygons, points) {
for (const { lat, lon } of points) {
if (!Number.isFinite(lat) || !Number.isFinite(lon)) continue;
const at = polygons.findIndex(([outer]) => ringHolds(outer, lat, lon));
if (at > 0) return [polygons[at], ...polygons.filter((_, i) => i !== at)];
if (at === 0) return polygons;
}
return polygons;
}
/** Whether a base answer is final (a real outline, or "try again later"). */
function keepBase(result) {
if (result === undefined) return true;
if (result?.rateLimited === true) return true;
if (isUnavailableCapability(result)) return false;
return Boolean(result?.ring) && !result.synthesized;
}
/** Street polylines → one ribbon polygon per chain (longest first). */
export function streetFootprint(street) {
const polygons = street.lines
.filter((line) => line.length >= 2)
.map((line) => [bufferCorridor(line, STREET_HALF_WIDTH_M)]);
if (!polygons.length) return null;
return {
ring: polygons[0][0],
polygons,
lines: street.lines,
kind: 'area',
heightM: null,
synthesized: false,
outlineSource: 'openfreemap',
};
}
export function createOutlineRungs({
tiles = createOpenFreeMapOutlineSource(),
nominatim = createNominatimOutlineClient(),
} = {}) {
/** Decoded tiles around a point, or null when the tiles cannot be read. */
async function tilesAround(lat, lon, radiusM, signal) {
try {
const { tiles: decoded } = await tiles.fetchBounds(
boxAround(lat, lon, radiusM),
{ zoom: OUTLINE_TILE_ZOOM, signal },
);
return decoded;
} catch (error) {
signal?.throwIfAborted();
if (error?.name === 'AbortError') throw error;
return null;
}
}
async function streetRung(ctx) {
const name = ctx.matchName || ctx.target;
if (!isNamedAsk(name)) return null;
const decoded = await tilesAround(
ctx.lat,
ctx.lon,
STREET_RADIUS_M,
ctx.signal,
);
if (!decoded) return null;
const street = streetFromTiles(decoded, {
name,
lat: ctx.lat,
lon: ctx.lon,
});
return street ? streetFootprint(street) : null;
}
async function buildingRung(ctx) {
const decoded = await tilesAround(
ctx.lat,
ctx.lon,
BUILDING_MAX_DISTANCE_M * 2,
ctx.signal,
);
const building = decoded && buildingFromTiles(decoded, ctx);
if (!building) return null;
return {
ring: building.ring,
polygons: [building.rings],
kind: 'building',
heightM: building.heightM,
synthesized: false,
outlineSource: 'openfreemap',
};
}
/** The open area enclosing `seed` (the named feature) or the anchor. */
async function groundsRung(ctx, seed = null) {
const at = seed || { lat: ctx.lat, lon: ctx.lon };
const decoded = await tilesAround(at.lat, at.lon, 600, ctx.signal);
const area =
decoded &&
enclosingAreaFromTiles(decoded, {
lat: at.lat,
lon: at.lon,
minAreaM2: GROUNDS_MIN_AREA_M2,
});
if (!area) return null;
return {
ring: area.ring,
polygons: [area.rings],
kind: 'area',
heightM: null,
synthesized: false,
outlineSource: 'openfreemap',
};
}
async function nominatimRung(ctx, kind) {
const query = outlineQueryText(
kind === 'city' || kind === 'admin'
? ctx.matchName || ctx.target
: ctx.target || ctx.matchName,
);
if (!isNamedAsk(query)) return null;
const bias = ctx.view || { lat: ctx.lat, lon: ctx.lon };
const outline = await nominatim.lookup(
{ query, kind, lat: bias.lat, lon: bias.lon },
{ signal: ctx.signal },
);
if (!outline?.polygons) return outline; // null | undefined | unavailable | rateLimited
const polygons = anchorPartFirst(outline.polygons, [
{ lat: ctx.lat, lon: ctx.lon },
outline.center || {},
]);
const ring = polygons[0][0];
// A grounds ask answered by a building-sized polygon is the dome, not the
// grounds: report no grounds outline rather than the wrong shape.
// A grounds ask answered by a building-sized polygon found the named
// building, not its grounds: keep its position to find what encloses it.
if (ctx.groundsLike && ringAreaM2(ring) < GROUNDS_MIN_AREA_M2)
return { groundsSeed: ringCentroid(ring) };
return {
ring,
polygons,
kind:
kind === 'building' || outline.class === 'building'
? 'building'
: 'area',
heightM: null,
synthesized: false,
outlineSource: 'nominatim',
outlineName: outline.name,
};
}
/**
* Run the ladder for one resolved anchor.
*
* @param {object} ctx scope, target, matchName, lat, lon, named anchor facts,
* view centre, signal, and `credit(kind)`.
* @param {{base: () => Promise<any>, finish: (fp: object, scope: string) => any}} ladder
*/
async function resolve(ctx, { base, finish }) {
const { scope } = ctx;
const named = ctx.fromName && isNamedAsk(ctx.target);
// A cancelled ask (board cleared, superseded) stops between rungs, so it
// never spends a Nominatim request.
const alive = () => ctx.signal?.throwIfAborted();
const accept = (fp, asScope = scope) => {
alive();
if (!fp) return null;
const patch = finish(fp, asScope);
if (patch?.ring) {
ctx.credit?.(fp.outlineSource === 'openfreemap' ? 'tiles' : 'osm');
// One line per answered outline, like the resolver's own trace.
console.log(
`[Outline] "${ctx.target || ''}": scope=${asScope} → ${fp.outlineSource}` +
` (${patch.footprintKind}, ${patch.polygons?.length || 1} part(s))`,
);
return { ...patch, outlineSource: fp.outlineSource };
}
return null;
};
// Tiles first for streets and for a building pointed at rather than named.
if (scope === 'street') {
const street = accept(await streetRung(ctx), 'street');
if (street) return street;
} else if (scope === 'building' && !named && !ctx.around) {
const building = accept(await buildingRung(ctx), 'building');
if (building) return building;
}
alive();
const baseResult = await base();
alive();
if (keepBase(baseResult)) return baseResult;
// A transient or busy Nominatim answer is kept aside while the tile
// fallbacks run; it is returned only if nothing else answers.
let retryLater = null;
let groundsSeed = null;
const kind = named ? nominatimKindFor(ctx) : null;
if (kind) {
const found = await nominatimRung(ctx, kind);
alive();
if (found?.groundsSeed) {
groundsSeed = found.groundsSeed;
} else if (found?.ring) {
const patch = accept(found, ctx.groundsLike ? 'compound' : scope);
if (patch) return patch;
} else if (found === undefined || found?.rateLimited) {
retryLater = { value: found };
}
}
// Grounds nobody named in a polygon: the open area enclosing the point.
if (ctx.groundsLike && named) {
// Around the named building when Nominatim found it, else the anchor.
let grounds = groundsSeed
? accept(await groundsRung(ctx, groundsSeed), 'compound')
: null;
grounds ||= accept(await groundsRung(ctx), 'compound');
if (grounds) return grounds;
}
// A named building nobody outlined: the footprint at its point. Never for
// a grounds ask, where the building is the wrong shape.
if (scope === 'building' && named && !ctx.around && !ctx.groundsLike) {
const building = accept(await buildingRung(ctx), 'building');
if (building) return building;
}
// Keep the mark retryable only when nothing else could answer. A
// definitive miss or an approximation from the configured ladder stands
// rather than inviting repeated lookups.
if (retryLater && isUnavailableCapability(baseResult))
return retryLater.value;
return baseResult;
}
return { resolve };
}
-424
View File
@@ -1,424 +0,0 @@
// OVERPASS-FREE OUTLINE RUNGS.
//
// Streets and pointed-at buildings come from OpenFreeMap tiles before the
// existing ladder; cities, landmarks, grounds and named buildings fall to the
// server's guarded Nominatim outline after it. With default settings no
// request reaches any Overpass endpoint.
import test from 'node:test';
import assert from 'node:assert/strict';
import { mkdtempSync, readFileSync, rmSync } from 'node:fs';
import os from 'node:os';
import path from 'node:path';
import { Readable } from 'node:stream';
import {
createOutlineRungs,
isNamedAsk,
nominatimKindFor,
outlineQueryText,
} from './outlineRungs.js';
import { createAnnotationResolver } from './resolver.js';
import { decodeOpenFreeMapOutlineTile } from '../sources/openFreeMapOutlines.js';
import { createNominatimOutlineClient } from '../sources/nominatimOutlines.js';
import { createApplicationRequestServices } from '../services/requests.js';
import { overpassProxy } from '../../server/providers/overpass.js';
import { geocodeProxy } from '../../server/providers/regional/place.js';
import {
createGateStateStore,
createNominatimGate,
} from '../../server/providers/regional/nominatimGate.js';
const TILES = [6745, 6746].map((y) =>
decodeOpenFreeMapOutlineTile(
readFileSync(
new URL(`../data/fixtures/ofm-outlines-austin-14-3743-${y}.pbf`, import.meta.url),
),
14,
3743,
y,
),
);
const fixtureTiles = () => {
const asked = [];
return {
asked,
fetchBounds: async (box, options) => {
asked.push({ box, zoom: options.zoom });
return { tiles: TILES };
},
};
};
const CAPITOL = { lat: 30.27472, lon: -97.74035 };
const square = (lat, lon, halfDeg) => [
[lon - halfDeg, lat - halfDeg],
[lon + halfDeg, lat - halfDeg],
[lon + halfDeg, lat + halfDeg],
[lon - halfDeg, lat + halfDeg],
[lon - halfDeg, lat - halfDeg],
];
function fakeNominatim(answer) {
const asks = [];
return {
asks,
lookup: async (ask) => {
asks.push(ask);
return typeof answer === 'function' ? answer(ask) : answer;
},
};
}
/** A ladder whose base answers `baseAnswer` and whose finish passes rings through. */
function ladder(baseAnswer) {
const calls = { base: 0, finished: [] };
return {
calls,
base: async () => {
calls.base += 1;
return baseAnswer;
},
finish: (fp, scope) => {
calls.finished.push(scope);
return { ring: fp.ring, polygons: fp.polygons, footprintKind: fp.kind, synthesized: false };
},
};
}
const UNAVAILABLE = { unavailable: true, retryable: false, code: 'OVERPASS_NOT_CONFIGURED' };
const ctx = (overrides) => ({
scope: 'auto',
target: 'X',
matchName: 'X',
fromName: true,
pointLike: false,
groundsLike: false,
around: false,
...CAPITOL,
view: { lat: CAPITOL.lat, lon: CAPITOL.lon },
credit() {},
...overrides,
});
test('ask wording helpers', () => {
assert.equal(isNamedAsk('this building'), false);
assert.equal(isNamedAsk('here'), false);
assert.equal(isNamedAsk('Texas Capitol'), true);
assert.equal(outlineQueryText('the Texas Capitol grounds'), 'Texas Capitol');
assert.equal(nominatimKindFor({ scope: 'city' }), 'city');
assert.equal(nominatimKindFor({ scope: 'state' }), 'admin');
assert.equal(nominatimKindFor({ scope: 'compound' }), 'landmark');
assert.equal(nominatimKindFor({ scope: 'building', groundsLike: true }), 'landmark');
assert.equal(nominatimKindFor({ scope: 'neighborhood' }), null);
assert.equal(nominatimKindFor({ scope: 'street' }), null);
assert.equal(nominatimKindFor({ scope: 'compound', pointLike: true }), null);
});
test('a street comes from tiles before the configured ladder, as merged ribbons', async () => {
const tiles = fixtureTiles();
const nominatim = fakeNominatim(null);
const credits = [];
const run = ladder(UNAVAILABLE);
const result = await createOutlineRungs({ tiles, nominatim }).resolve(
ctx({ scope: 'street', target: 'Congress Avenue', matchName: 'Congress Avenue', credit: (k) => credits.push(k) }),
run,
);
assert.ok(result.ring.length > 6);
assert.equal(result.polygons.length, 1);
assert.equal(result.outlineSource, 'openfreemap');
assert.equal(run.calls.base, 0, 'the ladder is not consulted');
assert.deepEqual(run.calls.finished, ['street']);
assert.deepEqual(credits, ['tiles']);
assert.equal(tiles.asked[0].zoom, 14);
assert.equal(nominatim.asks.length, 0, 'streets never reach Nominatim');
});
test('a street the tiles do not name falls through to the ladder', async () => {
const run = ladder(UNAVAILABLE);
const result = await createOutlineRungs({ tiles: fixtureTiles(), nominatim: fakeNominatim(null) }).resolve(
ctx({ scope: 'street', target: 'Lombard Street', matchName: 'Lombard Street' }),
run,
);
assert.equal(result, UNAVAILABLE);
assert.equal(run.calls.base, 1);
});
test('"circle this building" picks the tile building at the point first', async () => {
const nominatim = fakeNominatim(null);
const run = ladder(UNAVAILABLE);
const result = await createOutlineRungs({ tiles: fixtureTiles(), nominatim }).resolve(
ctx({ scope: 'building', target: 'this building', fromName: false }),
run,
);
assert.equal(result.footprintKind, 'building');
assert.equal(run.calls.base, 0);
assert.equal(nominatim.asks.length, 0);
});
test('a named building asks Nominatim, then falls back to the tile building', async () => {
const nominatim = fakeNominatim(null);
const result = await createOutlineRungs({ tiles: fixtureTiles(), nominatim }).resolve(
ctx({ scope: 'building', target: 'Texas State Capitol' }),
ladder(UNAVAILABLE),
);
assert.deepEqual(nominatim.asks.map((a) => a.kind), ['building']);
assert.equal(result.footprintKind, 'building');
assert.equal(result.outlineSource, 'openfreemap');
});
test('a city with no configured Overpass is outlined by Nominatim', async () => {
const ring = square(43.6, 1.44, 0.05);
const nominatim = fakeNominatim({ polygons: [[ring]], class: 'boundary' });
const credits = [];
const run = ladder(UNAVAILABLE);
const result = await createOutlineRungs({ tiles: fixtureTiles(), nominatim }).resolve(
ctx({ scope: 'city', target: 'Toulouse, France', matchName: 'Toulouse', credit: (k) => credits.push(k) }),
run,
);
assert.deepEqual(result.ring, ring);
assert.equal(result.outlineSource, 'nominatim');
assert.deepEqual(nominatim.asks[0], { query: 'Toulouse', kind: 'city', lat: CAPITOL.lat, lon: CAPITOL.lon });
assert.deepEqual(run.calls.finished, ['city'], 'scope caps and drift bound still apply');
assert.deepEqual(credits, ['osm']);
});
test('a multi-part city leads with the part holding the anchor, not the largest', async () => {
const sea = square(33.5, 139.5, 0.8); // larger, far from the anchor
const land = square(CAPITOL.lat, CAPITOL.lon, 0.2);
const nominatim = fakeNominatim({ polygons: [[sea], [land]], class: 'boundary' });
const result = await createOutlineRungs({ tiles: fixtureTiles(), nominatim }).resolve(
ctx({ scope: 'city', target: 'Tokyo', matchName: 'Tokyo' }),
ladder(UNAVAILABLE),
);
assert.deepEqual(result.ring, land, 'the main ring (centroid, scope cap) is the anchored part');
assert.equal(result.polygons.length, 2, 'every part is still drawn');
});
test('a real outline from the ladder (bundled or operator Overpass) is kept', async () => {
const nominatim = fakeNominatim(null);
const outline = { ring: square(0, 0, 1), synthesized: false };
const result = await createOutlineRungs({ tiles: fixtureTiles(), nominatim }).resolve(
ctx({ scope: 'city', target: 'Austin' }),
ladder(outline),
);
assert.equal(result, outline);
assert.equal(nominatim.asks.length, 0);
});
test('a transient ladder failure stays retryable without spending a lookup', async () => {
const nominatim = fakeNominatim(null);
const result = await createOutlineRungs({ tiles: fixtureTiles(), nominatim }).resolve(
ctx({ scope: 'compound', target: 'Zilker Park' }),
ladder(undefined),
);
assert.equal(result, undefined);
assert.equal(nominatim.asks.length, 0);
});
test('no Nominatim outline leaves the honest point; busy stays retryable', async () => {
const rungs = (answer) => createOutlineRungs({ tiles: fixtureTiles(), nominatim: fakeNominatim(answer) });
assert.equal(await rungs(null).resolve(ctx({ scope: 'compound', target: 'Nowhere Park' }), ladder(UNAVAILABLE)), UNAVAILABLE);
const busy = { rateLimited: true, retryAfterMs: 5000 };
assert.equal(await rungs(busy).resolve(ctx({ scope: 'compound', target: 'Zilker Park' }), ladder(UNAVAILABLE)), busy);
// After an operator's definitive miss, a busy Nominatim does not invite retries.
assert.equal(await rungs(busy).resolve(ctx({ scope: 'compound', target: 'Zilker Park' }), ladder(null)), null);
});
test('grounds: a building-sized answer is not the grounds; a real enclosure is', async () => {
const dome = { polygons: [[square(30.2747, -97.7403, 0.0003)]], class: 'office' };
const grounds = { polygons: [[square(30.2747, -97.7403, 0.002)]], class: 'leisure' };
// The dome is refused; the open area enclosing the point stands in.
const small = await createOutlineRungs({ tiles: fixtureTiles(), nominatim: fakeNominatim(dome) }).resolve(
ctx({ scope: 'building', target: 'Texas Capitol grounds', groundsLike: true }),
ladder(UNAVAILABLE),
);
assert.equal(small.outlineSource, 'openfreemap');
assert.equal(small.footprintKind, 'area');
// Without enclosing tile cover, the honest point remains.
const bare = await createOutlineRungs({
tiles: { fetchBounds: async () => ({ tiles: [] }) },
nominatim: fakeNominatim(dome),
}).resolve(ctx({ scope: 'building', target: 'Texas Capitol grounds', groundsLike: true }), ladder(UNAVAILABLE));
assert.equal(bare, UNAVAILABLE);
const nominatim = fakeNominatim(grounds);
const run = ladder(UNAVAILABLE);
const big = await createOutlineRungs({ tiles: fixtureTiles(), nominatim }).resolve(
ctx({ scope: 'building', target: 'the Texas Capitol grounds', groundsLike: true }),
run,
);
assert.equal(big.footprintKind, 'area');
assert.deepEqual(run.calls.finished, ['compound']);
assert.deepEqual(nominatim.asks.map((a) => [a.query, a.kind]), [['Texas Capitol', 'landmark']]);
});
test('point-like, around and neighbourhood asks never reach Nominatim', async () => {
for (const extra of [
{ scope: 'compound', pointLike: true, target: 'Tejano Monument' },
{ scope: 'compound', around: true, target: 'Zilker Park' },
{ scope: 'neighborhood', target: 'Mission District' },
]) {
const nominatim = fakeNominatim(null);
await createOutlineRungs({ tiles: fixtureTiles(), nominatim }).resolve(ctx(extra), ladder(UNAVAILABLE));
assert.equal(nominatim.asks.length, 0, JSON.stringify(extra));
}
});
// ------------------------------------------------------------ end to end
function routes(...plugins) {
const found = new Map();
for (const plugin of plugins)
plugin.configureServer({ middlewares: { use: (path, handler) => found.set(path, handler) } });
return found;
}
async function call(handlers, url, init = {}) {
const parsed = new URL(url, 'http://localhost');
const mount = [...handlers.keys()]
.filter((path) => parsed.pathname === path || parsed.pathname.startsWith(`${path}/`))
.sort((a, b) => b.length - a.length)[0];
if (!mount) return new Response('{}', { status: 404 });
const req = Readable.from(init.body ? [Buffer.from(init.body)] : []);
Object.assign(req, {
method: init.method || 'GET',
url: parsed.pathname.slice(mount.length) + parsed.search || '/',
headers: {},
socket: { remoteAddress: '127.0.0.1' },
});
let status;
let headers;
let body;
await handlers.get(mount)(
req,
{
on() {},
writeHead(s, h) {
status = s;
headers = h;
},
end(b) {
body = b;
},
},
() => {
status = 404;
body = '{}';
},
);
return new Response(body, { status, headers });
}
const NOMINATIM_ROWS = {
Toulouse: [
{ category: 'place', type: 'city', name: 'Toulouse', lat: '43.6', lon: '1.44', geojson: { type: 'Point', coordinates: [1.44, 43.6] } },
{ category: 'boundary', type: 'administrative', addresstype: 'city', name: 'Toulouse', lat: '43.6', lon: '1.44', osm_type: 'relation', osm_id: 35738, geojson: { type: 'Polygon', coordinates: [square(43.6, 1.44, 0.05)] } },
],
'Central Park': [
{ category: 'leisure', type: 'park', name: 'Central Park', lat: '40.78', lon: '-73.965', osm_type: 'relation', osm_id: 8, geojson: { type: 'Polygon', coordinates: [square(40.78, -73.965, 0.01)] } },
],
};
const PLACES = {
Toulouse: { lat: 43.6, lng: 1.44, name: 'Toulouse', label: 'Toulouse, France', types: ['locality', 'political'] },
'Central Park': { lat: 40.78, lng: -73.965, name: 'Central Park', label: 'Central Park, New York', types: ['park'] },
'Congress Avenue': { lat: 30.268, lng: -97.7427, name: 'Congress Avenue', label: 'Congress Ave, Austin', types: ['route'] },
};
const viewerAt = (lat, lon, height = 4000) => ({
camera: { positionCartographic: { latitude: (lat * Math.PI) / 180, longitude: (lon * Math.PI) / 180, height } },
});
test('default settings: outline, street and building asks send nothing to Overpass', async (t) => {
const prior = process.env.OVERPASS_UPSTREAMS;
delete process.env.OVERPASS_UPSTREAMS;
t.after(() => {
if (prior !== undefined) process.env.OVERPASS_UPSTREAMS = prior;
});
// Any request that escapes the injected transports fails the test.
t.mock.method(globalThis, 'fetch', async (url) => assert.fail(`unexpected network request ${url}`));
const upstream = [];
const stateDir = mkdtempSync(path.join(os.tmpdir(), 'gev-outline-rungs-'));
t.after(() => rmSync(stateDir, { recursive: true, force: true }));
const gate = createNominatimGate({
settings: { endpoint: 'https://nominatim.openstreetmap.org/search', isPublic: true, dailyCap: 50 },
fetchImpl: async (url) => {
upstream.push(String(url));
const q = new URL(url).searchParams.get('q');
return Response.json(NOMINATIM_ROWS[q] || []);
},
usage: createGateStateStore({ file: path.join(stateDir, 'state.json') }),
sleep: async () => {},
});
const handlers = routes(overpassProxy(), geocodeProxy({ gate }));
const local = [];
const transport = (url, init) => {
local.push(`${init?.method || 'GET'} ${String(url).split('?')[0]}`);
return call(handlers, url, init);
};
const services = createApplicationRequestServices({ fetchImpl: transport });
const resolver = createAnnotationResolver({
featureSource: services.features,
outlineRungs: createOutlineRungs({
tiles: fixtureTiles(),
nominatim: createNominatimOutlineClient({ fetchImpl: transport }),
}),
});
const placeSearch = { geocode: async (q) => ({ place: PLACES[q] || null, answered: true }) };
const resolve = async (viewer, args) => {
const result = await resolver.resolveAnnotationTarget({ viewer, placeSearch, footprint: true, deferFootprint: true, ...args });
return { result, outline: await result.resolveOutline() };
};
const paris = await resolve(viewerAt(43.6, 1.44, 40_000), { target: 'Toulouse' });
assert.ok(paris.outline?.ring?.length >= 4, 'city outlined');
assert.equal(paris.outline.outlineSource, 'nominatim');
const park = await resolve(viewerAt(40.78, -73.965), { target: 'Central Park' });
assert.equal(park.outline?.outlineSource, 'nominatim');
const street = await resolve(viewerAt(30.27, -97.742), { target: 'Congress Avenue' });
assert.equal(street.outline?.outlineSource, 'openfreemap');
const building = await resolve(viewerAt(30.2747, -97.7403, 800), {
latitude: CAPITOL.lat,
longitude: CAPITOL.lon,
entityKind: 'building',
target: 'this building',
});
assert.equal(building.outline?.footprintKind, 'building');
assert.equal(building.outline?.outlineSource, 'openfreemap');
assert.ok(upstream.every((url) => url.startsWith('https://nominatim.openstreetmap.org/search?')));
assert.equal(upstream.length, 2, 'one upstream request per explicit ask that needed one');
assert.ok(!upstream.some((url) => /overpass/i.test(url)));
// The only Overpass traffic is the local capability probe, answered locally.
assert.ok(
local.filter((line) => line.includes('/api/overpass')).every((line) => line === 'GET /api/overpass/status'),
local.join('\n'),
);
});
test('a cancelled ask stops between rungs and never reaches Nominatim', async () => {
const nominatim = fakeNominatim({ polygons: [[square(43.6, 1.44, 0.05)]], class: 'boundary' });
const rungs = createOutlineRungs({ tiles: fixtureTiles(), nominatim });
// Cancelled while the configured ladder ran.
const controller = new AbortController();
await assert.rejects(
rungs.resolve(ctx({ scope: 'city', target: 'Toulouse', signal: controller.signal }), {
base: async () => {
controller.abort();
return UNAVAILABLE;
},
finish: (fp) => fp,
}),
{ name: 'AbortError' },
);
// Cancelled before it started: not even the ladder runs.
const run = ladder(UNAVAILABLE);
await assert.rejects(
rungs.resolve(ctx({ scope: 'city', target: 'Toulouse', signal: AbortSignal.abort() }), run),
{ name: 'AbortError' },
);
assert.equal(run.calls.base, 0);
assert.equal(nominatim.asks.length, 0);
});
@@ -1,145 +0,0 @@
// Tile fallbacks still run after a busy or failed Nominatim answer, and
// grounds asks phrased "around" still reach their outline rungs.
import test from 'node:test';
import assert from 'node:assert/strict';
import { readFileSync } from 'node:fs';
import { createOutlineRungs, nominatimKindFor } from './outlineRungs.js';
import { decodeOpenFreeMapOutlineTile } from '../sources/openFreeMapOutlines.js';
const TILES = [6745, 6746].map((y) =>
decodeOpenFreeMapOutlineTile(
readFileSync(
new URL(
`../data/fixtures/ofm-outlines-austin-14-3743-${y}.pbf`,
import.meta.url,
),
),
14,
3743,
y,
),
);
const UNAVAILABLE = { unavailable: true, retryable: false };
const CAPITOL = { lat: 30.27472, lon: -97.74035 };
function setup(answer) {
const tileCalls = [];
const asks = [];
const rungs = createOutlineRungs({
tiles: {
fetchBounds: async (box) => {
tileCalls.push(box);
return { tiles: TILES };
},
},
nominatim: {
lookup: async (ask) => {
asks.push(ask);
return answer;
},
},
});
return { rungs, tileCalls, asks };
}
const ladder = {
base: async () => UNAVAILABLE,
finish: (fp) => ({ ring: fp.ring, footprintKind: fp.kind }),
};
const ctx = (overrides) => ({
scope: 'building',
target: 'Texas State Capitol',
matchName: 'Texas State Capitol',
fromName: true,
...CAPITOL,
view: CAPITOL,
...overrides,
});
for (const [label, answer] of [
['busy', { rateLimited: true, retryAfterMs: 5000 }],
['transient', undefined],
]) {
test(`a ${label} Nominatim answer does not skip the tile fallbacks`, async () => {
const building = setup(answer);
const found = await building.rungs.resolve(ctx(), ladder);
assert.equal(building.asks.length, 1);
assert.ok(building.tileCalls.length > 0, 'tiles were consulted');
assert.equal(found.footprintKind, 'building');
const grounds = setup(answer);
const lawn = await grounds.rungs.resolve(
ctx({ target: 'the Texas Capitol grounds', groundsLike: true }),
ladder,
);
assert.equal(lawn.footprintKind, 'area');
assert.equal(lawn.outlineSource, 'openfreemap');
});
}
test('with no tile answer either, the busy result keeps the mark retryable', async () => {
const busy = { rateLimited: true, retryAfterMs: 5000 };
const rungs = createOutlineRungs({
tiles: { fetchBounds: async () => ({ tiles: [] }) },
nominatim: { lookup: async () => busy },
});
assert.equal(await rungs.resolve(ctx(), ladder), busy);
});
test('"around the Capitol grounds" is still a grounds ask', async () => {
assert.equal(
nominatimKindFor({ scope: 'building', groundsLike: true, around: true }),
'landmark',
);
assert.equal(nominatimKindFor({ scope: 'compound', around: true }), null);
const { rungs, asks } = setup(null);
const lawn = await rungs.resolve(
ctx({
target: 'the Texas Capitol grounds',
groundsLike: true,
around: true,
}),
ladder,
);
assert.equal(asks.length, 1);
assert.equal(lawn.outlineSource, 'openfreemap');
});
test('grounds are found around the named building when the anchor is off the lawn', async () => {
// A place-search anchor ~250 m from the dome, outside the Capitol lawn.
const offLawn = { lat: 30.27265, lon: -97.73925 };
const d = 0.0003;
const dome = {
polygons: [
[
[
[CAPITOL.lon - d, CAPITOL.lat - d],
[CAPITOL.lon + d, CAPITOL.lat - d],
[CAPITOL.lon + d, CAPITOL.lat + d],
[CAPITOL.lon - d, CAPITOL.lat + d],
[CAPITOL.lon - d, CAPITOL.lat - d],
],
],
],
class: 'office',
};
const withoutSeed = setup(null);
assert.equal(
await withoutSeed.rungs.resolve(
ctx({
...offLawn,
target: 'the Texas Capitol grounds',
groundsLike: true,
}),
ladder,
),
UNAVAILABLE,
'the anchor alone is not inside the grounds',
);
const { rungs } = setup(dome);
const lawn = await rungs.resolve(
ctx({ ...offLawn, target: 'the Texas Capitol grounds', groundsLike: true }),
ladder,
);
assert.equal(lawn?.outlineSource, 'openfreemap');
assert.equal(lawn.footprintKind, 'area');
});
-407
View File
@@ -1,407 +0,0 @@
// Area annotations draw bundled US Census place and neighborhood outlines
// (San Francisco DataSF first, then Who's On First) with no network lookup.
import { test } from 'node:test';
import assert from 'node:assert/strict';
import { readFileSync } from 'node:fs';
import { FEATURE_SOURCE_METHODS } from '../sources/featureSource.js';
import { createAnnotationResolver } from './resolver.js';
function viewerAt(lat, lon, height = 20_000) {
return {
camera: {
positionCartographic: {
latitude: (lat * Math.PI) / 180,
longitude: (lon * Math.PI) / 180,
height,
},
},
};
}
/** A resolver whose boundary lookups and geocoder record every call. */
function harness(place = null, { outlineRungs } = {}) {
const lookups = [];
const featureSource = {};
for (const method of FEATURE_SOURCE_METHODS)
featureSource[method] = async () => {
lookups.push(method);
return null;
};
const geocodes = [];
const placeSearch = {
async geocode(query) {
geocodes.push(query);
return { place };
},
async textSearch() {
geocodes.push('text-search');
return [];
},
};
const { resolveAnnotationTarget } = createAnnotationResolver({
featureSource,
...(outlineRungs ? { outlineRungs } : {}),
});
const resolve = (target, viewer, extra = {}) =>
resolveAnnotationTarget({
placeSearch,
viewer,
target,
footprint: true,
deferFootprint: true,
...extra,
});
return { resolve, lookups, geocodes };
}
const AUSTIN = viewerAt(30.2672, -97.7431, 3000);
const SAN_FRANCISCO = viewerAt(37.76, -122.42, 3000);
const LONDON = viewerAt(51.51, -0.16, 3000);
function assertBundledArea(resolved, label) {
assert.equal(resolved?.source, 'bundled', label);
assert.equal(resolved.label, label);
assert.equal(resolved.footprintKind, 'area');
assert.equal(resolved.synthesized, false);
assert.equal(resolved.resolveOutline, undefined, 'drawn at once');
assert.deepEqual(resolved.ring[0], resolved.ring.at(-1), 'ring closed');
assert.ok(resolved.polygons.length >= 1);
}
test('a city name that is only a neighborhood alias goes to the geocoder', async () => {
const { resolve, geocodes } = harness();
// Sé lists "São Paulo" among its names; the ask means the city.
const saoPaulo = viewerAt(-23.55, -46.633, 60_000);
const city = await resolve('São Paulo', saoPaulo);
assert.notEqual(city?.label, 'Se');
assert.ok(geocodes.includes('São Paulo'));
// A district ask still takes the alias.
const district = await resolve('São Paulo', saoPaulo, {
entityKind: 'district',
});
assert.equal(district?.label, 'Se');
});
test('"Austin" over Austin draws the Census place with no lookup', async () => {
const { resolve, lookups, geocodes } = harness();
for (const target of ['Austin', 'Austin, Texas', 'City of Austin']) {
const resolved = await resolve(target, AUSTIN);
assertBundledArea(resolved, 'Austin');
assert.ok(resolved.polygons.length >= 4, 'every part of the city');
}
// A qualified name needs no view.
const springfield = await resolve('Springfield, Illinois', AUSTIN);
assertBundledArea(springfield, 'Springfield');
assert.ok(Math.abs(springfield.lat - 39.78) < 0.2);
assert.deepEqual(geocodes, []);
assert.deepEqual(lookups, []);
});
test("the SF DataSF pack answers before Who's On First in San Francisco", async () => {
const { resolve, lookups, geocodes } = harness();
const sf = JSON.parse(
readFileSync(
new URL(
'../data/local_data/neighborhoods/san-francisco.json',
import.meta.url,
),
),
);
const mission = sf.features.find((f) => f.properties.name === 'Mission');
const missionRing = mission.geometry.coordinates[0];
for (const target of ['the Mission District', 'Mission', 'the Mission']) {
const resolved = await resolve(target, SAN_FRANCISCO, {
entityKind: 'district',
});
assertBundledArea(resolved, 'Mission');
assert.deepEqual(resolved.polygons[0][0], missionRing, target);
}
// A name only WOF has in San Francisco falls through to it.
const dogpatch = await resolve('Dogpatch', SAN_FRANCISCO);
assertBundledArea(dogpatch, 'Dogpatch');
// A park asked for as a compound is not a neighborhood of its name.
await resolve('Mission Dolores Park', SAN_FRANCISCO, {
entityKind: 'compound',
});
assert.equal(geocodes.filter((q) => q !== 'text-search').length, 1);
assert.deepEqual(lookups, []);
});
test("Who's On First neighborhoods resolve near the view", async () => {
const { resolve, geocodes } = harness();
assertBundledArea(await resolve('Notting Hill', LONDON), 'Notting Hill');
assertBundledArea(
await resolve('Le Marais', viewerAt(48.857, 2.352, 3000)),
'Le Marais',
);
assertBundledArea(
await resolve('Williamsburg', viewerAt(40.71, -73.96, 3000)),
'Williamsburg',
);
assert.deepEqual(geocodes, []);
});
test('campuses, qualified hoods and far names still go to the geocoder', async () => {
const { resolve, geocodes } = harness();
// Stanford is a CDP, but a campus ask wants the campus.
const stanford = await resolve('Stanford', viewerAt(37.43, -122.17, 3000), {
entityKind: 'compound',
});
assert.notEqual(stanford?.source, 'bundled');
await resolve('the Capitol grounds', AUSTIN);
await resolve('Notting Hill', AUSTIN);
assert.equal(geocodes.filter((q) => q !== 'text-search').length, 3);
// A state settles a place name wherever the camera is: not Brooklyn's
// Williamsburg, the Virginia city.
const virginia = await resolve(
'Williamsburg, Virginia',
viewerAt(40.71, -73.96, 3000),
);
assertBundledArea(virginia, 'Williamsburg');
assert.ok(Math.abs(virginia.lat - 37.27) < 0.1, 'Virginia');
});
test('a geocoded city or neighborhood upgrades to the bundled outline', async () => {
const city = harness({
lat: 30.2672,
lng: -97.7431,
label: 'Austin, TX, USA',
name: 'Austin',
types: ['locality', 'political'],
});
// Seen from far away: the view does not settle the name, the geocoder does.
const resolved = await city.resolve('Austin', viewerAt(39, -100, 5_000_000));
assert.equal(resolved.source, 'geocode');
const outline = await resolved.resolveOutline();
assert.equal(outline.adminArea, 'Austin');
assert.ok(outline.polygons.length >= 4);
assert.deepEqual(city.lookups, [], 'no boundary lookup');
const hood = harness({
lat: 51.5118,
lng: -0.2046,
label: 'Notting Hill, London, UK',
name: 'Notting Hill',
types: ['neighborhood', 'political'],
});
const nottingHill = await hood.resolve(
'Notting Hill',
viewerAt(48, 10, 5_000_000),
);
const hoodOutline = await nottingHill.resolveOutline();
assert.equal(hoodOutline.adminArea, 'Notting Hill');
assert.deepEqual(hood.lookups, []);
});
test('a district ask prefers a neighborhood; a place must hold the view', async () => {
const { resolve, geocodes } = harness();
// "Richmond" over SF's Richmond District: never Richmond, California.
const richmond = await resolve('Richmond', viewerAt(37.78, -122.48, 3000), {
entityKind: 'district',
});
assertBundledArea(richmond, 'Richmond District');
assert.ok(Math.abs(richmond.lat - 37.78) < 0.05, 'not the East Bay city');
// Omitted kind at a street-level view: the district in view, too.
const bare = await resolve('Richmond', viewerAt(37.78, -122.48, 3000));
assertBundledArea(bare, 'Richmond District');
// Over the East Bay city, the city.
const wide = await resolve('Richmond', viewerAt(37.94, -122.35, 40_000));
assert.equal(wide?.source, 'bundled');
assert.ok(wide.lat > 37.88, 'Richmond, California');
assert.deepEqual(geocodes, []);
});
test('landmark-named neighborhoods wait for the landmark ladder', async () => {
const manhattan = viewerAt(40.78, -73.965, 3000);
const cases = [
['Central Park', {}],
['Central Park', { entityKind: 'compound' }],
['Central Park', { intent: 'around_the_thing' }],
['Hyde Park', {}],
['Notting Hill', { intent: 'around_the_thing' }],
];
for (const [target, extra] of cases) {
const { resolve, geocodes } = harness();
const viewer = target === 'Central Park' ? manhattan : LONDON;
const resolved = await resolve(target, viewer, extra);
assert.notEqual(
resolved?.source,
'bundled',
`${target} ${JSON.stringify(extra)}`,
);
assert.ok(geocodes.length >= 1, `${target} is geocoded`);
}
// Named as a district, the WOF neighborhood answers.
const { resolve } = harness();
const park = await resolve('Central Park', manhattan, {
entityKind: 'district',
});
assert.equal(park?.source, 'bundled');
// "Park Slope" ends in no landmark word: still a neighborhood.
const slope = await resolve('Park Slope', viewerAt(40.671, -73.98, 3000));
assertBundledArea(slope, 'Park Slope');
});
/** Outline rungs whose Nominatim answers `answer` (null: nothing found). */
function rungsAnswering(answer) {
const asked = [];
return {
asked,
resolve: async (ctx, { base, finish }) => {
asked.push({ scope: ctx.scope, target: ctx.target });
const baseResult = await base();
if (baseResult?.ring && !baseResult.synthesized) return baseResult;
return answer ? finish(answer, ctx.scope) : baseResult;
},
};
}
const PARIS_PLACE = {
lat: 48.8566,
lng: 2.3522,
label: 'Paris, France',
name: 'Paris',
types: ['locality', 'political'],
};
const square = (lat, lon, d) => [
[lon - d, lat - d],
[lon + d, lat - d],
[lon + d, lat + d],
[lon - d, lat + d],
[lon - d, lat - d],
];
test('a coarse bundled admin-1 city waits for a finer outline', async () => {
const overParis = viewerAt(48.8566, 2.3522, 40_000);
const ring = square(48.8566, 2.3522, 0.05);
const rungs = rungsAnswering({ ring, kind: 'area', heightM: null });
const fine = harness(PARIS_PLACE, { outlineRungs: rungs });
const resolved = await fine.resolve('Paris', overParis);
assert.equal(
resolved.source,
'geocode',
'the 11-vertex admin-1 is not drawn at once',
);
const outline = await resolved.resolveOutline();
assert.deepEqual(outline.ring, ring, 'the finer outline wins');
assert.deepEqual(rungs.asked, [{ scope: 'city', target: 'Paris' }]);
// Nothing finer (Nominatim off, busy or capped): the bundled shape, not a point.
const coarse = harness(PARIS_PLACE, { outlineRungs: rungsAnswering(null) });
const fallback = await (
await coarse.resolve('Paris', overParis)
).resolveOutline();
assert.equal(fallback.adminArea, 'Paris');
assert.ok(fallback.ring.length < 32);
// No geocoder answer at all: the bundled shape at once.
const offline = harness(null, { outlineRungs: rungsAnswering(null) });
const direct = await offline.resolve('Paris', overParis);
assert.equal(direct.source, 'bundled');
assert.equal(direct.label, 'Paris');
});
test('detailed admin-1 units and admin asks keep the bundled shape', async () => {
const { resolve } = harness(null, { outlineRungs: rungsAnswering(null) });
for (const [name, lat, lon] of [
['Berlin', 52.52, 13.4],
['Bavaria', 48.9, 11.4],
['Texas', 31, -99],
]) {
const resolved = await resolve(name, viewerAt(lat, lon, 400_000));
assert.equal(resolved?.source, 'bundled', name);
}
// Countries are never coarse cities, however few vertices.
const singapore = await resolve('Singapore', viewerAt(1.35, 103.82, 60_000));
assert.equal(singapore?.source, 'bundled');
// An explicit admin ask takes the admin unit as it is.
const dc = await resolve('Paris', viewerAt(48.8566, 2.3522, 40_000), {
entityKind: 'state',
});
assert.equal(dc?.source, 'bundled');
});
test('a campus ask reaches the campus, not the Census place of its town', async () => {
const stanfordTown = {
lat: 37.4241,
lng: -122.1661,
label: 'Stanford, CA, USA',
name: 'Stanford',
types: ['locality', 'political'],
};
const campus = square(37.4275, -122.1697, 0.01);
for (const ask of ['Stanford University', 'Stanford campus']) {
const rungs = rungsAnswering({ ring: campus, kind: 'area', heightM: null });
const { resolve } = harness(stanfordTown, { outlineRungs: rungs });
const resolved = await resolve(ask, viewerAt(37.43, -122.17, 8000));
assert.notEqual(resolved?.source, 'bundled', ask);
const outline = await resolved.resolveOutline();
assert.deepEqual(outline.ring, campus, ask);
assert.equal(
rungs.asked[0].scope,
'compound',
`${ask}: past the Census rung`,
);
}
// "Stanford" said to be a compound is the campus too.
const kindRungs = rungsAnswering({
ring: campus,
kind: 'area',
heightM: null,
});
const byKind = harness(stanfordTown, { outlineRungs: kindRungs });
const compound = await (
await byKind.resolve('Stanford', viewerAt(37.43, -122.17, 8000), {
entityKind: 'compound',
})
).resolveOutline();
assert.deepEqual(compound.ring, campus);
assert.equal(kindRungs.asked[0].scope, 'compound');
// "Stanford" alone is the town (the Stanford CDP), and a town named after a
// college stays a town.
const { resolve } = harness(stanfordTown, {
outlineRungs: rungsAnswering(null),
});
const town = await (
await resolve('Stanford', viewerAt(39, -100, 5_000_000))
).resolveOutline();
assert.equal(town.adminArea, 'Stanford');
const station = harness(
{
lat: 30.628,
lng: -96.3344,
label: 'College Station, TX, USA',
name: 'College Station',
types: ['locality', 'political'],
},
{ outlineRungs: rungsAnswering(null) },
);
const collegeStation = await (
await station.resolve('College Station', viewerAt(39, -100, 5_000_000))
).resolveOutline();
assert.equal(collegeStation.adminArea, 'College Station');
});
test('a grounds ask never takes the state the geocoder typed', async () => {
const texas = {
lat: 31.0,
lng: -99.0,
label: 'Texas, USA',
name: 'Texas',
types: ['administrative_area_level_1', 'political'],
};
for (const [ask, extra] of [
['the Texas Capitol grounds', {}],
['Texas Capitol', { entityKind: 'compound' }],
// The grounds wording is only in the label.
['Texas Capitol', { labelHint: 'Capitol grounds' }],
]) {
const rungs = rungsAnswering(null);
const { resolve } = harness(texas, { outlineRungs: rungs });
const resolved = await resolve(ask, viewerAt(31, -99, 3_000_000), extra);
assert.notEqual(resolved?.source, 'bundled', ask);
const outline = await resolved.resolveOutline();
assert.notEqual(outline?.adminArea, 'Texas', ask);
assert.equal(rungs.asked[0].scope, 'compound', ask);
}
});
+12 -299
View File
@@ -19,50 +19,18 @@ import {
lookupNaturalRegionOutline,
findNaturalRegion,
} from '../data/naturalEarthRegions.js';
import {
findAdminArea,
findAdminAreaAt,
polygonsContain as polygonsContainPoint,
} from '../data/adminBoundaries.js';
import {
findNeighborhoodArea,
findNeighborhoodAreaAt,
findPlaceArea,
findPlaceAreaAt,
NEIGHBORHOOD_NEAR_KM,
normalizePlaceName,
PLACE_NEAR_KM,
} from '../data/placeBoundaries.js';
import { findAdminArea, findAdminAreaAt } from '../data/adminBoundaries.js';
import {
registerDynamicCredit,
NATURAL_EARTH_CREDIT,
US_CENSUS_CREDIT,
OSM_CREDIT,
OPENMAPTILES_CREDIT,
WOF_CREDIT,
} from '../data/dataCredits.js';
import { createOutlineRungs } from './outlineRungs.js';
import { unavailablePlaceSearch } from '../search/placeSearch.js';
import { isPickedWorldPosition } from '../data/scenePick.js';
/** Ms the analyst region lookup waits on geocode + admin boundary. */
export const REGION_FALLBACK_BUDGET_MS = 3_000;
/**
* Names that end like a landmark rather than a district: "Central Park",
* "Lake Merritt"-style suffixes are checked on the last word only, so "Park
* Slope" and "Notting Hill" stay districts.
*/
const LANDMARK_NAME =
/\b(?:park|gardens?|square|plaza|lake|river|bay|beach|pier|harbou?r|airport|station|stadium|arena|cemetery|zoo|university|college|campus|palace|castle|cathedral|church|temple|mosque|bridge|market|mall|cent(?:er|re)|tower|museum|reservoir|forest|woods?|heath|common|green)$/i;
/** The name part of an ask, before any comma qualifier. */
function stripQualifiers(query) {
return String(query || '')
.split(',')[0]
.trim();
}
/** Entity facts that rule out a state/county reading of the ask. */
const NON_ADMIN_ENTITY_KINDS = new Set([
'building',
@@ -79,8 +47,6 @@ export function createAnnotationResolver({
boundarySource,
signal: lifetime,
}),
// Overpass-free rungs (OpenFreeMap tiles, guarded Nominatim) around the ladder.
outlineRungs = createOutlineRungs(),
} = {}) {
requireFeatureSource(featureSource);
lifetime?.throwIfAborted();
@@ -210,11 +176,6 @@ export function createAnnotationResolver({
// locality suffixes; the types classify the entity (point-like vs area-like).
let placesPrimary = null;
let placeTypes = [];
// A coarse bundled admin-1 shape that carries a city's name (Natural
// Earth's 11-vertex "Paris"): kept as the fallback while the finer rungs
// (Census, Nominatim) try first.
let coarseCityShape = null;
const askNamesAdmin = Boolean(adminScopeFromAsk(target, entityKind));
// Instrumentation (logged once per target at the end): which sources were tried and what they returned.
const trace = {
query: String(target || '').trim(),
@@ -240,26 +201,7 @@ export function createAnnotationResolver({
() => null,
);
signal?.throwIfAborted();
if (admin && !askNamesAdmin && isCoarseCityShape(admin))
coarseCityShape = admin;
else if (admin) return bundledAdminTarget(viewer, admin, trace.query);
// Bundled US places and neighborhoods named at or near the view. A
// campus, grounds or monument ask ("Stanford" as a compound, "the
// Capitol grounds") wants that feature, not a district of its name,
// and "the area around X" is drawn by the ladder below.
if (
entityKind !== 'compound' &&
intent !== 'around_the_thing' &&
!isGroundsLikeQuery(trace.query) &&
!isMonumentLikeQuery(trace.query)
) {
const area = await bundledAreaNear(trace.query, center, {
entityKind,
viewRadiusKm: viewportProximity(viewer)?.radiusKm ?? null,
});
signal?.throwIfAborted();
if (area) return bundledAdminTarget(viewer, area, trace.query);
}
if (admin) return bundledAdminTarget(viewer, admin, trace.query);
}
if (!Number.isFinite(lat) || !Number.isFinite(lon)) {
@@ -387,10 +329,7 @@ export function createAnnotationResolver({
}
}
if (!Number.isFinite(lat) || !Number.isFinite(lon))
return coarseCityShape
? bundledAdminTarget(viewer, coarseCityShape, trace.query)
: null;
if (!Number.isFinite(lat) || !Number.isFinite(lon)) return null;
// Monument/memorial/statue names geocode unreliably — Google scatters them across the city (e.g.
// several Texas Capitol monuments landed blocks-to-miles apart, looking "all over the map"). When
@@ -455,9 +394,7 @@ export function createAnnotationResolver({
`osmSnap=${trace.osmSnap} → FINAL source=rejected (proximity gate, ${drift.driftKm.toFixed(0)}km > ${drift.limitKm.toFixed(0)}km)`,
);
}
return coarseCityShape
? bundledAdminTarget(viewer, coarseCityShape, trace.query)
: null;
return null;
}
}
@@ -477,23 +414,7 @@ export function createAnnotationResolver({
// an admin boundary for a city/state, the enclosing compound for a mall/campus, a single
// building for a premise. The voice model's entityKind (an entity FACT) refines an
// unresolved 'auto' scope only — real geocode types always win.
// A campus or grounds ask ("Stanford University", "Stanford campus",
// "the Capitol grounds", in the target or its label) or a compound by
// entity fact wants that compound even when the geocoder types the town
// or the state: never a Census place, neighborhood or admin boundary.
const typedScope = refineScope(scopeFromTypes(geocodeTypes), entityKind);
const compoundAsk =
entityKind === 'compound' ||
isCampusAsk(target) ||
isGroundsLikeQuery(target) ||
isGroundsLikeQuery(labelHint);
const baseScope =
compoundAsk &&
['country', 'state', 'county', 'city', 'neighborhood'].includes(
typedScope,
)
? 'compound'
: typedScope;
const baseScope = refineScope(scopeFromTypes(geocodeTypes), entityKind);
// Point-like targets (monuments/statues/memorials/…) resolve POINT-FIRST: only an
// (almost) exactly-named, monument-scale polygon may replace the point; a nearby polygon
// sharing locality words must not (docs/field-test-rootcause-2026-06-30.md §1).
@@ -522,7 +443,7 @@ export function createAnnotationResolver({
* The inline path awaits it right here; progressive callers (deferFootprint) invoke it
* AFTER the anchor mark is drawn and upgrade the mark in place.
*/
const resolveBaseOutline = async () => {
const resolveOutline = async () => {
let scope = baseScope;
const isAdmin =
scope === 'country' ||
@@ -550,7 +471,6 @@ export function createAnnotationResolver({
);
if (ne) {
registerDynamicCredit(viewer, NATURAL_EARTH_CREDIT);
traceBundledOutline(ne.name, 'natural-earth');
const neCentroid = ringCentroid(ne.ring);
return {
ring: ne.ring,
@@ -606,18 +526,7 @@ export function createAnnotationResolver({
lon,
scope,
).catch(() => null);
if (admin && !askNamesAdmin && isCoarseCityShape(admin))
coarseCityShape ||= admin;
else if (admin) return bundledAdminOutline(viewer, admin);
}
// A geocoded US city, town or CDP: the Census place that holds it.
if (scope === 'city') {
const place = await findPlaceAreaAt(
[target, matchName],
lat,
lon,
).catch(() => null);
if (place) return bundledAdminOutline(viewer, place);
if (admin) return bundledAdminOutline(viewer, admin);
}
// Pure admin: only an admin boundary is correct — never fall back to a
// building/landuse (a city is never a single building).
@@ -628,19 +537,7 @@ export function createAnnotationResolver({
// resolve here instantly to a REAL boundary, sidestepping the slow/flaky live-Overpass
// path that times out and falls back to points (see docs/field-test-2-analysis.md).
const ext = await lookupNeighborhoodRing(lat, lon, matchName);
if (ext) {
traceBundledOutline(ext.name || matchName, 'datasf');
fp = { ring: ext.ring, kind: 'area', heightM: null };
}
// THEN a bundled Who's On First neighborhood carrying the name at the point.
if (!fp) {
const wof = await findNeighborhoodAreaAt(
[target, matchName],
lat,
lon,
).catch(() => null);
if (wof) return bundledAdminOutline(viewer, wof);
}
if (ext) fp = { ring: ext.ring, kind: 'area', heightM: null };
// Else fall through to the OSM admin/place → named-landuse → synthesis ladder. Each
// returns a footprint, null (definitively no polygon), or undefined (transient
// upstream failure). Synthesize a blob (the "Mission" problem, research §3b) ONLY
@@ -766,11 +663,6 @@ export function createAnnotationResolver({
}
}
}
return finishOutline(fp, scope);
};
/** Validate a footprint for its scope and re-centre it (resolveOutline's patch). */
const finishOutline = (fp, scope) => {
// Scope sanity: reject a real footprint whose area is wildly wrong for the asked
// scope (a building/compound/neighborhood intent must never draw a state-sized
// blob). Synthesized discs are deliberately sized and exempt.
@@ -801,51 +693,17 @@ export function createAnnotationResolver({
footprintKind: fp.kind,
buildingHeight: fp.heightM,
synthesized: Boolean(fp.synthesized),
...(Array.isArray(fp.polygons) ? { polygons: fp.polygons } : {}),
lat: centroid.lat,
lon: centroid.lon,
height: sampleGroundHeight(viewer, centroid.lon, centroid.lat),
};
};
// Streets and pointed-at buildings from tiles, then this ladder, then the
// guarded Nominatim outline (see outlineRungs.js).
const outlineContext = {
scope: baseScope,
target,
matchName,
lat,
lon,
fromName: fromGeocode || source === 'places' || source === 'osm-local',
pointLike,
groundsLike,
around: intent === 'around_the_thing',
view: viewportProximity(viewer),
signal,
credit: (kind) =>
registerDynamicCredit(
viewer,
kind === 'tiles' ? OPENMAPTILES_CREDIT : OSM_CREDIT,
),
};
const resolveOutline = async () => {
const result = await outlineRungs.resolve(outlineContext, {
base: resolveBaseOutline,
finish: finishOutline,
});
// Nothing finer answered (no polygon, Nominatim off, busy or capped):
// the coarse bundled shape beats a point.
if (coarseCityShape && !(result?.ring && !result.synthesized))
return bundledAdminOutline(viewer, coarseCityShape);
return result;
};
let ring = null;
let footprintKind = null; // 'building' | 'area'
let buildingHeight = null; // meters, only for buildings
let outlineUnavailable = false;
let synthesized = false; // true = buffered/approximate area, render dashed/feathered
let polygons = null; // every part of a multi-part outline, when known
if (footprint && !deferFootprint) {
const fp = await resolveOutline();
outlineUnavailable = isUnavailableCapability(fp);
@@ -854,7 +712,6 @@ export function createAnnotationResolver({
footprintKind = fp.footprintKind;
buildingHeight = fp.buildingHeight;
synthesized = fp.synthesized;
polygons = fp.polygons || null;
// Re-center the anchor on the resolved footprint centroid.
lat = fp.lat;
lon = fp.lon;
@@ -882,7 +739,6 @@ export function createAnnotationResolver({
source,
synthesized,
outlineUnavailable,
...(polygons ? { polygons } : {}),
viewport: placeViewport,
...(footprint && deferFootprint ? { resolveOutline } : {}),
};
@@ -922,109 +778,13 @@ export function createAnnotationResolver({
const GROUNDS_RADIUS_M = 300; // "X grounds/compound/campus" loose disc when OSM has no polygon
const GROUNDS_RADIUS_MIN_M = 150; // viewport-derived grounds disc is clamped to this band so a tiny
const GROUNDS_RADIUS_MAX_M = 1200; // place can't shrink to a dot, nor a city-wide viewport balloon
const COARSE_CITY_MAX_KM2 = 5_000; // city-sized admin-1 units (see isCoarseCityShape)
const COARSE_CITY_MAX_VERTICES = 32;
/** Credit the pack a bundled boundary came from. */
function creditAdminSource(viewer, admin) {
const credit = {
'natural-earth': NATURAL_EARTH_CREDIT,
'us-census': US_CENSUS_CREDIT,
wof: WOF_CREDIT,
}[admin.source];
if (credit) registerDynamicCredit(viewer, credit);
}
/**
* The bundled US place or neighborhood an ask names at or near the view,
* from Census places, the San Francisco DataSF pack (first in SF), then
* Who's On First. How far "near" reaches follows the view (half its
* radius); a state-qualified place ("Springfield, Illinois") needs no
* view. A district ask takes a neighborhood first and a place only when
* the view is inside it ("Richmond" over SF is not Richmond, CA). Without
* an entity kind the candidate that holds the view wins, else the nearer.
* A name that ends like a landmark ("Central Park", "Lake Merritt") is
* left to the landmark ladder unless the ask is a district.
*/
async function bundledAreaNear(
query,
center,
{ entityKind = null, viewRadiusKm = null } = {},
) {
const district = entityKind === 'district';
if (!district && LANDMARK_NAME.test(stripQualifiers(query))) return null;
const reach = Number.isFinite(viewRadiusKm) ? viewRadiusKm / 2 : null;
const clamp = (km, lo, hi) =>
km === null ? hi : Math.min(hi, Math.max(lo, km));
const placeReach = district ? 0 : clamp(reach, 3, PLACE_NEAR_KM);
const place = await findPlaceArea(query, {
near: center,
nearKm: placeReach,
}).catch(() => null);
const holds = (area) =>
Boolean(center) &&
polygonsContainPoint(area.polygons, center.lat, center.lon);
// A district ask only takes a place the view is inside (or one a state
// qualifier named outright).
const placeOk =
place && (!district || holds(place) || /,/.test(query)) ? place : null;
if (!center) return placeOk;
let hood = null;
const sf = await lookupNeighborhoodRing(center.lat, center.lon, query, {
exact: true,
}).catch(() => null);
if (sf) hood = sfNeighborhoodArea(sf);
else
hood = await findNeighborhoodArea(query, {
near: center,
nearKm: clamp(reach, 1, NEIGHBORHOOD_NEAR_KM),
}).catch(() => null);
// Outside a district ask, an alternate name unrelated to the name is too
// weak: Se in Sao Paulo lists "Sao Paulo" among its names, and "outline
// Sao Paulo" means the city ("Richmond" for Richmond District still counts).
if (
!district &&
hood?.byAlias &&
!normalizePlaceName(hood.name).includes(
normalizePlaceName(stripQualifiers(query)),
)
)
hood = null;
if (!hood || !placeOk) return district ? hood || placeOk : placeOk || hood;
if (district) return hood;
const hoodHolds = holds(hood);
const placeHolds = holds(placeOk);
if (hoodHolds !== placeHolds) return hoodHolds ? hood : placeOk;
const distance = (area) =>
approximateDistanceM(
center.lat,
center.lon,
area.label.lat,
area.label.lon,
);
return distance(hood) <= distance(placeOk) ? hood : placeOk;
}
/** An SF DataSF ring in the bundled-area shape `bundledAdminTarget` takes. */
function sfNeighborhoodArea({ ring, name }) {
const lons = ring.map((p) => p[0]);
const lats = ring.map((p) => p[1]);
return {
kind: 'neighborhood',
name,
region: null,
source: 'datasf',
polygons: [[ring]],
ring,
bbox: [
Math.min(...lons),
Math.min(...lats),
Math.max(...lons),
Math.max(...lats),
],
label: ringCentroid(ring),
candidates: 1,
};
registerDynamicCredit(
viewer,
admin.source === 'us-census' ? US_CENSUS_CREDIT : NATURAL_EARTH_CREDIT,
);
}
/**
@@ -1036,7 +796,6 @@ export function createAnnotationResolver({
*/
function bundledAdminOutline(viewer, admin) {
creditAdminSource(viewer, admin);
traceBundledOutline(admin.name, admin.source, admin.polygons?.length);
const { lat, lon } = admin.label;
return {
ring: closeRing([...admin.ring]), // copy: closeRing mutates, the pack is shared
@@ -1051,31 +810,6 @@ export function createAnnotationResolver({
};
}
/**
* Whether a bundled unit is too coarse to stand for a city of its name: a
* Natural Earth admin-1 at city size (up to 5,000 km²) drawn with fewer
* than 32 vertices. Paris (11), Buenos Aires (11), Washington DC (9),
* Vienna and Prague (23) and Seoul (28) are; Berlin (48), Hamburg (43),
* Bremen (39) and every larger state are not. Countries never are.
*/
function isCoarseCityShape(admin) {
if (admin?.source !== 'natural-earth' || admin.kind !== 'state')
return false;
if (!(admin.areaKm2 <= COARSE_CITY_MAX_KM2)) return false;
const vertices = (admin.polygons || []).reduce(
(n, poly) => n + poly.reduce((m, ring) => m + ring.length, 0),
0,
);
return vertices < COARSE_CITY_MAX_VERTICES;
}
/** One trace line per outline a bundled pack answered (same shape as outlineRungs). */
function traceBundledOutline(name, source, parts = 1) {
console.log(
`[Outline] "${name}": bundled → ${source} (${parts || 1} part(s))`,
);
}
/** A complete resolved target for a bundled administrative unit (no outline pending). */
function bundledAdminTarget(viewer, admin, query) {
const outline = bundledAdminOutline(viewer, admin);
@@ -1827,19 +1561,6 @@ export function createAnnotationResolver({
/** A target that reads like an enclosing GROUNDS / COMPOUND / CAMPUS (e.g. "Texas Capitol grounds")
* rather than a single building — used to synthesize a loose-footprint disc when OSM has no real
* polygon, instead of failing outright. */
/**
* A university, college or campus named as such: "Stanford University",
* "Boston College", "University of Texas", "Stanford campus". Towns named
* after one ("College Station", "University Park") are not.
*/
function isCampusAsk(query) {
const text = stripQualifiers(String(query || '')).trim();
return (
/\b(?:university|college|campus)$/i.test(text) ||
/^(?:the\s+)?(?:university|college)\s+of\b/i.test(text)
);
}
function isGroundsLikeQuery(query) {
return /\b(grounds|compound|campus|complex|quad|plaza)\b/i.test(
String(query || ''),
@@ -2358,14 +2079,6 @@ export function createAnnotationResolver({
).catch(() => null);
if (admin) return { name: q, ring: [...admin.ring] };
}
if (scope === 'city') {
const place = await findPlaceAreaAt(
[q, geo.primaryName],
geo.lat,
geo.lon,
).catch(() => null);
if (place) return { name: q, ring: [...place.ring] };
}
const fp = await fetchAdminArea(geo.lat, geo.lon, q, scope, signal).catch(
() => null,
);
+305
View File
@@ -0,0 +1,305 @@
/**
* Embed mode and views in the running app.
*
* With `?embed=1` the app shows only the globe: clean view, with the HUD,
* welcome and setup prompts hidden. A page that frames it changes what it
* shows by posting `{ type: 'gev:view', id, view }`; the app applies the view
* through its own actions and answers `{ type: 'gev:view-applied', id, ok,
* steps }`. It announces `{ type: 'gev:ready' }` once it can take views.
*
* Annotations in a share link are drawn once the link has been restored,
* embedded or not.
*/
import * as Cesium from 'cesium';
import { announceNavigationAuthority } from '../navigationPolicy.js';
import { annotationsFromParams, createView } from '../view/index.js';
export const EMBED_VIEW_MESSAGE = 'gev:view';
export const EMBED_APPLIED_MESSAGE = 'gev:view-applied';
export const EMBED_READY_MESSAGE = 'gev:ready';
const FOLLOW_LAYERS = {
aircraft: 'flights',
military_aircraft: 'military',
satellite: 'satellites',
};
const FOLLOW_ATTEMPTS = 20;
const FOLLOW_RETRY_MS = 1000;
const FLIGHT_SECONDS = 2;
/**
* Whether this page shows the app embedded: `?embed=1` when another page
* frames it, or inline when a panel page loads the app into itself and sets
* `globalThis.GEV_EMBED_INLINE` first.
*/
export function isEmbedded(location = globalThis.location) {
return (
isEmbeddedInline() ||
new URLSearchParams(location?.search || '').get('embed') === '1'
);
}
/** Whether a panel page loaded the app into itself. */
export function isEmbeddedInline() {
return globalThis.GEV_EMBED_INLINE === true;
}
// A panel keeps drawing at about this rate when its host stops animation
// frames; see keepPanelRendering.
const PANEL_FRAME_MS = 33;
const MISSED_FRAMES_MS = 250;
/** A render error's details, including the plain objects workers report. */
function describeError(error) {
if (error instanceof Error)
return `${error.name}: ${error.message}\n${error.stack}`;
try {
return JSON.stringify(error, Object.getOwnPropertyNames(error ?? {}));
} catch {
return String(error);
}
}
/**
* Keep the globe drawing in an inline panel. Some hosts report a panel on
* screen as hidden, which stops the browser's animation frames and with them
* Cesium's render loop; while frames stop arriving, draw from a timer.
* Returns a function that stops it.
*/
export function keepPanelRendering(
viewer,
{ windowRef = globalThis.window, now = () => performance.now() } = {},
) {
let lastFrame = now();
let frameRequest = null;
const onFrame = () => {
lastFrame = now();
frameRequest = windowRef.requestAnimationFrame(onFrame);
};
frameRequest = windowRef.requestAnimationFrame(onFrame);
// Cesium stops drawing after a render error, and so does the timer.
// Report the error in full: workers report plain objects, which Cesium
// prints as [object Object].
let failed = false;
const removeErrorListener = viewer.scene?.renderError?.addEventListener(
(_scene, error) => {
failed = true;
console.error(
"[God's Eye View panel] render error:",
describeError(error),
);
},
);
const timer = windowRef.setInterval(() => {
if (
failed ||
now() - lastFrame < MISSED_FRAMES_MS ||
viewer.isDestroyed?.()
)
return;
viewer.resize();
viewer.render();
}, PANEL_FRAME_MS);
return () => {
windowRef.clearInterval(timer);
windowRef.cancelAnimationFrame(frameRequest);
removeErrorListener?.();
};
}
const delay = (ms, signal) =>
new Promise((resolve) => {
const timer = setTimeout(resolve, ms);
signal?.addEventListener(
'abort',
() => {
clearTimeout(timer);
resolve();
},
{ once: true },
);
});
/** Fly the camera to a view's camera and resolve when the flight ends. */
function flyTo(viewer, camera) {
announceNavigationAuthority('embed-view');
return new Promise((resolve) => {
viewer.camera.flyTo({
destination: Cesium.Cartesian3.fromDegrees(
camera.lon,
camera.lat,
camera.altitude_m,
),
orientation: {
heading: Cesium.Math.toRadians(camera.heading_deg),
pitch: Cesium.Math.toRadians(camera.pitch_deg),
roll: 0,
},
duration: FLIGHT_SECONDS,
complete: () => resolve(true),
cancel: () => resolve(false),
});
});
}
/**
* Make the app show a view: style and map, exactly the view's layers,
* annotations, the camera, and the followed entity. Each step runs through the
* app's own actions (`run(name, args)`); a failed step is reported and the
* rest still run. Resolves to the steps and whether each succeeded.
*/
export async function applyView(
view,
{ viewer, dataManager, run, signal, retryMs = FOLLOW_RETRY_MS },
) {
const steps = [];
const act = async (name, args) => {
let result = null;
try {
result = await run(name, args);
} catch (error) {
result = { ok: false, error: error?.message || String(error) };
}
steps.push({
step: name,
ok: result?.ok !== false,
...(result?.error ? { error: result.error } : {}),
});
return result;
};
if (view.style) await act('set_visual_style', { style: view.style });
if (view.map) await act('set_map_stack', { stack: view.map });
const wanted = new Set(view.layers);
for (const layer of dataManager.getAll()) {
if (layer.enabled && !wanted.has(layer.id))
await act('set_layer_visibility', { layerId: layer.id, enabled: false });
}
for (const layerId of view.layers) {
if (!dataManager.isEnabled(layerId))
await act('set_layer_visibility', { layerId, enabled: true });
}
if (!view.follow) await act('stop_tracking', {});
// Annotations go first: drawing them may frame the marks, and the view's
// own camera should have the last word.
await act('clear_annotations', {});
if (view.annotations.length)
await act('annotate_map', { annotations: [...view.annotations] });
if (!view.follow) {
steps.push({ step: 'camera', ok: await flyTo(viewer, view.camera) });
return steps;
}
// A followed entity owns the camera, and a camera flight would end the
// follow, so fly to the view only while the entity is not there yet. It
// appears once its layer has data; retry until then.
const layerId = FOLLOW_LAYERS[view.follow.kind];
const follow = async () => {
try {
return (
(await run('track_entity', { query: view.follow.id, layerId }))?.ok ===
true
);
} catch {
return false;
}
};
let followed = await follow();
if (!followed) {
steps.push({ step: 'camera', ok: await flyTo(viewer, view.camera) });
for (let attempt = 1; attempt < FOLLOW_ATTEMPTS && !followed; attempt++) {
if (signal?.aborted) break;
await delay(retryMs, signal);
followed = await follow();
}
}
steps.push({ step: 'follow', ok: followed });
if (followed && view.follow.cockpit)
await act('control_cockpit', { action: 'enter', targetLayer: layerId });
return steps;
}
/**
* Install embed mode when the page asks for it, and draw any annotations the
* opening link carries. Returns a function that removes the message handler.
*/
export function installViews({
shell,
viewer,
dataManager,
run,
signal,
location = globalThis.location,
windowRef = globalThis.window,
}) {
const ready = Promise.resolve(shell.initialRestorePromise).catch(() => {});
const linked = annotationsFromParams(
new URLSearchParams(String(location?.hash || '').replace(/^#/, '')),
);
if (linked.length)
void ready.then(() => {
if (!signal?.aborted) void run('annotate_map', { annotations: linked });
});
if (!isEmbedded(location)) return () => {};
windowRef.document.body.classList.add('ui-embed');
shell.setCleanView?.(true);
const stopRendering = isEmbeddedInline()
? keepPanelRendering(viewer, { windowRef })
: () => {};
// A framing page talks to the app across frames; an inline panel shares
// the page with it and talks through the page's own window.
const peer = isEmbeddedInline() ? windowRef : windowRef.parent;
// Answers go back only to the origin that asked. An inline panel posts to
// its own window, and a sandboxed framing page has no origin to name.
const post = (message, origin = '*') => {
if (peer && (peer !== windowRef || isEmbeddedInline()))
peer.postMessage(message, origin);
};
// Views apply one at a time, in the order they arrive.
let queue = ready;
const onMessage = (event) => {
if (event.source !== peer || event.data?.type !== EMBED_VIEW_MESSAGE)
return;
const { id = null } = event.data;
const replyOrigin =
isEmbeddedInline() || !event.origin || event.origin === 'null'
? '*'
: event.origin;
let view;
try {
view = createView(event.data.view);
} catch (error) {
post(
{
type: EMBED_APPLIED_MESSAGE,
id,
ok: false,
error: error.message,
},
replyOrigin,
);
return;
}
queue = queue.then(async () => {
if (signal?.aborted) return;
const steps = await applyView(view, { viewer, dataManager, run, signal });
post(
{
type: EMBED_APPLIED_MESSAGE,
id,
ok: steps.every((step) => step.ok),
steps,
},
replyOrigin,
);
});
};
windowRef.addEventListener('message', onMessage);
void ready.then(() => {
if (!signal?.aborted) post({ type: EMBED_READY_MESSAGE });
});
return () => {
stopRendering();
windowRef.removeEventListener('message', onMessage);
};
}
+325
View File
@@ -0,0 +1,325 @@
import assert from 'node:assert/strict';
import test from 'node:test';
import {
applyView,
installViews,
isEmbedded,
keepPanelRendering,
} from './embed.js';
import { createView } from '../view/index.js';
const fakeViewer = () => {
const flights = [];
return {
flights,
camera: {
flyTo(options) {
flights.push(options);
options.complete();
},
},
};
};
const fakeLayers = (enabled) => ({
getAll: () =>
['flights', 'earthquakes', 'ais-live-vessels'].map((id) => ({
id,
enabled: enabled.has(id),
})),
isEnabled: (id) => enabled.has(id),
});
const recorder = (answers = {}) => {
const calls = [];
const run = async (name, args) => {
calls.push([name, args]);
const answer = answers[name];
return typeof answer === 'function'
? answer(calls)
: (answer ?? { ok: true });
};
return { calls, run };
};
test('embed mode is opt-in by query parameter', () => {
assert.equal(isEmbedded({ search: '?embed=1' }), true);
assert.equal(isEmbedded({ search: '?welcome=0' }), false);
});
test('a view is applied through the app actions in order', async () => {
const viewer = fakeViewer();
const { calls, run } = recorder();
const view = createView({
camera: { lat: 25, lon: 121, altitude_m: 300000, pitch_deg: -40 },
layers: ['ais-live-vessels'],
style: 'thermal',
map: 'osm',
annotations: [{ type: 'pin', target: 'Taipei 101' }],
});
const steps = await applyView(view, {
viewer,
dataManager: fakeLayers(new Set(['earthquakes'])),
run,
});
assert.deepEqual(
calls.map(([name, args]) => [
name,
args.layerId ?? args.style ?? args.stack ?? null,
args.enabled ?? null,
]),
[
['set_visual_style', 'thermal', null],
['set_map_stack', 'osm', null],
['set_layer_visibility', 'earthquakes', false],
['set_layer_visibility', 'ais-live-vessels', true],
['stop_tracking', null, null],
['clear_annotations', null, null],
['annotate_map', null, null],
],
);
assert.equal(viewer.flights.length, 1);
assert.ok(steps.every((step) => step.ok));
assert.ok(steps.some((step) => step.step === 'camera'));
});
test('a followed entity is retried until its layer has it', async () => {
let tries = 0;
const { run } = recorder({
track_entity: () => ({ ok: ++tries >= 3 }),
});
const view = createView({
camera: { lat: 0, lon: 0 },
follow: { kind: 'aircraft', id: 'abc123' },
});
const steps = await applyView(view, {
viewer: fakeViewer(),
dataManager: fakeLayers(new Set()),
run,
retryMs: 1,
});
assert.equal(tries, 3);
assert.deepEqual(steps.at(-1), { step: 'follow', ok: true });
});
test('an embedded page takes views only from its parent and answers it', async () => {
const posted = [];
const targets = [];
const parent = {
postMessage: (message, target) => {
posted.push(message);
targets.push(target);
},
};
const windowRef = new EventTarget();
windowRef.parent = parent;
windowRef.document = { body: { classList: new Set() } };
windowRef.document.body.classList.add = Set.prototype.add;
const shell = {
initialRestorePromise: Promise.resolve(),
clean: null,
setCleanView(value) {
this.clean = value;
},
};
const { calls, run } = recorder();
const remove = installViews({
shell,
viewer: fakeViewer(),
dataManager: fakeLayers(new Set()),
run,
location: { search: '?embed=1', hash: '' },
windowRef,
});
await new Promise((resolve) => setTimeout(resolve, 0));
assert.equal(shell.clean, true);
assert.deepEqual(posted, [{ type: 'gev:ready' }]);
const send = (source, data, origin = 'https://host.example') => {
const event = new Event('message');
Object.assign(event, { source, data, origin });
windowRef.dispatchEvent(event);
};
send({}, { type: 'gev:view', id: 1, view: { camera: { lat: 1, lon: 2 } } });
send(parent, { type: 'gev:view', id: 2, view: { camera: {} } });
send(parent, {
type: 'gev:view',
id: 3,
view: { camera: { lat: 1, lon: 2 }, layers: ['flights'] },
});
await new Promise((resolve) => setTimeout(resolve, 10));
assert.deepEqual(posted[1], {
type: 'gev:view-applied',
id: 2,
ok: false,
error: 'A view needs a camera lat and lon',
});
assert.equal(posted[2].id, 3);
assert.equal(posted[2].ok, true);
assert.ok(
calls.some(
([name, args]) =>
name === 'set_layer_visibility' && args.layerId === 'flights',
),
);
assert.equal(posted.filter((message) => message.id === 1).length, 0);
// Ready names no data and goes to any parent; answers go only to the
// origin that asked.
assert.deepEqual(targets, [
'*',
'https://host.example',
'https://host.example',
]);
remove();
});
test('an app loaded inline into a panel page talks through its own window', async () => {
const posted = [];
const windowRef = new EventTarget();
windowRef.parent = windowRef;
windowRef.postMessage = (message) => posted.push(message);
windowRef.document = { body: { classList: new Set() } };
windowRef.document.body.classList.add = Set.prototype.add;
Object.assign(windowRef, {
requestAnimationFrame: () => 1,
cancelAnimationFrame() {},
setInterval: () => 1,
clearInterval() {},
});
globalThis.GEV_EMBED_INLINE = true;
try {
assert.equal(isEmbedded({ search: '' }), true);
const remove = installViews({
shell: { initialRestorePromise: Promise.resolve() },
viewer: fakeViewer(),
dataManager: fakeLayers(new Set()),
run: recorder().run,
location: { search: '', hash: '' },
windowRef,
});
await new Promise((resolve) => setTimeout(resolve, 0));
assert.deepEqual(posted, [{ type: 'gev:ready' }]);
const event = new Event('message');
Object.assign(event, {
source: windowRef,
data: { type: 'gev:view', id: 7, view: { camera: { lat: 1, lon: 2 } } },
});
windowRef.dispatchEvent(event);
await new Promise((resolve) => setTimeout(resolve, 10));
assert.equal(posted[1].id, 7);
assert.equal(posted[1].ok, true);
remove();
} finally {
delete globalThis.GEV_EMBED_INLINE;
}
});
test('annotations in a link are drawn after it restores, embedded or not', async () => {
const { calls, run } = recorder();
const an = encodeURIComponent(
JSON.stringify([{ type: 'pin', target: 'Austin' }]),
);
installViews({
shell: { initialRestorePromise: Promise.resolve() },
viewer: fakeViewer(),
dataManager: fakeLayers(new Set()),
run,
location: { search: '', hash: `#v=2&lat=1&lon=2&an=${an}` },
windowRef: new EventTarget(),
});
await new Promise((resolve) => setTimeout(resolve, 0));
assert.deepEqual(calls, [
['annotate_map', { annotations: [{ type: 'pin', target: 'Austin' }] }],
]);
});
test('a panel keeps drawing from a timer while animation frames stop', () => {
let clock = 0;
let tick = null;
let frame = null;
const windowRef = {
requestAnimationFrame: (fn) => ((frame = fn), 1),
cancelAnimationFrame: () => (frame = null),
setInterval: (fn) => ((tick = fn), 1),
clearInterval: () => (tick = null),
};
const drawn = [];
const viewer = {
resize: () => drawn.push('resize'),
render: () => drawn.push('render'),
};
const stop = keepPanelRendering(viewer, { windowRef, now: () => clock });
// Frames arrive: the render loop draws, so the timer does nothing.
clock = 100;
frame();
tick();
assert.deepEqual(drawn, []);
// Frames stop, as in a panel its host reports hidden: the timer draws.
clock = 400;
tick();
assert.deepEqual(drawn, ['resize', 'render']);
stop();
assert.equal(tick, null);
assert.equal(frame, null);
});
test('a panel reports a render error in full and stops drawing', () => {
let listener = null;
let tick = null;
const windowRef = {
requestAnimationFrame: () => 1,
cancelAnimationFrame() {},
setInterval: (fn) => ((tick = fn), 1),
clearInterval() {},
};
const drawn = [];
const viewer = {
resize() {},
render: () => drawn.push('render'),
scene: {
renderError: {
addEventListener: (fn) => ((listener = fn), () => (listener = null)),
},
},
};
const logged = [];
const original = console.error;
console.error = (...args) => logged.push(args.join(' '));
try {
let clock = 0;
const stop = keepPanelRendering(viewer, { windowRef, now: () => clock });
listener(viewer.scene, { name: 'RuntimeError', message: 'worker failed' });
assert.match(logged[0], /render error: .*"message":"worker failed"/);
clock = 1000;
tick();
assert.deepEqual(drawn, []);
stop();
assert.equal(listener, null);
} finally {
console.error = original;
}
});
test('a followed aircraft owns the camera, and cockpit view is entered after it', async () => {
const viewer = fakeViewer();
const { calls, run } = recorder();
const steps = await applyView(
createView({
camera: { lat: 32.7, lon: -117.2 },
follow: { kind: 'military_aircraft', id: 'AE1234', cockpit: true },
}),
{ viewer, dataManager: fakeLayers(new Set()), run, retryMs: 0 },
);
// Following worked at once, so no camera flight ended it.
assert.equal(viewer.flights.length, 0);
assert.deepEqual(calls.slice(-2), [
['track_entity', { query: 'ae1234', layerId: 'military' }],
['control_cockpit', { action: 'enter', targetLayer: 'military' }],
]);
assert.ok(steps.every((step) => step.ok));
assert.throws(
() =>
createView({
camera: { lat: 0, lon: 0 },
follow: { kind: 'satellite', id: '25544', cockpit: true },
}),
/no cockpit view/,
);
});
+662
View File
@@ -0,0 +1,662 @@
/**
* The God's Eye View panel's script, run inside the panel page a host shows.
* It is sent as source text (see src/tools/globePanel.js), so it must not use
* imports or anything outside the function.
*
* The page lives on the host's site, and hosts may refuse other addresses,
* such as a server on the user's machine. So the panel never requests the
* app's server itself: every request for the app's own paths goes to the
* panel's MCP server as a call to `config.toolName`, which requests the path
* from the app's server. The app's built page, scripts, styles, images and
* data all arrive that way; map imagery from providers loads directly.
*/
export function panelRuntime(config) {
const status = document.getElementById('status');
const open = document.getElementById('open');
const expand = document.getElementById('expand');
const say = (text) => {
if (status.isConnected) status.textContent = text;
};
let blocked = null;
document.addEventListener('securitypolicyviolation', (event) => {
blocked = `${event.effectiveDirective} blocked ${event.blockedURI || 'a resource'}`;
});
// JSON-RPC with the host.
let nextId = 1;
const pending = new Map();
const send = (message) =>
window.parent.postMessage({ jsonrpc: '2.0', ...message }, '*');
const request = (method, params) =>
new Promise((resolve, reject) => {
const id = nextId++;
pending.set(id, { resolve, reject });
send({ id, method, params });
});
const notify = (method, params) => send({ method, params });
// Requests to the app's server, through the MCP server.
const MAX_CALLS = 6;
let calls = 0;
const waiting = [];
async function callTool(args) {
if (calls >= MAX_CALLS) await new Promise((go) => waiting.push(go));
calls += 1;
try {
const result = await request('tools/call', {
name: config.toolName,
// The server refuses the panel's requests without its key.
arguments: { ...args, key: config.panelKey },
});
if (result?.isError || !result?.structuredContent)
throw new Error(result?.content?.[0]?.text || 'The request failed');
return result.structuredContent;
} finally {
calls -= 1;
waiting.shift()?.();
}
}
const toBase64 = (bytes) => {
let binary = '';
for (let start = 0; start < bytes.length; start += 0x8000)
binary += String.fromCharCode(...bytes.subarray(start, start + 0x8000));
return btoa(binary);
};
const fromBase64 = (text) =>
Uint8Array.from(atob(text), (char) => char.charCodeAt(0));
async function bodyBytes(body) {
if (body == null) return null;
if (typeof body === 'string') return new TextEncoder().encode(body);
if (body instanceof ArrayBuffer) return new Uint8Array(body);
if (ArrayBuffer.isView(body))
return new Uint8Array(body.buffer, body.byteOffset, body.byteLength);
return new Uint8Array(await new Response(body).arrayBuffer());
}
const NULL_BODY = new Set([101, 204, 205, 304]);
const held = new Map();
/** Request a path on the app's server; resolves to a Response. */
function appFetch(path, init = {}) {
const method = (init.method || 'GET').toUpperCase();
// Built files never change while the panel runs, so each loads once.
const shared = method === 'GET' && path.startsWith(config.panelBase);
if (shared && held.has(path)) return held.get(path).then(respond);
const loading = (async () => {
const headers = {};
new Headers(init.headers || {}).forEach((value, name) => {
headers[name] = value;
});
const bytes = await bodyBytes(init.body);
let part = await callTool({
path,
method,
headers,
...(bytes ? { body: toBase64(bytes) } : {}),
});
const parts = [fromBase64(part.body)];
while (part.nextOffset !== undefined) {
if (init.signal?.aborted) break;
part = await callTool({ id: part.id, offset: part.nextOffset });
parts.push(fromBase64(part.body));
}
let body = new Blob(parts);
if (part.encoding === 'gzip')
body = await new Response(
body.stream().pipeThrough(new DecompressionStream('gzip')),
).blob();
return {
body: await body.arrayBuffer(),
status: part.status,
statusText: part.statusText,
headers: part.headers,
};
})();
if (shared) {
held.set(path, loading);
loading.catch(() => held.delete(path));
}
const result = loading.then(respond);
if (!init.signal) return result;
return Promise.race([
result,
new Promise((_, reject) => {
const abort = () =>
reject(new DOMException('The request was aborted', 'AbortError'));
if (init.signal.aborted) abort();
init.signal.addEventListener('abort', abort, { once: true });
}),
]);
}
function respond({ body, status, statusText, headers }) {
return new Response(NULL_BODY.has(status) ? null : body.slice(0), {
status,
statusText,
headers,
});
}
// The app's own paths: relative ones, absolute ones on this page's site
// (matched by scheme and host, since a host may use its own scheme), and
// ones under the app's address in the page, an https address for code
// that needs one (`config.appBaseUrl`).
const page = new URL(document.baseURI);
const appOrigin = new URL(config.appBaseUrl).origin;
function appPath(value) {
let url;
try {
url = new URL(String(value), document.baseURI);
} catch {
return null;
}
const own =
url.origin === appOrigin ||
(url.protocol === page.protocol && url.host === page.host);
return own ? url.pathname + url.search : null;
}
const fileUrls = new Map();
/**
* A data: URL holding an app file, for elements that load by URL. Hosts
* allow data: images where some refuse blob: ones. Files of the panel
* build never change and are kept; others, such as live camera frames
* that change address on every refresh, are not.
*/
function appFileUrl(path) {
if (fileUrls.has(path)) return fileUrls.get(path);
const loading = appFetch(path)
.then((response) => response.blob())
.then(
(blob) =>
new Promise((resolve, reject) => {
const reader = new FileReader();
reader.onload = () => resolve(reader.result);
reader.onerror = () => reject(reader.error);
reader.readAsDataURL(blob);
}),
);
if (path.startsWith(config.panelBase)) {
fileUrls.set(path, loading);
loading.catch(() => fileUrls.delete(path));
}
return loading;
}
function installRequests() {
const nativeFetch = window.fetch.bind(window);
window.fetch = async (input, init = {}) => {
const isRequest = input instanceof Request;
const path = appPath(isRequest ? input.url : input);
if (path === null) return nativeFetch(input, init);
if (!isRequest) return appFetch(path, init);
return appFetch(path, {
method: init.method || input.method,
headers: init.headers || input.headers,
body:
init.body ??
(['GET', 'HEAD'].includes(input.method)
? undefined
: await input.arrayBuffer()),
signal: init.signal || input.signal,
});
};
// XMLHttpRequest for app paths: answered from appFetch, with the
// instance's state replaced, and the usual events dispatched.
const xhr = XMLHttpRequest.prototype;
const nativeOpen = xhr.open;
const nativeSend = xhr.send;
const nativeHeader = xhr.setRequestHeader;
const nativeAbort = xhr.abort;
xhr.open = function (method, url, ...rest) {
const path = appPath(url);
if (path === null) {
this.__gev = null;
return nativeOpen.call(this, method, url, ...rest);
}
this.__gev = { method, path, headers: {}, controller: null };
setState(this, { readyState: 1 });
this.dispatchEvent(new Event('readystatechange'));
};
xhr.setRequestHeader = function (name, value) {
if (!this.__gev) return nativeHeader.call(this, name, value);
this.__gev.headers[name] = value;
};
xhr.abort = function () {
if (!this.__gev) return nativeAbort.call(this);
this.__gev.controller?.abort();
};
xhr.send = function (body) {
const state = this.__gev;
if (!state) return nativeSend.call(this, body);
state.controller = new AbortController();
appFetch(state.path, {
method: state.method,
headers: state.headers,
body,
signal: state.controller.signal,
})
.then(async (response) => {
const type = this.responseType;
const value =
type === 'arraybuffer'
? await response.arrayBuffer()
: type === 'blob'
? await response.blob()
: await response.text();
const text = typeof value === 'string' ? value : null;
let parsed = value;
if (type === 'json') {
try {
parsed = JSON.parse(text);
} catch {
parsed = null;
}
} else if (type === 'document')
parsed = new DOMParser().parseFromString(text, 'text/html');
const headers = [...response.headers]
.map(([name, val]) => `${name}: ${val}`)
.join('\r\n');
setState(this, {
readyState: 4,
status: response.status,
statusText: response.statusText,
response: parsed,
responseText: type === '' || type === 'text' ? text : '',
responseURL: new URL(state.path, document.baseURI).href,
getAllResponseHeaders: () => headers,
getResponseHeader: (name) => response.headers.get(name),
});
for (const name of ['readystatechange', 'load', 'loadend'])
this.dispatchEvent(
name === 'readystatechange'
? new Event(name)
: new ProgressEvent(name),
);
})
.catch((error) => {
setState(this, { readyState: 4, status: 0 });
const name = error?.name === 'AbortError' ? 'abort' : 'error';
for (const event of ['readystatechange', name, 'loadend'])
this.dispatchEvent(
event === 'readystatechange'
? new Event(event)
: new ProgressEvent(event),
);
});
};
function setState(target, values) {
for (const [name, value] of Object.entries(values))
Object.defineProperty(target, name, {
configurable: true,
...(typeof value === 'function' ? { value } : { get: () => value }),
});
}
const nativeBeacon = navigator.sendBeacon?.bind(navigator);
if (nativeBeacon)
navigator.sendBeacon = (url, data) => {
const path = appPath(url);
if (path === null) return nativeBeacon(url, data);
appFetch(path, { method: 'POST', body: data }).catch(() => {});
return true;
};
// Images name app files by URL; give them the file as a data: URL.
const image = Object.getOwnPropertyDescriptor(
HTMLImageElement.prototype,
'src',
);
Object.defineProperty(HTMLImageElement.prototype, 'src', {
...image,
set(value) {
const path = appPath(value);
if (path === null) return image.set.call(this, value);
appFileUrl(path).then(
(url) => image.set.call(this, url),
() => this.dispatchEvent(new Event('error')),
);
},
});
// Workers on the app's server start from a small worker here that is
// handed the script once it has loaded; messages wait until then.
const NativeWorker = window.Worker;
const BOOT =
'const early=[];let ready=false;' +
"addEventListener('message',e=>{if(e.data&&e.data.__gevScript){" +
'const go=()=>{ready=true;for(const d of early)dispatchEvent(new MessageEvent("message",{data:d}))};' +
'if(e.data.module)import(e.data.__gevScript).then(go);else{importScripts(e.data.__gevScript);go()}' +
'e.stopImmediatePropagation();return}' +
'if(!ready){early.push(e.data);e.stopImmediatePropagation()}});';
window.Worker = function Worker(url, options) {
const path = appPath(url);
if (path === null || String(url).startsWith('blob:'))
return new NativeWorker(url, options);
const worker = new NativeWorker(
URL.createObjectURL(new Blob([BOOT], { type: 'text/javascript' })),
options,
);
appFetch(path)
.then((response) => response.blob())
.then((blob) => URL.createObjectURL(blob))
.then((script) =>
worker.postMessage({
__gevScript: script,
module: options?.type === 'module',
}),
);
return worker;
};
window.Worker.prototype = NativeWorker.prototype;
// Some hosts drop the GPU contents of a panel's 2D canvases while they
// consider the panel off screen, and the overlays then cover the globe
// in black. Keep 2D canvases in memory instead, which hosts do not drop.
const getContext = HTMLCanvasElement.prototype.getContext;
HTMLCanvasElement.prototype.getContext = function (type, options) {
return type === '2d'
? getContext.call(this, type, { ...options, willReadFrequently: true })
: getContext.call(this, type, options);
};
// Markup the app adds as HTML text would request its images from this
// page's site at once; hold their addresses until the files are here.
const htmlWithHeldImages = (html) =>
String(html).replace(
/(<(?:img|source)\b[^>]*?\s)src=(["'])([^"']*)\2/gi,
(tag, before, quote, value) =>
appPath(value) === null
? tag
: `${before}data-gev-src=${quote}${value}${quote}`,
);
const fillHeldImages = (root) => {
for (const element of root?.querySelectorAll?.('[data-gev-src]') ?? []) {
const path = appPath(element.getAttribute('data-gev-src'));
element.removeAttribute('data-gev-src');
appFileUrl(path).then((url) => element.setAttribute('src', url));
}
};
const innerHtml = Object.getOwnPropertyDescriptor(
Element.prototype,
'innerHTML',
);
Object.defineProperty(Element.prototype, 'innerHTML', {
...innerHtml,
set(value) {
innerHtml.set.call(this, htmlWithHeldImages(value));
fillHeldImages(this);
},
});
const insertHtml = Element.prototype.insertAdjacentHTML;
Element.prototype.insertAdjacentHTML = function (position, html) {
insertHtml.call(this, position, htmlWithHeldImages(html));
fillHeldImages(this.parentNode ?? this);
};
// Markup and styles the app adds later name app files too.
new MutationObserver((records) => {
for (const record of records)
for (const node of record.addedNodes) {
if (node.nodeType !== 1) continue;
if (node.nodeName === 'STYLE') void inlineStyleUrls(node);
for (const element of [node, ...node.querySelectorAll('img[src]')]) {
if (element.nodeName !== 'IMG') continue;
const path = appPath(element.getAttribute('src'));
if (path !== null)
appFileUrl(path).then((url) => element.setAttribute('src', url));
}
}
}).observe(document.documentElement, { childList: true, subtree: true });
}
/** Replace url() references to app files in CSS with data: URLs. */
async function cssWithFiles(text, baseUrl) {
const found = new Map();
for (const match of text.matchAll(/url\(\s*(['"]?)([^'")]+)\1\s*\)/g)) {
const reference = match[2];
if (/^(data|blob):/.test(reference)) continue;
const path = appPath(new URL(reference, baseUrl).href);
if (path !== null && !found.has(reference))
found.set(
reference,
appFileUrl(path).catch(() => null),
);
}
let result = text;
for (const [reference, loading] of found) {
const url = await loading;
if (url) result = result.split(reference).join(url);
}
return result;
}
async function inlineStyleUrls(style) {
if (!/url\(/.test(style.textContent)) return;
style.textContent = await cssWithFiles(style.textContent, document.baseURI);
}
async function addStylesheet(href) {
const path = appPath(href);
const style = document.createElement('style');
document.head.appendChild(style);
if (path === null) {
const link = document.createElement('link');
link.rel = 'stylesheet';
link.href = href;
style.replaceWith(link);
return;
}
const text = await (await appFetch(path)).text();
style.textContent = await cssWithFiles(
text,
new URL(path, document.baseURI).href,
);
}
function runScript(text, type) {
const script = document.createElement('script');
if (type) script.type = type;
script.textContent = text;
document.body.appendChild(script);
}
/**
* Load God's Eye View into this page from the app's panel build, in
* inline embed mode, starting at the first view as its link would.
*/
async function startApp(url, view) {
window.GEV_EMBED_INLINE = true;
window.GEV_APP_BASE_URL = config.appBaseUrl;
try {
history.replaceState(
null,
'',
location.href.split('#')[0] + new URL(url).hash,
);
// The app restores this view from its link, following included, so
// it needs applying again only for what links cannot carry: cockpit.
if (queued === view && !view.follow?.cockpit) queued = null;
} catch {
// The view still arrives once the app is ready.
}
installRequests();
const response = await appFetch(config.panelBase);
if (!response.ok)
throw new Error(
response.status === 404
? 'the app has no panel build (npm run build:panel)'
: `the app answered ${response.status}`,
);
const built = new DOMParser().parseFromString(
await response.text(),
'text/html',
);
window.CESIUM_BASE_URL = `${config.panelBase}cesium/`;
const [prelude] = await Promise.all([
appFetch(config.panelBase + config.workerPreludePath).then((r) =>
r.text(),
),
...[...built.head.querySelectorAll('link[rel="stylesheet"]')].map(
(link) => addStylesheet(link.getAttribute('href')),
),
]);
// Cesium's script sets CESIUM_WORKERS, its workers as one script it
// starts from memory. Workers request files themselves, which this page
// cannot answer, so the prelude, holding those files, runs ahead of it.
let workers;
Object.defineProperty(window, 'CESIUM_WORKERS', {
configurable: true,
get: () => workers,
set: (script) => {
workers = prelude + script;
},
});
// Images in the markup load their files once added; give them the
// files instead of letting them request this page's site.
const images = [];
for (const image of built.body.querySelectorAll('img[src]')) {
const path = appPath(image.getAttribute('src'));
if (path === null) continue;
image.removeAttribute('src');
images.push([image, path]);
}
for (const node of [...built.body.childNodes]) {
if (node.nodeName !== 'SCRIPT')
document.body.insertBefore(document.adoptNode(node), status);
}
for (const [image, path] of images)
appFileUrl(path).then((url) => image.setAttribute('src', url));
const scripts = await Promise.all(
[...built.querySelectorAll('script')].map(async (script) => ({
type: script.type,
text: script.getAttribute('src')
? await (await appFetch(appPath(script.getAttribute('src')))).text()
: script.textContent,
})),
);
for (const { type, text } of scripts) runScript(text, type);
// Browsers cap live 3D contexts across pages and take one back when
// there are too many; say so rather than show no map.
document.addEventListener(
'webglcontextlost',
() => {
const notice = document.createElement('div');
notice.id = 'status';
notice.textContent =
"This panel lost its 3D graphics, likely because other God's Eye " +
"View panels in this conversation hold them. Use Open in God's " +
'Eye View above, or show it in a new conversation.';
document.body.appendChild(notice);
},
{ capture: true, once: true },
);
}
// Tool results: the first view starts the app, later ones move it.
let started = false;
let ready = false;
let queued = null;
let currentUrl = null;
const postView = (view) =>
window.postMessage({ type: 'gev:view', id: nextId++, view }, '*');
function show(result) {
const data = result?.structuredContent;
const view = data?.view;
const url = data?.url || view?.url;
if (!view || !url) return;
currentUrl = url;
open.hidden = false;
if (ready) return postView(view);
queued = view;
if (started) return;
started = true;
say("Loading God's Eye View…");
startApp(url, view).catch((error) =>
say(`God's Eye View could not load here: ${error?.message || error}.`),
);
setTimeout(() => {
if (!ready)
say(
"God's Eye View did not load here" +
(blocked ? ` (${blocked})` : '') +
". Use Open in God's Eye View above.",
);
}, config.loadTimeoutMs);
}
window.addEventListener('message', (event) => {
const message = event.data;
if (event.source === window) {
if (message?.type === 'gev:ready') {
ready = true;
status.remove();
if (queued) postView(queued);
queued = null;
}
return;
}
if (event.source !== window.parent || message?.jsonrpc !== '2.0') return;
if (
message.id !== undefined &&
pending.has(message.id) &&
!message.method
) {
const { resolve, reject } = pending.get(message.id);
pending.delete(message.id);
if (message.error) reject(message.error);
else resolve(message.result);
return;
}
if (message.method === 'ui/notifications/tool-result') show(message.params);
if (message.method === 'ui/notifications/host-context-changed')
showDisplayMode(message.params);
});
// Fullscreen, where the host offers it, gives the globe the room.
let displayMode = 'inline';
const showDisplayMode = (context) => {
if (context?.displayMode) displayMode = context.displayMode;
if (context?.availableDisplayModes)
expand.hidden = !context.availableDisplayModes.includes('fullscreen');
expand.textContent = displayMode === 'fullscreen' ? 'Collapse' : 'Expand';
};
expand.addEventListener('click', () => {
request('ui/request-display-mode', {
mode: displayMode === 'fullscreen' ? 'inline' : 'fullscreen',
}).then(
(result) => showDisplayMode({ displayMode: result?.mode }),
() => {},
);
});
open.addEventListener('click', () => {
if (!currentUrl) return;
request('ui/open-link', { url: currentUrl }).catch(() =>
window.open(currentUrl, '_blank', 'noopener'),
);
});
// The handshake the MCP Apps SDK's App.connect performs: hosts validate
// these parameters, and keep a view hidden until it is initialized.
request('ui/initialize', {
appCapabilities: { availableDisplayModes: ['inline', 'fullscreen'] },
appInfo: {
name: 'gods-eye-view',
title: "God's Eye View",
version: '1.0.0',
},
protocolVersion: config.protocolVersion,
}).then(
(result) => {
showDisplayMode(result?.hostContext);
send({ method: 'ui/notifications/initialized' });
notify('ui/notifications/size-changed', {
width: document.body.clientWidth,
height: config.panelHeight,
});
},
(error) =>
say(
"This client did not accept the God's Eye View panel" +
(error?.message ? `: ${error.message}` : '.'),
),
);
}
+13 -1
View File
@@ -4,6 +4,7 @@ import { initDrawTool } from '../annotations/drawTool.js';
import { initImageryBoxTool } from '../ui/imageryBoxTool.js';
import { createRecentImageryPanel } from '../ui/recentImagery.js';
import { initGevVoiceCommands } from '../voice/gevRealtime.js';
import { installViews, isEmbeddedInline } from './embed.js';
import { installScopeMask, destroyScopeMask } from '../scopeMask.js';
import {
installRenderGovernor,
@@ -118,7 +119,9 @@ export function createApplicationTools({
// GPU. Holder/data state is untouched, so return is seamless: restore
// the loop, refresh the one DOM surface we gated, render a frame.
const syncVisibilitySuspension = () => {
const hidden = document.hidden;
// A panel's host may report it hidden while it is on screen; the panel
// keeps drawing itself (see keepPanelRendering in embed.js).
const hidden = document.hidden && !isEmbeddedInline();
viewer.useDefaultRenderLoop = !hidden;
cockpitCloudEffects?.setSuspended?.(hidden);
if (!hidden) {
@@ -176,5 +179,14 @@ export function createApplicationTools({
delete window.__gevVoiceCommands;
});
debug.voiceCommands = voiceCommands;
defer(
installViews({
shell: styleManager,
viewer,
dataManager,
run: (name, args) => voiceCommands.runner(name, args, { signal }),
signal,
}),
);
return { sceneDirector, annotations, voiceCommands };
}
+33 -9
View File
@@ -86,23 +86,47 @@ test('Cockpit heading tape leaves the bottom exit row unobstructed', () => {
assert.doesNotMatch(css, /cockpit-compass-label/);
});
test('Cockpit vision cycle exposes exactly five real visual styles without NONE', () => {
assert.match(cycleVisionMode.toString(), /const modes = COCKPIT_VISION_MODES;/);
assert.match(setVisionMode.toString(), /const labels = \{\s*optical: inherited,\s*crt: 'CRT',\s*nvg: 'NVG',\s*thermal: 'FLIR',\s*noir: 'NOIR',?\s*\};/);
test('Cockpit vision cycle exposes every map preset exactly once', () => {
assert.match(
cycleVisionMode.toString(),
/const modes = COCKPIT_VISION_MODES;/,
);
assert.match(
setVisionMode.toString(),
/const labels = \{\s*optical: 'NORMAL',\s*crt: 'CRT',\s*nvg: 'NVG',\s*thermal: 'FLIR',\s*anime: 'ANIME',\s*noir: 'NOIR',\s*snow: 'SNOW',?\s*\};/,
);
assert.doesNotMatch(setVisionMode.toString(), /none: 'NONE'/);
assert.match(ui, /getInheritedVisionLabel: \(\) =>\s*\(?[\s\S]*?STYLE_STATUS_LABELS\[this\.activeStyle\]/);
assert.doesNotMatch(ui, /getInheritedVisionLabel/);
assert.match(html, /id="cockpit-vision-current-label"[^>]*>NORMAL<\/strong>/);
assert.match(ui, /const target = applyCockpitVisionStageIntensities\(\s*this\.stages,\s*next,\s*this\._cockpitVisionRestore,?\s*\);/);
assert.match(ui, /this\._cockpitVisionRestore = captureCockpitVisionBaseline\(\s*this\.stages,\s*this\.transitions,?\s*\);/);
assert.match(
cockpitEnter.toString(),
/this\.setVisionMode\(this\.getInitialVisionMode\(\)\)/,
'Cockpit must enter on the mode matching the active map preset',
);
assert.match(
readRadioSource(
new URL('./ui/cockpitCoordinator.js', import.meta.url),
'utf8',
),
/getInitialVisionMode:\s*\(\)\s*=>\s*cockpitVisionModeForStyle\(this\.activeStyle\)/,
);
assert.match(
ui,
/if \(next === 'optical'\) \{[\s\S]*?applyCockpitVisionStageIntensities\(\s*this\.stages,\s*next,\s*this\._cockpitVisionRestore,?\s*\);[\s\S]*?return;[\s\S]*?const target = applyCockpitVisionStageIntensities/,
'the inherited entry must restore the map shader while CRT, NVG, FLIR, and NOIR remain temporary Cockpit overrides',
/const target = applyCockpitVisionStageIntensities\(\s*this\.stages,\s*next,?\s*\);/,
);
assert.match(
ui,
/this\._cockpitVisionRestore = captureCockpitVisionBaseline\(\s*this\.stages,\s*this\.transitions,?\s*\);/,
);
assert.match(
ui,
/if \(next === 'optical'\) \{[\s\S]*?applyCockpitVisionStageIntensities\(\s*this\.stages,\s*next,?\s*\);[\s\S]*?this\._updateSliderPanel\(null,[\s\S]*?return;[\s\S]*?const target = applyCockpitVisionStageIntensities/,
'Normal must clear the inherited map shader while the other map presets remain temporary Cockpit overrides',
);
assert.match(
ui,
/_syncCockpitInheritedStyle\(\)[\s\S]*?name === this\.activeStyle \? 1 : 0[\s\S]*?this\.transitions\.delete\(name\)[\s\S]*?setVisionMode\(this\.cockpitView\.visionMode\)/,
'changing the map preset in Cockpit must refresh both the inherited label and restore baseline',
'changing the map preset in Cockpit must refresh the exit restore baseline without replacing Normal',
);
assert.match(ui, /setStyle\([\s\S]*?this\._syncCockpitInheritedStyle\(\);/);
});
+22 -5
View File
@@ -3,21 +3,40 @@ export const COCKPIT_VISION_MODES = Object.freeze([
'crt',
'nvg',
'thermal',
'anime',
'noir',
'snow',
]);
const TARGET_STYLE_BY_MODE = Object.freeze({
crt: 'retro',
nvg: 'surveillance',
thermal: 'thermal',
anime: 'anime',
noir: 'noir',
snow: 'snow',
});
/** Normalize a requested Cockpit vision mode to the inherited preset entry. */
const COCKPIT_MODE_BY_STYLE = Object.freeze({
normal: 'optical',
retro: 'crt',
surveillance: 'nvg',
thermal: 'thermal',
anime: 'anime',
noir: 'noir',
snow: 'snow',
});
/** Normalize a requested Cockpit vision mode to the unfiltered Normal entry. */
export function normalizeCockpitVisionMode(mode) {
return COCKPIT_VISION_MODES.includes(mode) ? mode : 'optical';
}
/** Start Cockpit on the equivalent supported map preset. */
export function cockpitVisionModeForStyle(style) {
return COCKPIT_MODE_BY_STYLE[String(style || '').toLowerCase()] || 'optical';
}
/** Settle pending map-style crossfades and return their intended final intensities. */
export function captureCockpitVisionBaseline(stages, transitions) {
const baseline = {};
@@ -34,12 +53,10 @@ export function captureCockpitVisionBaseline(stages, transitions) {
* Apply Cockpit-only stage intensities without changing any shader parameters.
* Returns the temporary style whose parameters should be shown, or null.
*/
export function applyCockpitVisionStageIntensities(stages, mode, restore = {}) {
export function applyCockpitVisionStageIntensities(stages, mode) {
const next = normalizeCockpitVisionMode(mode);
if (next === 'optical') {
for (const [name, intensity] of Object.entries(restore)) {
if (stages[name]) stages[name].uniforms.intensity = intensity;
}
for (const stage of Object.values(stages)) stage.uniforms.intensity = 0;
return null;
}
+75 -23
View File
@@ -3,6 +3,7 @@ import test from 'node:test';
import {
applyCockpitVisionStageIntensities,
captureCockpitVisionBaseline,
cockpitVisionModeForStyle,
COCKPIT_VISION_MODES,
normalizeCockpitVisionMode,
} from './cockpitVisionPolicy.js';
@@ -14,12 +15,31 @@ const createStages = () => ({
thermal: { uniforms: { intensity: 0, heat: 0.8 } },
});
test('Cockpit vision order exposes inherited, CRT, NVG, FLIR, and NOIR modes', () => {
assert.deepEqual(COCKPIT_VISION_MODES, ['optical', 'crt', 'nvg', 'thermal', 'noir']);
test('Cockpit vision order mirrors the unique map preset order', () => {
assert.deepEqual(COCKPIT_VISION_MODES, [
'optical',
'crt',
'nvg',
'thermal',
'anime',
'noir',
'snow',
]);
assert.equal(normalizeCockpitVisionMode('none'), 'optical');
assert.equal(normalizeCockpitVisionMode('unknown'), 'optical');
});
test('Cockpit entry selects the vision mode equivalent to the active map style', () => {
assert.equal(cockpitVisionModeForStyle('normal'), 'optical');
assert.equal(cockpitVisionModeForStyle('retro'), 'crt');
assert.equal(cockpitVisionModeForStyle('surveillance'), 'nvg');
assert.equal(cockpitVisionModeForStyle('thermal'), 'thermal');
assert.equal(cockpitVisionModeForStyle('anime'), 'anime');
assert.equal(cockpitVisionModeForStyle('noir'), 'noir');
assert.equal(cockpitVisionModeForStyle('snow'), 'snow');
assert.equal(cockpitVisionModeForStyle('unsupported'), 'optical');
});
test('Cockpit settles pending map crossfades before a temporary preset takes ownership', () => {
const stages = createStages();
const transitions = new Map([
@@ -29,49 +49,81 @@ test('Cockpit settles pending map crossfades before a temporary preset takes own
const restore = captureCockpitVisionBaseline(stages, transitions);
assert.deepEqual(restore, { noir: 1, retro: 0, surveillance: 0, thermal: 0 });
assert.equal(transitions.size, 0);
applyCockpitVisionStageIntensities(stages, 'nvg', restore);
applyCockpitVisionStageIntensities(stages, 'nvg');
assert.equal(stages.surveillance.uniforms.intensity, 1);
applyCockpitVisionStageIntensities(stages, 'optical', restore);
assert.equal(stages.noir.uniforms.intensity, 1);
applyCockpitVisionStageIntensities(stages, 'optical');
assert.equal(stages.noir.uniforms.intensity, 0);
assert.equal(stages.retro.uniforms.intensity, 0);
});
test('returning from a temporary preset restores the exact inherited intensities', () => {
test('Normal clears every temporary and inherited visual stage', () => {
const stages = createStages();
const restore = Object.fromEntries(
Object.entries(stages).map(([name, stage]) => [name, stage.uniforms.intensity]),
Object.entries(stages).map(([name, stage]) => [
name,
stage.uniforms.intensity,
]),
);
applyCockpitVisionStageIntensities(stages, 'thermal', restore);
applyCockpitVisionStageIntensities(stages, 'optical', restore);
applyCockpitVisionStageIntensities(stages, 'thermal');
applyCockpitVisionStageIntensities(stages, 'optical');
assert.deepEqual(
Object.fromEntries(Object.entries(stages).map(([name, stage]) => [name, stage.uniforms.intensity])),
restore,
Object.fromEntries(
Object.entries(stages).map(([name, stage]) => [
name,
stage.uniforms.intensity,
]),
),
{ noir: 0, retro: 0, surveillance: 0, thermal: 0 },
);
assert.deepEqual(restore, {
noir: 0.72,
retro: 0.18,
surveillance: 0,
thermal: 0,
});
});
test('temporary styles replace each other without changing the inherited restore snapshot', () => {
const stages = createStages();
const restore = Object.fromEntries(
Object.entries(stages).map(([name, stage]) => [name, stage.uniforms.intensity]),
Object.entries(stages).map(([name, stage]) => [
name,
stage.uniforms.intensity,
]),
);
applyCockpitVisionStageIntensities(stages, 'thermal', restore);
assert.equal(applyCockpitVisionStageIntensities(stages, 'crt', restore), 'retro');
applyCockpitVisionStageIntensities(stages, 'thermal');
assert.equal(applyCockpitVisionStageIntensities(stages, 'crt'), 'retro');
assert.equal(stages.retro.uniforms.intensity, 1);
assert.equal(stages.noir.uniforms.intensity, 0);
applyCockpitVisionStageIntensities(stages, 'optical', restore);
assert.equal(stages.noir.uniforms.intensity, 0.72);
assert.equal(stages.retro.uniforms.intensity, 0.18);
applyCockpitVisionStageIntensities(stages, 'optical');
assert.equal(stages.noir.uniforms.intensity, 0);
assert.equal(stages.retro.uniforms.intensity, 0);
assert.deepEqual(restore, {
noir: 0.72,
retro: 0.18,
surveillance: 0,
thermal: 0,
});
});
test('NOIR is a temporary Cockpit override and inherited restores the captured map style', () => {
test('NOIR is a temporary Cockpit override and Normal clears it', () => {
const stages = createStages();
const restore = Object.fromEntries(
Object.entries(stages).map(([name, stage]) => [name, stage.uniforms.intensity]),
Object.entries(stages).map(([name, stage]) => [
name,
stage.uniforms.intensity,
]),
);
assert.equal(applyCockpitVisionStageIntensities(stages, 'noir', restore), 'noir');
assert.equal(applyCockpitVisionStageIntensities(stages, 'noir'), 'noir');
assert.equal(stages.noir.uniforms.intensity, 1);
assert.equal(stages.retro.uniforms.intensity, 0);
applyCockpitVisionStageIntensities(stages, 'optical', restore);
assert.equal(stages.noir.uniforms.intensity, 0.72);
assert.equal(stages.retro.uniforms.intensity, 0.18);
applyCockpitVisionStageIntensities(stages, 'optical');
assert.equal(stages.noir.uniforms.intensity, 0);
assert.equal(stages.retro.uniforms.intensity, 0);
assert.deepEqual(restore, {
noir: 0.72,
retro: 0.18,
surveillance: 0,
thermal: 0,
});
});
+281
View File
@@ -0,0 +1,281 @@
import assert from 'node:assert/strict';
import test from 'node:test';
import path from 'node:path';
import { fileURLToPath } from 'node:url';
import { Readable } from 'node:stream';
import { readPinokioEnvironment } from '../scripts/pinokio-environment.mjs';
import { openAiRealtimeProxy } from 'gods-eye-view/server/providers/openai';
import {
makeCostRateLimiter,
resolvePerMinuteCap,
} from '../server/providers/common/rate-limit.js';
import { OPENAI_DEFAULT_PER_MIN } from '../server/providers/openai/rate-limit.js';
import { handleHudSummary } from '../server/providers/openai/hud-summary.js';
import {
googlePlacesContextProxy,
GOOGLE_DEFAULT_PER_MIN,
} from '../server/providers/places/google.js';
// Both limiters are cached at module scope on first use, so the env value that
// matters is the one in place when a route is first exercised — restoring the
// variable afterwards does not rebuild the limiter. Each route is therefore
// driven by exactly one test here; a second test wanting a different cap would
// need its own process, not just a different env value.
/** Set an env var for one test and put the old value back afterwards. */
function env(t, name, value) {
const old = process.env[name];
if (value === undefined) delete process.env[name];
else process.env[name] = value;
t.after(() => {
if (old === undefined) delete process.env[name];
else process.env[name] = old;
});
}
function install(plugin) {
const routes = new Map();
plugin.configureServer({
middlewares: { use: (route, handler) => routes.set(route, handler) },
restart: async () => {},
});
return routes;
}
function request(
handler,
{ url = '/', method = 'POST', remoteAddress = '127.0.0.1' } = {},
) {
return new Promise((resolve, reject) => {
const req = Readable.from([]);
Object.assign(req, {
method,
url,
headers: { host: 'localhost:4173', 'content-type': 'application/json' },
socket: { remoteAddress },
});
const headers = {};
const res = {
statusCode: 200,
setHeader(name, value) {
headers[name.toLowerCase()] = value;
},
end(body = '') {
resolve({ status: this.statusCode, headers, body: String(body) });
},
};
Promise.resolve(handler(req, res)).catch(reject);
});
}
test('an unset throttle resolves to the documented default, not to unlimited', () => {
// The defect #16 reports: the cap was opt-in, so an unconfigured server let
// anyone who could reach it mint tokens on the operator's key without bound.
assert.equal(resolvePerMinuteCap(undefined, 30), 30);
assert.equal(resolvePerMinuteCap('', 30), 30);
assert.equal(resolvePerMinuteCap(' ', 30), 30);
assert.ok(
typeof makeCostRateLimiter(undefined, 30) === 'function',
'an unconfigured cost endpoint must still be throttled',
);
});
test('an explicit 0 is the documented way to run unthrottled', () => {
assert.equal(resolvePerMinuteCap('0', 30), 0);
assert.equal(
makeCostRateLimiter('0', 30),
null,
'operators who mean "no throttle" must still be able to say so',
);
});
test('a configured value wins, and a fractional one floors', () => {
assert.equal(resolvePerMinuteCap('45', 30), 45);
assert.equal(resolvePerMinuteCap(45, 30), 45);
assert.equal(resolvePerMinuteCap('30.7', 30), 30);
});
test('a positive fraction keeps the guard rather than flooring to unlimited', () => {
for (const fraction of ['0.5', '0.99', 0.01, '1e-9']) {
assert.equal(resolvePerMinuteCap(fraction, 30), 1, String(fraction));
assert.equal(
typeof makeCostRateLimiter(fraction, 30),
'function',
`${fraction} must still throttle`,
);
}
// Only an explicit zero turns the limiter off.
assert.equal(makeCostRateLimiter('0', 30), null);
assert.equal(makeCostRateLimiter('0.0', 30), null);
});
test('an unreadable value keeps the guard instead of removing it', () => {
// `3O` (letter O) and `60/min` are the shapes a hand-edited .env produces.
// Reading either as "unlimited" would disarm the spend guard on a typo —
// the opposite of what the operator was reaching for when they typed it.
for (const typo of ['3O', '60/min', 'thirty', '-5', 'NaN']) {
assert.equal(
resolvePerMinuteCap(typo, 30),
30,
`${typo} must fall back to the default, not to unlimited`,
);
}
});
test('the default cap clears the app’s own demand with room to spare', () => {
// src/hud.js HUD_SUMMARY_INTERVAL_MS = 15000 -> 4 summaries/min per open
// tab, plus one token mint per voice session. The cap has to sit well above
// that or it would throttle the app rather than an abuser.
const hudRequestsPerMinutePerTab = 60_000 / 15_000;
assert.ok(
OPENAI_DEFAULT_PER_MIN >= hudRequestsPerMinutePerTab * 4,
`default ${OPENAI_DEFAULT_PER_MIN}/min leaves too little headroom over ${hudRequestsPerMinutePerTab}/min of ordinary use`,
);
});
test('the realtime token route throttles by default, and says so in a 429', async (t) => {
env(t, 'GEV_RATELIMIT_OPENAI_PER_MIN', undefined);
env(t, 'OPENAI_API_KEY', 'fixture-upstream-secret');
let upstreamCalls = 0;
t.mock.method(globalThis, 'fetch', async () => {
upstreamCalls += 1;
return Response.json({ value: 'fixture-ephemeral' });
});
const handler = install(openAiRealtimeProxy({})).get('/api/realtime/token');
const statuses = [];
for (let i = 0; i < OPENAI_DEFAULT_PER_MIN + 1; i += 1) {
statuses.push((await request(handler)).status);
}
assert.deepEqual(
statuses.slice(0, OPENAI_DEFAULT_PER_MIN),
Array(OPENAI_DEFAULT_PER_MIN).fill(200),
'the default must not throttle ordinary use',
);
const overLimit = await request(handler);
assert.equal(overLimit.status, 429);
assert.equal(overLimit.headers['retry-after'], '5');
assert.equal(JSON.parse(overLimit.body).error, 'Rate limit exceeded');
assert.equal(
overLimit.body.includes('fixture-upstream-secret'),
false,
'a throttled reply must not leak the server-held key',
);
assert.ok(
upstreamCalls <= OPENAI_DEFAULT_PER_MIN,
'a throttled request must be refused before it reaches OpenAI, or it still costs money',
);
});
test('the HUD summary route throttles the paid path, never the free one', async (t) => {
env(t, 'OPENAI_API_KEY', 'fixture-upstream-secret');
let upstreamCalls = 0;
t.mock.method(globalThis, 'fetch', async () => {
upstreamCalls += 1;
return Response.json({ output_text: 'five word hud summary here' });
});
// Its own client address: the limiter is cached at module scope and shared
// across the OpenAI routes, so the realtime case above has already spent
// 127.0.0.1's budget inside this process.
const from = '10.9.9.9';
const statuses = [];
for (let i = 0; i < OPENAI_DEFAULT_PER_MIN; i += 1)
statuses.push(
(await request(handleHudSummary, { remoteAddress: from })).status,
);
assert.deepEqual(
statuses,
Array(OPENAI_DEFAULT_PER_MIN).fill(200),
'the default must not throttle ordinary HUD use',
);
const overLimit = await request(handleHudSummary, { remoteAddress: from });
assert.equal(overLimit.status, 429);
assert.equal(overLimit.headers['retry-after'], '5');
assert.equal(JSON.parse(overLimit.body).error, 'Rate limit exceeded');
assert.equal(
overLimit.body.includes('fixture-upstream-secret'),
false,
'a throttled reply must not leak the server-held key',
);
assert.ok(
upstreamCalls <= OPENAI_DEFAULT_PER_MIN,
'a throttled request must be refused before it reaches OpenAI, or it still costs money',
);
// The keyless HUD answer is deliberately NOT throttled: it is composed
// locally, spends nothing upstream, and capping it would only take the free
// HUD away from an unconfigured install. Same exhausted client, no key.
delete process.env.OPENAI_API_KEY;
const free = await request(handleHudSummary, { remoteAddress: from });
assert.equal(free.status, 200, 'the free path answers past a spent budget');
assert.equal(JSON.parse(free.body).configured, false);
assert.equal(
upstreamCalls <= OPENAI_DEFAULT_PER_MIN,
true,
'the free path reaches no provider',
);
});
test('the Google places route throttles by default and keeps its places[] contract', async (t) => {
env(t, 'GEV_RATELIMIT_GOOGLE_PER_MIN', undefined);
let upstreamCalls = 0;
const handler = install(
googlePlacesContextProxy({
resolveApiKey: () => 'fixture-google-server-key',
fetchImpl: async () => {
upstreamCalls += 1;
return Response.json({ places: [] });
},
}),
).get('/api/google/nearby-places');
const url = '/?lat=37.77493&lon=-122.41942';
const statuses = [];
for (let i = 0; i < GOOGLE_DEFAULT_PER_MIN; i += 1) {
statuses.push((await request(handler, { url, method: 'GET' })).status);
}
assert.ok(
statuses.every((s) => s === 200),
'the default must not throttle ordinary use',
);
const overLimit = await request(handler, { url, method: 'GET' });
assert.equal(overLimit.status, 429);
assert.equal(overLimit.headers['retry-after'], '5');
// Every error response on this route carries places[]; the client reads it
// unconditionally, so a throttled reply must not be the one shape that omits it.
assert.deepEqual(JSON.parse(overLimit.body).places, []);
assert.equal(
upstreamCalls,
GOOGLE_DEFAULT_PER_MIN,
'a throttled request must be refused before it reaches Google, or it still costs money',
);
});
test('the code defaults match the caps the Pinokio build ships', () => {
// These two numbers used to live only in pinokio/_ENVIRONMENT, and the value
// this repo suggested elsewhere had already drifted away from them. Now that
// they are the server's defaults, the packaged app and an unconfigured
// checkout must agree, or a Pinokio user's throttle silently stops matching
// the one everybody else gets. If you deliberately want them to differ, this
// assertion is the place to say so.
const shipped = readPinokioEnvironment(
path.join(
path.dirname(fileURLToPath(import.meta.url)),
'..',
'pinokio',
'_ENVIRONMENT',
),
);
assert.equal(
Number(shipped.GEV_RATELIMIT_OPENAI_PER_MIN),
OPENAI_DEFAULT_PER_MIN,
);
assert.equal(
Number(shipped.GEV_RATELIMIT_GOOGLE_PER_MIN),
GOOGLE_DEFAULT_PER_MIN,
);
});
+16 -75
View File
@@ -58,30 +58,18 @@ export function normalizeAdminName(value) {
const PACKS = {
countries: {
url: new URL('./local_data/natural_earth/countries.json', import.meta.url),
importJson: () =>
import('./local_data/natural_earth/countries.json', {
with: { type: 'json' },
}),
},
admin1: {
url: new URL(
'./local_data/natural_earth/states_provinces.json',
import.meta.url,
),
importJson: () =>
import('./local_data/natural_earth/states_provinces.json', {
with: { type: 'json' },
}),
},
counties: {
url: new URL(
'./local_data/us_census_counties/counties.json',
import.meta.url,
),
importJson: () =>
import('./local_data/us_census_counties/counties.json', {
with: { type: 'json' },
}),
},
};
@@ -165,17 +153,14 @@ export function polygonsContain(polygons, lat, lon) {
/**
* A unit's geometry, decoded on first use. Longitudes are unwrapped around
* the largest part for the box so units cut at the antimeridian (Alaska,
* Chukotka) get a box on the right side of the globe. An entry from another
* encoding supplies `decode()` returning the same `[outer, ...holes][]`.
* Chukotka) get a box on the right side of the globe.
*/
export function geometryOf(entry) {
function geometryOf(entry) {
if (entry.geometry) return entry.geometry;
const decimals = entry.feature.d ?? entry.decimals;
const polygons = entry.decode
? entry.decode()
: entry.feature.polygons.map((poly) =>
poly.map((ring) => decodeRing(ring, decimals)),
);
const polygons = entry.feature.polygons.map((poly) =>
poly.map((ring) => decodeRing(ring, decimals)),
);
const ref = polygons[0]?.[0]?.[0]?.[0] ?? 0;
let west = Infinity;
let south = Infinity;
@@ -193,7 +178,7 @@ export function geometryOf(entry) {
if (lat > north) north = lat;
}
}
const label = Array.isArray(entry.feature.label)
const label = entry.feature.label
? { lon: entry.feature.label[0], lat: entry.feature.label[1] }
: ringCentroid(polygons[0][0]);
entry.geometry = {
@@ -223,7 +208,7 @@ function ringCentroid(ring) {
}
/** Km from a point to a unit's box (0 inside it). */
export function boxDistanceKm(bbox, lat, lon) {
function boxDistanceKm(bbox, lat, lon) {
const [west, south, east, north] = bbox;
const mid = (west + east) / 2;
const x = lon - 360 * Math.round((lon - mid) / 360);
@@ -300,7 +285,7 @@ function addKey(index, key, entry, tier) {
const loadAdmin1 = createRetryableLoader(async () => {
packLoads.admin1 += 1;
const pack = await loadBundledJson(PACKS.admin1.url, PACKS.admin1.importJson);
const pack = await loadBundledJson(PACKS.admin1.url);
const decimals = pack.meta?.decimals ?? 3;
const index = new Map();
for (const feature of pack.features || []) {
@@ -326,10 +311,7 @@ const loadAdmin1 = createRetryableLoader(async () => {
const loadCounties = createRetryableLoader(async () => {
packLoads.counties += 1;
const pack = await loadBundledJson(
PACKS.counties.url,
PACKS.counties.importJson,
);
const pack = await loadBundledJson(PACKS.counties.url);
const decimals = pack.meta?.decimals ?? 4;
const byFull = new Map();
const byName = new Map();
@@ -351,10 +333,7 @@ const loadCounties = createRetryableLoader(async () => {
const loadCountries = createRetryableLoader(async () => {
packLoads.countries += 1;
const pack = await loadBundledJson(
PACKS.countries.url,
PACKS.countries.importJson,
);
const pack = await loadBundledJson(PACKS.countries.url);
const index = new Map();
for (const feature of pack.features || []) {
const entry = {
@@ -420,8 +399,8 @@ const COUNTY_WORDS = [
];
const COUNTY_WORD_RE = new RegExp(`\\b(${COUNTY_WORDS.join('|')})\\b`);
const STATE_PREFIX_RE =
/^(free state|state|commonwealth|province|territory|prefecture|region|land|canton|department) of (.+)$/;
const STATE_SUFFIX_RE = /^(.+) (state|province|prefecture|territory)$/;
/^(?:free state|state|commonwealth|province|territory|prefecture|region|land|canton|department) of (.+)$/;
const STATE_SUFFIX_RE = /^(.+) (?:state|province|prefecture|territory)$/;
/**
* Split an ask into what it names and how.
@@ -482,17 +461,14 @@ export function parseAdminQuery(text) {
if (prefix)
return {
kind: 'state',
name: prefix[2],
name: prefix[1],
qualifiers,
postal: null,
countyWord: null,
full: head,
form: 'prefix',
adminType: prefix[1],
};
// South Africa's canonical province name is literally "Free State"; do not
// strip its final word as though the ask explicitly named an admin type.
const suffix = head === 'free state' ? null : STATE_SUFFIX_RE.exec(head);
const suffix = STATE_SUFFIX_RE.exec(head);
if (suffix)
return {
kind: 'state',
@@ -502,7 +478,6 @@ export function parseAdminQuery(text) {
countyWord: null,
full: head,
form: 'suffix',
adminType: suffix[2],
};
return {
kind: 'any',
@@ -514,30 +489,6 @@ export function parseAdminQuery(text) {
};
}
/** Explicit wording is stronger than a stale same-name geometry. */
function explicitStateTypeFits(entry, parsed) {
if (!parsed.adminType) return true;
const words = new Set(
normalizeAdminName(entry.feature?.type)
.split(/[^a-z]+/)
.filter(Boolean),
);
const accepted =
{
'free state': ['state'],
state: ['state'],
commonwealth: ['state'],
province: ['province'],
territory: ['territory'],
prefecture: ['prefecture'],
region: ['region'],
land: ['state'],
canton: ['canton'],
department: ['department', 'departamento'],
}[parsed.adminType] || [];
return accepted.some((word) => words.has(word));
}
/** A qualifier names the unit's country ("Bavaria, Germany") or the unit
* itself ("Washington, DC"). */
function qualifierMatches(entry, qualifier) {
@@ -554,7 +505,7 @@ function qualifierMatches(entry, qualifier) {
* Rank candidates: a unit containing `near` first, then the match tier,
* then distance from `near`, then larger area. Returns the ranked list.
*/
export function rank(candidates, near) {
function rank(candidates, near) {
const hasNear = Number.isFinite(near?.lat) && Number.isFinite(near?.lon);
const scored = candidates.map(({ entry, tier }) => {
const geometry = geometryOf(entry);
@@ -675,10 +626,6 @@ async function stateCandidates(parsed, { allowAmbiguous = false } = {}) {
list = list.filter((i) =>
parsed.qualifiers.every((q) => qualifierMatches(i.entry, q)),
);
// Preserve explicit identity at the selector seam. In particular,
// "Bagmati Province" must not select Natural Earth's historical NP-BA
// Bagmati Zone merely because the point and normalized name overlap.
list = list.filter((i) => explicitStateTypeFits(i.entry, parsed));
if (allowAmbiguous) return list;
// A bare name answers alone only for a prominent unit — a US state, a
// Canadian province, Bavaria — never a French département, an English
@@ -791,12 +738,6 @@ export async function findAdminAreaAt(names, lat, lon, scope) {
.map(parseAdminQuery)
.filter((p) => p?.name);
if (!parsedList.length) return null;
// If the ask explicitly names an admin type, do not let the geocoder's
// untyped alias reopen an incompatible historical same-name boundary.
const explicitStates = parsedList.filter(
(parsed) => parsed.kind === 'state' && parsed.adminType,
);
const stateParses = explicitStates.length ? explicitStates : parsedList;
const containing = (items) => {
const out = [];
for (const item of items)
@@ -841,7 +782,7 @@ export async function findAdminAreaAt(names, lat, lon, scope) {
return toResult(rank(found, { lat, lon })[0], found.length);
}
const states = [];
for (const parsed of stateParses)
for (const parsed of parsedList)
states.push(
...(await stateCandidates(
{ ...parsed, qualifiers: [] },
-22
View File
@@ -225,28 +225,6 @@ test('geocoder-confirmed lookup: the named unit that contains the point', async
assert.equal(await findAdminAreaAt(['Georgia'], 41.7, 44.8, 'state'), null);
const newYork = await findAdminAreaAt(['New York'], 42.9, -75.5, 'state');
assert.equal(newYork?.id, 'US-NY');
assert.equal(
(
await findAdminAreaAt(
['Ontario Province', 'Ontario'],
43.6532,
-79.3832,
'state',
)
)?.id,
'CA-ON',
);
// Natural Earth still carries the historical NP-BA Bagmati Zone. An ask
// for today's province must not silently select that incompatible geometry.
assert.equal(
await findAdminAreaAt(
['Bagmati Province', 'Bagmati'],
27.7172,
85.324,
'state',
),
null,
);
// A county the geocoder typed, named without the county word.
const travis = await findAdminAreaAt(
['Travis', 'Travis County'],
+3
View File
@@ -5,6 +5,9 @@ import { createAisStreamSource } from '../sources/live/standalone.js';
const aisLiveVesselsLayer = createApplicationVessels({
source: createAisStreamSource({
apiUrl: import.meta.env?.VITE_AIS_LIVE_API_URL || '/api/ais-live',
// Resolved against the document's address, which a panel host may serve
// from its own scheme.
origin: () => globalThis.document?.baseURI ?? 'http://localhost',
}),
options: {
maxRows: import.meta.env?.VITE_AIS_LIVE_MAX_ROWS,
+140 -868
View File
File diff suppressed because it is too large. Load diff
+7 -374
View File
@@ -174,95 +174,11 @@ test('analyst: nearest sorting attaches distanceKm ascending', async () => {
test('analyst: follow-up re-filters the remembered set without re-snapshot', async () => {
const eng = makeEngine();
const first = await eng.query({
layers: ['flights'],
scope: { kind: 'region', name: 'Texland' },
});
const r = await eng.query({
followUp: true,
filters: [{ field: 'onGround', op: 'eq', value: true }],
});
await eng.query({ layers: ['flights'], scope: { kind: 'region', name: 'Texland' } });
const r = await eng.query({ followUp: true, filters: [{ field: 'onGround', op: 'eq', value: true }] });
assert.equal(r.count, 1);
assert.equal(r.items[0].id, 'GND1');
assert.equal(r.coverage.followUp, true);
assert.equal(r.scopeLabel, first.scopeLabel);
assert.equal(r.display.scope, first.display.scope);
assert.equal(r.coverage.scope, first.coverage.scope);
});
test('analyst: follow-ups preserve partial and unanswered coverage', async () => {
const eng = createAnalystEngine({
getRecords: (key) => (key === 'flights' ? FLIGHTS : []),
getLayerSnapshot: (key) => ({
enabled: key === 'flights',
feedState: key === 'flights' ? 'nominal' : 'off',
}),
getViewContext: () => ({ lat: 30.27, lon: -97.74, viewRadiusKm: 150 }),
});
const first = await eng.query({
layers: ['flights', 'military'],
scope: { kind: 'anywhere' },
});
assert.equal(first.partial, true);
assert.deepEqual(first.unanswered, ['military']);
const followUp = await eng.query({ followUp: true, limit: 1 });
assert.equal(followUp.partial, true);
assert.deepEqual(followUp.unanswered, ['military']);
assert.deepEqual(
followUp.coverage.layersQueried.map(({ layerKey, status }) => [
layerKey,
status,
]),
first.coverage.layersQueried.map(({ layerKey, status }) => [
layerKey,
status,
]),
);
});
test('analyst: cancelled pre-commit work cannot replace follow-up memory', async () => {
const eng = makeEngine();
await eng.query({
layers: ['flights'],
scope: { kind: 'anywhere' },
filters: [{ field: 'id', op: 'eq', value: 'SWA1' }],
});
let current = true;
const stale = await eng.query(
{ layers: ['local-firms'], scope: { kind: 'anywhere' } },
{
isCurrent: () => current,
beforeCommit: async () => {
current = false;
},
},
);
assert.equal(stale.code, 'CANCELLED');
const followUp = await eng.query({ followUp: true });
assert.deepEqual(followUp.items.map((item) => item.id), ['SWA1']);
});
test('analyst: failed pre-commit work cannot replace follow-up memory', async () => {
const eng = makeEngine();
await eng.query({
layers: ['flights'],
scope: { kind: 'anywhere' },
filters: [{ field: 'id', op: 'eq', value: 'SWA1' }],
});
await assert.rejects(
eng.query(
{ layers: ['local-firms'], scope: { kind: 'anywhere' } },
{
beforeCommit: async () => {
throw new Error('projection failed');
},
},
),
/projection failed/,
);
const followUp = await eng.query({ followUp: true });
assert.deepEqual(followUp.items.map((item) => item.id), ['SWA1']);
});
test('analyst: unresolved region is an honest failure, not empty success', async () => {
@@ -284,39 +200,6 @@ test('analyst: a region lookup timeout is reported as region-timeout', async ()
assert.equal(r.coverage.scope, 'region:Texas:timeout');
});
test('analyst: region lookup receives and obeys the owning action signal', async () => {
const controller = new AbortController();
let receivedSignal = null;
const eng = createAnalystEngine({
getRecords: () => FLIGHTS,
resolveRegionRing: async (_name, signal) => {
receivedSignal = signal;
await new Promise((resolve) =>
signal.addEventListener('abort', resolve, { once: true }),
);
return null;
},
getViewContext: () => ({
lat: 30.27,
lon: -97.74,
viewRadiusKm: 150,
}),
});
const pending = eng.query(
{ layers: ['flights'], scope: { kind: 'region', name: 'Texas' } },
{
signal: controller.signal,
isCurrent: () => !controller.signal.aborted,
},
);
await Promise.resolve();
controller.abort();
const result = await pending;
assert.equal(receivedSignal, controller.signal);
assert.equal(result.code, 'CANCELLED');
assert.equal(result.cancelled, true);
});
test('analyst: route fields queryable from cached enrichment only', async () => {
const r = await makeEngine().query({
layers: ['flights'], scope: { kind: 'anywhere' },
@@ -379,132 +262,20 @@ test('helpers: haversine sanity + scope radius', () => {
assert.deepEqual(scoped.map((f) => f.id).sort(), ['GND1', 'SWA1']);
});
test('capped cohorts report returned/total/truncated and keep it on follow-up', async () => {
test('bounded loaded cohorts disclose truncation before filtering and retain it on follow-up', async () => {
const engine = createAnalystEngine({
getRecords: () => [{ id: 'sample dam', lat: 0, lon: 0, name: 'sample' }],
getRecordCoverage: () => ({ total: 3000, truncated: true }),
getRecordCoverage: () => ({ basis: 'bounded-loaded-records', recordsExamined: 1, loadedCount: 3000, sourceTruncated: true }),
getViewContext: () => ({ lat: 0, lon: 0, viewRadiusKm: 25 }),
});
const result = await engine.query({ layers: ['local-dams'], scope: { kind: 'anywhere' }, sortBy: 'distance' });
assert.equal(result.count, 1);
const layer = result.coverage.layersQueried[0];
assert.deepEqual({ returned: layer.returned, total: layer.total, truncated: layer.truncated }, { returned: 1, total: 3000, truncated: true });
assert.deepEqual(result.coverage.records, { returned: 1, total: 3000, truncated: true });
assert.match(result.display.caveat, /counted 1 of 3,000 loaded records/);
assert.equal(result.coverage.layersQueried[0].sourceTruncated, true);
assert.match(result.coverage.note, /omitted records may change the nearest item or count/);
const followUp = await engine.query({ followUp: true });
assert.equal(followUp.coverage.layersQueried[0].total, 3000);
assert.equal(followUp.coverage.layersQueried[0].loadedCount, 3000);
});
test('an uncapped cohort is complete and carries no caveat', async () => {
const result = await makeEngine().query({ layers: ['flights'], scope: { kind: 'anywhere' } });
assert.deepEqual(result.coverage.records, { returned: 4, total: 4, truncated: false });
assert.equal(result.display.caveat, undefined);
assert.equal(result.display.scope, 'anywhere in the loaded data');
});
test('unknown fields, operators, scopes and wrong-typed values are refused with the allowed values', async () => {
const engine = makeEngine();
const field = await engine.query({ layers: ['flights'], filters: [{ field: 'time', op: 'gte', value: 5 }] });
assert.equal(field.ok, false);
assert.equal(field.code, 'UNKNOWN_FIELD');
assert.ok(field.allowed.includes('altitudeM'));
const op = await engine.query({ layers: ['flights'], filters: [{ field: 'altitudeM', op: 'wat', value: 5 }] });
assert.equal(op.code, 'BAD_OPERATOR');
assert.deepEqual(op.allowed, ['gt', 'gte', 'lt', 'lte', 'eq', 'neq']);
const flag = await engine.query({ layers: ['flights'], filters: [{ field: 'military', op: 'eq', value: 'false' }] });
assert.equal(flag.code, 'BAD_VALUE', 'the string "false" must not match the TRUE record');
const num = await engine.query({ layers: ['flights'], filters: [{ field: 'altitudeM', op: 'gt', value: 'high' }] });
assert.equal(num.code, 'BAD_VALUE');
const scope = await engine.query({ layers: ['flights'], scope: { kind: 'polygon' } });
assert.equal(scope.code, 'UNKNOWN_SCOPE');
assert.deepEqual(scope.allowed, ['view', 'region', 'radius', 'anywhere']);
const sort = await engine.query({ layers: ['flights'], sortBy: 'loudness' });
assert.equal(sort.code, 'UNKNOWN_FIELD');
assert.equal(engine.hasMemory(), false, 'refusals never become follow-up memory');
});
test('a typed boolean filter still matches exactly', async () => {
const r = await makeEngine().query({ layers: ['flights'], scope: { kind: 'anywhere' }, filters: [{ field: 'military', op: 'eq', value: false }] });
assert.deepEqual(r.items.map((i) => i.id).sort(), ['GND1', 'N123', 'SWA1']);
});
test('distance honours sortDir: desc is farthest first', async () => {
const far = await makeEngine().query({ layers: ['flights'], scope: { kind: 'anywhere' }, sortBy: 'distance', sortDir: 'desc', limit: 1 });
assert.equal(far.items[0].id, 'N123');
const near = await makeEngine().query({ layers: ['flights'], scope: { kind: 'anywhere' }, sortBy: 'distance', limit: 1 });
assert.equal(near.items[0].id, 'SWA1');
});
test('a follow-up without a previous answer is refused, not a fresh flights query', async () => {
const r = await makeEngine().query({ followUp: true, filters: [] });
assert.equal(r.ok, false);
assert.equal(r.code, 'NO_RESULT_CONTEXT');
});
test('follow-ups never rewrite the remembered rows', async () => {
const engine = makeEngine();
await engine.query({ layers: ['flights'], scope: { kind: 'anywhere' } });
await engine.query({ followUp: true, sortBy: 'distance' });
const again = await engine.query({ followUp: true });
assert.equal(again.count, 4, 'a follow-up keeps the remembered set, not the view');
assert.ok(again.items.every((item) => item.distanceKm === undefined));
});
test('a superseded slow region query cannot overwrite the newer follow-up memory', async () => {
let releaseSlow;
const engine = createAnalystEngine({
getRecords: (key) => ({ flights: FLIGHTS, 'ais-live-vessels': SHIPS }[key] || []),
resolveRegionRing: () => new Promise((resolve) => { releaseSlow = () => resolve(TEXLAND); }),
getViewContext: () => ({ lat: 30.27, lon: -97.74, viewRadiusKm: 150 }),
});
let generation = 0;
const slowGeneration = ++generation;
const slow = engine.query({ layers: ['flights'], scope: { kind: 'region', name: 'Texland' } }, { isCurrent: () => generation === slowGeneration });
const newGeneration = ++generation;
const fresh = await engine.query({ layers: ['ais-live-vessels'], scope: { kind: 'anywhere' } }, { isCurrent: () => generation === newGeneration });
assert.equal(fresh.ok, true);
releaseSlow();
const stale = await slow;
assert.equal(stale.code, 'CANCELLED');
const followUp = await engine.query({ followUp: true });
assert.deepEqual(followUp.items.map((i) => i.layerKey), ['ais-live-vessels', 'ais-live-vessels']);
});
test('off, loading and unavailable layers are refused distinctly; a real zero stays ok', async () => {
const engineFor = (snapshot, rows = [], warming = false) => createAnalystEngine({
getRecords: () => rows,
getLayerSnapshot: () => snapshot,
isWarming: () => warming,
getViewContext: () => ({ lat: 0, lon: 0, viewRadiusKm: 25 }),
});
assert.equal((await engineFor({ enabled: false, feedState: 'off' }).query({ layers: ['earthquakes'] })).code, 'LAYER_OFF');
assert.equal((await engineFor({ enabled: true, feedState: 'loading' }).query({ layers: ['earthquakes'] })).code, 'NOT_READY');
assert.equal((await engineFor({ enabled: true, feedState: 'nominal' }, [], true).query({ layers: ['earthquakes'] })).code, 'NOT_READY');
assert.equal((await engineFor({ enabled: true, feedState: 'unavailable' }).query({ layers: ['earthquakes'] })).code, 'FEED_UNAVAILABLE');
const zero = await engineFor({ enabled: true, feedState: 'nominal' }, [{ id: 'q', lat: 50, lon: 50, magnitude: 2 }]).query({ layers: ['earthquakes'], filters: [{ field: 'magnitude', op: 'gte', value: 5 }] });
assert.equal(zero.ok, true);
assert.equal(zero.count, 0);
assert.equal(zero.coverage.layersQueried[0].status, 'ok');
});
test('time fields filter by ISO and derive ageHours; the feed window is exposed', async () => {
const now = Date.parse('2026-09-23T12:00:00Z');
const engine = createAnalystEngine({
now: () => now,
getRecords: () => [
{ id: 'recent', lat: 0, lon: 0, magnitude: 4, timeMs: now - 2 * 3_600_000 },
{ id: 'older', lat: 0, lon: 0, magnitude: 5, timeMs: now - 20 * 3_600_000 },
],
getViewContext: () => ({ lat: 0, lon: 0, viewRadiusKm: 25 }),
});
const iso = await engine.query({ layers: ['earthquakes'], scope: { kind: 'anywhere' }, filters: [{ field: 'timeMs', op: 'gte', value: '2026-09-23T06:00:00Z' }] });
assert.deepEqual(iso.items.map((i) => i.id), ['recent']);
const age = await engine.query({ layers: ['earthquakes'], scope: { kind: 'anywhere' }, filters: [{ field: 'ageHours', op: 'lte', value: 6 }] });
assert.deepEqual(age.items.map((i) => i.id), ['recent']);
assert.equal(age.items[0].ageHours, 2);
assert.equal(age.display.window, 'last 24 h');
assert.equal(age.coverage.layersQueried[0].window, 'last 24 h');
});
test('follow-up provenance stays attached to old rows after a feed recovers', async () => {
let state = 'stale';
@@ -522,141 +293,3 @@ test('follow-up provenance stays attached to old rows after a feed recovers', as
assert.equal(engine.hasMemory(), false);
assert.equal((await engine.query()).coverage.feedProvenance.overall, 'nominal');
});
test('a large loaded set sorts and summarizes without overflowing the stack', async () => {
const rows = Array.from({ length: 200_000 }, (_, i) => ({ id: `FIRE-${i}`, lat: 0, lon: 0, frp: i }));
const engine = createAnalystEngine({
getRecords: () => rows,
getViewContext: () => ({ lat: 0, lon: 0, viewRadiusKm: 25 }),
});
const result = await engine.query({ layers: ['local-firms'], scope: { kind: 'anywhere' }, sortBy: 'frp', limit: 1 });
assert.equal(result.count, 200_000);
assert.equal(result.items[0].frp, 199_999);
assert.equal(result.summary.frpMin, 0);
});
test('an enabled but empty layer says why on screen', async () => {
const engine = createAnalystEngine({
getRecords: () => [],
getLayerSnapshot: () => ({ enabled: true, feedState: 'nominal' }),
getEmptyNote: () => 'Fly below 400 km to a covered region',
getViewContext: () => ({ lat: 0, lon: 0, viewRadiusKm: 25 }),
});
const result = await engine.query({ layers: ['transit'] });
assert.equal(result.ok, true);
assert.equal(result.count, 0);
assert.equal(result.coverage.layersQueried[0].status, 'empty');
assert.match(result.display.caveat, /transit: Fly below 400 km/);
});
test('a radius around a named place without a centre is refused, not measured around the camera', async () => {
const r = await makeEngine().query({ layers: ['ais-live-vessels'], scope: { kind: 'radius', name: 'Singapore', km: 200 } });
assert.equal(r.ok, false);
assert.equal(r.code, 'BAD_SCOPE');
const ok = await makeEngine().query({ layers: ['ais-live-vessels'], scope: { kind: 'radius', name: 'Houston', km: 200, center: { lat: 29.6, lon: -95.0 } } });
assert.equal(ok.ok, true);
});
test('a field copied with its hinted unit still resolves', async () => {
const r = await makeEngine().query({ layers: ['local-firms'], scope: { kind: 'anywhere' }, sortBy: 'frp(MW)', filters: [{ field: 'frp (MW)', op: 'gt', value: 100 }] });
assert.equal(r.ok, true);
assert.deepEqual(r.items.map((i) => i.id), ['FIRE-3', 'FIRE-1']);
});
test('a written unit converts to the field unit or is refused', async () => {
const engine = makeEngine();
const feet = await engine.query({ layers: ['flights'], scope: { kind: 'anywhere' }, filters: [{ field: 'altitudeM(ft)', op: 'gt', value: 40000 }] });
assert.equal(feet.ok, true);
assert.deepEqual(feet.items.map((i) => i.id), ['RCH01'], '40,000 ft is 12,192 m, not 40,000 m');
const same = await engine.query({ layers: ['flights'], scope: { kind: 'anywhere' }, filters: [{ field: 'altitudeM (m)', op: 'gt', value: 12192 }] });
assert.deepEqual(same.items.map((i) => i.id), ['RCH01']);
const knots = await engine.query({ layers: ['flights'], scope: { kind: 'anywhere' }, filters: [{ field: 'speedMps(kts)', op: 'gt', value: 470 }] });
assert.deepEqual(knots.items.map((i) => i.id), ['RCH01'], '470 kn is ~242 m/s');
const wrong = await engine.query({ layers: ['flights'], filters: [{ field: 'altitudeM(kts)', op: 'gt', value: 1 }] });
assert.equal(wrong.code, 'BAD_UNIT');
const unitless = await engine.query({ layers: ['flights'], filters: [{ field: 'callsign(ft)', op: 'eq', value: 'X' }] });
assert.equal(unitless.code, 'BAD_UNIT');
const hinted = await engine.query({ layers: ['local-firms'], scope: { kind: 'anywhere' }, sortBy: 'frp(MW)' });
assert.equal(hinted.ok, true);
});
test('unambiguous numeric strings are numbers; blanks, booleans and words are not', async () => {
const engine = makeEngine();
const text = await engine.query({ layers: ['flights'], scope: { kind: 'anywhere' }, filters: [{ field: 'altitudeM', op: 'gt', value: '12192' }] });
assert.equal(text.ok, true);
assert.deepEqual(text.items.map((i) => i.id), ['RCH01']);
for (const value of ['', ' ', true, 'high', '12k'])
assert.equal((await engine.query({ layers: ['flights'], filters: [{ field: 'altitudeM', op: 'gt', value }] })).code, 'BAD_VALUE', JSON.stringify(value));
});
test('any supplied centre must be a real coordinate', async () => {
const engine = makeEngine();
for (const center of [{ lat: 0 }, { lat: 999, lon: 0 }, { lat: 10, lon: 200 }, { lat: '10', lon: 20 }])
assert.equal((await engine.query({ layers: ['flights'], scope: { kind: 'radius', km: 100, center } })).code, 'BAD_SCOPE', JSON.stringify(center));
assert.equal((await engine.query({ layers: ['flights'], scope: { kind: 'radius', km: 100, center: { lat: 30.2, lon: -97.7 } } })).ok, true);
});
test('mixed off/loading/unavailable layers refuse; a partly answerable query says so', async () => {
const snapshots = { earthquakes: { enabled: true, feedState: 'unavailable' }, 'local-firms': { enabled: true, feedState: 'loading' }, flights: { enabled: false, feedState: 'off' } };
const engine = createAnalystEngine({
getRecords: (key) => (key === 'ais-live-vessels' ? SHIPS : []),
getLayerSnapshot: (key) => snapshots[key] || { enabled: true, feedState: 'nominal' },
getViewContext: () => ({ lat: 0, lon: 0, viewRadiusKm: 25 }),
});
const none = await engine.query({ layers: ['earthquakes', 'local-firms', 'flights'] });
assert.equal(none.ok, false);
assert.equal(none.code, 'NOT_READY');
assert.deepEqual(none.layerStatus.map((l) => l.status), ['error', 'loading', 'off']);
assert.equal(engine.hasMemory(), false, 'a refusal never becomes follow-up memory');
const down = await engine.query({ layers: ['earthquakes', 'flights'] });
assert.equal(down.code, 'FEED_UNAVAILABLE');
const partial = await engine.query({ layers: ['ais-live-vessels', 'earthquakes'], scope: { kind: 'anywhere' } });
assert.equal(partial.ok, true);
assert.equal(partial.partial, true);
assert.deepEqual(partial.unanswered, ['earthquakes']);
assert.match(partial.display.caveat, /earthquakes error/);
});
test('returned items are copies: mutating them cannot change the remembered answer', async () => {
const engine = makeEngine();
const first = await engine.query({ layers: ['flights'], scope: { kind: 'anywhere' }, sortBy: 'altitudeM' });
for (const item of first.items) {
item.altitudeM = -1;
item.id = 'MUTATED';
}
const again = await engine.query({ followUp: true, sortBy: 'altitudeM' });
assert.deepEqual(again.items.map((i) => i.id), ['RCH01', 'SWA1', 'N123', 'GND1']);
assert.equal(FLIGHTS[1].altitudeM, 13500, 'the provider rows are untouched');
});
test('ranking 250k rows stays within a budget and yields to the page', async () => {
const rows = Array.from({ length: 250_000 }, (_, i) => ({ id: `F${i}`, lat: (i % 170) - 85, lon: (i % 350) - 175, frp: (i * 7919) % 100_003 }));
const engine = createAnalystEngine({ getRecords: () => rows, getViewContext: () => ({ lat: 0, lon: 0, viewRadiusKm: 25 }) });
let ticks = 0;
const timer = setInterval(() => ticks++, 0);
const started = performance.now();
const result = await engine.query({ layers: ['local-firms'], scope: { kind: 'anywhere' }, sortBy: 'frp', limit: 1 });
const elapsed = performance.now() - started;
clearInterval(timer);
assert.equal(result.count, 250_000);
assert.equal(result.items[0].frp, 100_002);
assert.ok(elapsed < 400, `took ${Math.round(elapsed)} ms`);
assert.ok(ticks > 0, 'the scan yielded between slices');
let current = true;
const pending = engine.query({ layers: ['local-firms'], scope: { kind: 'anywhere' }, sortBy: 'distance' }, { isCurrent: () => current });
current = false;
assert.equal((await pending).code, 'CANCELLED', 'a superseded scan stops at the next slice');
});
test('analyst: a follow-up naming other layers than the last answer is refused', async () => {
const engine = createAnalystEngine({
getRecords: (key) => ({ flights: FLIGHTS, 'local-firms': FIRES }[key] || []),
resolveRegionRing: async () => null,
getViewContext: () => ({ lat: 30.27, lon: -97.74, viewRadiusKm: 150 }),
});
await engine.query({ layers: ['flights'], scope: { kind: 'anywhere' } });
const r = await engine.query({ layers: ['local-firms'], followUp: true });
assert.equal(r.ok, false);
assert.equal(r.code, 'FOLLOW_UP_MISMATCH');
assert.equal(engine.hasMemory(), true, 'the earlier answer is still the last answer');
});
-162
View File
@@ -1,162 +0,0 @@
// Outline names are drawn by the DOM annotation overlay, never as Cesium
// text labels: the boundary data modules, the outline resolver and the
// annotation renderers must not import or construct Cesium label primitives,
// and drawing a bundled outline must create none at runtime.
import { test } from 'node:test';
import assert from 'node:assert/strict';
import { readFileSync } from 'node:fs';
import * as Cesium from 'cesium';
import { FEATURE_SOURCE_METHODS } from '../sources/featureSource.js';
import { createAnnotationResolver } from '../annotations/resolver.js';
import { createWorldAnnotationRenderer } from '../annotations/worldAnnotationRenderer.js';
const read = (path) => readFileSync(new URL(path, import.meta.url), 'utf8');
/** Pure data modules: no Cesium at all. */
const DATA_MODULES = [
'./adminBoundaries.js',
'./placeBoundaries.js',
'./neighborhoodPolygons.js',
'./local_data/us_census_places/files.js',
'./local_data/wof_neighborhoods/files.js',
];
/** Modules that use Cesium for outlines but must not draw text with it. */
const CESIUM_MODULES = [
'../annotations/resolver.js',
'../annotations/worldAnnotationRenderer.js',
'../annotations/hybridAnnotationRenderer.js',
];
/** The DOM overlay: callout elements, never Cesium label primitives. */
const OVERLAY_MODULES = ['../annotations/screenAnnotationRenderer.js'];
const LABEL_PRIMITIVES = [
/\bLabelCollection\b/,
/\bLabelGraphics\b/,
/\bLabelStyle\b/,
/\bLabelVisualizer\b/,
];
/** Entity label options, inline or assigned later. */
const ENTITY_LABELS = [
// `label: { text: … }` or `label: new …`
/\blabel\s*:\s*\{[^{}]*\btext\s*:/s,
/\blabel\s*:\s*new\b/,
// `label: labelOptions` (anything but `null`)
/\blabel\s*:\s*(?!null\b)[A-Za-z_$][\w$]*\s*[,}\n]/,
// `entity.label = …`
/\.label\s*=(?!=)/,
];
test('boundary data modules never import Cesium', () => {
for (const path of DATA_MODULES) {
const source = read(path);
assert.doesNotMatch(source, /from\s+['"]cesium['"]/, path);
assert.doesNotMatch(source, /import\(\s*['"]cesium['"]/, path);
}
});
test('outline modules and renderers construct no Cesium labels', () => {
for (const path of [...DATA_MODULES, ...CESIUM_MODULES, ...OVERLAY_MODULES])
for (const pattern of LABEL_PRIMITIVES)
assert.doesNotMatch(read(path), pattern, `${path}: ${pattern}`);
for (const path of [...DATA_MODULES, ...CESIUM_MODULES])
for (const pattern of ENTITY_LABELS)
assert.doesNotMatch(read(path), pattern, `${path}: ${pattern}`);
});
test('the static check catches a Cesium label however it is written', () => {
const offending = [
'viewer.scene.primitives.add(new Cesium.LabelCollection());',
"viewer.entities.add({ position, label: { text: 'Austin' } });",
'const graphics = new LabelGraphics({ text });',
'viewer.entities.add({ position, label: labelOptions });',
'entity.label = new Cesium.LabelGraphics(options);',
'entity.label = options;',
];
const all = [...LABEL_PRIMITIVES, ...ENTITY_LABELS];
for (const source of offending)
assert.ok(
all.some((pattern) => pattern.test(source)),
source,
);
// Data, not Cesium labels: a result's label point, a stripped proxy label.
for (const source of [
'label: geometry.label,',
'liveProxy(anno, { label: null });',
'if (entity.label === x) {}',
])
assert.ok(!ENTITY_LABELS.some((p) => p.test(source)), source);
});
test('drawing bundled place and neighborhood outlines creates no Cesium label', async (t) => {
// Cesium's material setup checks for these browser image types.
for (const name of [
'HTMLCanvasElement',
'HTMLImageElement',
'ImageBitmap',
'OffscreenCanvas',
]) {
if (name in globalThis) continue;
globalThis[name] = class {};
t.after(() => delete globalThis[name]);
}
const featureSource = {};
for (const method of FEATURE_SOURCE_METHODS)
featureSource[method] = async () => null;
const { resolveAnnotationTarget } = createAnnotationResolver({
featureSource,
});
const primitives = [];
const dataSources = [];
const viewer = {
camera: {
positionCartographic: Cesium.Cartographic.fromDegrees(
-97.7431,
30.2672,
3000,
),
},
scene: { primitives: { add: (p) => primitives.push(p) } },
dataSources: {
add: (ds) => dataSources.push(ds),
remove() {},
},
};
const renderer = createWorldAnnotationRenderer(viewer);
for (const [target, lon, lat] of [
['Austin', -97.7431, 30.2672],
['Notting Hill', -0.2, 51.511],
]) {
viewer.camera.positionCartographic = Cesium.Cartographic.fromDegrees(
lon,
lat,
3000,
);
const resolved = await resolveAnnotationTarget({
// The resolver reads only the camera (no scene to pick from).
viewer: { camera: viewer.camera },
target,
footprint: true,
deferFootprint: true,
});
assert.equal(resolved.source, 'bundled', target);
renderer.add({
id: target,
type: 'area',
label: resolved.label,
ring: resolved.ring,
polygons: resolved.polygons,
footprintKind: resolved.footprintKind,
synthesized: resolved.synthesized,
anchor: { lon: resolved.lon, lat: resolved.lat, height: 0 },
color: 'primary',
});
}
assert.equal(primitives.length, 0, 'no primitive collections');
const entities = dataSources.flatMap((ds) => ds.entities.values);
assert.ok(entities.length >= 2, 'the outlines were drawn');
for (const entity of entities) assert.equal(entity.label, undefined);
renderer.destroy();
});
+22 -17
View File
@@ -4,33 +4,38 @@
* The browser fetches the file from `url` — a `new URL('./….json',
* import.meta.url)` written in the calling module, which Vite serves as-is in
* dev and emits as an asset in the production build. Under Node (node:test)
* that URL is a `file:` URL, which `fetch` cannot read, so the caller's
* JSON-attributed import runs instead.
* that URL is a `file:` URL, which `fetch` cannot read, so the file is read
* from disk instead.
*
* The attributed import cannot serve the browser: the Vite dev server answers
* it with a JavaScript module, which the browser rejects for a JSON import.
*
* A pack split into many files passes no `importJson` (a literal import per
* file would also bundle every file as a script chunk); under Node its file
* is then read directly.
* Nothing here imports the JSON as a module. A JSON-attributed import cannot
* serve the browser (the Vite dev server answers it with a JavaScript module,
* which the browser rejects), and a literal `import('./….json')` makes the
* production build emit a JavaScript copy of every pack that the browser
* never loads.
*
* @module data/bundledJson
*/
/**
* @param {URL} url - Resolved against the calling module's `import.meta.url`.
* @param {(() => Promise<{default: any}>)|null} [importJson] - The caller's
* literal `import('./….json', { with: { type: 'json' } })`, used only under
* Node; without it Node reads the file.
* @param {{signal?: AbortSignal}} [options]
* @returns {Promise<any>} The parsed JSON.
*/
export async function loadBundledJson(url, importJson, { signal } = {}) {
if (url.protocol === 'file:') {
if (importJson) return (await importJson()).default;
const fs = globalThis.process.getBuiltinModule('node:fs');
return JSON.parse(fs.readFileSync(url, 'utf8'));
}
export async function loadBundledJson(url, { signal } = {}) {
signal?.throwIfAborted();
if (url.protocol === 'file:') {
// Node's built-in, reached without an import, so the browser build never
// bundles or rewrites it; only Node, reading `file:` URLs, gets here. The
// read is synchronous so the result settles as promptly as the cached
// module import it replaces; callers already memoize each pack.
const getBuiltinModule = globalThis.process?.getBuiltinModule;
if (typeof getBuiltinModule !== 'function')
throw new Error(
`God's Eye View needs Node 24.14 or newer to read ${url.pathname.split('/').pop()} (running ${globalThis.process?.version ?? 'an unknown runtime'})`,
);
const { readFileSync } = getBuiltinModule('node:fs');
return JSON.parse(readFileSync(url, 'utf8'));
}
const response = await fetch(url, { signal });
if (!response.ok) {
throw new Error(`HTTP ${response.status} for ${url.pathname}`);
+57 -10
View File
@@ -2,13 +2,14 @@ import { test } from 'node:test';
import assert from 'node:assert/strict';
import { loadBundledJson } from './bundledJson.js';
const MARINE = new URL('./local_data/natural_earth/marine.json', import.meta.url);
const importMarine = () =>
import('./local_data/natural_earth/marine.json', { with: { type: 'json' } });
const MARINE = new URL(
'./local_data/natural_earth/marine.json',
import.meta.url,
);
test('bundled JSON: a file URL uses the JSON import under Node', async (t) => {
test('bundled JSON: a file URL is read from disk under Node', async (t) => {
const fetchMock = t.mock.method(globalThis, 'fetch');
const pack = await loadBundledJson(MARINE, importMarine);
const pack = await loadBundledJson(MARINE);
assert.ok(pack.features.some((ft) => ft.name === 'Gulf of Mexico'));
assert.equal(fetchMock.mock.callCount(), 0);
});
@@ -21,16 +22,62 @@ test('bundled JSON: a served URL is fetched as plain JSON', async (t) => {
headers: { 'content-type': 'application/json' },
});
});
const url = new URL('http://localhost/src/data/local_data/natural_earth/marine.json');
const importJson = () => assert.fail('the browser path must not import JSON');
assert.deepEqual(await loadBundledJson(url, importJson), { features: [] });
const url = new URL(
'http://localhost/src/data/local_data/natural_earth/marine.json',
);
assert.deepEqual(await loadBundledJson(url), { features: [] });
assert.deepEqual(requests, [url.href]);
});
test('bundled JSON: an HTTP error rejects so the retryable loader can retry', async (t) => {
t.mock.method(globalThis, 'fetch', async () => new Response('', { status: 404 }));
t.mock.method(
globalThis,
'fetch',
async () => new Response('', { status: 404 }),
);
await assert.rejects(
loadBundledJson(new URL('http://localhost/missing.json'), importMarine),
loadBundledJson(new URL('http://localhost/missing.json')),
/HTTP 404/,
);
});
test('bundled JSON: an aborted load rejects before reading', async () => {
const controller = new AbortController();
controller.abort();
await assert.rejects(loadBundledJson(MARINE, { signal: controller.signal }), {
name: 'AbortError',
});
});
test('bundled JSON: no data pack is imported as a module', async () => {
// A literal JSON import in app code makes the production build emit a
// JavaScript copy of the pack beside the fetched JSON, which the browser
// never loads. Tests are not bundled, so they may still import JSON.
const { spawnSync } = await import('node:child_process');
const result = spawnSync(
'git',
[
'grep',
'-nE',
String.raw`import\(\s*['"][^'"]*local_data/[^'"]*\.json`,
'--',
'src',
':!*.test.mjs',
],
{ cwd: new URL('../../', import.meta.url), encoding: 'utf8' },
);
// git grep exits 1 when nothing matches.
assert.equal(result.status, 1, result.stdout);
});
test('bundled JSON: an older Node says which version is needed', async (t) => {
// Node before 20.16 / 22.3 has no process.getBuiltinModule.
const original = process.getBuiltinModule;
t.after(() => {
process.getBuiltinModule = original;
});
process.getBuiltinModule = undefined;
await assert.rejects(loadBundledJson(MARINE), {
message: /needs Node 24\.14 or newer to read marine\.json/,
});
});
+25
View File
@@ -128,6 +128,14 @@ test('the catalog applies the fair cap to configured packs end to end', async ()
sources.slice(0, 3).map((s) => s.id),
['env-0', 'env-1', 'env-2'],
);
assert.equal(sources[0].pack, 'env');
const [trimmed] = getSources.trimmedPacks();
assert.deepEqual(
{ ...trimmed, region: undefined },
{ pack: 'env', available: 12, served: 9, region: undefined },
);
assert.equal(trimmed.region.south, 30);
assert.ok(Math.abs(trimmed.region.north - 30.11) < 1e-9);
} finally {
for (const key of Object.keys(process.env)) {
if (!(key in saved)) delete process.env[key];
@@ -135,3 +143,20 @@ test('the catalog applies the fair cap to configured packs end to end', async ()
Object.assign(process.env, saved);
}
});
test('a pack trimmed to its nearest cameras reports how many it offered', async () => {
const { prioritizeSources } =
await import('../../server/providers/cctv/normalize.js');
const cameras = Array.from({ length: 5 }, (_, i) => ({
id: `c${i}`,
lat: i,
lon: 0,
}));
const kept = prioritizeSources(cameras, 2, [{ lat: 0, lon: 0 }]);
assert.deepEqual(
kept.map((camera) => camera.id),
['c0', 'c1'],
);
assert.equal(kept.available, 5);
assert.deepEqual(kept.region, { west: 0, south: 0, east: 0, north: 4 });
});
+3 -14
View File
@@ -377,26 +377,15 @@ export const NATURAL_EARTH_CREDIT = {
'<a href="https://www.naturalearthdata.com" target="_blank" rel="noopener">Natural Earth</a> (public domain)',
};
/** Registered when the first bundled US county or place outline resolves
* (public domain — credited as a courtesy). */
/** Registered when the first bundled US county outline resolves (public
* domain — credited as a courtesy). */
export const US_CENSUS_CREDIT = {
key: 'us-census-counties',
html:
'US county and place boundaries from the ' +
'US county boundaries from the ' +
'<a href="https://www.census.gov/geographies/mapping-files/time-series/geo/cartographic-boundary.html" target="_blank" rel="noopener">U.S. Census Bureau</a> (public domain)',
};
/** Registered when the first bundled Who's On First neighborhood outline
* resolves. The link leads to the pack's per-source credits and licences and
* says how the boundaries were modified. */
export const WOF_CREDIT = {
key: 'wof-neighborhoods',
html:
'Neighborhood boundaries: ' +
'<a href="https://whosonfirst.org/docs/licenses/" target="_blank" rel="noopener">Who\'s On First</a> and contributors, simplified (' +
'<a href="https://github.com/bilawalsidhu/gods-eye-view/blob/main/src/data/local_data/wof_neighborhoods/ATTRIBUTION.md" target="_blank" rel="noopener">sources and licences</a>)',
};
/**
* Per-feed transit credit, registered the first time that feed's vehicles
* render (see `src/data/transitFeeds.js` for the license of each).
-1
View File
@@ -25,7 +25,6 @@ test('firms analyst record: full record maps every contract field', () => {
frp: 1520.4,
confidence: 0.9,
satellite: 'N21',
sensor: 'VIIRS',
acqTime: 1_753_600_000_000,
});
});
-11
View File
@@ -41,14 +41,3 @@
2026-09-25. Each record names its XYZ tile and expected traffic eligibility.
Modified, test-only OpenStreetMap data; same ODbL attribution and database
share-alike terms as the OpenFreeMap fixtures above.
- `ofm-outlines-austin-14-3743-{6745,6746}.pbf` — OpenFreeMap / OpenMapTiles
z14 tiles from
`https://tiles.openfreemap.org/planet/20260913_164504_pt/{z}/{x}/{y}.pbf`,
retrieved 2026-09-28. Trimmed to the `transportation_name` features for
Congress Avenue, South Congress Avenue and East 11th Street, and the
`building` features within about 150 m of the Texas Capitol and the
`landcover` park polygon around it, preserving
original geometry and dictionaries. Used by the street and building outline
tests. Same OpenFreeMap / OpenMapTiles / OpenStreetMap attribution and ODbL
1.0 share-alike terms as the fixtures above. Test-only.
Loaded 100 of 602 files, more files were not shown because too many files have changed in this diff. Show more