mirror of
https://github.com/bilawalsidhu/gods-eye-view.git
synced 2026-11-06 19:34:01 +00:00
Compare commits
6
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
 manjunath22466
|
01979bba1e | ||
|
manjunath22466
|
e57b60e511 | ||
|
manjunath22466
|
177b5951a0 | ||
|
manjunath22466
|
095981eee7 | ||
|
Bilawal Sidhu
|
9559e7a282 | ||
|
Bilawal Sidhu
|
a19897ad15 |
No files matched your search
+27
-4
@@ -64,6 +64,13 @@ OPENAI_REALTIME_REASONING_EFFORT=low
|
||||
# Keep only a short conversational window. Current map state is fetched live.
|
||||
OPENAI_REALTIME_CONTEXT_TOKENS=3000
|
||||
OPENAI_REALTIME_CONTEXT_RETENTION=0.5
|
||||
# Voice-card captions of what you said. Billed separately by OpenAI and
|
||||
# counted in the session cost meter and cap; off disables them.
|
||||
OPENAI_REALTIME_TRANSCRIBE_MODEL=gpt-4o-mini-transcribe
|
||||
# The voice debug log (.gev-logs/realtime-conversations.jsonl, rotated at
|
||||
# 32 MB, never expired) omits transcripts and typed or spoken text by default.
|
||||
# Set to 1 to keep them while debugging; delete the log when you are done.
|
||||
GEV_VOICE_LOG_CONTENT=
|
||||
OPENAI_HUD_SUMMARY_MODEL=gpt-5-nano
|
||||
# Optional: opt-in per-IP rate limit for the OpenAI cost endpoints
|
||||
# (/api/realtime/token + /api/openai/hud-summary), in requests per minute.
|
||||
@@ -187,7 +194,23 @@ AISSTREAM_API_KEY=
|
||||
# Optional: comma-separated Overpass URLs; use only instances you run or pay for.
|
||||
OVERPASS_UPSTREAMS=
|
||||
|
||||
# Optional: who may frame the app in embed mode (?embed=1), as CSP
|
||||
# frame-ancestors sources such as "https://example.com". Unset allows any
|
||||
# page; every other page of the app can never be framed.
|
||||
# GEV_EMBED_FRAME_ANCESTORS=
|
||||
# Optional: place-lookup endpoints. Lookups try bundled data first, then the
|
||||
# endpoint set here, or the public service under its usage limits when unset.
|
||||
# NOMINATIM_URL: search endpoint for last-resort place search and voice
|
||||
# outlines (e.g. https://nominatim.example.org/search). A configured URL
|
||||
# replaces the public service: nothing falls back to nominatim.openstreetmap.org,
|
||||
# and an invalid value disables Nominatim. Unset uses the public
|
||||
# nominatim.openstreetmap.org: one request at a time, 1.1 s apart, a short
|
||||
# queue, cached answers (outlines 90 days, places 30 days, no-results 24 h in
|
||||
# .gev-cache/nominatim) and a daily cap per install. Set it empty to disable.
|
||||
# The public service's limit applies to all installs of this app together; the
|
||||
# per-install cap bounds each install, not the total, so heavy or shared
|
||||
# deployments should point this at their own server.
|
||||
# NOMINATIM_URL=
|
||||
# Public-instance requests per UTC day, outlines and search together. Not
|
||||
# applied to your own NOMINATIM_URL.
|
||||
# NOMINATIM_DAILY_CAP=50
|
||||
# PHOTON_URL: keyless place search endpoint used by the browser
|
||||
# (e.g. https://photon.example.org/api/). Unset uses photon.komoot.io; set it
|
||||
# empty to disable.
|
||||
# PHOTON_URL=
|
||||
+111
-55
@@ -1,32 +1,37 @@
|
||||
# Changelog
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
## [0.2.0] — 2026-10-02 — God's Eye View through Your Agent
|
||||
|
||||
- Show God's Eye View inside AI conversations. Answers that can be shown
|
||||
include a view (camera, layers, style, map, marks, something to follow),
|
||||
and `show_in_gods_eye_view` displays it as the live globe in clients that
|
||||
support MCP Apps, such as Claude Desktop and the Codex and ChatGPT desktop
|
||||
apps, and as a link everywhere else. The panel loads the app's panel build
|
||||
(`npm run build:panel`) and its data through the MCP server, so a local
|
||||
server needs no HTTPS or tunnel. `?embed=1` shows the globe alone for pages
|
||||
that frame the app.
|
||||
|
||||
- Add tools for language-model clients and a local MCP server, served over
|
||||
stdio by `npm run mcp` and over HTTP at `/mcp` on the development and preview
|
||||
servers for local requests only. `npm run mcp`
|
||||
serves earthquake, active-fire, launch, aircraft, ship, satellite, camera, radio,
|
||||
place, routing, bike-share, transit, traffic, weather, cyclone, fire-perimeter,
|
||||
terrain, installation and map-feature queries, plus a situation brief, over
|
||||
stdio to clients such as Claude Code, reading
|
||||
from a running app. Tools are defined
|
||||
once in `gods-eye-view/tools`, reuse the layers' source factories, and are
|
||||
exposed through the protocol adapter in `gods-eye-view/tools/mcp`. The
|
||||
catalog also covers weather maps, wind, recent imagery, submarine cables,
|
||||
license plate reader cameras, datacenters and dams, the Bhote Koshi flood,
|
||||
military awareness and links that open the app over an area with layers on.
|
||||
Voice offers the same queries next to its app actions.
|
||||
- The voice card shows analyst counts ("At least 250,000 aircraft …") with
|
||||
their scope, feed window and caveats, and marks partial answers. Analyst
|
||||
speech uses the same deterministic headline, preserves lower bounds and says
|
||||
when an answer is partial. Follow-ups retain unanswered-layer coverage, and
|
||||
keep the original scope label when they narrow a remembered result.
|
||||
Contacts-window answers read one immutable panel snapshot, mark a 20,000-row
|
||||
layer cap as a lower bound, and commit that displayed cohort only after
|
||||
cancellation checks pass. Concurrent analyst calls from one model response
|
||||
remain independent; a stopped or replaced user turn still cancels them.
|
||||
- Voice answers are shorter and fill slow work instead of going silent. Slow
|
||||
or multi-step requests may open with one short spoken plan; instant ones
|
||||
just happen. Tool results carry a code-built `say` line and a `display`
|
||||
card: sources and notes stay on screen, a caveat is spoken only when it
|
||||
changes the answer, and "at least" and partial answers are always said. A card above the mic shows captions, the plan's steps and the
|
||||
result. When a tool that reports steps (layers, place lookups, outlines,
|
||||
nearest-aircraft) is still running after 1.5 s its current step is spoken
|
||||
once; others say "still working" once after 8 s. A soft
|
||||
tick plays in push-to-talk. Holding Space while the assistant talks cuts it
|
||||
off. Each turn writes latency and speech figures to the debug log
|
||||
(`node scripts/voice-turn-spans.mjs`). Captions of what you say are
|
||||
counted in the voice cost meter; the debug log leaves out what was said
|
||||
unless `GEV_VOICE_LOG_CONTENT=1`.
|
||||
- Point and ask: hold Space with the cursor on an aircraft, ship or spot on
|
||||
the map and say "what's that", "track that one" or "how many flights
|
||||
around here". A reticle marks the point and the voice card shows what
|
||||
"this" meant. In open mic it works when you have just moved the cursor
|
||||
there. "The second one" picks from the last numbered list. Asking about
|
||||
"the last one" resolves to the last of the five rows actually shown, and
|
||||
referent-only tracking calls are accepted while empty targets are rejected.
|
||||
Asking about
|
||||
bare ground up close gives the assistant a small crop of the map around the
|
||||
point.
|
||||
|
||||
- Public Overpass instances are no longer used by default. Street Traffic
|
||||
roads come from TomTom flow tiles, OpenFreeMap vector tiles, or both, chosen
|
||||
@@ -177,6 +182,7 @@
|
||||
geocode and admin-boundary fallback answers `region-timeout` after 3 s
|
||||
instead of holding the reply.
|
||||
|
||||
|
||||
- Transit and Directions rows repaint as soon as their data lands again:
|
||||
`refreshLayerStats()` now lives on the layer lifecycle, not only on the
|
||||
compatibility facade. `scripts/qa-radio.mjs` uses it instead of a private
|
||||
@@ -210,7 +216,57 @@
|
||||
(`src/ui/imagerySplit.js`, `src/maps/imageryComparison.js`), and
|
||||
`MapSourceController.subscribe()` reports every settled map switch.
|
||||
|
||||
### Local receiver feeds
|
||||
## Unreleased — voice reaches every layer; honest analyst answers
|
||||
|
||||
- Voice can switch on every shipped data layer (the two scene-driven
|
||||
Bhote Koshi layers stay scene-only), including transit, Recent
|
||||
Imagery, wind, the weather layers, cyclones and mapped installations, and
|
||||
understands their spoken names ("buses", "hurricanes", "military bases").
|
||||
One voice manifest (`src/voice/layerManifest.js`) generates the tool enums
|
||||
and aliases, and a unit test fails when a new layer has no entry.
|
||||
- `analyst_query` counts up to 250,000 loaded records per layer instead of the
|
||||
first 2,000, and says when a layer went past that (`complete: false`). It
|
||||
also answers over bikeshare stations, transit vehicles, rocket launches,
|
||||
cyclones, ALPR cameras, mapped installations, local ADS-B aircraft and fire perimeters; fire
|
||||
records carry their sensor (VIIRS or MODIS).
|
||||
- Unknown fields, operators, units, value types, scope kinds and invalid
|
||||
centres are refused with the allowed values instead of answering zero or widening to anywhere. A
|
||||
layer that is off, still loading or unavailable is reported as such, not
|
||||
as a count of zero; a partly answerable query is marked `partial`.
|
||||
Distance sorts honour `sortDir`, a written unit such as `altitudeM(ft)`
|
||||
converts, and large rankings no longer sort the whole set.
|
||||
- Results carry `lat`/`lon` for each item and a `display` object with the
|
||||
scope, the feed window (earthquakes and fires cover the last 24 h) and any
|
||||
caveat for the screen. The model is asked to answer in one short phrase.
|
||||
- The weather and Recent Imagery panels can be opened by voice.
|
||||
|
||||
## Unreleased — voice outlines without Overpass
|
||||
|
||||
- Voice outlines work again without Overpass. Bundled US Census places and
|
||||
Who's On First neighborhoods (lazy-loaded; `src/data/local_data/`) answer
|
||||
first, then streets, grounds and pointed-at buildings from OpenFreeMap
|
||||
tiles, then Nominatim for one explicit ask at a time. Traffic and outlines
|
||||
share pending fetch/decode ownership and a projection-aware retained-byte
|
||||
ceiling for the same immutable tile; consumer caches and request lifetimes
|
||||
remain independent, and eviction or teardown retires queued idle projections.
|
||||
Public-provider answers are cached and capped per
|
||||
install. Public Nominatim requests now reserve their daily slot and paced
|
||||
start atomically across processes sharing that installation's state file,
|
||||
and fail closed when durable state is unavailable or invalid. Operator-selected
|
||||
endpoints remain outside the public cap, spacing and state policy, and an
|
||||
upstream cooldown is isolated to the endpoint that returned it. Anything else
|
||||
stays a point marked "outline unavailable". Administrative Nominatim
|
||||
outlines carry the selected record's verified country/admin-1/admin-2
|
||||
identity; ambiguous, missing, contradictory or retired evidence is not
|
||||
promoted into a verified level.
|
||||
- Who's On First rebuilds accept only exact raw registry licence values in the
|
||||
reviewed open-licence allowlist. New, unknown, restricted or noncommercial
|
||||
source licences stay excluded until they are reviewed explicitly.
|
||||
- New settings: `NOMINATIM_URL` (your own Nominatim instead of the public
|
||||
one; empty disables it), `NOMINATIM_DAILY_CAP` (public requests per day,
|
||||
default 50) and `PHOTON_URL` (browser place search; empty disables it).
|
||||
|
||||
## Unreleased — local receiver feeds
|
||||
|
||||
- The Local ADS-B layer also reads local 1090 MHz and 978 MHz UAT decoder
|
||||
feeds: the `aircraft.json` that dump1090-fa, readsb, tar1090 or skyaware978
|
||||
@@ -239,7 +295,7 @@
|
||||
- Records carry `band` (`1090`/`978`) and `source` (`webusb`/`feed`).
|
||||
- See `docs/LOCAL-RECEIVERS.md`.
|
||||
|
||||
### Local RTL-SDR and Local ADS-B
|
||||
## Unreleased — local RTL-SDR and Local ADS-B
|
||||
|
||||
- Add a Local RTL-SDR card to the Radio panel. It connects a USB RTL-SDR in
|
||||
desktop Chrome or Edge through WebUSB and receives broadcast FM (tune, seek,
|
||||
@@ -264,7 +320,7 @@
|
||||
- Add `@jtarrio/webrtlsdr` and `@jtarrio/signals` (Apache-2.0); see
|
||||
`THIRD_PARTY_NOTICES.md`.
|
||||
|
||||
### Weather review
|
||||
## Unreleased — weather review
|
||||
|
||||
- On 3D Tiles, draw a 4096×2048 detail window around the view on each
|
||||
observed-weather shell except global infrared, sampled by the shell's own
|
||||
@@ -530,26 +586,26 @@ Analyst records for loaded satellites, datacenters and dams, with explicit bound
|
||||
|
||||
- Split application scene, controls, catalog, tools and HTML into reusable components; configure application request services and sources without changing global fetch. Preserve standalone markup and voice behavior. Explicit annotation navigation may resolve a distant named target.
|
||||
|
||||
### Satellite pass prediction
|
||||
## Satellite pass prediction
|
||||
|
||||
- Bisect pass rise/set to ~0.2 s and fit peak elevation with a parabola.
|
||||
- Mark passes visible from Earth-shadow and civil-twilight checks.
|
||||
- Add `getNextSatellitePass(noradId, options)` for any loaded catalog satellite.
|
||||
- `next_iss_pass` retains the next geometric pass and adds visibility metadata. `next_satellite_pass` adds bounded loaded-catalog name/NORAD lookup and optional visible-only filtering (Rehaan Delmotra, #451; maintainer adaptation).
|
||||
|
||||
### Voice component boundaries
|
||||
## Voice component boundaries
|
||||
|
||||
- Separate voice controls, Realtime connection requests and the action runner.
|
||||
- Allow compatible endpoints and server-selected models through construction options.
|
||||
- Cancel pending token/SDP requests on Stop or teardown and reject expired secrets.
|
||||
|
||||
### Configurable geospatial services
|
||||
## Configurable geospatial services
|
||||
|
||||
- Compose geocoding, place context and routes through independent providers.
|
||||
- Allow compatible endpoint configuration without changing voice tools or annotation behavior.
|
||||
- Isolate configured source caches and reject results after cancellation.
|
||||
|
||||
### ALPR camera locations
|
||||
## ALPR camera locations
|
||||
|
||||
- Port Manjunath's (@manjunath22466) cyan camera badges, coral selection brackets,
|
||||
gradient direction wedges and animated tactical labels into the reusable ALPR
|
||||
@@ -569,32 +625,32 @@ Analyst records for loaded satellites, datacenters and dams, with explicit bound
|
||||
- Separate the request adapter, camera model, presentation, and instance lifecycle.
|
||||
Source cancellation also guards late response bodies and rejects invalid query bounds.
|
||||
|
||||
### Release disabled infrastructure rendering
|
||||
## Release disabled infrastructure rendering
|
||||
|
||||
- Remove built Data Center, Dam and Submarine Cable entities when their layers
|
||||
are disabled, avoiding retained visualizer work and entity memory.
|
||||
- Keep parsed datasets cached for re-enable; rebuild entities without refetching.
|
||||
|
||||
### Camera layer components
|
||||
## Camera layer components
|
||||
|
||||
- Separate camera source requests, placement, frames, projection, cards and calibration.
|
||||
- Own visibility listeners and pending initialization within each layer lifetime.
|
||||
- Preserve existing camera catalogs, URL families, geometry and playback behavior.
|
||||
|
||||
### Traffic and bikeshare components
|
||||
## Traffic and bikeshare components
|
||||
|
||||
- Separate traffic loading, animation, styling and lifecycle into factory-owned components.
|
||||
- Give each flow source its own bounded decode cache and cancellation checks.
|
||||
- Separate bikeshare registry, station requests, rendering, selection and proximity handling.
|
||||
|
||||
### Installation and context components
|
||||
## Installation and context components
|
||||
|
||||
- Separate mapped-site requests, records, placement, selection and viewport lifecycle.
|
||||
- Separate proximity queries, subject tracking, navigation/history, panel and direction rendering.
|
||||
- Retain source and ground-floor ownership in standalone composition; reject malformed
|
||||
installation snapshots and ignore failures from cancelled requests.
|
||||
|
||||
### Satellite and mission layer components
|
||||
## Satellite and mission layer components
|
||||
|
||||
- Separate catalog loading, orbit calculations, display, tracking and interaction
|
||||
into instance-owned satellite components.
|
||||
@@ -602,20 +658,20 @@ Analyst records for loaded satellites, datacenters and dams, with explicit bound
|
||||
operations, retaining existing layer controls and satellite coordination.
|
||||
- Cancel late mission source work and reject malformed launch snapshots.
|
||||
|
||||
### Fire layer components
|
||||
## Fire layer components
|
||||
|
||||
- Split fire source loading, state, rendering, cards, selection and viewport work
|
||||
into reusable components with application-owned scene services.
|
||||
- Cancel late refreshes, retain good data after malformed responses, and preserve
|
||||
selection identity without repeating a user-selection notification on refresh.
|
||||
|
||||
### Earthquake components
|
||||
## Earthquake components
|
||||
|
||||
- Separate earthquake snapshot loading, record validation, and display ownership.
|
||||
- Cancel pending earthquake refreshes on disable or destruction, retaining the
|
||||
last good snapshot after malformed or failed refreshes.
|
||||
|
||||
### September 8, 2026
|
||||
## September 8, 2026
|
||||
|
||||
Earthquake refreshes validate the complete feed and construct replacement entities before clearing the previous snapshot. Malformed rows and duplicate rendered IDs retain the last good entities, overlays, count and timestamp and report a malformed response; unknown magnitude is excluded from M2.5+ rendering.
|
||||
|
||||
@@ -626,7 +682,7 @@ Launch payloads with missing records now say PAYLOAD DATA UNAVAILABLE. Missing n
|
||||
This changelog records public product changes. For the authoritative description
|
||||
of current runtime behavior, see [`docs/CURRENT-STATE.md`](docs/CURRENT-STATE.md).
|
||||
|
||||
### Other changes
|
||||
## [Unreleased]
|
||||
|
||||
|
||||
- Add ECMWF IFS model selection to Wind (#464, thanks @beneduzi), with model-scoped forecast-step caches, cancellation of replaced requests, and separate issue/valid timestamps.
|
||||
@@ -647,7 +703,7 @@ of current runtime behavior, see [`docs/CURRENT-STATE.md`](docs/CURRENT-STATE.md
|
||||
|
||||
- Separate submarine cable sources and rendering components, and export bundled geography lookup modules.
|
||||
|
||||
#### Added
|
||||
### Added
|
||||
|
||||
- DISPLAY ▸ Draw: draw on the world by hand. Pick Area, Line or Pin, click the
|
||||
vertices, double-click or press Enter to finish, label and colour it; Backspace
|
||||
@@ -657,7 +713,7 @@ of current runtime behavior, see [`docs/CURRENT-STATE.md`](docs/CURRENT-STATE.md
|
||||
together. While you are drawing, the draw tool owns the pointer and no layer
|
||||
selects what you click through (#235 — thanks @cora-fresh-labs).
|
||||
|
||||
#### Fixed
|
||||
### Fixed
|
||||
|
||||
- Keep traffic-road bounds crossing the antimeridian monotonic and inside the
|
||||
longitude range accepted by the Overpass request path, preserving the small
|
||||
@@ -724,7 +780,7 @@ of current runtime behavior, see [`docs/CURRENT-STATE.md`](docs/CURRENT-STATE.md
|
||||
toggle. Camera departure cancels pending work, arrival checks the final view,
|
||||
and superseded requests cannot keep a newer view loading.
|
||||
|
||||
#### Added
|
||||
### Added
|
||||
|
||||
- Add MODIS NRT (Terra+Aqua, ~1 km) active fires to the FIRMS layer, sharing the
|
||||
existing `FIRMS_MAP_KEY` and 30-minute cache.
|
||||
@@ -852,9 +908,9 @@ of current runtime behavior, see [`docs/CURRENT-STATE.md`](docs/CURRENT-STATE.md
|
||||
- Extract shared surface keyboard handling for the welcome launcher and Provider
|
||||
Settings, preserving Tab/Escape behavior and releasing the listener on teardown.
|
||||
|
||||
#### Added
|
||||
### Added
|
||||
|
||||
#### Security
|
||||
### Security
|
||||
|
||||
- The CCTV media route no longer forwards a client `Range` header to the upstream
|
||||
camera host as it arrived. A single `bytes=` range is canonicalized and
|
||||
@@ -885,7 +941,7 @@ of current runtime behavior, see [`docs/CURRENT-STATE.md`](docs/CURRENT-STATE.md
|
||||
|
||||
- Cancel the active location lookup when its controls are disposed.
|
||||
|
||||
#### Fixed
|
||||
### Fixed
|
||||
|
||||
- `DATA_SOURCES.md` states what the project does with camera frame content: a
|
||||
successful upstream response is relayed as the provider served it, nothing in
|
||||
@@ -964,7 +1020,7 @@ of current runtime behavior, see [`docs/CURRENT-STATE.md`](docs/CURRENT-STATE.md
|
||||
fetching, AIS records/tracks and shared request helpers; preserve existing
|
||||
routes, local setup, fallback behavior and rendering.
|
||||
|
||||
#### Added
|
||||
### Added
|
||||
|
||||
- **Directions layer** — keyless A→B directions without a geocoder or a
|
||||
microphone (thanks @spcpza). The row's chips arm a globe click for A and B
|
||||
@@ -992,7 +1048,7 @@ of current runtime behavior, see [`docs/CURRENT-STATE.md`](docs/CURRENT-STATE.md
|
||||
voice routes since launch, previously uncredited), with the OpenStreetMap
|
||||
credit and the "fix the map" link the service's usage policy asks for.
|
||||
|
||||
#### Changed
|
||||
### Changed
|
||||
|
||||
- The interface asks Google Fonts for only the icon glyphs it draws, instead of
|
||||
the whole variable icon font, and no longer requests a second icon family that
|
||||
@@ -1007,7 +1063,7 @@ of current runtime behavior, see [`docs/CURRENT-STATE.md`](docs/CURRENT-STATE.md
|
||||
- Rename standalone browser startup to `src/standalone/` and add a Node-only
|
||||
`gods-eye-view/build/vite` export with checked package ownership.
|
||||
|
||||
#### Development
|
||||
### Development
|
||||
|
||||
- The CCTV launcher and preview-server tests resolve their temporary fixture
|
||||
root through `fs.realpath`, so they pass on macOS, where the system temp
|
||||
@@ -1041,7 +1097,7 @@ of current runtime behavior, see [`docs/CURRENT-STATE.md`](docs/CURRENT-STATE.md
|
||||
infrastructure modules and their consumer tests. Package boundary checks keep
|
||||
those exports separate from app startup and local Node services.
|
||||
|
||||
#### Fixed
|
||||
### Fixed
|
||||
|
||||
- Reduce terrain-height timeouts when Re:Earth slows down. Batches are
|
||||
sized against measured response latency on both browser and server to reduce
|
||||
@@ -1132,7 +1188,7 @@ of current runtime behavior, see [`docs/CURRENT-STATE.md`](docs/CURRENT-STATE.md
|
||||
and upstream error bodies; response statuses and cache fallback remain intact.
|
||||
Includes the security fixes contributed by Tom-Neverwinter in PR #171.
|
||||
|
||||
#### Fixed
|
||||
### Fixed
|
||||
|
||||
- Map Source keyboard opening retries focus until the selected tile is visible.
|
||||
Leaving the disclosure, pointer interaction, or closing the tray cancels the
|
||||
@@ -1174,7 +1230,7 @@ of current runtime behavior, see [`docs/CURRENT-STATE.md`](docs/CURRENT-STATE.md
|
||||
that had since recovered. A miss is now cached only when every source
|
||||
consulted actually returned a verdict.
|
||||
|
||||
#### Added
|
||||
### Added
|
||||
|
||||
- Keyless place search. The LOCATION search box and the `fly_to_location` voice
|
||||
tool now resolve place names through Photon (komoot, over OpenStreetMap) when
|
||||
|
||||
+4
-1
@@ -96,7 +96,8 @@ The optional Local ADS-B layer and the broadcast-FM receiver use radio you recei
|
||||
- **Re:Earth Terrain.** Keyless (no API key). Used two ways: (1) `src/mapStackController.js` swaps in a `Cesium.CesiumTerrainProvider` pointed at Re:Earth's `cesium-mesh/ellipsoid` quantized-mesh endpoint for globe stacks without a Cesium ion token (e.g. OSM), replacing a flat `EllipsoidTerrainProvider`; falls back to the flat provider if the endpoint can't be reached. (2) The server-side `/api/terrain/heights` proxy (disk-cached, serve-stale) resolves per-point ellipsoidal ground height for entity placement. Both are best-effort with a keyless-safe fallback (bundled EGM96 geoid math) if Re:Earth is unreachable.
|
||||
- **OSRM on the FOSSGIS routing servers.** Keyless (no API key). Every route is requested by a person: clicking A and B in the Directions layer, or a voice "route from A to B". The server-side `/api/route` proxy sends an identifying `User-Agent`, pins the upstream host, accepts 2-12 coordinates, rejects any leg over 600 km or a total over 2,500 km, caps the buffered response at 8 MB, times the upstream out at 12 s, rate-limits per client, caches each route for 10 minutes, and coalesces identical requests that are in flight at the same time. Turn-by-turn maneuvers are asked for only when a caller wants them (`steps=1`), so the callers that do not read them send the smaller request. The "fix the map" link the policy asks for is in the Data attribution popover beside the OpenStreetMap credit.
|
||||
- **Mapped installations.** OpenFreeMap military polygons use a bundled Overture/OSM name index; wide views show named points. Optional operator-configured Overpass and explicit Google Places searches keep their own source paths. Mapped data does not establish current activity or complete coverage.
|
||||
- **Place search.** Named-place fly-to uses Google Geocoding when a Maps key is configured, then Photon's public instance, then `/api/geocode` (Nominatim search) when neither answers. `/api/geocode` uses the shared Nominatim forward-search queue at no more than one request per second, sends an identifying `User-Agent` and `Referer`, caches answers for five minutes, shares one upstream call between identical searches already in flight, bounds the queue so a burst is refused rather than held, and drops a queued search whose caller has given up. It asks for one result per search and is not used for systematic or bulk queries.
|
||||
- **Place search.** Named-place fly-to uses Google Geocoding when a Maps key is configured, then Photon (`PHOTON_URL`; unset uses the public instance, empty disables it), then `/api/geocode` (Nominatim search) when neither answers. `/api/geocode` and the voice outline route `/api/geocode/outline` share one Nominatim queue (`NOMINATIM_URL`; unset uses the public instance, empty disables it): one request at a time at least 1.1 s apart, an identifying `User-Agent` and `Referer`, a short queue that refuses a burst, a pause after 429/403/5xx answers (honouring `Retry-After` in full), at most one retry after a network failure, and a per-install daily cap on public-instance requests (`NOMINATIM_DAILY_CAP`, default 50). Processes using the same installation state file atomically reserve the next public start and daily slot; restarts share that count, spacing and any pause. Public access fails closed if the state file cannot be read, locked, written or semantically validated. This coordination does not span independent installations or state directories, and the public service's total limit still applies across all installs, so heavy or shared deployments should set their own `NOMINATIM_URL`. An operator-selected endpoint is explicitly outside the public cap, spacing and durable-state requirement; an upstream cooldown is kept with the endpoint that returned it rather than leaking across operator URLs. Answers are cached on disk (outlines 90 days, places 30 days, no-results 24 hours); failures are not cached. Outlines are looked up only for an explicit ask, after bundled data and any configured Overpass, and only results whose type suits the ask are used. Administrative outline responses expose country/admin-1/admin-2 identity only when the selected Nominatim row has a recognized, unambiguous address type consistent with its own name; numeric rank, ambiguous types, missing fields and contradictory names do not manufacture identity. It is not used for systematic or bulk queries.
|
||||
- **Street and building outlines.** Voice street highlights, pointed-at buildings and grounds use the OpenFreeMap vector tiles the traffic layer already loads. Traffic and outlines share one bounded pending fetch/decode owner per immutable XYZ tile; materialized outline geometry is charged to the retained-byte ceiling and capped by coordinate count. Consumer result caches and request lifetimes remain independent, while eviction or explicit teardown cancels queued idle projections and releases their decoded owner. No Overpass request is made unless `OVERPASS_UPSTREAMS` is configured.
|
||||
- **Cockpit regional briefing.** `/api/regional-brief` rounds aircraft coordinates into 0.1° cache cells, caches results for five minutes, and resolves physical regions from bundled public-domain Natural Earth polygons without reverse-geocoding requests. Google News RSS is queried with the resolved locality/region first; GDELT is used only when that RSS query fails or is empty. Google's published Google News terms restrict that source to personal, noncommercial use, so commercial deployments must disable/replace it or obtain separate permission; GDELT permits commercial dataset use with citation. The Data attribution popover identifies the active headline sources; article links retain publisher attribution. Headlines are location-query matches, not verified incidents, risk rankings, or evidence that a location is safe. Empty, partial, stale, and unavailable source states remain distinct. Open-Meteo supplies current conditions independently of the news source. `WX OFF` disables cockpit weather rendering only; the Local Info briefing still fetches its source-backed weather values and displays the required linked Open-Meteo credit.
|
||||
- **Dynamic weather presentation.** While cockpit mode is active, `/api/weather-effects` requests current Open-Meteo observations for the aircraft/camera location, rounds coordinates into 0.1° cache cells, caches results for five minutes, and may retain a stale observation for up to 30 minutes during a transient outage. WMO condition code selects the visual family; observed cloud cover, precipitation, visibility, wind speed, and wind direction bound its strength and motion. Missing or expired weather renders no synthetic atmospheric effect, and normal globe view never renders the weather overlay.
|
||||
|
||||
@@ -118,6 +119,8 @@ The [Bhote Koshi event pack](public/events/bhote-koshi-2026/README.md), under `p
|
||||
| **DataSF Analysis Neighborhoods** (41 SF neighborhood polygons) | `neighborhoods/` | **PDDL 1.0** (public domain) | ✅ (no restrictions) | "City & County of San Francisco — DataSF" (courtesy — not legally required) |
|
||||
| **Natural Earth countries, states and provinces** (260 admin-0/map units + 4,587 admin-1 polygons) | `natural_earth/` | **Public domain** | ✅ (no restrictions) | "Made with Natural Earth" (courtesy credit — not legally required) |
|
||||
| **US Census Bureau counties** (3,235 county polygons) | `us_census_counties/` | **Public domain** | ✅ (no restrictions) | "U.S. Census Bureau" (courtesy — not legally required) |
|
||||
| **US Census Bureau places** (32,629 city, town and CDP polygons) | `us_census_places/` | **Public domain** | ✅ (no restrictions) | "U.S. Census Bureau" (courtesy — not legally required) |
|
||||
| **Who's On First neighborhoods** (61,755 polygons) | `wof_neighborhoods/` | Per source: CC BY, ODC-By, Open Government licences, CC0, PDDL, public domain (folder `ATTRIBUTION.md`) | ✅ (attribution) | "Neighborhood boundaries: Who's On First and contributors"; per-source credits in the folder |
|
||||
| **CCTV ground heights** (3,445 cameras) | `cctv_ground_heights/` | Precomputed camera placement heights, aligned to work with Google Photorealistic 3D Tiles (folder README) | — | — |
|
||||
|
||||
### ⚠️ TeleGeography is bundled but NonCommercial
|
||||
|
||||
@@ -279,8 +279,6 @@ _Ask for radio near anywhere and the globe starts broadcasting — every station
|
||||
|
||||
---
|
||||
|
||||
**💬 Or ask from Claude and Codex.** God's Eye View is also an MCP server: ask Claude Desktop, Codex or ChatGPT desktop to show a place, and the live globe opens right in the conversation. Setup takes a minute — see [docs/MCP_SETUP.md](docs/MCP_SETUP.md).
|
||||
|
||||
## 🛰️ What's on the Globe
|
||||
|
||||
Nineteen layers and map sources. **Seventeen have a keyless path.** Some offer additional capabilities with a provider key. (🟢 no key · 🟡 free key · 🔴 metered.)
|
||||
|
||||
@@ -1,55 +0,0 @@
|
||||
/**
|
||||
* Lets other pages frame the app in embed mode (`?embed=1`). Every other
|
||||
* document keeps the server's `X-Frame-Options: DENY` and
|
||||
* `frame-ancestors 'none'`, which protect Provider Settings. Embed mode
|
||||
* shows only the globe, so for those documents framing is limited to
|
||||
* `ancestors` instead: `GEV_EMBED_FRAME_ANCESTORS`, or any frame by default.
|
||||
* "Any" sends no framing restriction at all: `frame-ancestors *` would still
|
||||
* refuse sandboxed frames with an opaque origin, which panel hosts use.
|
||||
*/
|
||||
|
||||
const FRAMING_HEADERS = new Set(['x-frame-options', 'content-security-policy']);
|
||||
|
||||
/** Whether a request is for an embed-mode document. */
|
||||
export function isEmbedDocumentRequest(url) {
|
||||
let parsed;
|
||||
try {
|
||||
parsed = new URL(url || '/', 'http://localhost');
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
return (
|
||||
parsed.searchParams.get('embed') === '1' &&
|
||||
(parsed.pathname === '/' || parsed.pathname.endsWith('.html'))
|
||||
);
|
||||
}
|
||||
|
||||
/** Vite plugin applying the embed framing policy on dev and preview servers. */
|
||||
export function embedFramingPlugin({
|
||||
ancestors = process.env.GEV_EMBED_FRAME_ANCESTORS || '*',
|
||||
} = {}) {
|
||||
const anywhere = ancestors.trim() === '*';
|
||||
const policy = `frame-ancestors ${ancestors}`;
|
||||
const install = (server) => {
|
||||
server.middlewares.use((req, res, next) => {
|
||||
if (!isEmbedDocumentRequest(req.url)) return next();
|
||||
// The server's own headers are written when the response is sent, so
|
||||
// intercept them rather than setting ours first.
|
||||
const setHeader = res.setHeader.bind(res);
|
||||
res.setHeader = (name, value) => {
|
||||
const key = String(name).toLowerCase();
|
||||
if (!FRAMING_HEADERS.has(key)) return setHeader(name, value);
|
||||
if (key === 'content-security-policy' && !anywhere)
|
||||
return setHeader(name, policy);
|
||||
return res;
|
||||
};
|
||||
if (!anywhere) setHeader('Content-Security-Policy', policy);
|
||||
next();
|
||||
});
|
||||
};
|
||||
return {
|
||||
name: 'embed-framing',
|
||||
configureServer: install,
|
||||
configurePreviewServer: install,
|
||||
};
|
||||
}
|
||||
-153
@@ -1,153 +0,0 @@
|
||||
/**
|
||||
* The MCP Apps panel build. A panel loads the app through its MCP server,
|
||||
* one tool call per file (see src/tools/globePanel.js), so the app must be
|
||||
* few files: one app script, one stylesheet, and Cesium's own script, which
|
||||
* carries its workers and starts them from memory. The build is served at
|
||||
* PANEL_BASE next to the app.
|
||||
*/
|
||||
|
||||
// The panel loads these paths (src/tools/globePanel.js); a test keeps the
|
||||
// two in step.
|
||||
export const PANEL_BASE = '/panel/';
|
||||
/** The script the panel runs ahead of Cesium's workers (workerFilesPrelude). */
|
||||
export const PANEL_WORKER_PRELUDE_PATH = 'cesium/worker-prelude.js';
|
||||
export const PANEL_OUT_DIR = 'dist/panel';
|
||||
|
||||
/** A browser Vite config changed to produce the panel build. */
|
||||
export function panelBuildConfig(config) {
|
||||
return {
|
||||
...config,
|
||||
base: PANEL_BASE,
|
||||
build: {
|
||||
...config.build,
|
||||
outDir: PANEL_OUT_DIR,
|
||||
emptyOutDir: true,
|
||||
modulePreload: false,
|
||||
cssCodeSplit: false,
|
||||
rollupOptions: {
|
||||
...config.build?.rollupOptions,
|
||||
output: { inlineDynamicImports: true },
|
||||
},
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
const CONTENT_TYPES = {
|
||||
'.css': 'text/css',
|
||||
'.geojsonl': 'application/geo+json-seq',
|
||||
'.glb': 'model/gltf-binary',
|
||||
'.html': 'text/html',
|
||||
'.jpg': 'image/jpeg',
|
||||
'.js': 'text/javascript',
|
||||
'.json': 'application/json',
|
||||
'.png': 'image/png',
|
||||
'.svg': 'image/svg+xml',
|
||||
'.wasm': 'application/wasm',
|
||||
'.woff2': 'font/woff2',
|
||||
'.xml': 'application/xml',
|
||||
};
|
||||
|
||||
/**
|
||||
* Vite plugin serving the panel build at PANEL_BASE on the dev server, as
|
||||
* the preview server and production builds serve it from the build output.
|
||||
*/
|
||||
export function panelBuildPlugin({ outDir = PANEL_OUT_DIR } = {}) {
|
||||
return {
|
||||
name: 'panel-build',
|
||||
configureServer(server) {
|
||||
server.middlewares.use(async (req, res, next) => {
|
||||
const { pathname } = new URL(req.url || '/', 'http://localhost');
|
||||
if (!pathname.startsWith(PANEL_BASE)) return next();
|
||||
const { readFile } = await import('node:fs/promises');
|
||||
const { extname, resolve, sep } = await import('node:path');
|
||||
const root = resolve(server.config.root, outDir);
|
||||
let relative;
|
||||
try {
|
||||
relative = decodeURIComponent(pathname.slice(PANEL_BASE.length));
|
||||
} catch {
|
||||
res.statusCode = 400;
|
||||
res.end('Bad path');
|
||||
return;
|
||||
}
|
||||
const file = resolve(root, relative || 'index.html');
|
||||
if (file !== root && !file.startsWith(root + sep)) return next();
|
||||
try {
|
||||
const body = await readFile(file);
|
||||
res.setHeader(
|
||||
'Content-Type',
|
||||
CONTENT_TYPES[extname(file)] || 'application/octet-stream',
|
||||
);
|
||||
res.end(body);
|
||||
} catch {
|
||||
res.statusCode = 404;
|
||||
res.end(
|
||||
relative ? 'Not found' : 'No panel build; run npm run build:panel',
|
||||
);
|
||||
}
|
||||
});
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Cesium files its workers load themselves, as paths under Cesium's base.
|
||||
* Workers request them from the panel page's own site, which has none of
|
||||
* the app's files, so the panel build embeds them in the workers script.
|
||||
*/
|
||||
export const PANEL_WORKER_FILES = Object.freeze([
|
||||
'Assets/approximateTerrainHeights.json',
|
||||
]);
|
||||
|
||||
/**
|
||||
* A script, run in each worker before Cesium's, that answers the worker's
|
||||
* requests for `files` (Cesium-relative path → text) from memory and lets
|
||||
* every other request through.
|
||||
*/
|
||||
export function workerFilesPrelude(files) {
|
||||
const answer = (FILES) => {
|
||||
const find = (url) => {
|
||||
const path = String(url).split(/[?#]/)[0];
|
||||
for (const name of Object.keys(FILES))
|
||||
if (path.endsWith(`/${name}`)) return FILES[name];
|
||||
return null;
|
||||
};
|
||||
const xhr = XMLHttpRequest.prototype;
|
||||
const { open, send, setRequestHeader } = xhr;
|
||||
xhr.open = function (method, url, ...rest) {
|
||||
this.__gevFile = find(url);
|
||||
if (this.__gevFile === null) return open.call(this, method, url, ...rest);
|
||||
};
|
||||
xhr.setRequestHeader = function (...args) {
|
||||
if (this.__gevFile === null) return setRequestHeader.apply(this, args);
|
||||
};
|
||||
xhr.send = function (body) {
|
||||
const text = this.__gevFile;
|
||||
if (text === null || text === undefined) return send.call(this, body);
|
||||
const type = this.responseType;
|
||||
const response =
|
||||
type === 'json'
|
||||
? JSON.parse(text)
|
||||
: type === 'arraybuffer'
|
||||
? new TextEncoder().encode(text).buffer
|
||||
: text;
|
||||
const state = {
|
||||
readyState: 4,
|
||||
status: 200,
|
||||
statusText: 'OK',
|
||||
response,
|
||||
responseText: typeof response === 'string' ? response : '',
|
||||
getAllResponseHeaders: () => 'content-type: application/json\r\n',
|
||||
getResponseHeader: (name) =>
|
||||
name.toLowerCase() === 'content-type' ? 'application/json' : null,
|
||||
};
|
||||
for (const [name, value] of Object.entries(state))
|
||||
Object.defineProperty(this, name, { configurable: true, value });
|
||||
setTimeout(() => {
|
||||
this.dispatchEvent(new Event('readystatechange'));
|
||||
this.dispatchEvent(new ProgressEvent('load'));
|
||||
this.dispatchEvent(new ProgressEvent('loadend'));
|
||||
});
|
||||
};
|
||||
};
|
||||
return `(${answer})(${JSON.stringify(files)});\n`;
|
||||
}
|
||||
+6
-10
@@ -1,7 +1,5 @@
|
||||
import { applicationHtmlPlugin } from './application-html.js';
|
||||
import cesium from 'vite-plugin-cesium';
|
||||
import { embedFramingPlugin } from './embed-framing.js';
|
||||
import { panelBuildPlugin } from './panel.js';
|
||||
|
||||
/** Build browser assets with explicit inputs; never load environment or providers. */
|
||||
export function createBrowserViteConfig({
|
||||
@@ -9,18 +7,13 @@ export function createBrowserViteConfig({
|
||||
publicDir,
|
||||
googleApiKey,
|
||||
cesiumToken,
|
||||
photonUrl,
|
||||
host = 'localhost',
|
||||
port = 4173,
|
||||
command,
|
||||
} = {}) {
|
||||
return {
|
||||
plugins: [
|
||||
cesium(),
|
||||
applicationHtmlPlugin(),
|
||||
...plugins,
|
||||
embedFramingPlugin(),
|
||||
panelBuildPlugin(),
|
||||
],
|
||||
plugins: [cesium(), applicationHtmlPlugin(), ...plugins],
|
||||
...(publicDir === undefined ? {} : { publicDir }),
|
||||
// A production build must not clean the dependency cache a running dev
|
||||
// server is still serving optimized module URLs from.
|
||||
@@ -47,7 +40,6 @@ export function createBrowserViteConfig({
|
||||
deny: ['.env', '.env.*', '*.{crt,pem}', '**/.git/**', '**/ENVIRONMENT'],
|
||||
},
|
||||
// These headers protect the document containing Provider Settings.
|
||||
// Embed-mode documents are framable instead; see embed-framing.js.
|
||||
headers: {
|
||||
'X-Frame-Options': 'DENY',
|
||||
'Content-Security-Policy': "frame-ancestors 'none'",
|
||||
@@ -56,6 +48,10 @@ export function createBrowserViteConfig({
|
||||
define: {
|
||||
'import.meta.env.GOOGLE_MAPS_API_KEY': JSON.stringify(googleApiKey),
|
||||
'import.meta.env.CESIUM_ION_TOKEN': JSON.stringify(cesiumToken),
|
||||
// Only when set: '' disables Photon in the browser, a URL replaces it.
|
||||
...(photonUrl === undefined
|
||||
? {}
|
||||
: { 'import.meta.env.PHOTON_URL': JSON.stringify(String(photonUrl)) }),
|
||||
},
|
||||
build: { chunkSizeWarningLimit: 1500 },
|
||||
};
|
||||
|
||||
+84
-44
@@ -1,42 +1,5 @@
|
||||
# God's Eye View Current State
|
||||
|
||||
## God's Eye View in conversations — October 2, 2026
|
||||
|
||||
Tool answers that can be shown in God's Eye View include a view: camera, layers,
|
||||
style, map, marks and an aircraft or satellite to follow, written in the
|
||||
share-link format (`gods-eye-view/view`). `show_in_gods_eye_view` shows a view as live
|
||||
God's Eye View inside clients that display MCP Apps, and as a link everywhere.
|
||||
The panel runs the app's panel build (`npm run build:panel`, served at
|
||||
`/panel/`) and loads it, with its data, through the MCP server, so it works
|
||||
in Claude Desktop and in the Codex and ChatGPT desktop apps with a local
|
||||
server and no HTTPS. `?embed=1` shows the app as the globe alone and takes
|
||||
new views from the page that frames it. MCP leads with the tools that find
|
||||
what to show. See [tools and the MCP server](TOOLS.md).
|
||||
|
||||
## Tools and local MCP server — October 1, 2026
|
||||
|
||||
`gods-eye-view/tools` defines queries that answer questions from the app's data,
|
||||
and `gods-eye-view/tools/mcp` exposes a composed catalog over the Model Context
|
||||
Protocol. `npm run mcp` serves Core's tools over stdio to a local MCP client,
|
||||
reading from a running app's `/api` routes (default `http://localhost:4173`).
|
||||
The development and preview servers also serve the tools over HTTP at `/mcp`,
|
||||
accepting only local requests with loopback hosts and origins.
|
||||
Queries cover earthquakes, active fires, recent launches, aircraft (in an
|
||||
area, by identifier, tracks, and type and route lookups), ships (in an area,
|
||||
by identifier, and tracks), satellites (next
|
||||
pass over a point, and those overhead now), public cameras (including a
|
||||
camera's current image), license plate reader cameras, radio stations, place search, routing, bike-share
|
||||
stations, transit vehicles, road traffic flow, weather, weather map images
|
||||
(radar, satellite, lightning), wind, the most recent satellite image of an
|
||||
area, submarine cables, datacenters and dams, the Bhote Koshi flood event pack, regional briefs, tropical cyclones, fire
|
||||
perimeters, terrain height, military installations and map features, plus a
|
||||
combined situation brief, military awareness around a point, the app's heads-up display caption, and a link that
|
||||
opens the app over an area. Tools reuse the layers' portable source factories, take a
|
||||
shared `area` argument (place name, bounding box, or point and radius) and cap
|
||||
lists at 25 rows by default. Voice offers the same queries next to its app
|
||||
actions: the session lists them, and the browser runs them through the same
|
||||
catalog, loaded on first use. See [tools and the MCP server](TOOLS.md).
|
||||
|
||||
## Cyber HUD — September 23, 2026
|
||||
|
||||
Display > HUD > Layout includes Cyber, also available through the HUD voice
|
||||
@@ -259,7 +222,7 @@ Consult the linked official advisory for safety decisions.
|
||||
|
||||
Voice and HUD snapshots reuse the existing feedState classifier. Analyst follow-ups retain their original data provenance; current-view results append provenance without replacing legacy fields. HUD context and deterministic telemetry include non-nominal feed state.
|
||||
|
||||
Satellite and local infrastructure layers expose on-demand analyst records through their current factory owners. Analyst counts and ranks explicitly cover only bounded examined loaded records (default 2,000 per new layer, core satellite rows before dense extras); omitted records can change nearest/count and satellite distance is ground distance. Existing tools and result fields remain available.
|
||||
Analyst records come on demand from each layer's `getAnalystRecords`: flights, military, local ADS-B, vessels, satellites, launches, earthquakes, FIRMS (VIIRS and MODIS, with a `sensor` field), fire perimeters, cyclones, transit, bikeshare, ALPR cameras, mapped installations, datacenters and dams. `analyst_query` counts the whole loaded set (up to 250,000 records per layer; past that `coverage.records` reports `returned`, `total` and `truncated: true` and the result carries `complete: false`). Ranking keeps a top-k and scans in 20,000-row slices that yield to the page and stop when superseded. It refuses unknown fields, operators, units that do not convert (`altitudeM(ft)` converts), wrong-typed values, invalid centres, unknown scope kinds and sort fields with `code` and `allowed`, refuses a follow-up with no previous answer (`NO_RESULT_CONTEXT`) or naming other layers than that answer (`FOLLOW_UP_MISMATCH`), and tells `LAYER_OFF`, `NOT_READY` and `FEED_UNAVAILABLE` apart from a real zero; when only some layers can answer the result is `partial` with `unanswered` ids. The Contacts-window path applies the same filters, ranking and item shape to one immutable panel snapshot. Each aircraft layer retains at most 20,000 rows; reaching that bound sets `complete: false` and the spoken/card count becomes “At least”, never an exact claim. A superseded query never replaces follow-up memory, while sibling analyst calls from one assistant response do not cancel one another. Items carry `lat`/`lon`; the feed window (earthquakes and FIRMS: last 24 h), the precise scope and any caveat sit in `display`, which is on-screen detail rather than something the model recites. "In view" is still a radius around the view centre, not the visible footprint. Satellite distance is ground distance.
|
||||
|
||||
Keyless terrain tiles retry when Re:Earth throttles them. The browser fetches
|
||||
`terrain.reearth.land/cesium-mesh/ellipsoid/{z}/{x}/{y}.terrain` directly; no
|
||||
@@ -334,6 +297,48 @@ replacement session; audio meters release failed initialization and reject revok
|
||||
frames. Delayed action results and post-capture continuations cannot resume a
|
||||
stopped conversation or send output into a replacement. See [voice ownership](VOICE-OWNERSHIP.md).
|
||||
|
||||
Voice speech: `src/voice/speech.js` adds `{ say, display, referents?, schedule? }`
|
||||
to layer toggles, fly_to_location, frame_overhead, select_nearest_aircraft,
|
||||
get_entity_context, annotate_map and analyst_query. Analyst speech and the card
|
||||
share one deterministic count/scope headline; lower bounds use "At least" and
|
||||
partial answers name the unanswered layers. `get_current_view_state` lists enabled layers only.
|
||||
The voice card nested in the mic control (`voiceCard.js`) shows captions, plan
|
||||
steps and the result. `resultDisplay.js` is the one adapter from a result to
|
||||
the card: builder results show as built, and analyst results show a title
|
||||
with their scope, window and caveats. Caveats are on the card; the model speaks
|
||||
one only when it changes the answer, and always says lower bounds and partial
|
||||
answers. `narration.js` speaks one
|
||||
code-authored line for the current step of a tool still running at 1.5 s
|
||||
(out of band, never after the result). Tools that report steps: layer
|
||||
toggles, place lookups, select_nearest_aircraft and annotate_map (finding,
|
||||
outlining). Other tools get one "still working" line at 8 s. The
|
||||
earcon plays in push-to-talk only. A claimed Space hold while the assistant
|
||||
speaks is a barge-in (WebRTC cancel + audio clear), refused while Radio holds
|
||||
the speaker (`mayVoiceClaimSpeaker()`). Progress lines stay outside the reply
|
||||
lifecycle and Radio handoff. `turn.span` debug records carry generation and
|
||||
playback latency, silence, preamble and word-count figures. Caption
|
||||
transcription (`OPENAI_REALTIME_TRANSCRIBE_MODEL`, default on) is metered into
|
||||
the session cost and cap. The debug log omits transcripts and text unless
|
||||
`GEV_VOICE_LOG_CONTENT=1`.
|
||||
|
||||
Point and ask: `pointerContext.js` reads what the cursor is on. A Space hold
|
||||
is the pointing gesture: its keydown snapshot (released with the key) is sent
|
||||
as an inert `pointer_context` item with the target, with reticle and chip.
|
||||
Open-mic speech and typed turns count only a pointer that moved on the map in
|
||||
the last 5 s and send only the kind (`pointing:"aircraft"`/`"ground"`); the chip appears when a tool
|
||||
resolves it. Losing the pointer sends one `target:"none"` item. `deixis.js` resolves `'pointer'` in
|
||||
track_entity/fly_to_location queries, get_entity_context scope, analyst_query
|
||||
`scope.kind` and annotate_map targets, plus `referent:n` from the latest
|
||||
numbered result (`referents.js`, cleared at session end). A new result set (an analyst count) replaces
|
||||
the numbered list even when it is empty. Only the five rows numbered on the
|
||||
card are resolvable; `-1` means the fifth/last visible row. `track_entity`
|
||||
accepts either a named query or a referent, while an empty target is rejected. A reticle marks the
|
||||
point during the turn and the voice card shows "THIS"/"HERE"; both hide in
|
||||
Clean-UI and recording. A ground-only pointer lookup at local/city scale sends a ringed 512 px crop in
|
||||
the one retained image slot, dropped if the camera moved or a newer capture or
|
||||
turn started; screenX/screenY against the crop are mapped back to the canvas. Harness: `qa-voice-routing.mjs --layer deixis`,
|
||||
`--layer deixis-live`, `--layer behavior --deixis-only`.
|
||||
|
||||
The application shell composes focused state owners for navigation, destination
|
||||
lookup/orbit, Cockpit, visual settings, panel layout, aircraft display and layer
|
||||
bindings. Keyboard/display subscriptions have a separate lifetime; existing
|
||||
@@ -400,9 +405,11 @@ Reference feed construction is exported through `sources/reference`; the cable s
|
||||
|
||||
Source factories have dedicated `layers/<family>/source` exports. ALPR and earthquake record normalization and CCTV source policy no longer pull rendering into source consumers. Catalog construction and voice feed reads use their focused owners; compatibility entries remain available. Settings filesystem hardening lives under `server/standalone/`. Import-direction checks complement export ownership checks; source behavior, settings policy and rendering are unchanged.
|
||||
|
||||
Voice controls bind to a protocol-independent session factory. The default WebRTC adapter preserves the existing Realtime connection, push-to-talk, cost controls and radio handoff. Session subscriptions expose state, transcript, action call/result, interruption and completion events; stopping or replacing a session cancels pending actions. Alternate adapters can use the same controls and action runner.
|
||||
Voice controls bind to a protocol-independent session factory. The default WebRTC adapter preserves the existing Realtime connection, push-to-talk, cost controls and radio handoff. Session subscriptions expose state, transcript, action call/progress/result, interruption, completion and turn-metrics events; stopping or replacing a session cancels pending actions. Alternate adapters can use the same controls and action runner.
|
||||
|
||||
Voice action argument schemas have one portable owner under `src/voice/actionSchemas.js`. The Realtime provider builds 29 tools, retaining the existing 28 names and argument contracts apart from the three additive analyst-layer enum values using separate description-only metadata. Description customization cannot replace argument types, enum values or required fields.
|
||||
Voice action argument schemas have one portable owner under `src/voice/actionSchemas.js`. The Realtime provider builds 30 tools using separate description-only metadata. Description customization cannot replace argument types, enum values or required fields.
|
||||
|
||||
`src/voice/layerManifest.js` declares each catalog layer's voice capabilities: spoken aliases, `toggle`, in-view `context`, `track`, and analyst `query` fields with type and unit, or `noQuery` with the reason. The `set_layer_visibility`, `show_data_layers_menu`, `get_entity_context` and `analyst_query` layer enums, the runner's alias table and the analyst field hints are generated from it. `src/voice/layerManifest.test.mjs` fails when a catalog layer is neither listed nor named in `VOICE_OFF_LAYERS` (today only the two scene-driven Bhote Koshi layers). Global Context is listed with `toggle: false`; `set_context_mode` reaches it. `set_panel_open` also opens `weather-panel` and `recent-imagery-panel`.
|
||||
|
||||
Portable source exports provide Radio Browser station normalization, CCTV feed types and regional records independently of HTTP middleware. Radio normalization accepts an explicit URL policy; the standalone directory retains its existing HTTPS rules. Tile coordinate validation accepts explicit zoom bounds with unchanged traffic defaults. Cache, request and rendering owners remain unchanged.
|
||||
|
||||
@@ -445,7 +452,18 @@ clears caches and removes map-stack listeners. Standalone setup supplies the
|
||||
existing service clients. HUD summaries, regional brief, cockpit weather, location
|
||||
framing and annotation boundaries receive their service instances explicitly.
|
||||
Compatibility entrypoints retain defaults for direct callers; normal startup does
|
||||
not change global service slots. Analyst follow-up memory belongs to its voice runner.
|
||||
not change global service slots. Analyst follow-up memory belongs to its voice
|
||||
runner and ends with the voice session: stopping voice invalidates writes still in
|
||||
flight and forgets the last answer, the pointer and the numbered list; removing
|
||||
voice disposes the runner.
|
||||
|
||||
Analyst result memory is committed only after any Contacts-window projection
|
||||
finishes and the call is still current. Cancelled or superseded queries cannot
|
||||
return success or replace follow-up memory; Contacts follow-ups filter the same
|
||||
cohort that was shown, and partial/unanswered coverage remains attached when a
|
||||
remembered result is narrowed.
|
||||
The narrowed answer retains the remembered scope label and detail, including
|
||||
the Contacts-window attribution, unless the follow-up supplies a new scope.
|
||||
|
||||
Application startup constructs fresh layer instances from explicit source objects.
|
||||
Standalone composition selects the existing providers. Controls, launch orbit
|
||||
@@ -3869,7 +3887,7 @@ and unreachable upstream (502/504) separately from road geometry.
|
||||
- **Analyst → track handoff carries a key, not just a label** (2026-08-21). `analyst_query` items include `icao24`/`mmsi` alongside the display `id`, and contact lookup uses the shared tiered ranking in `src/data/contactMatch.js`: hex exact → callsign exact → registration exact → callsign prefix → registration prefix → callsign substring → registration substring. The tiers keep an exact callsign ahead of a colliding registration regardless of feed order, and registrations compare separator-insensitively (`G-ABCD`/`GABCD`, `05-8152`/`058152`, `N123AB`/`N-123AB`).
|
||||
- **A typed command supersedes the turn it interrupts** (2026-08-21). `sendTextCommand` defers its `response.create` behind an active response instead of colliding with it, marks that response superseded so a late function call from it is refused rather than dispatched, and drops the old turn's queued follow-up. A refused call is still ANSWERED — a terminal `{ok:false, superseded:true}` `function_call_output` — because an unanswered `function_call` strands a pending call and deadlocks the model; the refusal creates no response of its own. A burst of typed commands coalesces into one response while keeping both conversation items.
|
||||
- **Subject reconciliation across satellite catalog rebuilds** (2026-08-21). A dense↔core toggle or TLE refresh clears and repopulates the catalog, so the published subject is re-resolved against the new satrec; a subject that did NOT survive releases the slot. The per-frame refresh cannot do this itself — `_getTrackedFramePosition` returns early once the satellite has no catalog entry — so the reconcile runs at rebuild completion. An empty catalog is a rebuild in flight, not a disappearance.
|
||||
- **One aircraft-proximity engine** (2026-08-22). `collectAircraftProximityWindow` (`src/data/militaryAwareness.js`) is the single computation behind both the Contacts panel window and the voice analyst's entity-centred "how many nearby", so the panel readout and the spoken count for one centre are identical by construction. **Invariant: do not re-derive a proximity count anywhere else.** They diverged before because the panel read live billboard positions (20,000 cap) while the analyst used last-fix coordinates over a 2,000-record slice — 111 on screen, 15 spoken. Explicit regions and arbitrary points deliberately keep the general record engine. Entity-centred results carry `window: {engine:'contacts-window', centeredOn, radiusKm, flights, military, aircraft}`.
|
||||
- **One aircraft-proximity snapshot** (2026-08-22; hardened 2026-09-30). `collectAircraftProximityWindow` (`src/data/militaryAwareness.js`) computes the Contacts panel cohort once. The voice analyst reads the exact retained snapshot and evaluation timestamp instead of re-scanning live layers, so the panel readout and spoken count for one centre cannot cross snapshots. **Invariant: do not re-derive a proximity count anywhere else.** Each layer retains 20,000 rows and samples one row beyond the cap; a saturated cohort reports `complete:false`, `truncated:true`, and is spoken as a lower bound. They diverged before because the panel read live billboard positions while the analyst used last-fix coordinates over a 2,000-record slice — 111 on screen, 15 spoken. Explicit regions, arbitrary points, and a radius different from the panel radius deliberately keep the general record engine. Entity-centred results carry `window: {engine:'contacts-window', centeredOn, radiusKm, flights, military, aircraft, complete, evaluatedAt}`.
|
||||
- **Centre precedence for nearby asks**: explicit place in the question > Contacts subject (a selected non-contact entity never silently becomes the centre) > an entity the user names > the current view, said aloud. Contacts active with no subject uses the view rather than reading an empty panel.
|
||||
- **Contact-match ties break on hex ascending.** `track_entity` is a mutation fulfilling "follow that one", and the model's observed answer to a non-ok track result is to retry with guesses rather than ask, so the lookup always commits rather than returning an ambiguity. What it owes the caller is stability: hex is unique and always present, so the same query resolves to the same contact for as long as both are loaded.
|
||||
- **Visual grounding**: at `local` view scale with no structured identity, the client captures the Cesium canvas (≤1200px JPEG, black-frame detection, double-render for freshness) and sends it as `input_image` with a strict "do not invent labels" instruction.
|
||||
@@ -3886,8 +3904,8 @@ and unreachable upstream (502/504) separately from road geometry.
|
||||
- **Reliability**: tool-call dedupe (2.5s window across call/item/args keys); response-create queueing that respects active responses and defers follow-ups when the user starts speaking; per-tool follow-up instructions so the agent confirms only what actually happened (zoom confirms only on `ok=true`).
|
||||
- **Aircraft identity honesty:** “What is this aircraft?” reads callsign, operator, registration, type, and route only from the selected contact context. Missing operator, route, or type enrichment is named explicitly rather than silently omitted or inferred from the callsign.
|
||||
- **Diagnostics**: every client/server event is posted to `/api/realtime/debug-log` and appended to `.gev-logs/realtime-conversations.jsonl` (gitignored) with secret/image redaction; last 30 errors persist in `localStorage` (`gev-realtime-errors`); `window.__gevVoiceCommands.getDiagnostics()` in the console.
|
||||
- **Counting semantics ("near")** — a CONTRACT; new count-bearing tool work inherits it. Three honest numbers exist for one question: the Contacts cohort (250 km around the subject, what the panel shows), `analyst_query`'s count of _currently-loaded_ records for the requested scope, and the layer-wide loaded total in `coverage.layersQueried[].records`. They diverge legitimately — the flights layer loads by viewport, so after a camera dive the loaded set can hold a fraction of the cohort (field case: panel 42, analyst 8). The contract:
|
||||
1. **Contacts ACTIVE** → "near / nearby / how many aircraft" means the **Contacts window** — the panel's numbers, spoken verbatim. Mechanism: `contactsWindow` (`{centeredOn, radiusKm, flights, military, vessels}`), carried by both `analyst_query` and `get_current_view_state`, derived by `contactsWindowFromSnapshot()` from the same snapshot the panel renders so the two cannot drift. A cohort whose feed cannot answer reports `'unknown'`, never a confident zero.
|
||||
- **Counting semantics ("near")** — a CONTRACT; new count-bearing tool work inherits it. Three honest numbers exist for one question: the Contacts cohort (250 km around the subject, what the panel shows), `analyst_query`'s count of _currently-loaded_ records for the requested scope, and the layer-wide loaded total in `coverage.layersQueried[].total`. They diverge legitimately — the flights layer loads by viewport, so after a camera dive the loaded set can hold a fraction of the cohort (field case: panel 42, analyst 8). The contract:
|
||||
1. **Contacts ACTIVE** → "near / nearby / how many aircraft" means the **Contacts window** — the panel's numbers, spoken verbatim. Mechanism: `contactsWindow` (`{centeredOn, radiusKm, flights, military, vessels, complete}`), carried by both `analyst_query` and `get_current_view_state`, derived by `contactsWindowFromSnapshot()` from the same snapshot the panel renders so the two cannot drift. Voice filtering and follow-up memory use that immutable snapshot too. A cohort whose feed cannot answer reports `'unknown'`, never a confident zero; a cohort at the 20,000-row retention bound reports a lower bound.
|
||||
2. **Contacts OFF** → "nearby" means **in view**; "near \<place\>" means a radius around that place. A radius query with Contacts active and no explicit centre is centred on the **active contact**, not the camera.
|
||||
3. **Every count names its scope in words** — "42 in your window", "8 in view", "about 30 within 250 km of Austin" — never a bare number. `analyst_query` returns `scopeLabel` so this is mechanical. Two different numbers with named scopes are not a contradiction.
|
||||
4. **The loaded-data caveat is stated once when relevant**: counts cover loaded data, and the flights layer loads where you look (appended to `coverage.note` for radius/view scopes over viewport-loaded layers).
|
||||
@@ -3984,6 +4002,28 @@ are omitted rather than framing the wrong part of the globe.
|
||||
Berlin inside Brandenburg). Each pack loads on the first ask that needs it,
|
||||
never at start-up; the analyst region lookup uses the same packs. Gate:
|
||||
`scripts/qa-admin-outlines.mjs`.
|
||||
- Bundled place and neighborhood outlines (`src/data/placeBoundaries.js`):
|
||||
US Census places (`src/data/local_data/us_census_places/`, one file per
|
||||
state, public domain, `scripts/build-census-places.mjs`) and Who's On First
|
||||
neighborhoods (`src/data/local_data/wof_neighborhoods/`, quadtree tiles,
|
||||
per-source licences, `scripts/build-wof-neighborhoods.py`). After the admin
|
||||
packs, an area ask named at or near the view is drawn from, in order, a
|
||||
Census place ("Austin"; a state qualifier narrows the search to that state
|
||||
wherever the camera is: "Springfield, Illinois"), the San Francisco DataSF
|
||||
pack, then WOF ("Notting Hill", "Le Marais"). "Near" reaches half the view
|
||||
radius. A district ask takes a neighborhood first and a place only when the
|
||||
view is inside it; without an entity kind, the candidate holding the view
|
||||
wins, else the nearer. "City of X" asks for an incorporated place; when
|
||||
several places still fit and nothing decides, the geocoder answers.
|
||||
Compound, grounds, monument and "around" asks skip these, as do names that
|
||||
end like a landmark ("Central Park") unless asked as a district, and a
|
||||
qualified neighborhood ("Williamsburg, Virginia") goes to the geocoder. A
|
||||
geocoded city or neighborhood is drawn from the same packs when one carries
|
||||
the name and holds the point, before any boundary lookup. Only the index and
|
||||
the state files or tiles around the point are fetched (files under 4 KB are
|
||||
inlined into the bundle), and the recently used ones are kept (12 state
|
||||
files, 24 tiles). Gate:
|
||||
`node scripts/qa-voice-routing.mjs --layer behavior --behavior outlines`.
|
||||
- Renderer contract: `src/annotations/hybridAnnotationRenderer.js` routes draped `area`/`route` geometry to world-space Cesium rendering and reticles/pins/arrows/callouts to the screen-space SVG renderer. Area labels are screen-space callouts so all captions share one visual language; progressive outline upgrades convert the existing screen group in place when the anchor snaps to the resolved centroid.
|
||||
- Tooling: voice has `annotate_map` and `clear_annotations` tools. Annotations accumulate and persist by default; clearing is explicit only. Partial failures, approximate synthesized zones, and route fallbacks are returned as structured tool results so the voice layer can be honest.
|
||||
- Console/dev API: `window.__gevAnnotations.tour()`, `.demo()`, `.annotate()`, `.clear()`, `.count()`, and `.list()` are the deterministic no-mic test surface.
|
||||
|
||||
@@ -1,71 +0,0 @@
|
||||
# MCP setup
|
||||
|
||||
Use God's Eye View from Claude and Codex: tools, links, and the live globe
|
||||
inside the conversation. Agents: follow these steps in order and check each
|
||||
one before going on.
|
||||
|
||||
## 1. Run the app
|
||||
|
||||
```sh
|
||||
npm install
|
||||
npm run build:panel # the in-conversation globe; rerun after app changes
|
||||
npm run dev # http://localhost:5173, keep it running
|
||||
```
|
||||
|
||||
The MCP server reads from this app.
|
||||
|
||||
## 2. Add the server to a client
|
||||
|
||||
The server is `server/mcp/stdio.js` in this checkout. Use absolute paths
|
||||
(clients do not load your shell profile), with each argument as its own entry.
|
||||
|
||||
| Client | How |
|
||||
| --- | --- |
|
||||
| Claude Code | `claude mcp add gods-eye-view -- node /abs/path/gods-eye-view/server/mcp/stdio.js --api-base http://localhost:5173` |
|
||||
| Codex CLI | `codex mcp add gods-eye-view -- node /abs/path/gods-eye-view/server/mcp/stdio.js --api-base http://localhost:5173` |
|
||||
| Codex and ChatGPT desktop | Same `~/.codex/config.toml` as the CLI (block below), or Settings → MCP servers → Add server (STDIO). |
|
||||
| Claude Desktop | Settings → Developer → Edit Config, then add the JSON below. |
|
||||
|
||||
```toml
|
||||
[mcp_servers.gods-eye-view]
|
||||
command = "node"
|
||||
args = ["/abs/path/gods-eye-view/server/mcp/stdio.js", "--api-base", "http://localhost:5173"]
|
||||
```
|
||||
|
||||
```json
|
||||
{
|
||||
"mcpServers": {
|
||||
"gods-eye-view": {
|
||||
"command": "node",
|
||||
"args": ["/abs/path/gods-eye-view/server/mcp/stdio.js", "--api-base", "http://localhost:5173"]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Windows with the checkout in WSL:** use `"command": "wsl.exe"` and put
|
||||
`"-e"` and the absolute Linux path to `node` first in `args`, for example
|
||||
`["-e", "/home/you/.local/share/mise/installs/node/24/bin/node", "/home/you/gods-eye-view/server/mcp/stdio.js", "--api-base", "http://localhost:5173"]`.
|
||||
|
||||
## 3. Restart and try it
|
||||
|
||||
Fully quit the client (on Windows, from the tray icon) and reopen it, then
|
||||
start a new chat; clients keep the server they started with. Ask: *"Show San
|
||||
Diego in God's Eye View with military flights on."*
|
||||
|
||||
- Claude Desktop (a chat, not the Code tab), Codex and ChatGPT desktop show
|
||||
the live globe; the first load takes 10–20 seconds.
|
||||
- Claude Code and the Codex CLI answer with data and a link.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
- **The server does not appear:** run the command from step 2 in a terminal;
|
||||
it should print `God's Eye View MCP server reading http://localhost:5173`
|
||||
and wait. A path or `args` mistake fails here.
|
||||
- **The panel says it could not load:** `npm run build:panel` was not run, or
|
||||
the dev server is down.
|
||||
- **Logs:** the server writes each call, and why one failed, to stderr. Claude
|
||||
Desktop keeps it in its logs folder as `mcp-server-gods-eye-view.log`; in
|
||||
Codex, right-click the panel → DevTools for the panel's console.
|
||||
|
||||
See [tools and the MCP server](TOOLS.md) for what the tools do.
|
||||
-291
@@ -1,291 +0,0 @@
|
||||
# Tools and the MCP server
|
||||
|
||||
Tools answer questions from God's Eye View data for language-model clients.
|
||||
They are defined once and exposed through adapters: the Model Context Protocol
|
||||
(MCP) and function calling, which voice uses.
|
||||
|
||||
To use them from Claude or Codex, see [MCP setup](MCP_SETUP.md).
|
||||
|
||||
## Layers
|
||||
|
||||
| Owner | Responsibility |
|
||||
| -------------------- | ------------------------------------------------------------------------------------------------- |
|
||||
| `src/tools/` | Tool definitions, catalog composition, argument validation, shared `area` and result helpers |
|
||||
| `src/tools/queries/` | Queries, one file per domain, reading only portable source contracts |
|
||||
| `src/tools/mcp/` | MCP protocol (JSON-RPC) and a stateless HTTP transport; knows the catalog interface, not queries |
|
||||
| `src/tools/functions.js` | Function-calling adapter: tool records and results for function-calling clients |
|
||||
| `src/tools/services.js` | The default services: the layers' source factories and request services, given a resolving fetch |
|
||||
| `server/mcp/` | Node composition: points the sources at a running app's `/api` routes; serves stdio and `/mcp` |
|
||||
| `server/standalone/voiceTools.js`, `src/standalone/toolCatalog.js` | Standalone voice composition: the session's tool list and the browser catalog |
|
||||
|
||||
Dependencies point downward only. `gods-eye-view/tools` and
|
||||
`gods-eye-view/tools/mcp` are portable exports: they reach no application,
|
||||
rendering, Node, Cesium or browser-global code, which
|
||||
`npm run check:boundaries` enforces. In the application, only voice reaches
|
||||
them: `withToolCatalog` in `src/voice/gevRealtime.js` imports the
|
||||
function-calling adapter, and the standalone entry supplies the catalog.
|
||||
|
||||
## Definitions and composition
|
||||
|
||||
`defineTool({ name, kind, title, description, inputSchema, requires, run })`
|
||||
validates and freezes a tool. `kind` is `query` (answers from data, read-only)
|
||||
or `action`. `inputSchema` uses a JSON Schema subset that `src/tools/schema.js`
|
||||
checks completely; unsupported keywords are rejected at definition time.
|
||||
`run(args, { services, signal })` resolves to `{ summary, data }`: one sentence
|
||||
for people and a structured object for programs. A tool may also return `images`,
|
||||
each `{ mimeType, data }` with base64 data; the MCP adapter sends them as image
|
||||
content. MCP results carry the summary and the data as JSON text, plus the data
|
||||
as `structuredContent`, for clients that read only one of them.
|
||||
|
||||
`composeCatalog({ tools, services, replace, interceptors })` builds a catalog:
|
||||
|
||||
- **Tools**: an application adds its own tools to `coreTools`. Reusing a name
|
||||
fails unless the name is listed in `replace`.
|
||||
- **Services**: each tool names the services it reads in `requires`. Tools
|
||||
whose services are not supplied are left out.
|
||||
- **Interceptors**: `(call, next) => next(call)` functions wrap every call,
|
||||
outermost first. They can observe, reject or change a call.
|
||||
- **Composite tools**: `run` receives `tools`, with `has(name)` and
|
||||
`call(name, args)`, to call other tools through the same catalog, so
|
||||
replaced tools and interceptors apply. Interceptors see such calls with
|
||||
`parent`, the calling tool's name.
|
||||
|
||||
### Surfaces
|
||||
|
||||
`src/tools/surfaces.js` lists which tools MCP and voice offer. A tool is on
|
||||
both unless `TOOL_SURFACES` turns it off; edit an entry to turn a tool on or
|
||||
off for one surface. MCP leaves out place search, routing, plain
|
||||
weather and wind, the regional brief, the HUD caption, radio, bike share and
|
||||
transit, which assistants already cover or which add little without the
|
||||
globe. `catalogForSurface(catalog, surface, overrides)` is the
|
||||
view a surface exposes: it lists and calls only the tools it offers, while
|
||||
composite tools still reach the whole catalog. `toolsForSurface` gives the
|
||||
same selection as a list of definitions, such as for the voice session's tool
|
||||
list. Voice leaves out tools that answer with images, link to the app, or
|
||||
repeat what its app actions answer.
|
||||
|
||||
Expected failures throw `ToolError` with one of `invalid_arguments`,
|
||||
`unavailable`, `unsupported`, `malformed` or `retry_later`. Other errors are
|
||||
reported to clients without details.
|
||||
|
||||
## Services
|
||||
|
||||
`gods-eye-view/tools/services` builds the default set with
|
||||
`createToolServices({ fetchImpl, appUrl })`. Services are the portable source factories the layers already use, such as
|
||||
`createUsgsEarthquakeSource`, `createFirmsSource` and `createLaunchSource`,
|
||||
plus a `places` service with `resolve(name, { signal })`. Sources request
|
||||
relative `/api/...` paths through an injected `fetchImpl`, so the same tool
|
||||
code runs wherever an application routes those paths.
|
||||
The `weather`, `regional`, `terrain`, `summary` and `features` services are the
|
||||
application request services from `gods-eye-view/application/requests`.
|
||||
`situation_brief` and `military_awareness` run each section whose services are supplied and mark the
|
||||
others unavailable. `app` is `{ baseUrl }`, the address links open. The `bikeshare` service is `{ systems, getStations }`: the system registry and
|
||||
the GBFS source. `createGeocodePlaceService` resolves place names through `/api/geocode`;
|
||||
`createPlaceSearchService` searches `/api/google/*` and reports when no search key
|
||||
is configured; `createRouteService` plans routes through `/api/route`.
|
||||
|
||||
## Views
|
||||
|
||||
`gods-eye-view/view` (`src/view/index.js`) describes what the app shows,
|
||||
independent of how it is shown: a camera (lat, lon, altitude, heading,
|
||||
pitch), data layers, visual style, map imagery, and an aircraft, military
|
||||
aircraft or satellite to follow. `createView` builds and bounds one,
|
||||
`viewToParams` and `viewFromParams` write and read it in the share-link
|
||||
format the app restores, and `viewUrl` gives the address that opens it. The
|
||||
style names are the ones share links use. Ships cannot be followed from a
|
||||
link yet. An aircraft's `follow` may ask for its cockpit view
|
||||
(`cockpit: true`); links cannot carry that and open the app following it.
|
||||
|
||||
A view can also carry `annotations`, the marks the app's `annotate_map`
|
||||
action draws (pins, highlights, areas, arrows, routes and labels at a place
|
||||
name or coordinates). Links carry them in the `an` parameter, and the app
|
||||
draws them once the link has been restored.
|
||||
|
||||
Tools take a view as `VIEW_ARGUMENTS`: an `area` to frame from above, or a
|
||||
`camera`, plus `layers`, `style`, `map`, `follow` and `annotations`. A
|
||||
tilted camera over an area looks at its center from behind; camera fields
|
||||
given with an area override its framing. `resolveViewArguments` turns them
|
||||
into a view.
|
||||
|
||||
### The God's Eye View panel
|
||||
|
||||
`show_in_gods_eye_view` names an MCP Apps view (`io.modelcontextprotocol/ui`):
|
||||
`_meta.ui.resourceUri` is `ui://gods-eye-view/globe`, a `text/html;profile=mcp-app`
|
||||
resource from `createGlobePanelResource({ runtime })` in
|
||||
`src/tools/globePanel.js`, with the panel's script from
|
||||
`src/app/globePanelRuntime.js`. Clients that display apps render it inside
|
||||
the conversation. The panel completes the MCP Apps handshake
|
||||
(`ui/initialize`, `ui/notifications/initialized`,
|
||||
`ui/notifications/size-changed`), and for each
|
||||
`ui/notifications/tool-result` carrying a view it loads the app in inline
|
||||
embed mode the first time and posts later views to that same app, so the
|
||||
globe changes without reloading. Its "Open in God's Eye View" button asks
|
||||
the host to open the link (`ui/open-link`).
|
||||
|
||||
Hosts serve panels from their own sites and may refuse other addresses;
|
||||
Codex, for one, refuses any address on the user's machine. So the panel
|
||||
never requests the app's server itself. It loads everything from the app's
|
||||
own paths through `panel_request`, a tool only the panel may call
|
||||
(`_meta.ui.visibility: ["app"]`): the MCP server requests the path from the
|
||||
app's server and returns the response, compressed and in parts when large.
|
||||
The same path works in every host. Only
|
||||
map imagery, tiles and fonts load directly, from the providers the
|
||||
resource's `csp` lists. `panel_request` refuses `/api/setup`, which writes
|
||||
provider keys.
|
||||
|
||||
The panel loads the app's panel build, which `npm run build:panel` writes to
|
||||
`dist/panel` and the servers serve at `/panel/`: one app script, one
|
||||
stylesheet, and Cesium's script, which carries its workers and starts them
|
||||
from memory (`CESIUM_WORKERS`). Files those workers load themselves are
|
||||
embedded in a prelude the panel runs ahead of them, since a worker's
|
||||
requests reach the panel page's own site. Rebuild it after changing
|
||||
the app.
|
||||
|
||||
Hosts differ in ways the panel works around, all inside the panel only:
|
||||
images and stylesheet files arrive as `data:` URLs, since some hosts refuse
|
||||
`blob:` images; code that needs an https address for the app gets
|
||||
`GEV_APP_BASE_URL`, since some hosts serve the page from their own scheme;
|
||||
the globe keeps drawing from a timer when the host reports the panel hidden
|
||||
and stops animation frames; and 2D canvases are kept in memory
|
||||
(`willReadFrequently`), since a host that treats the panel as off screen may
|
||||
drop their GPU contents and show the overlays as black over the globe.
|
||||
|
||||
Tools declare a UI resource with `defineTool({ ui: { resourceUri } })`, and
|
||||
an app-only tool with `ui: { visibility: ['app'] }`;
|
||||
`createMcpServer({ resources })` serves `resources/list` and
|
||||
`resources/read`.
|
||||
|
||||
### Embed mode
|
||||
|
||||
`?embed=1` shows only the globe: clean view, with the HUD, panels, welcome
|
||||
and setup prompts hidden; provider attribution stays. A page that frames it
|
||||
changes the view by posting `{ type: 'gev:view', id, view }` to the frame.
|
||||
The app applies it through its own actions (style, map, exactly the view's
|
||||
layers, annotations, then the followed entity, retried until its layer has
|
||||
it, or the camera when nothing is followed or the entity is not there yet,
|
||||
since a camera flight would end the follow; then cockpit view when asked) and
|
||||
answers `{ type: 'gev:view-applied', id, ok, steps }`. It
|
||||
posts `{ type: 'gev:ready' }` once it can take views, and only its parent
|
||||
page can send them. See `src/app/embed.js`.
|
||||
|
||||
The development and preview servers let other pages frame embed-mode
|
||||
documents only; every other document keeps `X-Frame-Options: DENY` and
|
||||
`frame-ancestors 'none'`. `GEV_EMBED_FRAME_ANCESTORS` restricts which pages
|
||||
may frame them (any by default).
|
||||
|
||||
Answers that have something to show include `data.view`: the view that
|
||||
shows them, with the matching layers on, an area framed from above, and a
|
||||
single aircraft or satellite followed, plus `url` to open it (null when the
|
||||
app's address is not configured). `suggestView` in `src/tools/views.js`
|
||||
builds one.
|
||||
|
||||
## The `area` argument
|
||||
|
||||
Location-scoped tools take `area` as exactly one of a `place` name, a `bbox`
|
||||
(`[west, south, east, north]`, crossing the antimeridian when west exceeds
|
||||
east), or `lat`, `lon` and `radius_km`. Lists default to 25 rows, at most 200,
|
||||
and report `total`, `returned` and `truncated`.
|
||||
|
||||
## MCP
|
||||
|
||||
`createMcpServer({ catalog, name, version, instructions, descriptions, decorate })`
|
||||
implements `initialize`, `ping`, `tools/list` and `tools/call` for protocol
|
||||
revisions 2025-11-25, 2025-06-18 and 2025-03-26. `descriptions` overrides a
|
||||
tool's title or description for this surface; `decorate(definition, tool)`
|
||||
merges extra fields into each listed definition. `createMcpHttpHandler(server)`
|
||||
returns a `Request`-to-`Response` handler for stateless Streamable HTTP: one
|
||||
JSON-RPC message per POST, answered with JSON. The host owns routing and any
|
||||
access control in front of it.
|
||||
|
||||
## Voice
|
||||
|
||||
Voice offers the catalog's queries next to its app actions.
|
||||
`toFunctionTools(tools, { exclude })` turns tools into
|
||||
`{ type: 'function', name, description, parameters }` records, and
|
||||
`toFunctionOutput(name, result)` turns a result into
|
||||
`{ ok, tool, summary, data }`, counting images in `images_omitted` instead of
|
||||
sending them.
|
||||
|
||||
The voice session token endpoint takes its tool list as `realtime.tools`.
|
||||
`realtimeSessionTools(additional)` appends function tools to the app actions,
|
||||
skipping names an action already uses, so `next_satellite_pass` stays the
|
||||
action. The standalone server supplies the core queries that
|
||||
`src/tools/surfaces.js` offers on voice, which leaves out tools that answer
|
||||
with images, link to the app, or repeat what voice's app actions answer
|
||||
(aircraft, ships, earthquakes, fires, datacenters, dams and satellites
|
||||
overhead, which `analyst_query` covers).
|
||||
|
||||
In the browser, `initGevVoiceCommands({ toolCatalog })` takes a function that
|
||||
resolves a catalog. App action names go to the action runner; other names the
|
||||
catalog has go to `catalog.call` with the call's abort signal. The standalone
|
||||
entry composes the catalog with `createToolServices` over the page's fetch and
|
||||
loads it the first time voice calls a query.
|
||||
|
||||
## Running locally
|
||||
|
||||
Start the app (`npm run dev` or `npm run preview`), then register the stdio
|
||||
server with an MCP client, for example Claude Code:
|
||||
|
||||
```bash
|
||||
claude mcp add gods-eye-view -- npm --prefix /path/to/gods-eye-view run --silent mcp
|
||||
```
|
||||
|
||||
`npm run mcp -- --api-base http://localhost:4173` selects another server.
|
||||
|
||||
The development and preview servers also serve the same tools over HTTP at
|
||||
`/mcp`, for clients that connect by URL:
|
||||
|
||||
```bash
|
||||
claude mcp add --transport http gods-eye-view http://localhost:4173/mcp
|
||||
```
|
||||
|
||||
The route accepts only requests from this machine that name a loopback host
|
||||
and, when a browser sends an `Origin`, come from a loopback origin. This is
|
||||
local transport safety, not authentication. The
|
||||
local server makes no requests other than to the app's `/api` routes and the
|
||||
public feeds the sources already use.
|
||||
|
||||
## Tools
|
||||
|
||||
| Tool | Reads | Returns |
|
||||
| --------------------- | ------------- | ------------------------------------------------------------- |
|
||||
| `get_earthquakes` | `earthquakes` | USGS M2.5+ events in the last 24 hours, strongest first |
|
||||
| `get_active_fires` | `fires` | NASA FIRMS detections in an area, highest radiative power first |
|
||||
| `get_recent_launches` | `launches` | Launch Library 2 launches in the last 30 days, newest first |
|
||||
| `aircraft_in_area` | `aircraft` | Aircraft in an area, nearest first; `military: true` reads the `military` feed |
|
||||
| `find_aircraft` | `aircraft` | Aircraft anywhere by callsign, ICAO address or registration |
|
||||
| `get_aircraft_track` | `aircraft` | Recent positions of one aircraft, thinned to 200 points |
|
||||
| `get_aircraft_info` | `aircraft` | Aircraft type and registration, and flight route, from adsbdb |
|
||||
| `vessels_in_area` | `vessels` | Ships reported by AIS in an area, nearest first, optionally by type |
|
||||
| `find_vessel` | `vessels` | Ships anywhere by MMSI, IMO number or name |
|
||||
| `get_vessel_track` | `vessels` | Recent positions of one ship, thinned to 200 points |
|
||||
| `next_satellite_pass` | `satellites` | Next pass over a place or point (default the ISS), with naked-eye visibility |
|
||||
| `satellites_overhead` | `satellites` | Satellites in a CelesTrak group above a place or point now, highest first |
|
||||
| `find_cctv_cameras` | `cctv` | Public cameras in an area, nearest first |
|
||||
| `get_cctv_snapshot` | `cctv` | The current image from one camera, returned as image content |
|
||||
| `find_alpr_cameras` | `alpr` | OpenStreetMap-mapped license plate readers in a US/Canadian area up to 3° |
|
||||
| `find_radio_stations` | `radio` | Radio Browser stations by area and/or search terms, with stream URLs |
|
||||
| `search_places` | `placeSearch` | Points of interest matching a query within an area (Google Places) |
|
||||
| `places_nearby` | `placeSearch` | Notable places around a place or point (Google Places) |
|
||||
| `plan_route` | `routing` | Walking, driving or cycling route over OpenStreetMap, with a simplified path |
|
||||
| `get_bike_share` | `bikeshare` | Live GBFS stations in an area, with bikes and docks available |
|
||||
| `get_transit_vehicles` | `transit` | Live GTFS-Realtime vehicle positions in an area, optionally one route |
|
||||
| `get_traffic_flow` | `traffic` | TomTom flow in a city-sized area: speed vs free flow, congested and closed road |
|
||||
| `get_weather` | `weather` | Current conditions at a place or point |
|
||||
| `get_weather_map` | `weatherMaps` | The latest NOAA radar, satellite or lightning map image over an area |
|
||||
| `get_wind` | `wind` | GFS or IFS model wind 10 m above ground at a location |
|
||||
| `get_recent_imagery` | `imagery` | The most recent clear Landsat/Sentinel-2 image of an area (VIIRS fallback) |
|
||||
| `find_submarine_cables` | `cables` | TeleGeography cables and landing points by area or name (CC BY-NC-SA 3.0) |
|
||||
| `find_infrastructure` | `infrastructure` | OpenStreetMap datacenters or dams in an area, nearest first (ODbL) |
|
||||
| `get_bhote_koshi_flood` | `events` | The 2026 Bhote Koshi flood: evidence trail in story order, flood path and imagery dates (CC BY-NC 4.0) |
|
||||
| `get_regional_brief` | `regional` | What and where a location is, its weather and recent headlines |
|
||||
| `get_cyclones` | `cyclones` | Active NHC/CPHC tropical cyclones, optionally in an area |
|
||||
| `get_fire_perimeters` | `perimeters` | Mapped WFIGS wildfire perimeters in an area, largest first |
|
||||
| `get_terrain_height` | `terrain` | Ground, geoid and ellipsoid heights at up to 20 points |
|
||||
| `find_military_installations` | `installations` | OpenStreetMap military sites in an area of at most 10° per side |
|
||||
| `get_map_features` | `features` | Administrative areas, named places or monuments at a location (needs Overpass) |
|
||||
| `situation_brief` | `weather` | Weather, earthquakes, fires, aircraft, ships and cyclones for an area, by section |
|
||||
| `military_awareness` | `military` | Military and other aircraft, ships and military installations within 250 km of a point, by section |
|
||||
| `get_hud_caption` | `weather`, `summary` | The app's heads-up display caption for an area |
|
||||
| `show_in_gods_eye_view` | `app` | A view in God's Eye View: the live panel in clients with MCP Apps, and a link everywhere; takes another answer's view or an area or camera, layers, style, map, marks and something to follow |
|
||||
| `panel_request` | `app` | Panel only: loads a path from the app's server for the God's Eye View panel |
|
||||
@@ -14,6 +14,12 @@ The session adapter, action runner and backend interfaces remain unchanged.
|
||||
| `realtimeCost.js` | Next-session preferences and the current session's model-bound meter |
|
||||
| `realtimeViewport.js` | One retained image, bounded deletion identities and capture generation |
|
||||
| `realtimeDiagnostics.js` | Bounded error history and sanitized optional diagnostics |
|
||||
| `turnMetrics.js` | Per-turn latency and speech spans written to the debug log |
|
||||
| `narration.js` | Progress lines and the earcon while tools run; spoken through the turns owner |
|
||||
| `pointerContext.js` | Pointer position, dwell pick and the current turn's pointer snapshot (session-scoped) |
|
||||
| `referents.js` | Numbered items from the latest result for `referent:n` (session-scoped) |
|
||||
| `resultDisplay.js` | What the voice card shows for a result and which items it numbers (read by the card and `referents.js`) |
|
||||
| `gevActions.js` runner | Analyst follow-up memory (session: `resetConversation()` on stop, `dispose()` on removal) |
|
||||
|
||||
Pure input policy, preferences and protocol-response policy have separate modules.
|
||||
Owners receive named readers, operations or focused collaborators. Cross-owner
|
||||
@@ -29,7 +35,7 @@ Action and post-capture continuations retain their conversation identity before
|
||||
publishing output, queueing a response or changing status.
|
||||
|
||||
Behavioral invariants remain in force: a 500 ms Space hold claims push-to-talk;
|
||||
short control taps and text entry remain native; a click-started session stays
|
||||
short control taps and text entry remain native; a claimed hold while the assistant speaks interrupts it (never during a Radio handoff); a click-started session stays
|
||||
open-mic. Every accepted or refused tool call receives its terminal output once.
|
||||
New typed input supersedes old intent. Radio waits for the spoken confirmation and
|
||||
verified muted playback. A live cost meter stays bound to its negotiated model,
|
||||
|
||||
Generated
+2
-2
@@ -1,12 +1,12 @@
|
||||
{
|
||||
"name": "gods-eye-view",
|
||||
"version": "0.2.0",
|
||||
"version": "0.1.1",
|
||||
"lockfileVersion": 3,
|
||||
"requires": true,
|
||||
"packages": {
|
||||
"": {
|
||||
"name": "gods-eye-view",
|
||||
"version": "0.2.0",
|
||||
"version": "0.1.1",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"@jtarrio/signals": "^0.10.0",
|
||||
|
||||
+3
-9
@@ -1,7 +1,7 @@
|
||||
{
|
||||
"name": "gods-eye-view",
|
||||
"private": true,
|
||||
"version": "0.2.0",
|
||||
"version": "0.1.1",
|
||||
"description": "A real-time intelligence console for planet Earth — photorealistic 3D globe, live aircraft/ships/satellites/earthquakes/CCTV, and hands-free voice control. Runs in a browser.",
|
||||
"type": "module",
|
||||
"license": "MIT",
|
||||
@@ -41,14 +41,12 @@
|
||||
"dev:secure": "./scripts/dev-secure.sh",
|
||||
"opensky:import": "./scripts/opensky-import-client.sh",
|
||||
"build": "vite build",
|
||||
"build:panel": "node scripts/build-panel.mjs",
|
||||
"preview": "vite preview",
|
||||
"test": "node scripts/run-unit-tests.mjs",
|
||||
"test:track": "node scripts/track-regression.mjs",
|
||||
"qa:map-source-tray": "node scripts/qa-map-source-tray.mjs",
|
||||
"check:boundaries": "node scripts/check-import-directions.mjs && node scripts/check-package-boundaries.mjs",
|
||||
"qa:transit": "node scripts/qa-transit.mjs",
|
||||
"mcp": "node server/mcp/stdio.js"
|
||||
"qa:transit": "node scripts/qa-transit.mjs"
|
||||
},
|
||||
"dependencies": {
|
||||
"@jtarrio/signals": "^0.10.0",
|
||||
@@ -252,10 +250,6 @@
|
||||
"./layers/transit/standalone": "./src/data/transit.js",
|
||||
"./layers/transit/source": "./src/layers/transit/source.js",
|
||||
"./sources/transit-service": "./src/sources/transitService.js",
|
||||
"./sources/map-features": "./src/sources/overpassFeatures.js",
|
||||
"./tools": "./src/tools/index.js",
|
||||
"./tools/mcp": "./src/tools/mcp/index.js",
|
||||
"./tools/services": "./src/tools/services.js",
|
||||
"./view": "./src/view/index.js"
|
||||
"./sources/map-features": "./src/sources/overpassFeatures.js"
|
||||
}
|
||||
}
|
||||
@@ -18,8 +18,7 @@ For commercial use, obtain separate permission or exclude both this pack and the
|
||||
derived coordinate dataset in `src/data/bhoteKoshiFloodPath.js` from the source
|
||||
and build. Deleting this directory alone does not remove the compiled river
|
||||
data. Excluding the Nepal scene also requires removing its registrations and
|
||||
imports before building, including the `get_bhote_koshi_flood` tool in
|
||||
`src/tools/queries/events.js`.
|
||||
imports before building.
|
||||
|
||||
Sources:
|
||||
|
||||
|
||||
@@ -111,8 +111,8 @@ const cacheDir = path.resolve(
|
||||
);
|
||||
const only = args.includes('--only') ? args[args.indexOf('--only') + 1] : null;
|
||||
|
||||
async function fetchPinned(name) {
|
||||
const { url, sha256 } = SOURCES[name];
|
||||
export async function fetchPinned(name, source = SOURCES[name]) {
|
||||
const { url, sha256 } = source;
|
||||
const file = path.join(cacheDir, path.basename(new URL(url).pathname));
|
||||
let bytes = await readFile(file).catch(() => null);
|
||||
if (!bytes) {
|
||||
@@ -137,7 +137,7 @@ const EARTH_RADIUS_KM = 6371;
|
||||
const toRad = (d) => (d * Math.PI) / 180;
|
||||
|
||||
/** Spherical ring area (km²); matches src/data/naturalEarthRegions.js. */
|
||||
function ringAreaKm2(ring) {
|
||||
export function ringAreaKm2(ring) {
|
||||
const n = ring.length;
|
||||
if (n < 3) return 0;
|
||||
let sum = 0;
|
||||
@@ -151,7 +151,7 @@ function ringAreaKm2(ring) {
|
||||
}
|
||||
|
||||
/** Signed planar (shoelace) area: positive counter-clockwise in lon/lat. */
|
||||
function signedArea(ring) {
|
||||
export function signedArea(ring) {
|
||||
let sum = 0;
|
||||
for (let i = 0, j = ring.length - 1; i < ring.length; j = i++)
|
||||
sum += ring[j][0] * ring[i][1] - ring[i][0] * ring[j][1];
|
||||
@@ -209,15 +209,16 @@ function douglasPeucker(points, tolerance) {
|
||||
* Simplify a closed ring to an OPEN rounded ring. The ring is split at its
|
||||
* farthest vertex from the start so Douglas-Peucker has two anchored halves
|
||||
* (a closed ring has no chord to measure against). Returns null when fewer
|
||||
* than four distinct vertices survive.
|
||||
* than `minVertices` distinct vertices survive (four by default; a pack that
|
||||
* keeps triangles passes three).
|
||||
*/
|
||||
function simplifyRing(ring, tolerance, decimals) {
|
||||
function simplifyRing(ring, tolerance, decimals, minVertices = 4) {
|
||||
let open = ring.slice();
|
||||
const first = open[0];
|
||||
const last = open[open.length - 1];
|
||||
if (open.length > 1 && first[0] === last[0] && first[1] === last[1])
|
||||
open = open.slice(0, -1);
|
||||
if (open.length < 4) return null;
|
||||
if (open.length < minVertices) return null;
|
||||
let far = 1;
|
||||
let farDist = -1;
|
||||
for (let i = 1; i < open.length; i++) {
|
||||
@@ -247,7 +248,7 @@ function simplifyRing(ring, tolerance, decimals) {
|
||||
out[0][1] === out[out.length - 1][1]
|
||||
)
|
||||
out.pop();
|
||||
return out.length >= 4 ? out : null;
|
||||
return out.length >= minVertices ? out : null;
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -255,16 +256,17 @@ function simplifyRing(ring, tolerance, decimals) {
|
||||
* area floors (the largest part always stays), orient outers
|
||||
* counter-clockwise and holes clockwise, sort parts by area descending.
|
||||
*/
|
||||
function simplifyPolygons(polygons, params, tolerance, decimals) {
|
||||
export function simplifyPolygons(polygons, params, tolerance, decimals) {
|
||||
const minVertices = params.minRingVertices ?? 4;
|
||||
const parts = [];
|
||||
for (const poly of polygons) {
|
||||
const outer = simplifyRing(poly[0], tolerance, decimals);
|
||||
const outer = simplifyRing(poly[0], tolerance, decimals, minVertices);
|
||||
if (!outer) continue;
|
||||
const area = ringAreaKm2(outer);
|
||||
if (signedArea(outer) < 0) outer.reverse();
|
||||
const holes = [];
|
||||
for (const hole of poly.slice(1)) {
|
||||
const ring = simplifyRing(hole, tolerance, decimals);
|
||||
const ring = simplifyRing(hole, tolerance, decimals, minVertices);
|
||||
if (!ring || ringAreaKm2(ring) < params.minHoleKm2) continue;
|
||||
if (signedArea(ring) > 0) ring.reverse();
|
||||
holes.push(ring);
|
||||
@@ -287,18 +289,22 @@ function simplifyPolygons(polygons, params, tolerance, decimals) {
|
||||
* and a fifth of the minimum tolerance, or failing that unsimplified at two
|
||||
* more decimals; the caller records `d`.
|
||||
*/
|
||||
function simplifyUnit(polygons, params) {
|
||||
export function unitTolerance(polygons, params) {
|
||||
const sourceArea = polygons.reduce(
|
||||
(sum, poly) => sum + ringAreaKm2(poly[0]),
|
||||
0,
|
||||
);
|
||||
const tolerance = Math.min(
|
||||
return Math.min(
|
||||
params.maxToleranceDeg,
|
||||
Math.max(
|
||||
params.minToleranceDeg,
|
||||
(params.toleranceFactor * Math.sqrt(sourceArea)) / 111.32,
|
||||
),
|
||||
);
|
||||
}
|
||||
|
||||
export function simplifyUnit(polygons, params) {
|
||||
const tolerance = unitTolerance(polygons, params);
|
||||
const coarse = simplifyPolygons(polygons, params, tolerance, params.decimals);
|
||||
if (coarse.length) return { polygons: coarse, decimals: params.decimals };
|
||||
const fine = simplifyPolygons(
|
||||
@@ -334,7 +340,7 @@ function encodeRing(ring, decimals) {
|
||||
return out;
|
||||
}
|
||||
|
||||
function encodeUnit({ polygons, decimals }) {
|
||||
export function encodeUnit({ polygons, decimals }) {
|
||||
return polygons.map((poly) => poly.map((ring) => encodeRing(ring, decimals)));
|
||||
}
|
||||
|
||||
@@ -362,7 +368,7 @@ function bboxOf(polygons) {
|
||||
// ── zip / shapefile / dbf ───────────────────────────────────────────────
|
||||
|
||||
/** Read named entries from a zip archive (stored or deflated). */
|
||||
function unzip(buffer) {
|
||||
export function unzip(buffer) {
|
||||
let eocd = buffer.length - 22;
|
||||
while (eocd >= 0 && buffer.readUInt32LE(eocd) !== 0x06054b50) eocd--;
|
||||
if (eocd < 0) throw new Error('zip: end of central directory not found');
|
||||
@@ -394,7 +400,7 @@ function unzip(buffer) {
|
||||
}
|
||||
|
||||
/** Polygon records (shape type 5) as arrays of closed [lon, lat] rings. */
|
||||
function readShp(buffer) {
|
||||
export function readShp(buffer) {
|
||||
const shapes = [];
|
||||
let offset = 100;
|
||||
while (offset < buffer.length) {
|
||||
@@ -431,7 +437,7 @@ function readShp(buffer) {
|
||||
}
|
||||
|
||||
/** dBASE records as plain objects (character and numeric fields). */
|
||||
function readDbf(buffer) {
|
||||
export function readDbf(buffer) {
|
||||
const count = buffer.readUInt32LE(4);
|
||||
const headerLength = buffer.readUInt16LE(8);
|
||||
const recordLength = buffer.readUInt16LE(10);
|
||||
@@ -458,7 +464,7 @@ function readDbf(buffer) {
|
||||
}
|
||||
|
||||
/** Group shapefile rings into polygons: clockwise outers, holes assigned by containment. */
|
||||
function shpPolygons(rings) {
|
||||
export function shpPolygons(rings) {
|
||||
const outers = [];
|
||||
const holes = [];
|
||||
for (const ring of rings) {
|
||||
@@ -475,7 +481,7 @@ function shpPolygons(rings) {
|
||||
return polygons;
|
||||
}
|
||||
|
||||
function pointInRing(ring, x, y) {
|
||||
export function pointInRing(ring, x, y) {
|
||||
let inside = false;
|
||||
for (let i = 0, j = ring.length - 1; i < ring.length; j = i++) {
|
||||
const [xi, yi] = ring[i];
|
||||
@@ -742,18 +748,24 @@ async function writePack(relative, pack) {
|
||||
);
|
||||
}
|
||||
|
||||
if (!only || only === 'countries')
|
||||
await writePack(
|
||||
'src/data/local_data/natural_earth/countries.json',
|
||||
await buildCountries(),
|
||||
);
|
||||
if (!only || only === 'admin1')
|
||||
await writePack(
|
||||
'src/data/local_data/natural_earth/states_provinces.json',
|
||||
await buildAdmin1(),
|
||||
);
|
||||
if (!only || only === 'counties')
|
||||
await writePack(
|
||||
'src/data/local_data/us_census_counties/counties.json',
|
||||
await buildCounties(),
|
||||
);
|
||||
const isMain =
|
||||
process.argv[1] &&
|
||||
path.resolve(process.argv[1]) === fileURLToPath(import.meta.url);
|
||||
|
||||
if (isMain) {
|
||||
if (!only || only === 'countries')
|
||||
await writePack(
|
||||
'src/data/local_data/natural_earth/countries.json',
|
||||
await buildCountries(),
|
||||
);
|
||||
if (!only || only === 'admin1')
|
||||
await writePack(
|
||||
'src/data/local_data/natural_earth/states_provinces.json',
|
||||
await buildAdmin1(),
|
||||
);
|
||||
if (!only || only === 'counties')
|
||||
await writePack(
|
||||
'src/data/local_data/us_census_counties/counties.json',
|
||||
await buildCounties(),
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,418 @@
|
||||
#!/usr/bin/env node
|
||||
/**
|
||||
* Build the bundled US places pack read by `src/data/placeBoundaries.js`:
|
||||
*
|
||||
* src/data/local_data/us_census_places/index.json state list, boxes, meta
|
||||
* src/data/local_data/us_census_places/<ST>.json one file per state
|
||||
* src/data/local_data/us_census_places/files.js their URLs, for bundlers
|
||||
*
|
||||
* Source: US Census Bureau 2025 cartographic boundary places
|
||||
* (cb_2025_us_place_500k, public domain), pinned by URL and SHA-256, so a
|
||||
* rerun reproduces the pack byte for byte. The download is cached outside the
|
||||
* repository (default: the system temp directory).
|
||||
*
|
||||
* Geometry uses the county pack's simplification and encoding
|
||||
* (scripts/build-admin-packs.mjs) with two differences: triangles are kept,
|
||||
* and a unit whose simplified rings cross themselves or each other is
|
||||
* simplified again at half the tolerance (finer precision last). Then
|
||||
* `scripts/pack_geometry.py repair census` checks every feature as a whole
|
||||
* MultiPolygon under GEOS and repairs the ones that are still invalid (rings
|
||||
* touching themselves, overlapping parts), so it needs Python with
|
||||
* shapely==2.1.2 on GEOS 3.13.1.
|
||||
*
|
||||
* Usage:
|
||||
* node scripts/build-census-places.mjs [--cache <dir>] [--python <exe>]
|
||||
*/
|
||||
|
||||
import { execFileSync } from 'node:child_process';
|
||||
import { mkdir, readdir, rm, writeFile } from 'node:fs/promises';
|
||||
import path from 'node:path';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import {
|
||||
PARAMS as ADMIN_PARAMS,
|
||||
encodeUnit,
|
||||
fetchPinned,
|
||||
readDbf,
|
||||
readShp,
|
||||
shpPolygons,
|
||||
simplifyPolygons,
|
||||
unitTolerance,
|
||||
unzip,
|
||||
} from './build-admin-packs.mjs';
|
||||
|
||||
const ROOT = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '..');
|
||||
const OUT = path.join(ROOT, 'src/data/local_data/us_census_places');
|
||||
|
||||
export const SOURCE = Object.freeze({
|
||||
url: 'https://www2.census.gov/geo/tiger/GENZ2025/shp/cb_2025_us_place_500k.zip',
|
||||
sha256: 'ce0e4019ecd4123d03d53aaa936eed0459b82e3e14b89a3dcd4d5e8b3308627d',
|
||||
});
|
||||
|
||||
/** County-pack parameters; triangles are valid rings here. */
|
||||
export const PARAMS = Object.freeze({
|
||||
...ADMIN_PARAMS.counties,
|
||||
minRingVertices: 3,
|
||||
maxRepairSteps: 8,
|
||||
});
|
||||
|
||||
/** Census LSAD codes for census designated places (unincorporated). */
|
||||
const CDP_LSAD = new Set(['55', '57', '62']);
|
||||
|
||||
// ── validity ─────────────────────────────────────────────────────────────
|
||||
|
||||
function orient(ax, ay, bx, by, cx, cy) {
|
||||
const v = (bx - ax) * (cy - ay) - (by - ay) * (cx - ax);
|
||||
return v > 0 ? 1 : v < 0 ? -1 : 0;
|
||||
}
|
||||
|
||||
function onSegment(ax, ay, bx, by, px, py) {
|
||||
return (
|
||||
Math.min(ax, bx) <= px &&
|
||||
px <= Math.max(ax, bx) &&
|
||||
Math.min(ay, by) <= py &&
|
||||
py <= Math.max(ay, by)
|
||||
);
|
||||
}
|
||||
|
||||
/** Whether segments ab and cd share any point (touching counts). */
|
||||
export function segmentsIntersect(a, b, c, d) {
|
||||
const o1 = orient(a[0], a[1], b[0], b[1], c[0], c[1]);
|
||||
const o2 = orient(a[0], a[1], b[0], b[1], d[0], d[1]);
|
||||
const o3 = orient(c[0], c[1], d[0], d[1], a[0], a[1]);
|
||||
const o4 = orient(c[0], c[1], d[0], d[1], b[0], b[1]);
|
||||
if (o1 !== o2 && o3 !== o4) return true;
|
||||
if (o1 === 0 && onSegment(a[0], a[1], b[0], b[1], c[0], c[1])) return true;
|
||||
if (o2 === 0 && onSegment(a[0], a[1], b[0], b[1], d[0], d[1])) return true;
|
||||
if (o3 === 0 && onSegment(c[0], c[1], d[0], d[1], a[0], a[1])) return true;
|
||||
if (o4 === 0 && onSegment(c[0], c[1], d[0], d[1], b[0], b[1])) return true;
|
||||
return false;
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether segments of two different rings cross or overlap. Rings may touch
|
||||
* at a point (a hole meeting its outer ring at a vertex is valid).
|
||||
*/
|
||||
function segmentsCross(s, t) {
|
||||
const [a, b, c, d] = [s.a, s.b, t.a, t.b];
|
||||
const o1 = orient(a[0], a[1], b[0], b[1], c[0], c[1]);
|
||||
const o2 = orient(a[0], a[1], b[0], b[1], d[0], d[1]);
|
||||
const o3 = orient(c[0], c[1], d[0], d[1], a[0], a[1]);
|
||||
const o4 = orient(c[0], c[1], d[0], d[1], b[0], b[1]);
|
||||
if (o1 && o2 && o3 && o4) return o1 !== o2 && o3 !== o4;
|
||||
if (o1 === 0 && o2 === 0) {
|
||||
// Collinear: overlapping by more than a point.
|
||||
const key = a[0] !== b[0] ? 0 : 1;
|
||||
const lo = Math.max(Math.min(a[key], b[key]), Math.min(c[key], d[key]));
|
||||
const hi = Math.min(Math.max(a[key], b[key]), Math.max(c[key], d[key]));
|
||||
return hi > lo;
|
||||
}
|
||||
// One endpoint on the other segment: a crossing only when the rings pass
|
||||
// through each other there; a touch at a point is allowed.
|
||||
return false;
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether the rings of one part ([outer, ...holes], open, integer
|
||||
* coordinates) are simple: no ring crosses or touches itself away from its
|
||||
* shared vertices, no two rings cross or overlap (they may touch at a point),
|
||||
* and no ring is flat.
|
||||
*/
|
||||
export function partIsValid(rings) {
|
||||
const segments = [];
|
||||
for (let r = 0; r < rings.length; r++) {
|
||||
const ring = rings[r];
|
||||
if (ring.length < 3) return false;
|
||||
let twice = 0;
|
||||
for (let i = 0, j = ring.length - 1; i < ring.length; j = i++)
|
||||
twice += ring[j][0] * ring[i][1] - ring[i][0] * ring[j][1];
|
||||
if (twice === 0) return false;
|
||||
for (let i = 0; i < ring.length; i++) {
|
||||
const a = ring[i];
|
||||
const b = ring[(i + 1) % ring.length];
|
||||
segments.push({
|
||||
r,
|
||||
i,
|
||||
n: ring.length,
|
||||
a,
|
||||
b,
|
||||
minX: Math.min(a[0], b[0]),
|
||||
maxX: Math.max(a[0], b[0]),
|
||||
});
|
||||
}
|
||||
}
|
||||
segments.sort((s, t) => s.minX - t.minX);
|
||||
for (let x = 0; x < segments.length; x++) {
|
||||
const s = segments[x];
|
||||
for (let y = x + 1; y < segments.length; y++) {
|
||||
const t = segments[y];
|
||||
if (t.minX > s.maxX) break;
|
||||
if (s.r === t.r) {
|
||||
const gap = Math.abs(s.i - t.i);
|
||||
// Neighbours share exactly one vertex; that is not a crossing — unless
|
||||
// they fold back over each other.
|
||||
if (gap === 1 || gap === s.n - 1) {
|
||||
if (s.n > 3 && collinearOverlap(s, t)) return false;
|
||||
continue;
|
||||
}
|
||||
}
|
||||
if (segmentsCross(s, t)) return false;
|
||||
}
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
function collinearOverlap(s, t) {
|
||||
const shared =
|
||||
s.a === t.a || s.a === t.b ? s.a : s.b === t.a || s.b === t.b ? s.b : null;
|
||||
if (!shared) return false;
|
||||
const p = s.a === shared ? s.b : s.a;
|
||||
const q = t.a === shared ? t.b : t.a;
|
||||
if (orient(shared[0], shared[1], p[0], p[1], q[0], q[1]) !== 0) return false;
|
||||
// Collinear and pointing the same way from the shared vertex: a spike.
|
||||
return (
|
||||
(p[0] - shared[0]) * (q[0] - shared[0]) +
|
||||
(p[1] - shared[1]) * (q[1] - shared[1]) >
|
||||
0
|
||||
);
|
||||
}
|
||||
|
||||
function toIntegers(polygons, decimals) {
|
||||
const f = 10 ** decimals;
|
||||
return polygons.map((poly) =>
|
||||
poly.map((ring) =>
|
||||
ring.map(([lon, lat]) => [Math.round(lon * f), Math.round(lat * f)]),
|
||||
),
|
||||
);
|
||||
}
|
||||
|
||||
export function polygonsValid(polygons, decimals) {
|
||||
return toIntegers(polygons, decimals).every(partIsValid);
|
||||
}
|
||||
|
||||
/**
|
||||
* Simplify a place like `simplifyUnit`, then halve the tolerance until every
|
||||
* part is valid; then one more decimal; then the source rings at two more.
|
||||
* @returns {{polygons: Array, decimals: number, repaired: boolean, valid: boolean}}
|
||||
*/
|
||||
export function simplifyPlace(source, params = PARAMS) {
|
||||
let tolerance = unitTolerance(source, params);
|
||||
const attempts = [];
|
||||
for (let step = 0; step <= params.maxRepairSteps; step++) {
|
||||
attempts.push([tolerance, params.decimals]);
|
||||
tolerance /= 2;
|
||||
}
|
||||
attempts.push([params.minToleranceDeg / 5, params.decimals + 1]);
|
||||
attempts.push([0, params.decimals + 1]);
|
||||
attempts.push([0, params.decimals + 2]);
|
||||
let first = null;
|
||||
for (let i = 0; i < attempts.length; i++) {
|
||||
const [tol, decimals] = attempts[i];
|
||||
const polygons = simplifyPolygons(source, params, tol, decimals);
|
||||
if (!polygons.length) continue;
|
||||
first ||= { polygons, decimals };
|
||||
if (polygonsValid(polygons, decimals))
|
||||
return { polygons, decimals, repaired: i > 0, valid: true };
|
||||
}
|
||||
return first
|
||||
? { ...first, repaired: false, valid: false }
|
||||
: {
|
||||
polygons: [],
|
||||
decimals: params.decimals,
|
||||
repaired: false,
|
||||
valid: false,
|
||||
};
|
||||
}
|
||||
|
||||
// ── names ────────────────────────────────────────────────────────────────
|
||||
|
||||
const GOVERNMENT =
|
||||
/\s+(?:metropolitan government|unified government|consolidated government|metro government|city and county|urban county)\b.*$/i;
|
||||
|
||||
/**
|
||||
* Everyday names for records whose Census name is formal: "Nashville-Davidson
|
||||
* metropolitan government (balance)" is also "Nashville"; "Urban Honolulu"
|
||||
* is "Honolulu".
|
||||
*/
|
||||
export function placeAliases(name) {
|
||||
const out = new Set();
|
||||
let base = String(name).replace(/\s*\(balance\)\s*$/i, '');
|
||||
if (base !== name) out.add(base);
|
||||
const stripped = base.replace(GOVERNMENT, '');
|
||||
if (stripped !== base) {
|
||||
out.add(stripped);
|
||||
base = stripped;
|
||||
}
|
||||
if (base !== name && /[-/]/.test(base)) out.add(base.split(/[-/]/)[0].trim());
|
||||
const urban = /^Urban (.+)$/.exec(name);
|
||||
if (urban) out.add(urban[1]);
|
||||
out.delete(name);
|
||||
return [...out].filter(Boolean).sort();
|
||||
}
|
||||
|
||||
// ── pack ─────────────────────────────────────────────────────────────────
|
||||
|
||||
function bboxOf(polygons) {
|
||||
let w = Infinity;
|
||||
let s = Infinity;
|
||||
let e = -Infinity;
|
||||
let n = -Infinity;
|
||||
for (const poly of polygons)
|
||||
for (const [lon, lat] of poly[0]) {
|
||||
if (lon < w) w = lon;
|
||||
if (lon > e) e = lon;
|
||||
if (lat < s) s = lat;
|
||||
if (lat > n) n = lat;
|
||||
}
|
||||
return [w, s, e, n];
|
||||
}
|
||||
|
||||
function union(boxes) {
|
||||
return [
|
||||
Math.min(...boxes.map((b) => b[0])),
|
||||
Math.min(...boxes.map((b) => b[1])),
|
||||
Math.max(...boxes.map((b) => b[2])),
|
||||
Math.max(...boxes.map((b) => b[3])),
|
||||
];
|
||||
}
|
||||
|
||||
/**
|
||||
* The box of a state's places. Alaska crosses the antimeridian: its box is
|
||||
* taken with western longitudes shifted past 180 when that is narrower.
|
||||
*/
|
||||
function unionBox(boxes) {
|
||||
const plain = union(boxes);
|
||||
const shifted = union(
|
||||
boxes.map(([w, s, e, n]) =>
|
||||
e < 0 ? [w + 360, s, e + 360, n] : [w, s, e, n],
|
||||
),
|
||||
);
|
||||
return shifted[2] - shifted[0] < plain[2] - plain[0] ? shifted : plain;
|
||||
}
|
||||
|
||||
async function build() {
|
||||
const params = PARAMS;
|
||||
const source = await fetchPinned('places', SOURCE);
|
||||
const files = unzip(source.bytes);
|
||||
const base = path.basename(new URL(SOURCE.url).pathname, '.zip');
|
||||
const shapes = readShp(files.get(`${base}.shp`));
|
||||
const records = readDbf(files.get(`${base}.dbf`));
|
||||
if (shapes.length !== records.length)
|
||||
throw new Error('places: shp/dbf record counts differ');
|
||||
const states = new Map();
|
||||
const stats = { records: records.length, repaired: [], invalid: [] };
|
||||
records.forEach((record, index) => {
|
||||
const unit = simplifyPlace(shpPolygons(shapes[index]), params);
|
||||
if (!unit.polygons.length) throw new Error(`no geometry: ${record.GEOID}`);
|
||||
if (unit.repaired) stats.repaired.push(record.GEOID);
|
||||
if (!unit.valid) stats.invalid.push(record.GEOID);
|
||||
const alt = placeAliases(record.NAME);
|
||||
const lsad = record.NAMELSAD.startsWith(record.NAME)
|
||||
? record.NAMELSAD.slice(record.NAME.length).trim()
|
||||
: '';
|
||||
let state = states.get(record.STUSPS);
|
||||
if (!state) {
|
||||
state = {
|
||||
st: record.STUSPS,
|
||||
fips: record.STATEFP,
|
||||
name: record.STATE_NAME,
|
||||
features: [],
|
||||
boxes: [],
|
||||
};
|
||||
states.set(record.STUSPS, state);
|
||||
}
|
||||
state.boxes.push(bboxOf(unit.polygons));
|
||||
state.features.push({
|
||||
geoid: record.GEOID,
|
||||
name: record.NAME,
|
||||
...(alt.length ? { alt } : {}),
|
||||
full: record.NAMELSAD,
|
||||
...(lsad ? { lsad } : {}),
|
||||
...(CDP_LSAD.has(record.LSAD) ? { cdp: true } : {}),
|
||||
...(unit.decimals !== params.decimals ? { d: unit.decimals } : {}),
|
||||
polygons: encodeUnit(unit),
|
||||
});
|
||||
});
|
||||
if (stats.invalid.length)
|
||||
throw new Error(`places still invalid: ${stats.invalid.join(', ')}`);
|
||||
|
||||
await mkdir(OUT, { recursive: true });
|
||||
for (const file of await readdir(OUT))
|
||||
if (/^[A-Z]{2}\.json$/.test(file)) await rm(path.join(OUT, file));
|
||||
const list = [...states.values()].sort((a, b) => a.st.localeCompare(b.st));
|
||||
let bytes = 0;
|
||||
for (const state of list) {
|
||||
state.features.sort((a, b) => a.geoid.localeCompare(b.geoid));
|
||||
const text =
|
||||
JSON.stringify({
|
||||
st: state.st,
|
||||
state: state.name,
|
||||
fips: state.fips,
|
||||
features: state.features,
|
||||
}) + '\n';
|
||||
bytes += Buffer.byteLength(text);
|
||||
await writeFile(path.join(OUT, `${state.st}.json`), text);
|
||||
}
|
||||
const round = (v) => Math.round(v * 1e4) / 1e4;
|
||||
const index = {
|
||||
meta: {
|
||||
title: 'US Census Bureau cartographic boundary places',
|
||||
source: base,
|
||||
url: SOURCE.url,
|
||||
sha256: source.digest,
|
||||
license: 'Public domain (U.S. Government work, 17 U.S.C. § 105)',
|
||||
schema:
|
||||
'states[]: st (USPS), fips, name, count, bbox [w, s, e, n] (e > 180 across the antimeridian). <st>.json: st, state, fips, features[]: geoid, name (NAME), alt?[] (everyday names for formal records), full (NAMELSAD), lsad? (city, town, CDP, …), cdp? (census designated place), d? (decimals when not meta.decimals), polygons[][ring] (first ring outer, rest holes; each ring open and encoded as integers in 10^-d degrees, first vertex absolute then [dLon, dLat] deltas)',
|
||||
decimals: params.decimals,
|
||||
curation: params,
|
||||
script: 'scripts/build-census-places.mjs',
|
||||
},
|
||||
states: list.map((s) => ({
|
||||
st: s.st,
|
||||
fips: s.fips,
|
||||
name: s.name,
|
||||
count: s.features.length,
|
||||
bbox: unionBox(s.boxes).map(round),
|
||||
})),
|
||||
};
|
||||
await writeFile(
|
||||
path.join(OUT, 'index.json'),
|
||||
JSON.stringify(index, null, 1) + '\n',
|
||||
);
|
||||
await writeFile(path.join(OUT, 'files.js'), filesModule(list));
|
||||
const count = list.reduce((n, s) => n + s.features.length, 0);
|
||||
console.log(
|
||||
`us_census_places: ${count} places in ${list.length} files, ${bytes} bytes; ` +
|
||||
`${stats.repaired.length} re-simplified to stay valid`,
|
||||
);
|
||||
const args = process.argv.slice(2);
|
||||
const python = args.includes('--python')
|
||||
? args[args.indexOf('--python') + 1]
|
||||
: 'python3';
|
||||
execFileSync(
|
||||
python,
|
||||
[path.join(ROOT, 'scripts/pack_geometry.py'), 'repair', 'census'],
|
||||
{ stdio: 'inherit', env: { ...process.env, PYTHONDONTWRITEBYTECODE: '1' } },
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* The file list as literal URLs, so a bundler emits every state file as an
|
||||
* asset and the loader can fetch one by its code.
|
||||
*/
|
||||
function filesModule(list) {
|
||||
const lines = [
|
||||
'// Generated by scripts/build-census-places.mjs; do not edit.',
|
||||
"export const INDEX_URL = new URL('./index.json', import.meta.url);",
|
||||
'export const STATE_URLS = {',
|
||||
...list.map(
|
||||
(s) => ` ${s.st}: new URL('./${s.st}.json', import.meta.url),`,
|
||||
),
|
||||
'};',
|
||||
];
|
||||
return lines.join('\n') + '\n';
|
||||
}
|
||||
|
||||
const isMain =
|
||||
process.argv[1] &&
|
||||
path.resolve(process.argv[1]) === fileURLToPath(import.meta.url);
|
||||
if (isMain) await build();
|
||||
@@ -1,46 +0,0 @@
|
||||
#!/usr/bin/env node
|
||||
/**
|
||||
* Build the MCP Apps panel: the app under dist/panel, served at /panel/.
|
||||
* See build/panel.js.
|
||||
*/
|
||||
|
||||
import { readFile, rename, rm, writeFile } from 'node:fs/promises';
|
||||
import { join } from 'node:path';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import { build } from 'vite';
|
||||
import standaloneConfig from '../server/standalone/vite.config.js';
|
||||
import {
|
||||
PANEL_BASE,
|
||||
PANEL_OUT_DIR,
|
||||
PANEL_WORKER_FILES,
|
||||
PANEL_WORKER_PRELUDE_PATH,
|
||||
panelBuildConfig,
|
||||
workerFilesPrelude,
|
||||
} from '../build/panel.js';
|
||||
|
||||
const root = fileURLToPath(new URL('../', import.meta.url));
|
||||
const outDir = join(root, PANEL_OUT_DIR);
|
||||
|
||||
const config = standaloneConfig({ command: 'build', mode: 'production' });
|
||||
await build({
|
||||
configFile: false,
|
||||
root,
|
||||
...panelBuildConfig(config),
|
||||
});
|
||||
|
||||
// Cesium's plugin copies its files under the base path inside outDir.
|
||||
const nested = join(outDir, PANEL_BASE, 'cesium');
|
||||
await rm(join(outDir, 'cesium'), { recursive: true, force: true });
|
||||
await rename(nested, join(outDir, 'cesium'));
|
||||
await rm(join(outDir, PANEL_BASE.split('/')[1]), { recursive: true });
|
||||
|
||||
// Cesium's workers load these files themselves; embed them in a prelude the
|
||||
// panel runs ahead of Cesium's workers script.
|
||||
const files = {};
|
||||
for (const name of PANEL_WORKER_FILES)
|
||||
files[name] = await readFile(join(outDir, 'cesium', name), 'utf8');
|
||||
await writeFile(
|
||||
join(outDir, PANEL_WORKER_PRELUDE_PATH),
|
||||
workerFilesPrelude(files),
|
||||
);
|
||||
console.log(`Panel build written to ${PANEL_OUT_DIR}`);
|
||||
@@ -0,0 +1,382 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Maintainer-only Who's On First neighborhood pack build (Shapely 2.1).
|
||||
|
||||
Reads the pinned Who's On First neighbourhood, macrohood and microhood
|
||||
archives and the WOF sources registry, keeps current polygon records whose
|
||||
geometry source matches the reviewed open-licence allowlist, simplifies them
|
||||
to about 15 m, and
|
||||
writes src/data/local_data/wof_neighborhoods/:
|
||||
|
||||
index.json format, licences per source, tile list with boxes
|
||||
<tile>.json records whose box centre falls in that quadtree tile
|
||||
files.js their URLs, for bundlers
|
||||
ATTRIBUTION.md per-source credits
|
||||
|
||||
Usage:
|
||||
python scripts/build-wof-neighborhoods.py --cache /tmp/wof-neighborhoods [--verify]
|
||||
|
||||
Requires shapely==2.1.2 on GEOS 3.13.1 (checked). --verify rebuilds in memory
|
||||
and fails unless every shipped file matches byte for byte.
|
||||
|
||||
Downloads go to --cache and are checked against the pinned SHA-256. The
|
||||
neighbourhood archive is about 400 MB.
|
||||
"""
|
||||
import argparse
|
||||
import collections
|
||||
import hashlib
|
||||
import json
|
||||
import math
|
||||
import subprocess
|
||||
import tarfile
|
||||
from pathlib import Path
|
||||
|
||||
from shapely import make_valid
|
||||
from shapely.affinity import scale
|
||||
from shapely.geometry import MultiPolygon, shape
|
||||
|
||||
import pack_geometry
|
||||
|
||||
ROOT = Path(__file__).resolve().parent.parent
|
||||
PACK = ROOT / 'src/data/local_data/wof_neighborhoods'
|
||||
# The legacy distribution is frozen (last published 2025-10-14) and only
|
||||
# offers `-latest` names; the SHA-256 pins make a changed file fail loudly.
|
||||
DIST = 'https://data.geocode.earth/wof/dist/legacy'
|
||||
ARCHIVES = {
|
||||
'neighbourhood': {
|
||||
'url': DIST + '/whosonfirst-data-neighbourhood-latest.tar.bz2',
|
||||
'sha256': '8e5e413aacf2244e4e4167fbd9be860bc407ae6de8b733e47f13abe6d3d10455',
|
||||
},
|
||||
'macrohood': {
|
||||
'url': DIST + '/whosonfirst-data-macrohood-latest.tar.bz2',
|
||||
'sha256': 'f5fbfabcb8229b83242dbd38530c27e61867e8b3b097e3393d9c841962a99679',
|
||||
},
|
||||
'microhood': {
|
||||
'url': DIST + '/whosonfirst-data-microhood-latest.tar.bz2',
|
||||
'sha256': 'e8197fa576e8831be8f091ac7e219d29224d89404ab0c56d377339e42a8f29ac',
|
||||
},
|
||||
}
|
||||
# The registry at a fixed commit.
|
||||
SOURCES = {
|
||||
'url': 'https://raw.githubusercontent.com/whosonfirst/whosonfirst-sources/'
|
||||
'e17af153c144fe6e08aa8db2607b4bf8aecd448d/data/sources-spec-latest.json',
|
||||
'sha256': 'd8a5dd873d26eee11583de4438185c046dec33848d5bde27047ca4bff44d9c39',
|
||||
}
|
||||
TOLERANCE_M = 15
|
||||
PRECISION = 5
|
||||
TILE_MAX_BYTES = 200_000
|
||||
TILE_MAX_DEPTH = 16
|
||||
LICENSE_POLICY = json.loads((ROOT / 'scripts/wof-open-licenses.json').read_text())
|
||||
ALLOWED_LICENSES = frozenset(LICENSE_POLICY['licenses'])
|
||||
REVIEWED_SOURCES = LICENSE_POLICY.get('reviewedSources', {})
|
||||
FIELDS = ['id', 'name', 'country', 'type', 'bbox', 'aliases', 'source', 'rings', 'label']
|
||||
|
||||
|
||||
def fetch(cache, name, url, sha256):
|
||||
path = cache / name
|
||||
if not path.exists():
|
||||
print('Downloading', url, flush=True)
|
||||
# Plain urllib requests are refused by the WOF host; curl is not.
|
||||
subprocess.run(['curl', '--fail', '-L', '-o', str(path), url], check=True)
|
||||
digest = hashlib.sha256()
|
||||
with open(path, 'rb') as handle:
|
||||
for chunk in iter(lambda: handle.read(1 << 20), b''):
|
||||
digest.update(chunk)
|
||||
if digest.hexdigest() != sha256:
|
||||
raise ValueError(f'{name}: SHA-256 {digest.hexdigest()} is not the pinned {sha256}')
|
||||
return path
|
||||
|
||||
|
||||
# ── geometry ────────────────────────────────────────────────────────────
|
||||
|
||||
def simplify(geometry):
|
||||
"""Simplify at TOLERANCE_M in local metres, snap to the pack grid, keep it
|
||||
valid, and order parts largest first. Returns (rings, label) where the
|
||||
label is a point inside the largest part, or ([], None)."""
|
||||
geometry = MultiPolygon(pack_geometry.polygon_parts(make_valid(geometry) if not geometry.is_valid else geometry))
|
||||
if geometry.is_empty:
|
||||
return [], None
|
||||
lat = geometry.centroid.y
|
||||
sx = 111320 * max(0.001, math.cos(math.radians(lat)))
|
||||
sy = 110574
|
||||
metric = scale(geometry, xfact=sx, yfact=sy, origin=(0, 0))
|
||||
metric = metric.simplify(TOLERANCE_M, preserve_topology=True)
|
||||
geometry = scale(metric, xfact=1 / sx, yfact=1 / sy, origin=(0, 0))
|
||||
# To whole pack units (10^-PRECISION degrees), where GEOS is exact.
|
||||
units = scale(geometry, xfact=10 ** PRECISION, yfact=10 ** PRECISION, origin=(0, 0))
|
||||
parts = pack_geometry.ordered_parts(pack_geometry.settle(units))
|
||||
if not parts:
|
||||
return [], None
|
||||
rings = [
|
||||
[pack_geometry.encode_polyline(pack_geometry.open_ring(p.exterior.coords))]
|
||||
+ [pack_geometry.encode_polyline(pack_geometry.open_ring(r.coords)) for r in p.interiors]
|
||||
for p in parts
|
||||
]
|
||||
# Check what ships: the geometry decoded from the encoded strings.
|
||||
decoded = [[pack_geometry.decode_polyline(r) for r in poly] for poly in rings]
|
||||
why = pack_geometry.problems(decoded)
|
||||
if why:
|
||||
raise ValueError(f'encoded geometry invalid: {why}')
|
||||
point = parts[0].representative_point()
|
||||
point = shape({'type': 'Point', 'coordinates': [round(point.x) / 10 ** PRECISION, round(point.y) / 10 ** PRECISION]})
|
||||
return rings, [point.x, point.y]
|
||||
|
||||
|
||||
# ── records ─────────────────────────────────────────────────────────────
|
||||
|
||||
def read_archive(path, records):
|
||||
with tarfile.open(path, 'r|bz2') as archive:
|
||||
for member in archive:
|
||||
if not member.name.endswith('.geojson') or '-alt-' in member.name:
|
||||
continue
|
||||
feature = json.load(archive.extractfile(member))
|
||||
if feature['geometry']['type'] not in ('Polygon', 'MultiPolygon'):
|
||||
records.pop(feature['properties']['wof:id'], None)
|
||||
continue
|
||||
records[feature['properties']['wof:id']] = feature
|
||||
|
||||
|
||||
def keep(properties):
|
||||
deprecated = properties.get('edtf:deprecated', 'uuuu') not in ('uuuu', '', None)
|
||||
superseded = bool(properties.get('wof:superseded_by', []))
|
||||
return properties.get('mz:is_current', -1) != 0 and not deprecated and not superseded
|
||||
|
||||
|
||||
def source_table(spec):
|
||||
lookup = {}
|
||||
for source in spec.values():
|
||||
for key in (source.get('name'), source.get('prefix')):
|
||||
if key:
|
||||
lookup[key] = source
|
||||
return lookup
|
||||
|
||||
|
||||
def verify_source_reviews(lookup):
|
||||
"""Pin exceptional source decisions to the exact reviewed registry evidence."""
|
||||
registry_commit = SOURCES['url'].split('/')[-3]
|
||||
for key, review in REVIEWED_SOURCES.items():
|
||||
source = lookup.get(key)
|
||||
if not source:
|
||||
raise ValueError(f'{key}: reviewed source is absent from the pinned registry')
|
||||
if review.get('registryCommit') != registry_commit:
|
||||
raise ValueError(f'{key}: review does not match the pinned registry commit')
|
||||
if review.get('registrySha256') != SOURCES['sha256']:
|
||||
raise ValueError(f'{key}: review does not match the pinned registry SHA-256')
|
||||
if review.get('rawLicenseType') != source.get('license_type'):
|
||||
raise ValueError(f'{key}: reviewed raw licence no longer matches the registry')
|
||||
text = source.get('license_text') or ''
|
||||
if review.get('licenseTextSha256') != hashlib.sha256(text.encode()).hexdigest():
|
||||
raise ValueError(f'{key}: reviewed licence text no longer matches the registry')
|
||||
if review['rawLicenseType'] not in ALLOWED_LICENSES:
|
||||
raise ValueError(f'{key}: reviewed raw licence is not admitted by policy')
|
||||
|
||||
|
||||
def licence_allowed(value):
|
||||
"""Only exact raw registry strings reviewed in the policy are accepted."""
|
||||
return isinstance(value, str) and value in ALLOWED_LICENSES
|
||||
|
||||
|
||||
def source_allowed(key, source):
|
||||
return (key != 'unknown' and bool(source) and
|
||||
licence_allowed(source.get('license_type')))
|
||||
|
||||
|
||||
def build_records(archives, lookup):
|
||||
raw = {}
|
||||
for kind in ('neighbourhood', 'macrohood', 'microhood'):
|
||||
read_archive(archives[kind], raw)
|
||||
print(kind, len(raw), flush=True)
|
||||
features = []
|
||||
excluded = collections.Counter()
|
||||
for wof_id in sorted(raw):
|
||||
properties = raw[wof_id]['properties']
|
||||
if not keep(properties):
|
||||
continue
|
||||
key = properties.get('src:geom', 'unknown')
|
||||
if not source_allowed(key, lookup.get(key)):
|
||||
excluded[key] += 1
|
||||
continue
|
||||
name = properties['wof:name']
|
||||
aliases = sorted({
|
||||
alias
|
||||
for k, values in properties.items()
|
||||
if k.startswith('name:') and isinstance(values, list)
|
||||
for alias in values
|
||||
if isinstance(alias, str) and alias != name
|
||||
})
|
||||
if not name.strip() and not aliases:
|
||||
continue # nothing to ask for it by
|
||||
geometry = shape(raw[wof_id]['geometry'])
|
||||
if geometry.is_empty:
|
||||
continue
|
||||
rings, label = simplify(geometry)
|
||||
if not rings:
|
||||
continue
|
||||
country = properties.get('iso:country', properties.get('wof:country'))
|
||||
features.append([
|
||||
wof_id, name, country, properties['wof:placetype'],
|
||||
[round(v, PRECISION) for v in geometry.bounds], aliases, key, rings, label,
|
||||
])
|
||||
return features, excluded
|
||||
|
||||
|
||||
# ── tiles ───────────────────────────────────────────────────────────────
|
||||
|
||||
def compact(value):
|
||||
return json.dumps(value, ensure_ascii=False, separators=(',', ':'))
|
||||
|
||||
|
||||
def tiles(features, key='', box=(-180.0, -90.0, 180.0, 90.0)):
|
||||
size = sum(len(compact(f).encode()) for f in features)
|
||||
if size <= TILE_MAX_BYTES or len(key) >= TILE_MAX_DEPTH:
|
||||
yield key or 'world', features
|
||||
return
|
||||
west, south, east, north = box
|
||||
mx, my = (west + east) / 2, (south + north) / 2
|
||||
quads = [[], [], [], []]
|
||||
for f in features:
|
||||
x = (f[4][0] + f[4][2]) / 2
|
||||
y = (f[4][1] + f[4][3]) / 2
|
||||
quads[(1 if x >= mx else 0) + (2 if y >= my else 0)].append(f)
|
||||
boxes = [(west, south, mx, my), (mx, south, east, my), (west, my, mx, north), (mx, my, east, north)]
|
||||
for i, quad in enumerate(quads):
|
||||
if quad:
|
||||
yield from tiles(quad, key + str(i), boxes[i])
|
||||
|
||||
|
||||
def union_box(features):
|
||||
return [
|
||||
min(f[4][0] for f in features), min(f[4][1] for f in features),
|
||||
max(f[4][2] for f in features), max(f[4][3] for f in features),
|
||||
]
|
||||
|
||||
|
||||
# ── credits ─────────────────────────────────────────────────────────────
|
||||
|
||||
def licence_label(key, source):
|
||||
licence = source['license_type']
|
||||
if key == 'karmashapes':
|
||||
return 'CC BY 4.0'
|
||||
return {
|
||||
'CC-BY': 'CC BY', 'Attribution 3.0 Unported': 'CC BY 3.0', 'Public domain': 'Public Domain',
|
||||
'Open Government License - British Columbia, v2.0': 'Open Government Licence - British Columbia, v2.0',
|
||||
}.get(licence, licence)
|
||||
|
||||
|
||||
def attribution(counts, lookup):
|
||||
lines = [
|
||||
'# Neighborhood boundary credits',
|
||||
'',
|
||||
"Neighborhood boundaries from [Who's On First](https://whosonfirst.org/docs/licenses/) "
|
||||
'and contributors, used under the licences below. Modified: boundaries simplified to about 15 m, repaired and re-encoded.',
|
||||
'',
|
||||
'| Source | Records | Licence | Terms |',
|
||||
'|---|---:|---|---|',
|
||||
]
|
||||
for key, count in sorted(counts.items(), key=lambda kv: (-kv[1], kv[0])):
|
||||
source = lookup[key]
|
||||
name = source.get('fullname') or key
|
||||
terms = source.get('license') or ''
|
||||
link = f'[terms]({terms})' if terms.startswith('http') else terms
|
||||
lines.append(f'| {name} (`{key}`) | {count:,} | {licence_label(key, source)} | {link} |')
|
||||
via = [(key, lookup[key]) for key in sorted(counts) if lookup[key].get('src:via')]
|
||||
if via:
|
||||
lines += ['', '## Upstream sources', '']
|
||||
for key, source in via:
|
||||
lines += [f'### {source.get("fullname") or key}', '']
|
||||
for v in source['src:via']:
|
||||
note = f' ({v["source_note"]})' if v.get('source_note') else ''
|
||||
context = f'{v["context"]}: ' if v.get('context') else ''
|
||||
lines.append(f'- {context}{v.get("source_name", "")}{note} {v.get("source_link", "")}'.rstrip())
|
||||
lines.append('')
|
||||
return '\n'.join(lines).rstrip() + '\n'
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
parser.add_argument('--cache', type=Path, required=True)
|
||||
parser.add_argument('--verify', action='store_true', help='rebuild in memory and compare with the shipped pack')
|
||||
args = parser.parse_args()
|
||||
pack_geometry.require_pinned_tooling()
|
||||
args.cache.mkdir(parents=True, exist_ok=True)
|
||||
archives = {
|
||||
kind: fetch(args.cache, Path(a['url']).name, a['url'], a['sha256'])
|
||||
for kind, a in ARCHIVES.items()
|
||||
}
|
||||
spec = json.loads(fetch(args.cache, 'sources-spec-latest.json', SOURCES['url'], SOURCES['sha256']).read_text())
|
||||
lookup = source_table(spec)
|
||||
verify_source_reviews(lookup)
|
||||
features, excluded = build_records(archives, lookup)
|
||||
if len({f[0] for f in features}) != len(features):
|
||||
raise ValueError('duplicate WOF id')
|
||||
|
||||
out = {}
|
||||
tile_list = []
|
||||
for key, members in tiles(features):
|
||||
members.sort(key=lambda f: f[0])
|
||||
out[f'{key}.json'] = compact({'tile': key, 'features': members}) + '\n'
|
||||
tile_list.append({'key': key, 'count': len(members), 'bbox': union_box(members)})
|
||||
counts = collections.Counter(f[6] for f in features)
|
||||
index = {
|
||||
'meta': {
|
||||
'title': "Who's On First neighbourhood, macrohood and microhood polygons",
|
||||
'archives': ARCHIVES,
|
||||
'sourcesRegistry': SOURCES,
|
||||
'license': 'Per source; see sources and ATTRIBUTION.md',
|
||||
'format': 'neighborhood-polylines-v1',
|
||||
'fields': FIELDS,
|
||||
'precision': PRECISION,
|
||||
'coordinateOrder': 'lon,lat',
|
||||
'toleranceM': TOLERANCE_M,
|
||||
'schema': (
|
||||
'tiles[]: key, count, bbox [w, s, e, n] of its records. <key>.json: tile, features[] as '
|
||||
'fields: WOF id, name, ISO country, placetype, bbox, aliases[] (other-language names), '
|
||||
'geometry source key, rings (polygons, largest first, each [outer, ...holes]; each ring '
|
||||
'an encoded polyline string: zigzag varint deltas of lon then lat at 10^-precision '
|
||||
'degrees), label [lon, lat] (a point inside the largest polygon)'
|
||||
),
|
||||
'sources': {
|
||||
key: {
|
||||
'name': lookup[key].get('fullname') or key,
|
||||
'license': licence_label(key, lookup[key]),
|
||||
'terms': lookup[key].get('license') or '',
|
||||
'records': counts[key],
|
||||
}
|
||||
for key in sorted(counts)
|
||||
},
|
||||
'script': 'scripts/build-wof-neighborhoods.py',
|
||||
},
|
||||
'tiles': tile_list,
|
||||
}
|
||||
out['index.json'] = json.dumps(index, ensure_ascii=False, indent=1) + '\n'
|
||||
out['ATTRIBUTION.md'] = attribution(counts, lookup)
|
||||
# Literal URLs, so a bundler emits every tile as an asset.
|
||||
out['files.js'] = '\n'.join([
|
||||
'// Generated by scripts/build-wof-neighborhoods.py; do not edit.',
|
||||
"export const INDEX_URL = new URL('./index.json', import.meta.url);",
|
||||
'export const TILE_URLS = {',
|
||||
*[f" t{t['key']}: new URL('./{t['key']}.json', import.meta.url)," for t in tile_list],
|
||||
'};',
|
||||
]) + '\n'
|
||||
if args.verify:
|
||||
# Reproducibility: a rebuild must match what ships, byte for byte.
|
||||
shipped = {p.name for p in PACK.glob('*.json')} | {'ATTRIBUTION.md', 'files.js'}
|
||||
differ = sorted(
|
||||
name for name in shipped | set(out)
|
||||
if not (PACK / name).exists() or name not in out
|
||||
or (PACK / name).read_bytes() != out[name].encode('utf-8')
|
||||
)
|
||||
if differ:
|
||||
raise SystemExit(f'rebuild differs from the shipped pack: {differ[:10]}')
|
||||
print(f'wof_neighborhoods: rebuild matches all {len(out)} shipped files')
|
||||
return
|
||||
PACK.mkdir(parents=True, exist_ok=True)
|
||||
for old in PACK.glob('*.json'):
|
||||
old.unlink()
|
||||
for name, text in out.items():
|
||||
(PACK / name).write_text(text, encoding='utf-8')
|
||||
print(f'wof_neighborhoods: {len(features)} records in {len(tile_list)} tiles; '
|
||||
f'excluded {sum(excluded.values())} with an unapproved licence: {dict(excluded)}', flush=True)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
@@ -37,10 +37,6 @@ const portableExport = (key) =>
|
||||
) ||
|
||||
[
|
||||
'./director',
|
||||
'./tools',
|
||||
'./tools/mcp',
|
||||
'./tools/services',
|
||||
'./view',
|
||||
'./voice/action-schemas',
|
||||
'./voice/session',
|
||||
'./data/lifecycle',
|
||||
|
||||
@@ -30,6 +30,11 @@
|
||||
"src/annotations/regionRing.test.mjs",
|
||||
"src/data/adminBoundaries.test.mjs",
|
||||
"scripts/build-admin-packs.mjs",
|
||||
"scripts/build-census-places.mjs",
|
||||
"src/data/placeBoundaries.test.mjs",
|
||||
"src/data/boundaryLabels.test.mjs",
|
||||
"src/data/packGeometry.test.mjs",
|
||||
"src/annotations/placeOutline.test.mjs",
|
||||
"scripts/qa-admin-outlines.mjs",
|
||||
"src/data/inputOwnership.test.mjs",
|
||||
"docs/CODE-BOUNDARIES.md",
|
||||
@@ -864,33 +869,5 @@
|
||||
"src/layers/traffic/motion.test.mjs",
|
||||
"scripts/qa-journey-recorder.mjs",
|
||||
"scripts/qa-alpr-journey.mjs",
|
||||
"scripts/qa-context-arrivals.mjs",
|
||||
"src/tools/catalog.test.mjs",
|
||||
"src/tools/area.test.mjs",
|
||||
"src/tools/queries/queries.test.mjs",
|
||||
"src/tools/mcp/mcp.test.mjs",
|
||||
"src/tools/localServer.test.mjs",
|
||||
"src/tools/queries/aviation.test.mjs",
|
||||
"src/tools/queries/space.test.mjs",
|
||||
"src/tools/queries/media.test.mjs",
|
||||
"src/tools/queries/places.test.mjs",
|
||||
"src/layers/transit/source.test.mjs",
|
||||
"src/tools/queries/mobility.test.mjs",
|
||||
"src/tools/queries/environment.test.mjs",
|
||||
"src/tools/queries/maritime.test.mjs",
|
||||
"src/tools/queries/share.test.mjs",
|
||||
"src/tools/localRoute.test.mjs",
|
||||
"src/tools/queries/atmosphere.test.mjs",
|
||||
"src/tools/queries/imagery.test.mjs",
|
||||
"src/tools/queries/cables.test.mjs",
|
||||
"src/tools/queries/surveillance.test.mjs",
|
||||
"src/tools/queries/infrastructure.test.mjs",
|
||||
"src/tools/queries/events.test.mjs",
|
||||
"src/tools/functions.test.mjs",
|
||||
"src/tools/surfaces.test.mjs",
|
||||
"src/tools/places.test.mjs",
|
||||
"src/view/index.test.mjs",
|
||||
"src/tools/views.test.mjs",
|
||||
"src/app/embed.test.mjs",
|
||||
"src/tools/globePanel.test.mjs"
|
||||
"scripts/qa-context-arrivals.mjs"
|
||||
]
|
||||
@@ -0,0 +1,247 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Maintainer-only topology check and repair for the bundled outline packs.
|
||||
|
||||
python scripts/pack_geometry.py check census|wof every shipped feature,
|
||||
decoded as a whole MultiPolygon, is valid under GEOS, keeps at least
|
||||
three distinct vertices per ring, and lists its largest part first
|
||||
python scripts/pack_geometry.py repair census rewrite the Census place
|
||||
files so every feature passes `check` (run by build-census-places.mjs)
|
||||
|
||||
Geometry is checked and repaired in the packs' own integer units (10^-d
|
||||
degrees), where GEOS arithmetic is exact: a vertex that lies on another edge
|
||||
after rounding is seen as the touch it is.
|
||||
|
||||
Requires Shapely 2.1.2 on GEOS 3.13.1, pinned so repairs are reproducible.
|
||||
"""
|
||||
import argparse
|
||||
import json
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
import shapely
|
||||
from shapely import make_valid, set_precision
|
||||
from shapely.geometry import MultiPolygon, Polygon
|
||||
from shapely.geometry.polygon import orient
|
||||
|
||||
ROOT = Path(__file__).resolve().parent.parent
|
||||
DATA = ROOT / 'src/data/local_data'
|
||||
SHAPELY_VERSION = '2.1.2'
|
||||
GEOS_VERSION = (3, 13, 1)
|
||||
|
||||
|
||||
def require_pinned_tooling():
|
||||
if shapely.__version__ != SHAPELY_VERSION or shapely.geos_version != GEOS_VERSION:
|
||||
raise RuntimeError(
|
||||
f'Use shapely=={SHAPELY_VERSION} on GEOS {".".join(map(str, GEOS_VERSION))}; '
|
||||
f'found {shapely.__version__} on GEOS {shapely.geos_version}'
|
||||
)
|
||||
|
||||
|
||||
def compact(value):
|
||||
return json.dumps(value, ensure_ascii=False, separators=(',', ':'))
|
||||
|
||||
|
||||
# ── encodings ────────────────────────────────────────────────────────────
|
||||
|
||||
def decode_deltas(encoded):
|
||||
"""[x0, y0, dx1, dy1, …] → [(x, y), …] in pack units (Census/admin packs)."""
|
||||
x = y = 0
|
||||
ring = []
|
||||
for i in range(0, len(encoded) - 1, 2):
|
||||
x += encoded[i]
|
||||
y += encoded[i + 1]
|
||||
ring.append((x, y))
|
||||
return ring
|
||||
|
||||
|
||||
def encode_deltas(coords):
|
||||
out = []
|
||||
px = py = 0
|
||||
for fx, fy in coords:
|
||||
x, y = round(fx), round(fy)
|
||||
out += [x - px, y - py]
|
||||
px, py = x, y
|
||||
return out
|
||||
|
||||
|
||||
def decode_polyline(encoded):
|
||||
"""Zigzag varint deltas, 5-bit groups offset by 63, lon then lat; pack units."""
|
||||
values = []
|
||||
last = [0, 0]
|
||||
i = 0
|
||||
while i < len(encoded):
|
||||
n = shift = 0
|
||||
while True:
|
||||
b = ord(encoded[i]) - 63
|
||||
i += 1
|
||||
n |= (b & 31) << shift
|
||||
shift += 5
|
||||
if b < 32:
|
||||
break
|
||||
axis = len(values) % 2
|
||||
last[axis] += ~(n >> 1) if n & 1 else n >> 1
|
||||
values.append(last[axis])
|
||||
return list(zip(values[::2], values[1::2]))
|
||||
|
||||
|
||||
def encode_polyline(coords):
|
||||
out = []
|
||||
last = [0, 0]
|
||||
for x, y in coords:
|
||||
for i, value in enumerate([round(x), round(y)]):
|
||||
n = value - last[i]
|
||||
last[i] = value
|
||||
n = (n << 1) ^ (n >> 63)
|
||||
while n >= 32:
|
||||
out.append(chr((32 | (n & 31)) + 63))
|
||||
n >>= 5
|
||||
out.append(chr(n + 63))
|
||||
return ''.join(out)
|
||||
|
||||
|
||||
# ── geometry ─────────────────────────────────────────────────────────────
|
||||
|
||||
def polygon_parts(geometry):
|
||||
if geometry.geom_type == 'Polygon':
|
||||
return [] if geometry.is_empty else [geometry]
|
||||
if hasattr(geometry, 'geoms'):
|
||||
return [p for g in geometry.geoms for p in polygon_parts(g)]
|
||||
return []
|
||||
|
||||
|
||||
def open_ring(coords):
|
||||
coords = [tuple(c[:2]) for c in coords]
|
||||
return coords[:-1] if len(coords) > 1 and coords[0] == coords[-1] else coords
|
||||
|
||||
|
||||
def ordered_parts(geometry):
|
||||
"""Polygon parts oriented (outer counter-clockwise), largest area first."""
|
||||
parts = [orient(p, 1.0) for p in polygon_parts(geometry)]
|
||||
return sorted(parts, key=lambda p: (-p.area, p.bounds))
|
||||
|
||||
|
||||
def on_grid(geometry):
|
||||
"""Snap to whole pack units keeping the result valid; polygon parts only."""
|
||||
snapped = set_precision(geometry, 1, mode='valid_output')
|
||||
return MultiPolygon(polygon_parts(snapped))
|
||||
|
||||
|
||||
def settle(geometry):
|
||||
"""A valid MultiPolygon of `geometry` (in pack units) on whole units."""
|
||||
if not geometry.is_valid:
|
||||
geometry = make_valid(geometry)
|
||||
geometry = on_grid(MultiPolygon(polygon_parts(geometry)))
|
||||
for _ in range(3):
|
||||
if geometry.is_valid:
|
||||
break
|
||||
geometry = on_grid(make_valid(geometry))
|
||||
return geometry
|
||||
|
||||
|
||||
def rings_ok(polygons):
|
||||
return all(len(set(ring)) >= 3 for poly in polygons for ring in poly)
|
||||
|
||||
|
||||
def problems(polygons):
|
||||
"""Why decoded `polygons` ([[outer, *holes], …]) fail the pack contract."""
|
||||
if not polygons or not rings_ok(polygons):
|
||||
return 'ring with fewer than three distinct vertices'
|
||||
try:
|
||||
geometry = MultiPolygon([Polygon(p[0], p[1:]) for p in polygons])
|
||||
except Exception as error: # noqa: BLE001 - malformed input is a finding
|
||||
return f'unbuildable: {error}'
|
||||
if not geometry.is_valid:
|
||||
return shapely.is_valid_reason(geometry)
|
||||
areas = [part.area for part in geometry.geoms]
|
||||
if max(areas) > areas[0]:
|
||||
return 'largest part is not first'
|
||||
return None
|
||||
|
||||
|
||||
# ── packs ────────────────────────────────────────────────────────────────
|
||||
|
||||
def census_features():
|
||||
folder = DATA / 'us_census_places'
|
||||
decimals = json.loads((folder / 'index.json').read_text())['meta']['decimals']
|
||||
for path in sorted(folder.glob('[A-Z][A-Z].json')):
|
||||
pack = json.loads(path.read_text(encoding='utf-8'))
|
||||
yield path, pack, decimals
|
||||
|
||||
|
||||
def census_polygons(feature):
|
||||
return [[decode_deltas(r) for r in poly] for poly in feature['polygons']]
|
||||
|
||||
|
||||
def wof_features():
|
||||
folder = DATA / 'wof_neighborhoods'
|
||||
index = json.loads((folder / 'index.json').read_text(encoding='utf-8'))
|
||||
for tile in index['tiles']:
|
||||
pack = json.loads((folder / f'{tile["key"]}.json').read_text(encoding='utf-8'))
|
||||
for feature in pack['features']:
|
||||
yield feature, index['meta']['precision']
|
||||
|
||||
|
||||
def check(pack):
|
||||
bad = []
|
||||
total = 0
|
||||
if pack == 'census':
|
||||
for _, data, decimals in census_features():
|
||||
for feature in data['features']:
|
||||
total += 1
|
||||
why = problems(census_polygons(feature))
|
||||
if why:
|
||||
bad.append((feature['geoid'], feature['name'], why))
|
||||
else:
|
||||
for feature, precision in wof_features():
|
||||
total += 1
|
||||
polygons = [[decode_polyline(r) for r in poly] for poly in feature[7]]
|
||||
why = problems(polygons)
|
||||
if why:
|
||||
bad.append((feature[0], feature[1], why))
|
||||
for row in bad[:20]:
|
||||
print('invalid', *row)
|
||||
print(f'{pack}: {total} features, {len(bad)} failing')
|
||||
return not bad
|
||||
|
||||
|
||||
def repair_census():
|
||||
repaired = 0
|
||||
for path, data, decimals in census_features():
|
||||
changed = False
|
||||
for feature in data['features']:
|
||||
polygons = census_polygons(feature)
|
||||
if not problems(polygons):
|
||||
continue
|
||||
geometry = MultiPolygon([Polygon(p[0], p[1:]) for p in polygons if len(set(p[0])) >= 3])
|
||||
parts = ordered_parts(settle(geometry))
|
||||
feature['polygons'] = [
|
||||
[encode_deltas(open_ring(p.exterior.coords))]
|
||||
+ [encode_deltas(open_ring(r.coords)) for r in p.interiors]
|
||||
for p in parts
|
||||
]
|
||||
why = problems(census_polygons(feature))
|
||||
if why:
|
||||
raise ValueError(f'{feature["geoid"]} {feature["name"]}: still {why}')
|
||||
repaired += 1
|
||||
changed = True
|
||||
if changed:
|
||||
path.write_text(compact(data) + '\n', encoding='utf-8')
|
||||
print(f'us_census_places: {repaired} features repaired to valid GEOS topology')
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
parser.add_argument('command', choices=['check', 'repair'])
|
||||
parser.add_argument('pack', choices=['census', 'wof'])
|
||||
args = parser.parse_args()
|
||||
require_pinned_tooling()
|
||||
if args.command == 'repair':
|
||||
if args.pack != 'census':
|
||||
parser.error('the WOF build repairs its own geometry')
|
||||
repair_census()
|
||||
sys.exit(0 if check('census') else 1)
|
||||
sys.exit(0 if check(args.pack) else 1)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
+73
-166
@@ -10,8 +10,7 @@
|
||||
"src/data/localGeojsonCore.js",
|
||||
"src/data/localGeojsonLod.js",
|
||||
"src/data/inputOwnership.js",
|
||||
"src/data/infrastructureOverlayEntry.js",
|
||||
"src/sources/infrastructureData.js"
|
||||
"src/data/infrastructureOverlayEntry.js"
|
||||
],
|
||||
"external": ["cesium"]
|
||||
},
|
||||
@@ -28,12 +27,7 @@
|
||||
"vite-build": {
|
||||
"runtime": "node",
|
||||
"exports": ["./build/vite", "./build/html"],
|
||||
"modules": [
|
||||
"build/vite.js",
|
||||
"build/application-html.js",
|
||||
"build/embed-framing.js",
|
||||
"build/panel.js"
|
||||
],
|
||||
"modules": ["build/vite.js", "build/application-html.js"],
|
||||
"external": ["vite-plugin-cesium"]
|
||||
},
|
||||
"live-providers": {
|
||||
@@ -250,6 +244,9 @@
|
||||
"server/providers/regional/http.js",
|
||||
"server/providers/regional/news.js",
|
||||
"server/providers/regional/place.js",
|
||||
"server/providers/regional/nominatimGate.js",
|
||||
"server/providers/regional/nominatimCache.js",
|
||||
"server/providers/regional/outlineSelect.js",
|
||||
"server/providers/regional/query.js",
|
||||
"server/providers/regional/weather-effects.js",
|
||||
"server/providers/regional/weather.js",
|
||||
@@ -288,6 +285,7 @@
|
||||
"src/data/installationFeedback.js",
|
||||
"src/voice/voiceCost.js",
|
||||
"src/voice/actionSchemas.js",
|
||||
"src/voice/layerManifest.js",
|
||||
"server/providers/openai/toolDescriptions.js",
|
||||
"src/sources/rateLimit.js"
|
||||
],
|
||||
@@ -519,7 +517,6 @@
|
||||
"src/data/detectionPolicy.js",
|
||||
"src/data/installationFeedback.js",
|
||||
"src/data/layerState.js",
|
||||
"src/data/layerStateCoordinator.js",
|
||||
"src/data/layerStateTokenReservations.json",
|
||||
"src/data/militaryAwarenessEngine.js",
|
||||
"src/data/scenePick.js",
|
||||
@@ -656,6 +653,7 @@
|
||||
"src/ui/styles/command-dock-trays.css",
|
||||
"src/ui/styles/command-dock-sliding.css",
|
||||
"src/ui/styles/voice-cost.css",
|
||||
"src/ui/styles/voice-card.css",
|
||||
"src/ui/styles/first-run.css",
|
||||
"src/ui/styles/provider-settings.css",
|
||||
"src/ui/styles/weather.css",
|
||||
@@ -841,7 +839,6 @@
|
||||
"src/data/contactMatch.js",
|
||||
"src/data/iconOrientation.js",
|
||||
"src/data/layerState.js",
|
||||
"src/data/layerStateCoordinator.js",
|
||||
"src/data/layerStateTokenReservations.json",
|
||||
"src/data/modelVisualAnchor.js",
|
||||
"src/data/motionModel.js",
|
||||
@@ -930,7 +927,6 @@
|
||||
"src/data/issPass.js",
|
||||
"src/data/satelliteClass.js",
|
||||
"src/data/satellitePass.js",
|
||||
"src/sources/tle.js",
|
||||
"src/layers/satellites/catalog.js",
|
||||
"src/layers/satellites/controls.js",
|
||||
"src/layers/satellites/index.js",
|
||||
@@ -1088,7 +1084,6 @@
|
||||
"src/layers/bikeshare/ingestion.js",
|
||||
"src/layers/bikeshare/lifecycle.js",
|
||||
"src/layers/bikeshare/model.js",
|
||||
"src/sources/gbfsStations.js",
|
||||
"src/layers/bikeshare/policy.js",
|
||||
"src/layers/bikeshare/queries.js",
|
||||
"src/layers/bikeshare/registry.js",
|
||||
@@ -1295,7 +1290,19 @@
|
||||
"src/voice/realtimeFacade.js",
|
||||
"src/voice/realtimeRadio.js",
|
||||
"src/voice/realtimeTurns.js",
|
||||
"src/voice/realtimeConnection.js"
|
||||
"src/voice/realtimeConnection.js",
|
||||
"src/voice/turnMetrics.js",
|
||||
"src/voice/speechLint.js",
|
||||
"src/voice/voiceCard.js",
|
||||
"src/voice/voiceCardPresentation.js",
|
||||
"src/voice/resultDisplay.js",
|
||||
"src/voice/referents.js",
|
||||
"src/voice/layerManifest.js",
|
||||
"src/voice/speech.js",
|
||||
"src/voice/narration.js",
|
||||
"src/voice/earcon.js",
|
||||
"src/voice/pointerContext.js",
|
||||
"src/voice/pointerReticle.js"
|
||||
],
|
||||
"external": []
|
||||
},
|
||||
@@ -1338,6 +1345,9 @@
|
||||
"src/annotations/hybridAnnotationRenderer.js",
|
||||
"src/annotations/index.js",
|
||||
"src/annotations/resolver.js",
|
||||
"src/annotations/outlineRungs.js",
|
||||
"src/sources/openFreeMapOutlines.js",
|
||||
"src/sources/nominatimOutlines.js",
|
||||
"src/annotations/screenAnnotationRenderer.js",
|
||||
"src/annotations/worldAnnotationRenderer.js",
|
||||
"src/app/catalog.js",
|
||||
@@ -1395,6 +1405,7 @@
|
||||
"src/data/aisLiveVessels.js",
|
||||
"src/data/alprCameras.js",
|
||||
"src/data/analystEngine.js",
|
||||
"src/voice/layerManifest.js",
|
||||
"src/data/bikeshare.js",
|
||||
"src/data/cctv.js",
|
||||
"src/data/cctvCards.js",
|
||||
@@ -1443,7 +1454,6 @@
|
||||
"src/data/issPass.js",
|
||||
"src/data/labelArbiter.js",
|
||||
"src/data/layerState.js",
|
||||
"src/data/layerStateCoordinator.js",
|
||||
"src/data/layerStateTokenReservations.json",
|
||||
"src/data/lifecycle.js",
|
||||
"src/data/localGeojson.js",
|
||||
@@ -1465,6 +1475,9 @@
|
||||
"src/data/motionModel.js",
|
||||
"src/data/naturalEarthRegions.js",
|
||||
"src/data/adminBoundaries.js",
|
||||
"src/data/placeBoundaries.js",
|
||||
"src/data/local_data/us_census_places/files.js",
|
||||
"src/data/local_data/wof_neighborhoods/files.js",
|
||||
"src/data/local_data/natural_earth/states_provinces.json",
|
||||
"src/data/local_data/natural_earth/countries.json",
|
||||
"src/data/local_data/us_census_counties/counties.json",
|
||||
@@ -1536,7 +1549,6 @@
|
||||
"src/layers/bikeshare/ingestion.js",
|
||||
"src/layers/bikeshare/lifecycle.js",
|
||||
"src/layers/bikeshare/model.js",
|
||||
"src/sources/gbfsStations.js",
|
||||
"src/layers/bikeshare/policy.js",
|
||||
"src/layers/bikeshare/queries.js",
|
||||
"src/layers/bikeshare/registry.js",
|
||||
@@ -1671,7 +1683,6 @@
|
||||
"src/layers/satellites/labels.js",
|
||||
"src/layers/satellites/lifecycle.js",
|
||||
"src/layers/satellites/orbits.js",
|
||||
"src/sources/tle.js",
|
||||
"src/layers/satellites/records.js",
|
||||
"src/layers/satellites/policy.js",
|
||||
"src/layers/satellites/rendering.js",
|
||||
@@ -1864,6 +1875,14 @@
|
||||
"src/voice/commands.js",
|
||||
"src/voice/control.js",
|
||||
"src/voice/gevActions.js",
|
||||
"src/voice/speech.js",
|
||||
"src/voice/earcon.js",
|
||||
"src/voice/narration.js",
|
||||
"src/voice/turnMetrics.js",
|
||||
"src/voice/speechLint.js",
|
||||
"src/voice/voiceCard.js",
|
||||
"src/voice/voiceCardPresentation.js",
|
||||
"src/voice/resultDisplay.js",
|
||||
"src/voice/gevRealtime.js",
|
||||
"src/voice/layerSummary.js",
|
||||
"src/voice/realtimeBackend.js",
|
||||
@@ -2027,6 +2046,12 @@
|
||||
"src/maps/imageryHost.js",
|
||||
"src/ui/recentImagery.js",
|
||||
"src/ui/imageryBoxTool.js",
|
||||
"src/voice/pointerContext.js",
|
||||
"src/voice/pointerReticle.js",
|
||||
"src/voice/deixis.js",
|
||||
"src/voice/referents.js",
|
||||
"src/voice/pointerPick.js",
|
||||
"src/data/pickedContext.js",
|
||||
"src/layers/alpr/tileRecords.js",
|
||||
"src/sources/capability.js",
|
||||
"src/sources/openFreeMap.js",
|
||||
@@ -2040,12 +2065,7 @@
|
||||
"src/sources/requestDeadline.js",
|
||||
"src/data/local_data/osm_military_names/names.json",
|
||||
"src/layers/installations/namedMarkers.js",
|
||||
"src/data/infrastructureOverlayEntry.js",
|
||||
"src/sources/infrastructureData.js",
|
||||
"src/tools/functions.js",
|
||||
"src/voice/actionSchemas.js",
|
||||
"src/view/index.js",
|
||||
"src/app/embed.js"
|
||||
"src/data/infrastructureOverlayEntry.js"
|
||||
],
|
||||
"external": [
|
||||
"hls.js",
|
||||
@@ -2077,6 +2097,9 @@
|
||||
"src/data/bhoteKoshiShotPaths.js",
|
||||
"src/annotations/annotationResolver.js",
|
||||
"src/annotations/resolver.js",
|
||||
"src/annotations/outlineRungs.js",
|
||||
"src/sources/openFreeMapOutlines.js",
|
||||
"src/sources/nominatimOutlines.js",
|
||||
"src/app/catalog.js",
|
||||
"src/app/constructCatalog.js",
|
||||
"src/layers/wind/index.js",
|
||||
@@ -2152,6 +2175,9 @@
|
||||
"src/data/motionModel.js",
|
||||
"src/data/naturalEarthRegions.js",
|
||||
"src/data/adminBoundaries.js",
|
||||
"src/data/placeBoundaries.js",
|
||||
"src/data/local_data/us_census_places/files.js",
|
||||
"src/data/local_data/wof_neighborhoods/files.js",
|
||||
"src/data/local_data/natural_earth/states_provinces.json",
|
||||
"src/data/local_data/natural_earth/countries.json",
|
||||
"src/data/local_data/us_census_counties/counties.json",
|
||||
@@ -2207,7 +2233,6 @@
|
||||
"src/layers/bikeshare/ingestion.js",
|
||||
"src/layers/bikeshare/lifecycle.js",
|
||||
"src/layers/bikeshare/model.js",
|
||||
"src/sources/gbfsStations.js",
|
||||
"src/layers/bikeshare/policy.js",
|
||||
"src/layers/bikeshare/queries.js",
|
||||
"src/layers/bikeshare/registry.js",
|
||||
@@ -2342,7 +2367,6 @@
|
||||
"src/layers/satellites/labels.js",
|
||||
"src/layers/satellites/lifecycle.js",
|
||||
"src/layers/satellites/orbits.js",
|
||||
"src/sources/tle.js",
|
||||
"src/layers/satellites/records.js",
|
||||
"src/layers/satellites/policy.js",
|
||||
"src/layers/satellites/rendering.js",
|
||||
@@ -2531,8 +2555,7 @@
|
||||
"src/sources/requestDeadline.js",
|
||||
"src/data/local_data/osm_military_names/names.json",
|
||||
"src/layers/installations/namedMarkers.js",
|
||||
"src/data/infrastructureOverlayEntry.js",
|
||||
"src/sources/infrastructureData.js"
|
||||
"src/data/infrastructureOverlayEntry.js"
|
||||
],
|
||||
"external": [
|
||||
"hls.js",
|
||||
@@ -2548,8 +2571,16 @@
|
||||
"application-operations": {
|
||||
"exports": ["./application/operations"],
|
||||
"modules": [
|
||||
"src/sources/openFreeMap.js",
|
||||
"src/sources/vectorTiles.js",
|
||||
"src/sources/phaseTiming.js",
|
||||
"src/sources/militaryTileGeometry.js",
|
||||
"src/data/tomtomTiles.js",
|
||||
"src/annotations/annotationResolver.js",
|
||||
"src/annotations/resolver.js",
|
||||
"src/annotations/outlineRungs.js",
|
||||
"src/sources/openFreeMapOutlines.js",
|
||||
"src/sources/nominatimOutlines.js",
|
||||
"src/app/operations.js",
|
||||
"src/sources/sourceSlot.js",
|
||||
"src/app/surfaceServices.js",
|
||||
@@ -2560,6 +2591,9 @@
|
||||
"src/data/local_data/neighborhoods/san-francisco.json",
|
||||
"src/data/naturalEarthRegions.js",
|
||||
"src/data/adminBoundaries.js",
|
||||
"src/data/placeBoundaries.js",
|
||||
"src/data/local_data/us_census_places/files.js",
|
||||
"src/data/local_data/wof_neighborhoods/files.js",
|
||||
"src/data/local_data/natural_earth/states_provinces.json",
|
||||
"src/data/local_data/natural_earth/countries.json",
|
||||
"src/data/local_data/us_census_counties/counties.json",
|
||||
@@ -2626,7 +2660,6 @@
|
||||
"modules": [
|
||||
"src/data/installationFeedback.js",
|
||||
"src/data/layerState.js",
|
||||
"src/data/layerStateCoordinator.js",
|
||||
"src/data/layerStateTokenReservations.json",
|
||||
"src/loadingFeedback.js",
|
||||
"src/navigationPolicy.js",
|
||||
@@ -2758,7 +2791,11 @@
|
||||
},
|
||||
"voice-action-schemas": {
|
||||
"exports": ["./voice/action-schemas"],
|
||||
"modules": ["src/voice/actionSchemas.js", "src/hudLayoutPolicy.js"],
|
||||
"modules": [
|
||||
"src/voice/actionSchemas.js",
|
||||
"src/voice/layerManifest.js",
|
||||
"src/hudLayoutPolicy.js"
|
||||
],
|
||||
"external": []
|
||||
},
|
||||
"voice-session": {
|
||||
@@ -2771,7 +2808,14 @@
|
||||
"modules": [
|
||||
"src/voice/sessionCommands.js",
|
||||
"src/voice/session.js",
|
||||
"src/voice/control.js"
|
||||
"src/voice/control.js",
|
||||
"src/voice/voiceCard.js",
|
||||
"src/voice/voiceCardPresentation.js",
|
||||
"src/voice/resultDisplay.js",
|
||||
"src/voice/referents.js",
|
||||
"src/voice/layerManifest.js",
|
||||
"src/voice/speech.js",
|
||||
"src/voice/pointerReticle.js"
|
||||
],
|
||||
"external": []
|
||||
},
|
||||
@@ -3094,142 +3138,5 @@
|
||||
"src/layers/perimeters/source.js"
|
||||
],
|
||||
"external": []
|
||||
},
|
||||
"tools": {
|
||||
"exports": ["./tools"],
|
||||
"modules": [
|
||||
"src/tools/index.js",
|
||||
"src/tools/catalog.js",
|
||||
"src/tools/schema.js",
|
||||
"src/tools/area.js",
|
||||
"src/tools/results.js",
|
||||
"src/tools/places.js",
|
||||
"src/tools/queries/hazards.js",
|
||||
"src/tools/queries/space.js",
|
||||
"src/tools/queries/aviation.js",
|
||||
"src/tools/queries/media.js",
|
||||
"src/tools/queries/places.js",
|
||||
"src/tools/queries/mobility.js",
|
||||
"src/tools/queries/environment.js",
|
||||
"src/tools/queries/brief.js",
|
||||
"src/tools/queries/maritime.js",
|
||||
"src/tools/queries/share.js",
|
||||
"src/data/layerState.js",
|
||||
"src/data/layerStateTokenReservations.json",
|
||||
"src/sources/gbfsStations.js",
|
||||
"src/sources/tle.js",
|
||||
"src/data/satellitePass.js",
|
||||
"src/tools/queries/atmosphere.js",
|
||||
"src/layers/wind/inspection.js",
|
||||
"src/layers/wind/model.js",
|
||||
"src/layers/wind/fields.js",
|
||||
"src/tools/queries/imagery.js",
|
||||
"src/tools/queries/cables.js",
|
||||
"src/tools/queries/surveillance.js",
|
||||
"src/tools/queries/infrastructure.js",
|
||||
"src/tools/queries/events.js",
|
||||
"src/data/bhoteKoshiFloodPath.js",
|
||||
"src/tools/functions.js",
|
||||
"src/tools/surfaces.js",
|
||||
"src/layers/transit/policy.js",
|
||||
"src/data/transitFeeds.js",
|
||||
"src/layers/transit/movement.js",
|
||||
"src/data/contactPlayback.js",
|
||||
"src/data/layerSnapshot.js",
|
||||
"src/data/feedState.js",
|
||||
"src/loadingFeedback.js",
|
||||
"src/data/installationFeedback.js",
|
||||
"src/view/index.js",
|
||||
"src/tools/views.js",
|
||||
"src/tools/globePanel.js",
|
||||
"src/tools/queries/panelRequest.js"
|
||||
],
|
||||
"external": ["satellite.js"]
|
||||
},
|
||||
"tools-mcp": {
|
||||
"exports": ["./tools/mcp"],
|
||||
"modules": [
|
||||
"src/tools/mcp/index.js",
|
||||
"src/tools/mcp/protocol.js",
|
||||
"src/tools/mcp/http.js",
|
||||
"src/tools/catalog.js",
|
||||
"src/tools/schema.js"
|
||||
],
|
||||
"external": []
|
||||
},
|
||||
"tools-services": {
|
||||
"exports": ["./tools/services"],
|
||||
"modules": [
|
||||
"src/tools/services.js",
|
||||
"src/layers/bikeshare/registry.js",
|
||||
"src/layers/bikeshare/source.js",
|
||||
"src/layers/cctv/source.js",
|
||||
"src/layers/cctv/sourcePolicy.js",
|
||||
"src/layers/cyclones/source.js",
|
||||
"src/sources/httpBody.js",
|
||||
"src/layers/earthquakes/source.js",
|
||||
"src/layers/earthquakes/records.js",
|
||||
"src/layers/firms/source.js",
|
||||
"src/layers/installations/source.js",
|
||||
"src/sources/requestDeadline.js",
|
||||
"src/data/militaryNames.js",
|
||||
"src/sources/militaryTileGeometry.js",
|
||||
"src/data/bundledJson.js",
|
||||
"src/data/retryableLoad.js",
|
||||
"src/data/local_data/osm_military_names/names.json",
|
||||
"src/sources/openFreeMap.js",
|
||||
"src/sources/vectorTiles.js",
|
||||
"src/sources/phaseTiming.js",
|
||||
"src/data/tomtomTiles.js",
|
||||
"src/sources/capability.js",
|
||||
"src/data/militaryInstallationData.js",
|
||||
"src/layers/launches/source.js",
|
||||
"src/layers/perimeters/source.js",
|
||||
"src/layers/radio/source.js",
|
||||
"src/layers/radio/policy.js",
|
||||
"src/layers/satellites/source.js",
|
||||
"src/layers/traffic/source.js",
|
||||
"src/layers/traffic/roadModes.js",
|
||||
"src/data/flowMatch.js",
|
||||
"src/layers/traffic/flowSource.js",
|
||||
"src/layers/traffic/flowDecode.js",
|
||||
"src/sources/overpassRoads.js",
|
||||
"src/layers/transit/source.js",
|
||||
"src/sources/transitHistory.js",
|
||||
"src/services/requests.js",
|
||||
"src/sources/overpassFeatures.js",
|
||||
"src/sources/featureSource.js",
|
||||
"src/sources/overpassFeaturesRecords.js",
|
||||
"src/sources/featureGeometry.js",
|
||||
"src/sources/live/standalone.js",
|
||||
"src/sources/live/contract.js",
|
||||
"src/sources/live/aircraft.js",
|
||||
"src/sources/live/vessels.js",
|
||||
"src/tools/places.js",
|
||||
"src/tools/catalog.js",
|
||||
"src/tools/schema.js",
|
||||
"src/layers/weather/source.js",
|
||||
"src/layers/wind/source.js",
|
||||
"src/sources/windBody.js",
|
||||
"src/layers/recentImagery/catalog.js",
|
||||
"src/layers/recentImagery/model.js",
|
||||
"src/layers/submarineCables/bundledSource.js",
|
||||
"src/layers/alpr/source.js",
|
||||
"src/layers/alpr/policy.js",
|
||||
"src/layers/alpr/records.js",
|
||||
"src/layers/alpr/tileRecords.js",
|
||||
"src/sources/infrastructureData.js",
|
||||
"src/sources/eventPacks.js"
|
||||
],
|
||||
"external": ["@mapbox/vector-tile", "pbf"]
|
||||
},
|
||||
"view": {
|
||||
"exports": ["./view"],
|
||||
"modules": [
|
||||
"src/view/index.js",
|
||||
"src/data/layerState.js",
|
||||
"src/data/layerStateTokenReservations.json"
|
||||
],
|
||||
"external": []
|
||||
}
|
||||
}
|
||||
@@ -3,7 +3,7 @@
|
||||
* qa-admin-outlines.mjs — bundled state/province/county outlines, driven like a
|
||||
* user would drive them.
|
||||
*
|
||||
* Run: node scripts/qa-admin-outlines.mjs http://localhost:4173 [--headful] [--no-video]
|
||||
* Run: node scripts/qa-admin-outlines.mjs http://localhost:4173 [--headful] [--no-video] [--offline-imagery]
|
||||
*
|
||||
* Journey: frame Texas at state scale and outline "Texas"; pan and zoom toward
|
||||
* Austin in smooth 2–3 s moves; outline "Travis County, Texas"; pan north and
|
||||
@@ -33,7 +33,7 @@ import sharp from 'sharp';
|
||||
const args = process.argv.slice(2);
|
||||
if (args.includes('--help')) {
|
||||
console.log(
|
||||
'Usage: node scripts/qa-admin-outlines.mjs <dev-server-url> [--headful] [--no-video]',
|
||||
'Usage: node scripts/qa-admin-outlines.mjs <dev-server-url> [--headful] [--no-video] [--offline-imagery]',
|
||||
);
|
||||
process.exit(0);
|
||||
}
|
||||
@@ -41,6 +41,15 @@ const url = args.find((arg) => !arg.startsWith('--'));
|
||||
if (!url || !['http:', 'https:'].includes(new URL(url).protocol))
|
||||
throw new Error('Supply a running dev server URL; see --help');
|
||||
const recordVideo = !args.includes('--no-video');
|
||||
const offlineImagery = args.includes('--offline-imagery');
|
||||
const offlineImageryTile = offlineImagery
|
||||
? await fs.readFile(
|
||||
new URL(
|
||||
'../node_modules/cesium/Build/Cesium/Assets/Textures/NaturalEarthII/0/0/0.jpg',
|
||||
import.meta.url,
|
||||
),
|
||||
)
|
||||
: null;
|
||||
const shots = path.resolve('qa-shots');
|
||||
const framesDir = path.join(shots, 'admin-outlines-frames');
|
||||
await fs.mkdir(shots, { recursive: true });
|
||||
@@ -61,6 +70,8 @@ const result = {
|
||||
pageErrors: [],
|
||||
screenshots: [],
|
||||
video: null,
|
||||
offlineImagery,
|
||||
offlineImageryRequests: 0,
|
||||
};
|
||||
|
||||
const browser = await puppeteer.launch({
|
||||
@@ -87,6 +98,35 @@ try {
|
||||
sessionStorage.setItem('gev:first-run-mission-session:v1', 'dismissed');
|
||||
});
|
||||
await page.setViewport({ width: 1280, height: 800 });
|
||||
await page.setRequestInterception(true);
|
||||
page.on('request', (request) => {
|
||||
let target;
|
||||
try {
|
||||
target = new URL(request.url());
|
||||
} catch {
|
||||
request.continue();
|
||||
return;
|
||||
}
|
||||
const knownImageryTile =
|
||||
request.method() === 'GET' &&
|
||||
((target.origin === 'https://services.arcgisonline.com' &&
|
||||
/^\/ArcGIS\/rest\/services\/World_Imagery\/MapServer\/tile\/\d+\/\d+\/\d+$/.test(
|
||||
target.pathname,
|
||||
)) ||
|
||||
(target.origin === 'https://tile.openstreetmap.org' &&
|
||||
/^\/\d+\/\d+\/\d+\.png$/.test(target.pathname)));
|
||||
if (offlineImagery && knownImageryTile) {
|
||||
result.offlineImageryRequests += 1;
|
||||
request.respond({
|
||||
status: 200,
|
||||
contentType: 'image/jpeg',
|
||||
headers: { 'Access-Control-Allow-Origin': '*' },
|
||||
body: offlineImageryTile,
|
||||
});
|
||||
return;
|
||||
}
|
||||
request.continue();
|
||||
});
|
||||
const client = await page.createCDPSession();
|
||||
await client.send('Network.enable');
|
||||
client.on('Network.requestWillBeSent', ({ request }) => {
|
||||
|
||||
@@ -0,0 +1,714 @@
|
||||
#!/usr/bin/env node
|
||||
/**
|
||||
* Traffic-warm OpenFreeMap outline acceptance.
|
||||
*
|
||||
* Each sample opens a fresh application page, loads the Texas Capitol z14 tile
|
||||
* through Street Traffic, then circles a building from that same tile. It
|
||||
* records application fetches separately from browser transfer bytes,
|
||||
* body-read/decode/extraction/install measures, and Long Tasks. A second ask
|
||||
* on the page proves the outline source's own consumer-cache path. A separate
|
||||
* page holds traffic's target response while the outline consumer subscribes,
|
||||
* proving that overlap still produces one application fetch and one decode.
|
||||
*
|
||||
* Acceptance is declared before measurement: cold outline decode, extraction,
|
||||
* and install must fit one 60 Hz frame at p95 (16.7 ms), no sample may create a
|
||||
* >=50 ms Long Task, and neither the first nor second outline ask may fetch or
|
||||
* decode the traffic-warm tile again. The target tile's traffic-idle outline
|
||||
* projection must also fit one frame; unrelated traffic activation Long Tasks
|
||||
* are reported separately as baseline evidence.
|
||||
*/
|
||||
import assert from 'node:assert/strict';
|
||||
import fs from 'node:fs/promises';
|
||||
import path from 'node:path';
|
||||
import puppeteer from 'puppeteer';
|
||||
|
||||
const args = process.argv.slice(2);
|
||||
const valueAfter = (flag, fallback) => {
|
||||
const index = args.indexOf(flag);
|
||||
return index >= 0 && args[index + 1] ? args[index + 1] : fallback;
|
||||
};
|
||||
if (args.includes('--help') || args.includes('-h')) {
|
||||
console.log(
|
||||
'Usage: node scripts/qa-openfreemap-same-tile.mjs [--url http://localhost:4173] [--runs 5] [--headful] [--out qa-shots/openfreemap-same-tile.json]',
|
||||
);
|
||||
process.exit(0);
|
||||
}
|
||||
|
||||
const APP_URL = valueAfter('--url', 'http://localhost:4173');
|
||||
const RUNS = Number(valueAfter('--runs', '5'));
|
||||
const OUTPUT = path.resolve(
|
||||
valueAfter('--out', 'qa-shots/openfreemap-same-tile.json'),
|
||||
);
|
||||
const FRAME_BUDGET_MS = 16.7;
|
||||
const LONG_TASK_MS = 50;
|
||||
const TARGET = Object.freeze({
|
||||
latitude: 30.27472,
|
||||
longitude: -97.74035,
|
||||
z: 14,
|
||||
x: 3743,
|
||||
y: 6745,
|
||||
});
|
||||
const TARGET_PATH = `/${TARGET.z}/${TARGET.x}/${TARGET.y}.pbf`;
|
||||
|
||||
if (!Number.isInteger(RUNS) || RUNS < 3 || RUNS > 20)
|
||||
throw new Error('--runs must be an integer from 3 through 20');
|
||||
const targetUrl = new URL(APP_URL);
|
||||
if (!['http:', 'https:'].includes(targetUrl.protocol))
|
||||
throw new Error('--url must be HTTP(S)');
|
||||
|
||||
const percentile = (values, p) => {
|
||||
if (!values.length) return null;
|
||||
const sorted = values.slice().sort((a, b) => a - b);
|
||||
return sorted[Math.ceil(p * sorted.length) - 1];
|
||||
};
|
||||
const round = (value) =>
|
||||
value == null ? null : Number(Number(value).toFixed(3));
|
||||
const measures = (entries, name) =>
|
||||
entries.filter((entry) => entry.name === name).map((entry) => entry.ms);
|
||||
const sum = (values) => values.reduce((total, value) => total + value, 0);
|
||||
const sleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms));
|
||||
|
||||
await fs.mkdir(path.dirname(OUTPUT), { recursive: true });
|
||||
|
||||
const browser = await puppeteer.launch({
|
||||
headless: args.includes('--headful') ? false : 'new',
|
||||
protocolTimeout: 240_000,
|
||||
...(process.env.PUPPETEER_EXECUTABLE_PATH
|
||||
? { executablePath: process.env.PUPPETEER_EXECUTABLE_PATH }
|
||||
: {}),
|
||||
args: [
|
||||
'--no-sandbox',
|
||||
'--disable-setuid-sandbox',
|
||||
'--use-gl=angle',
|
||||
...(args.includes('--headful')
|
||||
? []
|
||||
: ['--use-angle=swiftshader', '--enable-unsafe-swiftshader']),
|
||||
'--disable-dev-shm-usage',
|
||||
'--disable-background-timer-throttling',
|
||||
'--disable-renderer-backgrounding',
|
||||
'--window-size=1440,1000',
|
||||
],
|
||||
});
|
||||
|
||||
const report = {
|
||||
appUrl: APP_URL,
|
||||
target: TARGET,
|
||||
targetPath: TARGET_PATH,
|
||||
acceptance: {
|
||||
coldCpuP95Ms: FRAME_BUDGET_MS,
|
||||
maxLongTaskMs: LONG_TASK_MS,
|
||||
trafficIdleProjectionMaxMs: FRAME_BUDGET_MS,
|
||||
coldApplicationFetches: 0,
|
||||
coldOutlineTransferBytes: 0,
|
||||
coldDecodes: 0,
|
||||
warmApplicationFetches: 0,
|
||||
warmDecodes: 0,
|
||||
overlapApplicationFetches: 1,
|
||||
overlapDecodes: 1,
|
||||
},
|
||||
renderer: null,
|
||||
samples: [],
|
||||
errors: [],
|
||||
};
|
||||
|
||||
function relevantMeasures(raw) {
|
||||
return raw
|
||||
.filter(
|
||||
(entry) =>
|
||||
entry.name.startsWith('roads:') || entry.name.startsWith('outlines:'),
|
||||
)
|
||||
.map((entry) => ({
|
||||
name: entry.name,
|
||||
ms: round(entry.duration),
|
||||
at: round(entry.startTime),
|
||||
detail: entry.detail || {},
|
||||
}));
|
||||
}
|
||||
|
||||
async function beginOutline(page, label) {
|
||||
await page.evaluate(
|
||||
({ label, target }) => {
|
||||
performance.clearMeasures();
|
||||
window.__qaOutlineLongTasks.length = 0;
|
||||
window.__qaOutlineStarted = performance.now();
|
||||
window.__qaOutlinePromise = Promise.resolve(
|
||||
window.__gevVoiceCommands.runner('annotate_map', {
|
||||
annotations: [
|
||||
{
|
||||
type: 'area',
|
||||
target: 'this building',
|
||||
label,
|
||||
entityKind: 'building',
|
||||
latitude: target.latitude,
|
||||
longitude: target.longitude,
|
||||
},
|
||||
],
|
||||
}),
|
||||
);
|
||||
},
|
||||
{ label, target: TARGET },
|
||||
);
|
||||
}
|
||||
|
||||
async function settleOutline(page, label) {
|
||||
return page.evaluate(async (label) => {
|
||||
const result = await window.__qaOutlinePromise;
|
||||
let annotation = null;
|
||||
const deadline = performance.now() + 30_000;
|
||||
while (performance.now() < deadline) {
|
||||
annotation = (window.__gevAnnotations?.list?.() || []).find(
|
||||
(item) => item.label === label,
|
||||
);
|
||||
if (annotation && !annotation.pendingOutline) break;
|
||||
await new Promise((resolve) => setTimeout(resolve, 25));
|
||||
}
|
||||
if (!annotation || annotation.pendingOutline)
|
||||
throw new Error(`${label}: outline did not settle`);
|
||||
const { scene } = window.__godsEyeView.viewer;
|
||||
await new Promise((resolve) => {
|
||||
const timer = setTimeout(resolve, 1500);
|
||||
const remove = scene.postRender.addEventListener(() => {
|
||||
clearTimeout(timer);
|
||||
remove();
|
||||
resolve();
|
||||
});
|
||||
scene.requestRender();
|
||||
});
|
||||
await new Promise((resolve) =>
|
||||
requestAnimationFrame(() => requestAnimationFrame(resolve)),
|
||||
);
|
||||
return {
|
||||
commandToRenderMs: performance.now() - window.__qaOutlineStarted,
|
||||
result,
|
||||
annotation: {
|
||||
pending: Boolean(annotation.pendingOutline),
|
||||
unavailable: Boolean(annotation.outlineUnavailable),
|
||||
synthesized: Boolean(annotation.synthesized),
|
||||
footprintKind: annotation.footprintKind || null,
|
||||
ringPoints: annotation.ring?.length || 0,
|
||||
parts: annotation.polygons?.length || (annotation.ring ? 1 : 0),
|
||||
},
|
||||
measures: performance.getEntriesByType('measure').map((entry) => ({
|
||||
name: entry.name,
|
||||
duration: entry.duration,
|
||||
startTime: entry.startTime,
|
||||
detail: entry.detail,
|
||||
})),
|
||||
longTasks: window.__qaOutlineLongTasks.slice(),
|
||||
};
|
||||
}, label);
|
||||
}
|
||||
|
||||
async function driveOutline(page, label) {
|
||||
await beginOutline(page, label);
|
||||
return settleOutline(page, label);
|
||||
}
|
||||
|
||||
try {
|
||||
for (let run = 0; run <= RUNS; run += 1) {
|
||||
const page = await browser.newPage();
|
||||
await page.setViewport({ width: 1440, height: 1000 });
|
||||
let overlapMode = false;
|
||||
let overlapApplicationRequests = 0;
|
||||
let heldTargetRequest = null;
|
||||
let resolveHeldTarget;
|
||||
const heldTarget = new Promise((resolve) => {
|
||||
resolveHeldTarget = resolve;
|
||||
});
|
||||
if (run === 0) {
|
||||
await page.setRequestInterception(true);
|
||||
page.on('request', (request) => {
|
||||
let isTarget = false;
|
||||
try {
|
||||
isTarget = new URL(request.url()).pathname.endsWith(TARGET_PATH);
|
||||
} catch {
|
||||
// Invalid URLs are passed through for the browser to handle.
|
||||
}
|
||||
if (overlapMode && isTarget) {
|
||||
overlapApplicationRequests += 1;
|
||||
if (!heldTargetRequest) {
|
||||
heldTargetRequest = request;
|
||||
resolveHeldTarget();
|
||||
return;
|
||||
}
|
||||
}
|
||||
void request.continue();
|
||||
});
|
||||
}
|
||||
await page.evaluateOnNewDocument(() => {
|
||||
sessionStorage.setItem('gev:first-run-mission-session:v1', 'dismissed');
|
||||
window.__qaOutlineLongTasks = [];
|
||||
try {
|
||||
new PerformanceObserver((list) => {
|
||||
for (const entry of list.getEntries())
|
||||
window.__qaOutlineLongTasks.push({
|
||||
at: entry.startTime,
|
||||
ms: entry.duration,
|
||||
});
|
||||
}).observe({ type: 'longtask', buffered: false });
|
||||
window.__qaLongTaskSupported = true;
|
||||
} catch {
|
||||
window.__qaLongTaskSupported = false;
|
||||
}
|
||||
});
|
||||
|
||||
let phase = 'startup';
|
||||
const network = [];
|
||||
const pending = new Map();
|
||||
const client = await page.createCDPSession();
|
||||
await client.send('Network.enable');
|
||||
client.on('Network.requestWillBeSent', (event) => {
|
||||
let url;
|
||||
try {
|
||||
url = new URL(event.request.url);
|
||||
} catch {
|
||||
return;
|
||||
}
|
||||
if (
|
||||
url.hostname !== 'tiles.openfreemap.org' ||
|
||||
!url.pathname.endsWith('.pbf')
|
||||
)
|
||||
return;
|
||||
const entry = {
|
||||
requestId: event.requestId,
|
||||
phase,
|
||||
url: url.origin + url.pathname,
|
||||
targetTile: url.pathname.endsWith(TARGET_PATH),
|
||||
cached: false,
|
||||
transferBytes: 0,
|
||||
responseBytes: 0,
|
||||
finished: false,
|
||||
};
|
||||
pending.set(event.requestId, entry);
|
||||
network.push(entry);
|
||||
});
|
||||
client.on('Network.requestServedFromCache', ({ requestId }) => {
|
||||
const entry = pending.get(requestId);
|
||||
if (entry) entry.cached = true;
|
||||
});
|
||||
client.on('Network.responseReceived', ({ requestId, response }) => {
|
||||
const entry = pending.get(requestId);
|
||||
if (!entry) return;
|
||||
entry.status = response.status;
|
||||
entry.cached ||= Boolean(
|
||||
response.fromDiskCache ||
|
||||
response.fromServiceWorker ||
|
||||
response.fromPrefetchCache,
|
||||
);
|
||||
});
|
||||
client.on('Network.dataReceived', ({ requestId, dataLength }) => {
|
||||
const entry = pending.get(requestId);
|
||||
if (entry) entry.responseBytes += dataLength;
|
||||
});
|
||||
client.on('Network.loadingFinished', ({ requestId, encodedDataLength }) => {
|
||||
const entry = pending.get(requestId);
|
||||
if (!entry) return;
|
||||
entry.transferBytes = entry.cached ? 0 : encodedDataLength;
|
||||
entry.finished = true;
|
||||
pending.delete(requestId);
|
||||
});
|
||||
client.on('Network.loadingFailed', ({ requestId, errorText }) => {
|
||||
const entry = pending.get(requestId);
|
||||
if (!entry) return;
|
||||
entry.failure = errorText;
|
||||
entry.finished = true;
|
||||
pending.delete(requestId);
|
||||
});
|
||||
|
||||
const pageErrors = [];
|
||||
page.on('pageerror', (error) => pageErrors.push(String(error.message)));
|
||||
const navigation = new URL(APP_URL);
|
||||
navigation.searchParams.set('welcome', '0');
|
||||
navigation.searchParams.set('trafficDebug', '1');
|
||||
navigation.searchParams.set('trafficRoads', 'osm');
|
||||
await page.goto(navigation.href, {
|
||||
waitUntil: 'domcontentloaded',
|
||||
timeout: 90_000,
|
||||
});
|
||||
await page.waitForFunction(
|
||||
() =>
|
||||
window.__godsEyeView?.viewer &&
|
||||
window.__godsEyeView?.dataManager &&
|
||||
window.__gevVoiceCommands?.runner &&
|
||||
window.__gevAnnotations &&
|
||||
document.getElementById('loading-screen')?.classList.contains('hidden'),
|
||||
{ timeout: 120_000, polling: 250 },
|
||||
);
|
||||
await page.evaluate(
|
||||
() => window.__godsEyeView.styleManager.initialRestorePromise,
|
||||
);
|
||||
await page.evaluate(() =>
|
||||
window.__godsEyeView.styleManager.setDetection({ enabled: false }),
|
||||
);
|
||||
await page.evaluate(async (target) => {
|
||||
const { viewer, dataManager } = window.__godsEyeView;
|
||||
await dataManager.setEnabled('traffic', false);
|
||||
viewer.camera.cancelFlight();
|
||||
viewer.camera.setView({
|
||||
destination: window.__CESIUM__.Cartesian3.fromDegrees(
|
||||
target.longitude,
|
||||
target.latitude,
|
||||
700,
|
||||
),
|
||||
orientation: {
|
||||
heading: 0,
|
||||
pitch: (-65 * Math.PI) / 180,
|
||||
roll: 0,
|
||||
},
|
||||
});
|
||||
viewer.camera.moveEnd.raiseEvent();
|
||||
window.__gevAnnotations.clear();
|
||||
performance.clearMeasures();
|
||||
}, TARGET);
|
||||
await sleep(750);
|
||||
|
||||
if (run === 0) {
|
||||
phase = 'overlap';
|
||||
overlapMode = true;
|
||||
await page.evaluate(() => {
|
||||
window.__qaOutlineLongTasks.length = 0;
|
||||
performance.clearMeasures();
|
||||
});
|
||||
await page.evaluate(() =>
|
||||
window.__godsEyeView.dataManager.setEnabled('traffic', true),
|
||||
);
|
||||
await Promise.race([
|
||||
heldTarget,
|
||||
sleep(30_000).then(() => {
|
||||
throw new Error('overlap phase did not intercept the target tile');
|
||||
}),
|
||||
]);
|
||||
await beginOutline(page, 'QA Capitol overlap');
|
||||
await sleep(100);
|
||||
await heldTargetRequest.continue();
|
||||
const overlap = await settleOutline(page, 'QA Capitol overlap');
|
||||
await sleep(750);
|
||||
const trafficStats = await page.evaluate(() =>
|
||||
window.__godsEyeView.dataManager.layers
|
||||
.get('traffic')
|
||||
.module.getStats(),
|
||||
);
|
||||
assert.equal(trafficStats.error, null, 'overlap traffic source error');
|
||||
overlap.measures = relevantMeasures(overlap.measures);
|
||||
overlap.network = network.filter(
|
||||
(entry) => entry.phase === 'overlap' && entry.targetTile,
|
||||
);
|
||||
const overlapDecodes = overlap.measures
|
||||
.filter(
|
||||
(entry) =>
|
||||
entry.name === 'roads:decode' &&
|
||||
String(entry.detail?.key).endsWith(
|
||||
`:${TARGET.z}/${TARGET.x}/${TARGET.y}`,
|
||||
),
|
||||
)
|
||||
.map((entry) => entry.ms);
|
||||
assert.equal(
|
||||
overlapApplicationRequests,
|
||||
1,
|
||||
'overlapping traffic and outline consumers issue one application request',
|
||||
);
|
||||
assert.equal(
|
||||
overlap.network.length,
|
||||
1,
|
||||
'overlapping traffic and outline consumers emit one target network request',
|
||||
);
|
||||
assert.equal(
|
||||
overlapDecodes.length,
|
||||
1,
|
||||
'overlapping traffic and outline consumers decode once',
|
||||
);
|
||||
assert.equal(overlap.annotation.unavailable, false);
|
||||
assert.equal(overlap.annotation.footprintKind, 'building');
|
||||
report.overlap = {
|
||||
applicationFetches: overlapApplicationRequests,
|
||||
targetNetworkRequests: overlap.network.length,
|
||||
transferBytes: sum(
|
||||
overlap.network.map((entry) => entry.transferBytes),
|
||||
),
|
||||
decodes: overlapDecodes.length,
|
||||
decodeMs: round(sum(overlapDecodes)),
|
||||
longTasks: overlap.longTasks.length,
|
||||
};
|
||||
await page.close();
|
||||
continue;
|
||||
}
|
||||
|
||||
phase = 'traffic';
|
||||
await page.evaluate(() => {
|
||||
window.__qaOutlineLongTasks.length = 0;
|
||||
performance.clearMeasures();
|
||||
});
|
||||
await page.evaluate(() =>
|
||||
window.__godsEyeView.dataManager.setEnabled('traffic', true),
|
||||
);
|
||||
await page.waitForFunction(
|
||||
() => {
|
||||
const stats = window.__godsEyeView.dataManager.layers
|
||||
.get('traffic')
|
||||
.module.getStats();
|
||||
return stats.count > 0 && !stats.loading && !stats.error;
|
||||
},
|
||||
{ timeout: 180_000, polling: 250 },
|
||||
);
|
||||
await sleep(1500);
|
||||
const trafficMeasures = relevantMeasures(
|
||||
await page.evaluate(() =>
|
||||
performance.getEntriesByType('measure').map((entry) => ({
|
||||
name: entry.name,
|
||||
duration: entry.duration,
|
||||
startTime: entry.startTime,
|
||||
detail: entry.detail,
|
||||
})),
|
||||
),
|
||||
);
|
||||
const trafficLongTasks = await page.evaluate(() =>
|
||||
window.__qaOutlineLongTasks.slice(),
|
||||
);
|
||||
const trafficTarget = network.filter(
|
||||
(entry) => entry.phase === 'traffic' && entry.targetTile,
|
||||
);
|
||||
const targetTile = `${TARGET.z}/${TARGET.x}/${TARGET.y}`;
|
||||
const targetProjectionMeasures = trafficMeasures.filter(
|
||||
(entry) =>
|
||||
entry.detail?.tile === targetTile &&
|
||||
['outlines:street-extract', 'outlines:polygon-extract'].includes(
|
||||
entry.name,
|
||||
),
|
||||
);
|
||||
const targetProjectionMs = sum(
|
||||
targetProjectionMeasures.map((entry) => entry.ms),
|
||||
);
|
||||
assert.ok(
|
||||
trafficTarget.length >= 1,
|
||||
`sample ${run}: traffic did not load ${TARGET_PATH}`,
|
||||
);
|
||||
assert.equal(
|
||||
targetProjectionMeasures.length,
|
||||
2,
|
||||
`sample ${run}: traffic did not prepare the target outline projection`,
|
||||
);
|
||||
assert.ok(
|
||||
targetProjectionMs <= FRAME_BUDGET_MS,
|
||||
`sample ${run}: traffic-idle target projection ${targetProjectionMs} ms exceeds ${FRAME_BUDGET_MS} ms`,
|
||||
);
|
||||
|
||||
phase = 'outline-cold';
|
||||
const cold = await driveOutline(page, `QA Capitol cold ${run}`);
|
||||
await sleep(750);
|
||||
cold.measures = relevantMeasures(cold.measures);
|
||||
cold.network = network.filter(
|
||||
(entry) => entry.phase === 'outline-cold' && entry.targetTile,
|
||||
);
|
||||
cold.longTasks = cold.longTasks.map((entry) => ({
|
||||
at: round(entry.at),
|
||||
ms: round(entry.ms),
|
||||
}));
|
||||
|
||||
await page.evaluate(() => window.__gevAnnotations.clear());
|
||||
phase = 'outline-warm';
|
||||
const warm = await driveOutline(page, `QA Capitol warm ${run}`);
|
||||
await sleep(750);
|
||||
warm.measures = relevantMeasures(warm.measures);
|
||||
warm.network = network.filter(
|
||||
(entry) => entry.phase === 'outline-warm' && entry.targetTile,
|
||||
);
|
||||
warm.longTasks = warm.longTasks.map((entry) => ({
|
||||
at: round(entry.at),
|
||||
ms: round(entry.ms),
|
||||
}));
|
||||
|
||||
const supported = await page.evaluate(() => window.__qaLongTaskSupported);
|
||||
assert.equal(supported, true, `sample ${run}: Long Tasks unsupported`);
|
||||
assert.deepEqual(pageErrors, [], `sample ${run}: browser errors`);
|
||||
assert.equal(cold.annotation.unavailable, false);
|
||||
assert.equal(cold.annotation.synthesized, false);
|
||||
assert.equal(cold.annotation.footprintKind, 'building');
|
||||
assert.ok(cold.annotation.ringPoints >= 4);
|
||||
assert.equal(warm.annotation.footprintKind, 'building');
|
||||
assert.ok(warm.annotation.ringPoints >= 4);
|
||||
|
||||
const coldDecodes = measures(cold.measures, 'roads:decode');
|
||||
const coldInstalls = measures(cold.measures, 'outlines:install');
|
||||
const warmDecodes = measures(warm.measures, 'roads:decode');
|
||||
assert.equal(
|
||||
cold.network.length,
|
||||
0,
|
||||
'shared decoded tile performs no second application fetch',
|
||||
);
|
||||
assert.equal(sum(cold.network.map((entry) => entry.transferBytes)), 0);
|
||||
assert.equal(coldDecodes.length, 0, 'shared decoded tile is not parsed again');
|
||||
assert.equal(
|
||||
measures(cold.measures, 'outlines:street-extract').length,
|
||||
0,
|
||||
'traffic idle preparation owns street extraction',
|
||||
);
|
||||
assert.equal(
|
||||
measures(cold.measures, 'outlines:polygon-extract').length,
|
||||
0,
|
||||
'traffic idle preparation owns polygon extraction',
|
||||
);
|
||||
assert.ok(coldInstalls.length >= 1, 'cold outline installs geometry');
|
||||
assert.equal(warm.network.length, 0, 'decoded-cache hit performs no fetch');
|
||||
assert.equal(warmDecodes.length, 0, 'decoded-cache hit performs no decode');
|
||||
|
||||
report.renderer ||= await page.evaluate(() => {
|
||||
const gl = window.__godsEyeView.viewer.scene.context._gl;
|
||||
const debug = gl.getExtension('WEBGL_debug_renderer_info');
|
||||
return debug
|
||||
? gl.getParameter(debug.UNMASKED_RENDERER_WEBGL)
|
||||
: gl.getParameter(gl.RENDERER);
|
||||
});
|
||||
if (args.includes('--headful')) {
|
||||
assert.doesNotMatch(
|
||||
String(report.renderer),
|
||||
/swiftshader|software rasterizer|llvmpipe/i,
|
||||
`sample ${run}: headful evidence requires a hardware renderer`,
|
||||
);
|
||||
}
|
||||
report.samples.push({
|
||||
run,
|
||||
pageErrors,
|
||||
traffic: {
|
||||
applicationFetches: trafficTarget.length,
|
||||
transferBytes: sum(trafficTarget.map((entry) => entry.transferBytes)),
|
||||
decodes: measures(trafficMeasures, 'roads:decode').length,
|
||||
targetProjectionMs: round(targetProjectionMs),
|
||||
longTasks: trafficLongTasks.map((entry) => ({
|
||||
at: round(entry.at),
|
||||
ms: round(entry.ms),
|
||||
})),
|
||||
measures: trafficMeasures,
|
||||
network: trafficTarget,
|
||||
},
|
||||
cold: {
|
||||
...cold,
|
||||
applicationFetches: cold.network.length,
|
||||
transferBytes: sum(cold.network.map((entry) => entry.transferBytes)),
|
||||
decodeMs: round(sum(coldDecodes)),
|
||||
bodyReadMs: round(sum(measures(cold.measures, 'roads:tile-body-read'))),
|
||||
streetExtractMs: round(
|
||||
sum(measures(cold.measures, 'outlines:street-extract')),
|
||||
),
|
||||
polygonExtractMs: round(
|
||||
sum(measures(cold.measures, 'outlines:polygon-extract')),
|
||||
),
|
||||
installMs: round(sum(coldInstalls)),
|
||||
mainThreadCpuMs: round(
|
||||
sum(coldDecodes) +
|
||||
sum(measures(cold.measures, 'outlines:street-extract')) +
|
||||
sum(measures(cold.measures, 'outlines:polygon-extract')) +
|
||||
sum(coldInstalls),
|
||||
),
|
||||
},
|
||||
warm: {
|
||||
...warm,
|
||||
applicationFetches: warm.network.length,
|
||||
transferBytes: sum(warm.network.map((entry) => entry.transferBytes)),
|
||||
decodeMs: round(sum(warmDecodes)),
|
||||
installMs: round(sum(measures(warm.measures, 'outlines:install'))),
|
||||
},
|
||||
});
|
||||
await page.close();
|
||||
}
|
||||
|
||||
const coldCpu = report.samples.map((sample) => sample.cold.mainThreadCpuMs);
|
||||
const outlineLongTasks = report.samples.flatMap((sample) => [
|
||||
...sample.cold.longTasks,
|
||||
...sample.warm.longTasks,
|
||||
]);
|
||||
const trafficLongTasks = report.samples.flatMap(
|
||||
(sample) => sample.traffic.longTasks,
|
||||
);
|
||||
report.summary = {
|
||||
samples: report.samples.length,
|
||||
overlapApplicationFetches: report.overlap?.applicationFetches ?? null,
|
||||
overlapDecodes: report.overlap?.decodes ?? null,
|
||||
coldCpuP50Ms: round(percentile(coldCpu, 0.5)),
|
||||
coldCpuP95Ms: round(percentile(coldCpu, 0.95)),
|
||||
coldCpuMaxMs: round(Math.max(...coldCpu)),
|
||||
coldCommandToRenderP95Ms: round(
|
||||
percentile(
|
||||
report.samples.map((sample) => sample.cold.commandToRenderMs),
|
||||
0.95,
|
||||
),
|
||||
),
|
||||
coldApplicationFetches: sum(
|
||||
report.samples.map((sample) => sample.cold.applicationFetches),
|
||||
),
|
||||
coldTransferBytes: sum(
|
||||
report.samples.map((sample) => sample.cold.transferBytes),
|
||||
),
|
||||
coldDecodes: sum(
|
||||
report.samples.map(
|
||||
(sample) => measures(sample.cold.measures, 'roads:decode').length,
|
||||
),
|
||||
),
|
||||
warmApplicationFetches: sum(
|
||||
report.samples.map((sample) => sample.warm.applicationFetches),
|
||||
),
|
||||
warmDecodes: sum(
|
||||
report.samples.map(
|
||||
(sample) => measures(sample.warm.measures, 'roads:decode').length,
|
||||
),
|
||||
),
|
||||
trafficIdleProjectionMaxMs: round(
|
||||
Math.max(
|
||||
...report.samples.map((sample) => sample.traffic.targetProjectionMs),
|
||||
),
|
||||
),
|
||||
trafficBaselineLongTasks: trafficLongTasks.length,
|
||||
trafficBaselineMaxLongTaskMs: round(
|
||||
trafficLongTasks.length
|
||||
? Math.max(...trafficLongTasks.map((entry) => entry.ms))
|
||||
: 0,
|
||||
),
|
||||
longTasks: outlineLongTasks.length,
|
||||
maxLongTaskMs: round(
|
||||
outlineLongTasks.length
|
||||
? Math.max(...outlineLongTasks.map((entry) => entry.ms))
|
||||
: 0,
|
||||
),
|
||||
};
|
||||
assert.ok(
|
||||
report.summary.coldCpuP95Ms <= FRAME_BUDGET_MS,
|
||||
`cold decode+extract+install p95 ${report.summary.coldCpuP95Ms} ms exceeds ${FRAME_BUDGET_MS} ms`,
|
||||
);
|
||||
assert.ok(
|
||||
report.summary.trafficIdleProjectionMaxMs <= FRAME_BUDGET_MS,
|
||||
'traffic-idle outline projection exceeded one frame',
|
||||
);
|
||||
assert.equal(report.summary.longTasks, 0, 'outline ask created a Long Task');
|
||||
assert.equal(report.summary.coldTransferBytes, 0);
|
||||
assert.equal(report.summary.coldApplicationFetches, 0);
|
||||
assert.equal(report.summary.coldDecodes, 0);
|
||||
assert.equal(report.summary.warmApplicationFetches, 0);
|
||||
assert.equal(report.summary.warmDecodes, 0);
|
||||
assert.equal(report.summary.overlapApplicationFetches, 1);
|
||||
assert.equal(report.summary.overlapDecodes, 1);
|
||||
report.passed = true;
|
||||
console.table(
|
||||
report.samples.map((sample) => ({
|
||||
run: sample.run,
|
||||
trafficPrepMs: sample.traffic.targetProjectionMs,
|
||||
appFetches: sample.cold.applicationFetches,
|
||||
transferBytes: sample.cold.transferBytes,
|
||||
bodyReadMs: sample.cold.bodyReadMs,
|
||||
decodeMs: sample.cold.decodeMs,
|
||||
streetMs: sample.cold.streetExtractMs,
|
||||
polygonMs: sample.cold.polygonExtractMs,
|
||||
installMs: sample.cold.installMs,
|
||||
cpuMs: sample.cold.mainThreadCpuMs,
|
||||
renderMs: round(sample.cold.commandToRenderMs),
|
||||
longTasks: sample.cold.longTasks.length,
|
||||
})),
|
||||
);
|
||||
console.log('OpenFreeMap same-tile acceptance passed:', report.summary);
|
||||
} catch (error) {
|
||||
report.passed = false;
|
||||
report.failure = String(error?.stack || error);
|
||||
console.error(error);
|
||||
process.exitCode = 1;
|
||||
} finally {
|
||||
await fs.writeFile(OUTPUT, `${JSON.stringify(report, null, 2)}\n`);
|
||||
await browser.close();
|
||||
}
|
||||
@@ -0,0 +1,411 @@
|
||||
#!/usr/bin/env node
|
||||
/**
|
||||
* qa-voice-bench.mjs — provider-neutral TEXT-mode voice-routing benchmark.
|
||||
*
|
||||
* Every provider gets the same prompt: production's realtimeInstructions()
|
||||
* and GEV_REALTIME_TOOLS (the Realtime adapter uses the exact session body the
|
||||
* token handler mints). Two suites, reported separately:
|
||||
* core the qa-voice-routing phrase table, graded with its semantics
|
||||
* (plus benchmark-only typed-argument checks)
|
||||
* coverage the newer-layers coverage probe, graded correct /
|
||||
* honest-refusal / hallucinated-capability / wrong-tool
|
||||
* dialogue two-turn dialogues in one context; turn 2 depends on turn 1
|
||||
* capability area handles / imagery / OSM / point-and-ask phrases (and two
|
||||
* dialogues) for the tools added after 4b56d0e9
|
||||
* Coverage rows are graded twice: v1 (original rubric, 4b56d0e9 tool schema)
|
||||
* and v2 (rubric for the current tool surface).
|
||||
*
|
||||
* Methodology: a fresh context per phrase (new Realtime/Live session, or a
|
||||
* stateless chat request); stub tool results; continuations until the model
|
||||
* stops calling tools (max 4). No app page is involved.
|
||||
*
|
||||
* Usage:
|
||||
* node scripts/qa-voice-bench.mjs --provider openai-realtime --model gpt-realtime-2 --reps 2
|
||||
* node scripts/qa-voice-bench.mjs --provider gemini-live --model gemini-3.8-live
|
||||
* node scripts/qa-voice-bench.mjs --provider ollama --model gemma4:e4b
|
||||
* node scripts/qa-voice-bench.mjs --provider openai-chat --base http://localhost:1234/v1 --model <id>
|
||||
* node scripts/qa-voice-bench.mjs --summarize --out qa-shots/voice-bench/<run>
|
||||
* Common flags: --out <dir> (reuse one dir to collect several models),
|
||||
* --suite core,coverage --only <substr> --budget <usd, cloud total>
|
||||
* --variant <label> --temperature <t> --num-ctx <n> --think --note <text>
|
||||
* --reps <n> --start-rep <k> (add repetitions k..k+n-1 to an existing run)
|
||||
* --replace (supersede earlier runs of this model for the suites run)
|
||||
* --compare <run dir> (add a before → after table against another run)
|
||||
*
|
||||
* Output (gitignored): <out>/results.jsonl, summary.json, summary.md.
|
||||
* Keys come from the environment or the macOS Keychain at runtime only.
|
||||
*/
|
||||
import fs from 'node:fs';
|
||||
import path from 'node:path';
|
||||
import os from 'node:os';
|
||||
import { execFileSync } from 'node:child_process';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import {
|
||||
CORE_PHRASES, COVERAGE_PROBE, DIALOGUES, CAPABILITY_EXTRAS,
|
||||
dialogueResult, coverageRubric,
|
||||
} from './voice-bench/phrases.mjs';
|
||||
import { gradeCore, gradeCoverage, gradeDialogue, expectLabel } from './voice-bench/grade.mjs';
|
||||
import { stubResult } from './voice-bench/stub.mjs';
|
||||
import { summarize, renderMarkdown, renderComparison } from './voice-bench/report.mjs';
|
||||
import { GEV_REALTIME_TOOLS, realtimeInstructions } from './voice-bench/productionConfig.mjs';
|
||||
import { chatNormalizationDiff } from './voice-bench/toolFormats.mjs';
|
||||
import { createOpenAiRealtimeAdapter } from './voice-bench/adapters/openaiRealtime.mjs';
|
||||
import { createGeminiLiveAdapter } from './voice-bench/adapters/geminiLive.mjs';
|
||||
import { createOpenAiChatAdapter, createOllamaAdapter } from './voice-bench/adapters/localChat.mjs';
|
||||
|
||||
const ROOT = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '..');
|
||||
const argv = process.argv.slice(2);
|
||||
const opt = (flag, fallback) => {
|
||||
const i = argv.indexOf(flag);
|
||||
return i >= 0 && argv[i + 1] !== undefined ? argv[i + 1] : fallback;
|
||||
};
|
||||
const has = (flag) => argv.includes(flag);
|
||||
|
||||
const PROVIDER = opt('--provider', null);
|
||||
const MODEL = opt('--model', null);
|
||||
const CLOUD = PROVIDER === 'openai-realtime' || PROVIDER === 'gemini-live';
|
||||
const REPS = Number(opt('--reps', CLOUD ? '2' : '1'));
|
||||
const START_REP = Number(opt('--start-rep', '1'));
|
||||
const SUITES = opt('--suite', 'core,coverage,dialogue,capability').split(',');
|
||||
const ONLY = opt('--only', null);
|
||||
const BUDGET = Number(opt('--budget', '30'));
|
||||
const VARIANT = opt('--variant', null);
|
||||
const OUT = path.resolve(ROOT, opt('--out', path.join('qa-shots', 'voice-bench', new Date().toISOString().replace(/[:.]/g, '-'))));
|
||||
const LEDGER = path.join(ROOT, 'qa-shots', 'voice-bench', 'cloud-spend.json');
|
||||
|
||||
fs.mkdirSync(OUT, { recursive: true });
|
||||
const RESULTS = path.join(OUT, 'results.jsonl');
|
||||
const NOTES = path.join(OUT, 'notes.json');
|
||||
|
||||
function readRows() {
|
||||
if (!fs.existsSync(RESULTS)) return [];
|
||||
// Several runs may append to one results file (a cloud run beside a local
|
||||
// one); skip a line another process is still writing.
|
||||
return fs.readFileSync(RESULTS, 'utf8').split('\n').filter(Boolean).flatMap((l) => {
|
||||
try { return [JSON.parse(l)]; } catch { return []; }
|
||||
});
|
||||
}
|
||||
function appendRow(row) {
|
||||
fs.appendFileSync(RESULTS, `${JSON.stringify(row)}\n`);
|
||||
}
|
||||
function readNotes() {
|
||||
try { return JSON.parse(fs.readFileSync(NOTES, 'utf8')); } catch { return []; }
|
||||
}
|
||||
/**
|
||||
* Re-grade stored turns with the CURRENT phrase tables and grader, so a
|
||||
* grader fix applies to every model already in the results file. The raw
|
||||
* calls and text in results.jsonl are the source of truth; `gradeAtRun`
|
||||
* keeps the verdict recorded at run time.
|
||||
*/
|
||||
function regrade(rows) {
|
||||
const core = new Map(CORE_PHRASES.map((p) => [p.phrase, p]));
|
||||
const cov = new Map(COVERAGE_PROBE.map((c) => [c.id, c]));
|
||||
const dia = new Map(DIALOGUES.map((d) => [d.id, d]));
|
||||
const cap = new Map(CAPABILITY_ITEMS.map((c) => [c.id, c]));
|
||||
// A run started with --replace supersedes earlier runs of the same
|
||||
// provider/model/variant for the suites it covers.
|
||||
const key = (r) => `${r.provider}:${r.model}#${r.variant || ''}`;
|
||||
const runs = rows.filter((r) => r.type === 'run');
|
||||
const superseded = new Set();
|
||||
for (const r of runs.filter((x) => x.replace)) {
|
||||
for (const older of runs) {
|
||||
if (older.at < r.at && key(older) === key(r)) superseded.add(`${older.runId}|${(r.suites || []).join(',')}`);
|
||||
}
|
||||
}
|
||||
const isSuperseded = (row) => [...superseded].some((s) => {
|
||||
const [runId, suites] = s.split('|');
|
||||
return runId === row.runId && suites.split(',').includes(row.suite);
|
||||
});
|
||||
return rows.filter((row) => row.type !== 'turn' || !isSuperseded(row)).map((row) => {
|
||||
if (row.type !== 'turn') return row;
|
||||
if (row.suite === 'dialogue' && dia.has(row.phraseId)) {
|
||||
return { ...row, gradeAtRun: row.grade, grade: gradeDialogue(dia.get(row.phraseId), [row.turn1?.calls || [], row.calls || []]) };
|
||||
}
|
||||
if (row.suite === 'core' && core.has(row.phraseId)) {
|
||||
return { ...row, gradeAtRun: row.grade, grade: gradeCore(core.get(row.phraseId), row.calls || []) };
|
||||
}
|
||||
if (row.suite === 'coverage' && cov.has(row.phraseId)) {
|
||||
const item = cov.get(row.phraseId);
|
||||
return {
|
||||
...row,
|
||||
gradeAtRun: row.grade,
|
||||
grade: gradeCoverage(item, row.calls || [], row.text, BASELINE_TOOLS),
|
||||
gradeV2: gradeCoverage(coverageRubric(item, 'v2'), row.calls || [], row.text, GEV_REALTIME_TOOLS),
|
||||
};
|
||||
}
|
||||
if (row.suite === 'capability' && cap.has(row.phraseId)) {
|
||||
const c = cap.get(row.phraseId);
|
||||
const grade = c.dialogue ? gradeDialogue(c.dialogue, [row.turn1?.calls || [], row.calls || []]) : gradeCore(c.phrase, row.calls || []);
|
||||
return { ...row, gradeAtRun: row.grade, grade };
|
||||
}
|
||||
return row;
|
||||
});
|
||||
}
|
||||
|
||||
function readRowsFrom(dir) {
|
||||
const file = path.join(dir, 'results.jsonl');
|
||||
if (!fs.existsSync(file)) return [];
|
||||
return fs.readFileSync(file, 'utf8').split('\n').filter(Boolean).flatMap((l) => {
|
||||
try { return [JSON.parse(l)]; } catch { return []; }
|
||||
});
|
||||
}
|
||||
|
||||
function writeSummary() {
|
||||
const summary = summarize(regrade(readRows()));
|
||||
// --compare <dir> (remembered in compare-with.txt): a before → after table
|
||||
// against another run folder, both re-graded with the current grader.
|
||||
const compareFile = path.join(OUT, 'compare-with.txt');
|
||||
if (has('--compare')) fs.writeFileSync(compareFile, `${path.resolve(ROOT, opt('--compare', ''))}\n`);
|
||||
let comparison = '';
|
||||
if (fs.existsSync(compareFile)) {
|
||||
const beforeDir = fs.readFileSync(compareFile, 'utf8').trim();
|
||||
const before = summarize(regrade(readRowsFrom(beforeDir)));
|
||||
const beforeHead = readRowsFrom(beforeDir).find((r) => r.type === 'run' && r.prompt?.gitHead)?.prompt?.gitHead || '4b56d0e9';
|
||||
const afterHead = readRows().find((r) => r.type === 'run' && r.prompt?.gitHead)?.prompt?.gitHead || '?';
|
||||
comparison = renderComparison(before, summary, {
|
||||
beforeLabel: `${path.relative(ROOT, beforeDir)} (prompt/tools @ ${beforeHead})`,
|
||||
afterLabel: `${path.relative(ROOT, OUT)} (prompt/tools @ ${afterHead})`,
|
||||
});
|
||||
summary.comparison = { beforeDir: path.relative(ROOT, beforeDir), before: before.models.map((m) => ({ key: m.key, coreFull: m.core.fullCallAccuracyPct, coverageV1Pct: m.coverage.acceptablePct, hcV1: m.coverage.counts['hallucinated-capability'], dialogueT2: m.dialogue.turn2FullPct, toolCallP50: m.latencyMs.toolCallP50, promptTokens: m.warmup?.promptTokens ?? m.tokens.inputPerRequestMean })) };
|
||||
fs.writeFileSync(path.join(OUT, 'comparison.md'), `${comparison}\n`);
|
||||
}
|
||||
fs.writeFileSync(path.join(OUT, 'summary.json'), JSON.stringify(summary, null, 2));
|
||||
const md = renderMarkdown(summary, { notes: readNotes(), payload: payloadChanges() });
|
||||
fs.writeFileSync(path.join(OUT, 'summary.md'), comparison ? md.replace('## Routing accuracy', `${comparison}\n## Routing accuracy`) : md);
|
||||
return summary;
|
||||
}
|
||||
/** Exactly what each provider receives differently from production's payload. */
|
||||
function payloadChanges() {
|
||||
const chat = chatNormalizationDiff(GEV_REALTIME_TOOLS)
|
||||
.map((c) => `\`${c.path}\`: ${JSON.stringify(c.from)} → ${JSON.stringify(c.to)}`);
|
||||
return [
|
||||
`openai-realtime: none — the session body is captured from production's token handler (${GEV_REALTIME_TOOLS.length} tools, instructions, tool_choice auto, reasoning low, truncation). Only difference from a mic session: response.create asks for output_modalities ["text"] (as qa-voice-routing does).`,
|
||||
'gemini-live: same instructions text as systemInstruction; the same tool schemas passed verbatim as functionDeclarations[].parametersJsonSchema (no keyword changes); tool_choice has no Live equivalent (AUTO is the default); responseModalities AUDIO + outputAudioTranscription (TEXT is rejected by the model).',
|
||||
`ollama / openai-chat: same instructions as the system message; tools wrapped as {type:"function",function:{name,description,parameters}} with ${chat.length} schema change(s): ${chat.join('; ') || 'none'}. Descriptions, enums and required lists are unchanged; nothing is stringified.`,
|
||||
];
|
||||
}
|
||||
|
||||
function ledgerSpent() {
|
||||
try { return JSON.parse(fs.readFileSync(LEDGER, 'utf8')).spentUsd || 0; } catch { return 0; }
|
||||
}
|
||||
function ledgerAdd(usd) {
|
||||
const spent = ledgerSpent() + usd;
|
||||
fs.mkdirSync(path.dirname(LEDGER), { recursive: true });
|
||||
fs.writeFileSync(LEDGER, JSON.stringify({ spentUsd: spent, updatedAt: new Date().toISOString() }));
|
||||
return spent;
|
||||
}
|
||||
const sh = (cmd, args) => {
|
||||
try { return execFileSync(cmd, args, { encoding: 'utf8', stdio: ['ignore', 'pipe', 'ignore'], timeout: 10_000 }).trim(); } catch { return null; }
|
||||
};
|
||||
|
||||
/** Snapshot of what else is using the machine (GPU-heavy servers, load). */
|
||||
function machineSnapshot() {
|
||||
const load = os.loadavg().map((x) => x.toFixed(2)).join(' ');
|
||||
const ollamaPs = sh('ollama', ['ps']);
|
||||
const procs = sh('ps', ['-Ao', 'pcpu=,rss=,comm=']) || '';
|
||||
const heavy = procs.split('\n').map((l) => l.trim().split(/\s+/)).filter((p) => Number(p[0]) > 20)
|
||||
.map((p) => `${path.basename(p.slice(2).join(' '))} ${p[0]}%cpu`).slice(0, 6);
|
||||
const gpuServers = procs.split('\n').filter((l) => /ollama runner|llama-server|LM Studio|lms|mlx|whisper/i.test(l))
|
||||
.map((l) => path.basename(l.trim().split(/\s+/).slice(2).join(' '))).slice(0, 6);
|
||||
return {
|
||||
at: new Date().toISOString(),
|
||||
loadavg: load,
|
||||
ollamaPs,
|
||||
busyProcesses: heavy,
|
||||
modelServers: gpuServers,
|
||||
summary: `load ${load}; busy: ${heavy.join(', ') || 'none'}`,
|
||||
};
|
||||
}
|
||||
|
||||
// ── Ollama lifecycle ────────────────────────────────────────────────────────
|
||||
|
||||
const OLLAMA = opt('--base', 'http://localhost:11434').replace(/\/v1\/?$/, '');
|
||||
|
||||
async function ollamaUnloadAll() {
|
||||
const ps = await fetch(`${OLLAMA}/api/ps`).then((r) => r.json()).catch(() => ({ models: [] }));
|
||||
for (const m of ps.models || []) {
|
||||
await fetch(`${OLLAMA}/api/generate`, { method: 'POST', body: JSON.stringify({ model: m.name, keep_alive: 0 }) }).then((r) => r.text()).catch(() => null);
|
||||
}
|
||||
return (ps.models || []).map((m) => m.name);
|
||||
}
|
||||
|
||||
async function ollamaLoad(model, numCtx) {
|
||||
const t = Date.now();
|
||||
const res = await fetch(`${OLLAMA}/api/generate`, {
|
||||
method: 'POST',
|
||||
body: JSON.stringify({ model, keep_alive: '30m', options: { num_ctx: numCtx } }),
|
||||
signal: AbortSignal.timeout(600_000),
|
||||
});
|
||||
const body = await res.json().catch(() => ({}));
|
||||
if (!res.ok || body.error) throw new Error(`load failed: ${body.error || res.status}`);
|
||||
return { wallMs: Date.now() - t, loadMs: body.load_duration ? body.load_duration / 1e6 : null };
|
||||
}
|
||||
|
||||
// ── Main ────────────────────────────────────────────────────────────────────
|
||||
|
||||
function buildAdapter() {
|
||||
const temperature = Number(opt('--temperature', '0.8'));
|
||||
switch (PROVIDER) {
|
||||
case 'openai-realtime':
|
||||
return createOpenAiRealtimeAdapter({ model: MODEL });
|
||||
case 'gemini-live':
|
||||
return createGeminiLiveAdapter({ model: MODEL, schemaMode: opt('--gemini-schema', 'jsonSchema') });
|
||||
case 'ollama':
|
||||
return createOllamaAdapter({ model: MODEL, baseUrl: OLLAMA, temperature, numCtx: Number(opt('--num-ctx', '32768')), think: has('--think') });
|
||||
case 'openai-chat':
|
||||
return createOpenAiChatAdapter({ model: MODEL, baseUrl: opt('--base', 'http://localhost:1234/v1'), temperature });
|
||||
default:
|
||||
throw new Error(`unknown --provider ${PROVIDER}`);
|
||||
}
|
||||
}
|
||||
|
||||
const COLD_PHRASE = 'Take me to Tokyo';
|
||||
|
||||
/** Capability suite: single phrases (optional pointer context) and dialogues. */
|
||||
const CAPABILITY_ITEMS = CAPABILITY_EXTRAS.map((c) => (c.turns
|
||||
? { suite: 'capability', id: c.id, text: c.turns.join(' → '), dialogue: c }
|
||||
: { suite: 'capability', id: c.context ? `${c.phrase} [pointer]` : c.phrase, text: c.phrase, phrase: c, context: c.context || null }));
|
||||
|
||||
/** The 4b56d0e9 tool schema, for grading coverage by the original yardstick. */
|
||||
const BASELINE_TOOLS = JSON.parse(fs.readFileSync(path.join(ROOT, 'scripts', 'voice-bench', 'baseline-tools-4b56d0e9.json'), 'utf8'));
|
||||
|
||||
/** What prompt this run actually sent (so before/after runs are identifiable). */
|
||||
function promptVersion() {
|
||||
let head = null;
|
||||
try { head = execFileSync('git', ['-C', ROOT, 'rev-parse', '--short', 'HEAD'], { encoding: 'utf8' }).trim(); } catch { /* not a checkout */ }
|
||||
return {
|
||||
gitHead: head,
|
||||
tools: GEV_REALTIME_TOOLS.length,
|
||||
toolJsonChars: JSON.stringify(GEV_REALTIME_TOOLS).length,
|
||||
instructionsChars: realtimeInstructions().length,
|
||||
};
|
||||
}
|
||||
|
||||
function workList() {
|
||||
const items = [];
|
||||
if (SUITES.includes('core')) {
|
||||
for (const p of CORE_PHRASES) items.push({ suite: 'core', id: p.phrase, text: p.phrase, phrase: p });
|
||||
}
|
||||
if (SUITES.includes('coverage')) {
|
||||
for (const c of COVERAGE_PROBE) items.push({ suite: 'coverage', id: c.id, text: c.utterance, item: c });
|
||||
}
|
||||
if (SUITES.includes('dialogue')) {
|
||||
for (const d of DIALOGUES) items.push({ suite: 'dialogue', id: d.id, text: d.turns.join(' → '), dialogue: d });
|
||||
}
|
||||
if (SUITES.includes('capability')) items.push(...CAPABILITY_ITEMS);
|
||||
return items.filter((w) => !ONLY || w.text.toLowerCase().includes(ONLY.toLowerCase()));
|
||||
}
|
||||
|
||||
async function main() {
|
||||
if (has('--note')) {
|
||||
const notes = readNotes();
|
||||
notes.push(opt('--note', ''));
|
||||
fs.writeFileSync(NOTES, JSON.stringify(notes, null, 2));
|
||||
}
|
||||
if (has('--summarize') || !PROVIDER) {
|
||||
writeSummary();
|
||||
console.log(`summary written to ${path.join(OUT, 'summary.md')}`);
|
||||
return;
|
||||
}
|
||||
if (!MODEL) throw new Error('--model is required');
|
||||
const adapter = buildAdapter();
|
||||
await adapter.open();
|
||||
|
||||
let warmup = null;
|
||||
if (PROVIDER === 'ollama') {
|
||||
const unloaded = await ollamaUnloadAll();
|
||||
const numCtx = Number(opt('--num-ctx', '32768'));
|
||||
const load = await ollamaLoad(MODEL, numCtx);
|
||||
// COLD turn: the first request after load pays the full-prompt prefill.
|
||||
// It is recorded separately (warmup.cold*) and primes the prefix cache
|
||||
// the way a live session's first turn would; every benchmark turn after
|
||||
// it is steady state.
|
||||
const w = await adapter.runTurn({ text: COLD_PHRASE, onCall: () => ({ ok: true }) });
|
||||
warmup = { unloaded, loadWallMs: load.wallMs, loadMs: load.loadMs, coldPhrase: COLD_PHRASE, prefillTokS: w.local.prefillTokS, promptTokens: w.local.promptEvalCount, coldFirstTokenMs: w.tFirstTokenMs, coldToolCallMs: w.tFirstCallMs, coldCalls: w.calls.map((c) => c.name), warmupTurnMs: w.tDoneMs, errors: w.errors };
|
||||
console.log(`warm-up: load ${Math.round(load.wallMs)}ms, prefill ${Math.round(w.local.prefillTokS || 0)} tok/s over ${w.local.promptEvalCount} tokens`);
|
||||
if (w.errors.length) console.log(` warm-up errors: ${w.errors.join('; ')}`);
|
||||
} else if (PROVIDER === 'openai-chat') {
|
||||
const w = await adapter.runTurn({ text: COLD_PHRASE, onCall: () => ({ ok: true }) });
|
||||
warmup = { loadMs: Number(opt('--load-ms', 'NaN')) || null, coldPhrase: COLD_PHRASE, prefillTokS: w.local.prefillTokS, promptTokens: w.local.promptEvalCount, coldFirstTokenMs: w.tFirstTokenMs, coldToolCallMs: w.tFirstCallMs, coldCalls: w.calls.map((c) => c.name), warmupTurnMs: w.tDoneMs, errors: w.errors };
|
||||
console.log(`warm-up: ${w.tDoneMs}ms prefill≈${Math.round(w.local.prefillTokS || 0)} tok/s`);
|
||||
if (w.errors.length) throw new Error(`warm-up failed, not benchmarking: ${w.errors[0]}`);
|
||||
}
|
||||
|
||||
const runId = `${Date.now().toString(36)}`;
|
||||
const machine = CLOUD ? null : machineSnapshot();
|
||||
appendRow({ type: 'run', runId, at: new Date().toISOString(), provider: PROVIDER, model: MODEL, variant: VARIANT, reps: REPS, suites: SUITES, settings: adapter.settings, machine, warmup, replace: has('--replace'), prompt: promptVersion(), methodology: 'fresh context per phrase/dialogue; stub tool results (canned results for dialogue turn 1); ≤4 continuations' });
|
||||
|
||||
const items = workList();
|
||||
let stopReason = null;
|
||||
let turnCosts = [];
|
||||
outer: for (let rep = START_REP; rep < START_REP + REPS; rep += 1) {
|
||||
console.log(`\n== ${PROVIDER} ${MODEL}${VARIANT ? ` (${VARIANT})` : ''} — rep ${rep}/${REPS}, ${items.length} phrases`);
|
||||
for (const w of items) {
|
||||
if (CLOUD) {
|
||||
const avg = turnCosts.length ? turnCosts.reduce((a, b) => a + b, 0) / turnCosts.length : 0.1;
|
||||
if (ledgerSpent() + avg > BUDGET) { stopReason = `cloud budget $${BUDGET} reached (ledger $${ledgerSpent().toFixed(2)})`; break outer; }
|
||||
}
|
||||
let r;
|
||||
let turn1 = null;
|
||||
let grade;
|
||||
if (w.dialogue) {
|
||||
const rs = await adapter.runDialogue({
|
||||
turns: w.dialogue.turns,
|
||||
onCall: (c, turn) => dialogueResult(w.dialogue, c.name, turn) ?? stubResult('core', c, GEV_REALTIME_TOOLS),
|
||||
});
|
||||
const [a, b] = rs;
|
||||
turn1 = { text: a.text.slice(0, 400), calls: a.calls, tFirstCallMs: a.tFirstCallMs, tDoneMs: a.tDoneMs, errors: a.errors };
|
||||
r = {
|
||||
...b,
|
||||
errors: [...a.errors, ...b.errors],
|
||||
timeout: a.timeout || b.timeout,
|
||||
costUsd: a.costUsd + b.costUsd,
|
||||
usage: Object.fromEntries(Object.keys(b.usage).map((k) => [k, (a.usage[k] || 0) + (b.usage[k] || 0)])),
|
||||
setupMs: a.setupMs,
|
||||
};
|
||||
grade = gradeDialogue(w.dialogue, [a.calls, b.calls]);
|
||||
} else {
|
||||
r = await adapter.runTurn({ text: w.text, context: w.context || undefined, onCall: (c) => stubResult(w.suite, c, GEV_REALTIME_TOOLS) });
|
||||
grade = w.suite === 'coverage' ? gradeCoverage(w.item, r.calls, r.text, GEV_REALTIME_TOOLS) : gradeCore(w.phrase, r.calls);
|
||||
}
|
||||
const row = {
|
||||
type: 'turn', runId, provider: PROVIDER, model: MODEL, variant: VARIANT, cloud: CLOUD, rep, suite: w.suite,
|
||||
phraseId: w.id, phrase: w.text,
|
||||
expect: w.dialogue ? `t1 ${expectLabel(w.dialogue.first)} → t2 ${expectLabel(w.dialogue.second)}` : w.suite === 'coverage' ? w.item.expect : expectLabel(w.phrase),
|
||||
context: w.context || undefined,
|
||||
expectNone: Boolean(w.phrase?.expectNone), supported: w.item ? w.item.supported : undefined,
|
||||
calls: r.calls, text: r.text.slice(0, 800), grade, turn1: turn1 || undefined,
|
||||
tFirstTokenMs: r.tFirstTokenMs, tFirstCallMs: r.tFirstCallMs, tDoneMs: r.tDoneMs, setupMs: r.setupMs,
|
||||
responses: r.responses, usage: r.usage, costUsd: r.costUsd, errors: r.errors, timeout: r.timeout,
|
||||
local: r.local || undefined,
|
||||
at: new Date().toISOString(),
|
||||
};
|
||||
appendRow(row);
|
||||
if (CLOUD) { ledgerAdd(r.costUsd); turnCosts.push(r.costUsd); }
|
||||
const verdict = w.suite === 'coverage' ? grade.verdict : (grade.ok ? 'PASS' : 'FAIL');
|
||||
console.log(` [${verdict}] ${w.text} → ${turn1 ? `${turn1.calls.map((c) => c.name).join(',') || '(none)'} ⇒ ` : ''}${r.calls.map((c) => c.name).join(',') || '(none)'} tool@${r.tFirstCallMs ?? '—'}ms done@${r.tDoneMs}ms $${r.costUsd.toFixed(4)}${r.errors.length ? ` ERR ${r.errors[0]}` : ''}${r.timeout ? ' TIMEOUT' : ''}`);
|
||||
if (r.errors.some((e) => /HTTP 4\d\d|close 100[78]|invalid|not found|model_not_found|does not exist/i.test(e)) && r.calls.length === 0 && !r.text) {
|
||||
const same = readRows().filter((x) => x.type === 'turn' && x.runId === runId).slice(-3);
|
||||
if (same.length === 3 && same.every((x) => x.errors?.length && !x.calls.length)) { stopReason = `3 consecutive hard errors: ${r.errors[0]}`; break outer; }
|
||||
}
|
||||
}
|
||||
writeSummary();
|
||||
}
|
||||
await adapter.close();
|
||||
if (PROVIDER === 'ollama' && !has('--keep-loaded')) {
|
||||
await fetch(`${OLLAMA}/api/generate`, { method: 'POST', body: JSON.stringify({ model: MODEL, keep_alive: 0 }) }).catch(() => null);
|
||||
}
|
||||
if (stopReason) {
|
||||
console.log(`STOPPED: ${stopReason}`);
|
||||
appendRow({ type: 'stop', runId, provider: PROVIDER, model: MODEL, reason: stopReason, at: new Date().toISOString() });
|
||||
}
|
||||
const summary = writeSummary();
|
||||
const mine = summary.models.find((m) => m.provider === PROVIDER && m.model === MODEL && (m.variant || null) === (VARIANT || null));
|
||||
if (mine) console.log(`\n${mine.key}: core ${mine.core.accuracyPct}% | coverage acceptable ${mine.coverage.acceptablePct}% | p50 tool ${mine.latencyMs.toolCallP50}ms | $${mine.costUsd}`);
|
||||
console.log(`results: ${RESULTS}\nsummary: ${path.join(OUT, 'summary.md')}\ncloud ledger: $${ledgerSpent().toFixed(3)}`);
|
||||
}
|
||||
|
||||
main().catch((e) => {
|
||||
console.error(e);
|
||||
process.exitCode = 1;
|
||||
});
|
||||
+928
-151
File diff suppressed because it is too large.
Load diff
@@ -0,0 +1,179 @@
|
||||
/**
|
||||
* Gemini Live adapter — BidiGenerateContent over WebSocket.
|
||||
*
|
||||
* Same prompt as production (system instruction = realtimeInstructions(),
|
||||
* tools = GEV_REALTIME_TOOLS as functionDeclarations with the production JSON
|
||||
* schema passed verbatim via parametersJsonSchema). The native-audio Live
|
||||
* model rejects a TEXT response modality (close 1007), so the session
|
||||
* requests AUDIO with output transcription on; tool calls arrive as
|
||||
* `toolCall` messages and the transcript stands in for the final words.
|
||||
*
|
||||
* Fresh session per phrase / dialogue. Each toolCall gets stub
|
||||
* `functionResponses`; the model continues on its own. A turn ends at a
|
||||
* `turnComplete` whose generation had no toolCall, or after 5 tool rounds
|
||||
* (mirrors the Realtime adapter's continuation cap), or 12 s of silence after
|
||||
* a tool round.
|
||||
*/
|
||||
import WebSocket from 'ws';
|
||||
import { providerKey } from '../keys.mjs';
|
||||
import { realtimeInstructions, GEV_REALTIME_TOOLS } from '../productionConfig.mjs';
|
||||
import { toGeminiDeclarations } from '../toolFormats.mjs';
|
||||
import { geminiUsageCostUsd } from '../pricing.mjs';
|
||||
import { newTurnResult } from './openaiRealtime.mjs';
|
||||
|
||||
const ENDPOINT = 'wss://generativelanguage.googleapis.com/ws/google.ai.generativelanguage.v1beta.GenerativeService.BidiGenerateContent';
|
||||
const MAX_TOOL_ROUNDS = 5;
|
||||
const IDLE_AFTER_TOOL_MS = 12_000;
|
||||
|
||||
export function createGeminiLiveAdapter({ model, schemaMode = 'jsonSchema', turnTimeoutMs = 60_000 }) {
|
||||
const instructions = realtimeInstructions();
|
||||
const functionDeclarations = toGeminiDeclarations(GEV_REALTIME_TOOLS, { mode: schemaMode });
|
||||
const adapter = {
|
||||
provider: 'gemini-live',
|
||||
model,
|
||||
settings: {
|
||||
responseModalities: ['AUDIO'],
|
||||
outputAudioTranscription: true,
|
||||
schemaMode,
|
||||
temperature: 'server default (not set; production has no Gemini config)',
|
||||
thinking: 'server default',
|
||||
input: 'realtimeInput.text',
|
||||
context: 'fresh session per phrase / dialogue',
|
||||
},
|
||||
async open() {},
|
||||
async close() {},
|
||||
async runTurn({ text, onCall, context }) {
|
||||
return (await adapter.runDialogue({ turns: [text], onCall, contexts: [context] }))[0];
|
||||
},
|
||||
runDialogue({ turns, onCall, contexts = [] }) {
|
||||
return new Promise((resolveAll) => {
|
||||
const tSetup = Date.now();
|
||||
const ws = new WebSocket(`${ENDPOINT}?key=${providerKey('gemini')}`);
|
||||
const results = [];
|
||||
let setupMs = null;
|
||||
let cur = null;
|
||||
let closed = false;
|
||||
const send = (o) => { if (ws.readyState === WebSocket.OPEN) ws.send(JSON.stringify(o)); };
|
||||
const shutdown = (error) => {
|
||||
if (closed) return;
|
||||
closed = true;
|
||||
if (cur) {
|
||||
clearTimeout(cur.timer);
|
||||
clearTimeout(cur.idle);
|
||||
if (error) cur.r.errors.push(error);
|
||||
if (cur.r.tDoneMs === null) cur.r.tDoneMs = Date.now() - cur.t0;
|
||||
results.push(cur.r);
|
||||
cur = null;
|
||||
}
|
||||
while (results.length < turns.length) {
|
||||
const r = newTurnResult();
|
||||
r.errors.push(error || 'dialogue aborted');
|
||||
results.push(r);
|
||||
}
|
||||
try { ws.terminate(); } catch { /* noop */ }
|
||||
resolveAll(results);
|
||||
};
|
||||
const startTurn = (turn) => {
|
||||
const r = newTurnResult();
|
||||
r.setupMs = turn === 0 ? setupMs : 0;
|
||||
cur = { r, t0: Date.now(), turn, toolRounds: 0, toolCallThisGeneration: false, idle: null };
|
||||
cur.timer = setTimeout(() => { cur.r.timeout = true; endTurn(); }, turnTimeoutMs);
|
||||
// Live has no system-role mid-session item; a context item (pointer_context)
|
||||
// rides as a JSON line ahead of the user's words in the same text turn.
|
||||
const ctx = contexts[turn] ? `${JSON.stringify(contexts[turn])}\n` : '';
|
||||
send({ realtimeInput: { text: `${ctx}${turns[turn]}` } });
|
||||
};
|
||||
const endTurn = () => {
|
||||
if (!cur) return;
|
||||
clearTimeout(cur.timer);
|
||||
clearTimeout(cur.idle);
|
||||
if (cur.r.tDoneMs === null) cur.r.tDoneMs = Date.now() - cur.t0;
|
||||
results.push(cur.r);
|
||||
const next = cur.turn + 1;
|
||||
const timedOut = cur.r.timeout;
|
||||
cur = null;
|
||||
if (next < turns.length && !timedOut) startTurn(next);
|
||||
else shutdown();
|
||||
};
|
||||
const mark = (key) => { if (cur && cur.r[key] === null) cur.r[key] = Date.now() - cur.t0; };
|
||||
|
||||
ws.on('open', () => {
|
||||
send({
|
||||
setup: {
|
||||
model: `models/${model}`,
|
||||
generationConfig: { responseModalities: ['AUDIO'] },
|
||||
systemInstruction: { parts: [{ text: instructions }] },
|
||||
tools: [{ functionDeclarations }],
|
||||
outputAudioTranscription: {},
|
||||
},
|
||||
});
|
||||
});
|
||||
ws.on('unexpected-response', (_req, res) => shutdown(`HTTP ${res.statusCode}`));
|
||||
ws.on('error', (e) => shutdown(`ws: ${e.message}`));
|
||||
ws.on('close', (code, reason) => {
|
||||
shutdown(results.length < turns.length ? `close ${code}: ${reason.toString().slice(0, 200)}` : undefined);
|
||||
});
|
||||
ws.on('message', (raw) => {
|
||||
let m;
|
||||
try { m = JSON.parse(raw.toString()); } catch { return; }
|
||||
if (m.setupComplete) {
|
||||
if (setupMs === null) {
|
||||
setupMs = Date.now() - tSetup;
|
||||
startTurn(0);
|
||||
}
|
||||
return;
|
||||
}
|
||||
if (!cur) return;
|
||||
clearTimeout(cur.idle);
|
||||
const r = cur.r;
|
||||
const lastAt = Date.now();
|
||||
const sc = m.serverContent;
|
||||
if (sc?.modelTurn?.parts?.length) {
|
||||
mark('tFirstTokenMs');
|
||||
for (const p of sc.modelTurn.parts) if (typeof p.text === 'string' && !p.thought) r.text += p.text;
|
||||
}
|
||||
if (sc?.outputTranscription?.text) {
|
||||
mark('tFirstTokenMs');
|
||||
r.text += sc.outputTranscription.text;
|
||||
}
|
||||
if (m.toolCall?.functionCalls?.length) {
|
||||
mark('tFirstTokenMs');
|
||||
mark('tFirstCallMs');
|
||||
cur.toolCallThisGeneration = true;
|
||||
cur.toolRounds += 1;
|
||||
const functionResponses = m.toolCall.functionCalls.map((fc) => {
|
||||
const c = { name: fc.name, args: fc.args || {}, atMs: Date.now() - cur.t0 };
|
||||
r.calls.push(c);
|
||||
return { id: fc.id, name: fc.name, response: onCall(c, cur.turn) };
|
||||
});
|
||||
send({ toolResponse: { functionResponses } });
|
||||
}
|
||||
if (m.usageMetadata) {
|
||||
const u = m.usageMetadata;
|
||||
r.usage.inputTokens += u.promptTokenCount || 0;
|
||||
r.usage.cachedTokens += u.cachedContentTokenCount || 0;
|
||||
r.usage.outputTokens += u.responseTokenCount || 0;
|
||||
r.usage.reasoningTokens += u.thoughtsTokenCount || 0;
|
||||
r.costUsd += geminiUsageCostUsd(model, u);
|
||||
}
|
||||
if (m.goAway) r.errors.push('goAway');
|
||||
if (sc?.turnComplete) {
|
||||
r.responses += 1;
|
||||
if (cur.toolCallThisGeneration && cur.toolRounds < MAX_TOOL_ROUNDS) {
|
||||
cur.toolCallThisGeneration = false;
|
||||
if (r.text && !r.text.endsWith(' | ')) r.text += ' | ';
|
||||
// The follow-up generation normally starts within a second; if
|
||||
// the model has nothing more to say, end the turn at its last
|
||||
// message rather than charging the wait to its latency.
|
||||
const c = cur;
|
||||
c.idle = setTimeout(() => { if (cur === c) { c.r.tDoneMs = lastAt - c.t0; endTurn(); } }, IDLE_AFTER_TOOL_MS);
|
||||
return;
|
||||
}
|
||||
endTurn();
|
||||
}
|
||||
});
|
||||
});
|
||||
},
|
||||
};
|
||||
return adapter;
|
||||
}
|
||||
@@ -0,0 +1,263 @@
|
||||
/**
|
||||
* Local-model adapters: stateless chat requests, so every phrase is a fresh
|
||||
* context by construction. Same system prompt (realtimeInstructions()) and
|
||||
* the production tools converted with toChatTools() (schema-normalized).
|
||||
*
|
||||
* openai-chat any OpenAI-compatible server (LM Studio :1234/v1, Ollama
|
||||
* :11434/v1). Streams SSE; timing is client-side.
|
||||
* ollama Ollama's native /api/chat. Same message and tool formats as
|
||||
* Chat Completions, but it can set num_ctx (the OpenAI shim
|
||||
* cannot — the ~15k-token prompt would be silently truncated
|
||||
* at a small default context) and turn thinking off, and it
|
||||
* reports server-side prefill/decode/load timings.
|
||||
*
|
||||
* Both mirror the Realtime loop: stub tool results, then continue until a
|
||||
* response has no tool calls, up to 4 continuations.
|
||||
*/
|
||||
import { realtimeInstructions, GEV_REALTIME_TOOLS } from '../productionConfig.mjs';
|
||||
import { toChatTools } from '../toolFormats.mjs';
|
||||
|
||||
const MAX_CONTINUATIONS = 4;
|
||||
|
||||
function emptyResult() {
|
||||
return {
|
||||
calls: [], text: '', errors: [], responses: 0,
|
||||
usage: { inputTokens: 0, cachedTokens: 0, outputTokens: 0, reasoningTokens: 0 },
|
||||
costUsd: 0, tFirstTokenMs: null, tFirstCallMs: null, tDoneMs: null, setupMs: 0, timeout: false,
|
||||
local: { prefillTokS: null, decodeTokS: null, promptEvalCount: null, evalCount: null, loadMs: null, firstResponseMs: null },
|
||||
};
|
||||
}
|
||||
|
||||
function parseArgs(raw) {
|
||||
if (raw && typeof raw === 'object') return raw;
|
||||
try { return JSON.parse(raw || '{}'); } catch { return { __unparsed: String(raw).slice(0, 400) }; }
|
||||
}
|
||||
|
||||
async function* lines(body) {
|
||||
const decoder = new TextDecoder();
|
||||
let buf = '';
|
||||
for await (const chunk of body) {
|
||||
buf += decoder.decode(chunk, { stream: true });
|
||||
let i;
|
||||
while ((i = buf.indexOf('\n')) >= 0) {
|
||||
yield buf.slice(0, i);
|
||||
buf = buf.slice(i + 1);
|
||||
}
|
||||
}
|
||||
if (buf.trim()) yield buf;
|
||||
}
|
||||
|
||||
/**
|
||||
* Shared multi-response loop over one or more user turns in ONE growing
|
||||
* message history. `request(messages, r, t0, first, deadline)` performs one
|
||||
* streamed completion and returns `{ content, toolCalls:[{id,name,args}] }`.
|
||||
*/
|
||||
async function runChatDialogue({ turns, onCall, request, turnTimeoutMs, contexts = [] }) {
|
||||
const messages = [{ role: 'system', content: realtimeInstructions() }];
|
||||
const results = [];
|
||||
for (let turn = 0; turn < turns.length; turn += 1) {
|
||||
const r = emptyResult();
|
||||
if (contexts[turn]) messages.push({ role: 'system', content: JSON.stringify(contexts[turn]) });
|
||||
messages.push({ role: 'user', content: turns[turn] });
|
||||
const t0 = Date.now();
|
||||
const deadline = t0 + turnTimeoutMs;
|
||||
try {
|
||||
for (let round = 0; round <= MAX_CONTINUATIONS; round += 1) {
|
||||
if (Date.now() > deadline) { r.timeout = true; break; }
|
||||
const out = await request(messages, r, t0, round === 0, deadline);
|
||||
r.responses += 1;
|
||||
if (out.content) r.text += (r.text ? ' | ' : '') + out.content.trim();
|
||||
if (!out.toolCalls.length) {
|
||||
messages.push({ role: 'assistant', content: out.content || '' });
|
||||
break;
|
||||
}
|
||||
messages.push({ role: 'assistant', content: out.content || '', tool_calls: out.toolCalls.map((tc) => tc.wire) });
|
||||
for (const tc of out.toolCalls) {
|
||||
const c = { name: tc.name, args: tc.args, atMs: tc.atMs };
|
||||
r.calls.push(c);
|
||||
messages.push({ role: 'tool', tool_call_id: tc.id, tool_name: tc.name, name: tc.name, content: JSON.stringify(onCall(c, turn)) });
|
||||
}
|
||||
}
|
||||
} catch (e) {
|
||||
if (e?.name === 'TimeoutError' || e?.name === 'AbortError') r.timeout = true;
|
||||
else r.errors.push(String(e?.message || e).slice(0, 300));
|
||||
}
|
||||
r.tDoneMs = Date.now() - t0;
|
||||
results.push(r);
|
||||
if (r.timeout || r.errors.length) {
|
||||
while (results.length < turns.length) {
|
||||
const skipped = emptyResult();
|
||||
skipped.errors.push('dialogue aborted after an earlier turn failed');
|
||||
results.push(skipped);
|
||||
}
|
||||
break;
|
||||
}
|
||||
}
|
||||
return results;
|
||||
}
|
||||
|
||||
// ── OpenAI-compatible (SSE) ─────────────────────────────────────────────────
|
||||
|
||||
export function createOpenAiChatAdapter({ model, baseUrl, temperature = 0.8, extraBody = {}, turnTimeoutMs = 180_000, apiKey = 'local' }) {
|
||||
const tools = toChatTools(GEV_REALTIME_TOOLS);
|
||||
const request = async (messages, r, t0, first, deadline) => {
|
||||
const res = await fetch(`${baseUrl.replace(/\/$/, '')}/chat/completions`, {
|
||||
method: 'POST',
|
||||
signal: AbortSignal.timeout(Math.max(1000, deadline - Date.now())),
|
||||
headers: { 'Content-Type': 'application/json', Authorization: `Bearer ${apiKey}` },
|
||||
body: JSON.stringify({
|
||||
model, messages, tools, tool_choice: 'auto', temperature, stream: true,
|
||||
stream_options: { include_usage: true }, ...extraBody,
|
||||
}),
|
||||
});
|
||||
if (!res.ok) throw new Error(`HTTP ${res.status}: ${(await res.text()).slice(0, 300)}`);
|
||||
const tStart = Date.now();
|
||||
let tFirst = null;
|
||||
let content = '';
|
||||
const byIndex = new Map();
|
||||
let usage = null;
|
||||
let firstCallDoneAt = null;
|
||||
for await (const line of lines(res.body)) {
|
||||
if (!line.startsWith('data:')) continue;
|
||||
const data = line.slice(5).trim();
|
||||
if (data === '[DONE]') break;
|
||||
let m;
|
||||
try { m = JSON.parse(data); } catch { continue; }
|
||||
if (m.usage) usage = m.usage;
|
||||
const choice = m.choices?.[0];
|
||||
const d = choice?.delta || {};
|
||||
const now = Date.now();
|
||||
if ((d.content || d.reasoning_content || d.reasoning || d.tool_calls?.length) && tFirst === null) {
|
||||
tFirst = now;
|
||||
if (r.tFirstTokenMs === null) r.tFirstTokenMs = now - t0;
|
||||
}
|
||||
if (d.content) content += d.content;
|
||||
for (const tc of d.tool_calls || []) {
|
||||
const idx = tc.index ?? byIndex.size;
|
||||
if (idx > 0 && byIndex.has(0) && firstCallDoneAt === null) firstCallDoneAt = now;
|
||||
const cur = byIndex.get(idx) || { id: tc.id || `call_${idx}`, name: '', argText: '' };
|
||||
if (tc.id) cur.id = tc.id;
|
||||
if (tc.function?.name) cur.name += tc.function.name;
|
||||
if (tc.function?.arguments) cur.argText += typeof tc.function.arguments === 'string' ? tc.function.arguments : JSON.stringify(tc.function.arguments);
|
||||
byIndex.set(idx, cur);
|
||||
}
|
||||
if (choice?.finish_reason && byIndex.size && firstCallDoneAt === null) firstCallDoneAt = now;
|
||||
}
|
||||
const tEnd = Date.now();
|
||||
if (byIndex.size && firstCallDoneAt === null) firstCallDoneAt = tEnd;
|
||||
if (byIndex.size && r.tFirstCallMs === null) r.tFirstCallMs = firstCallDoneAt - t0;
|
||||
if (usage) {
|
||||
r.usage.inputTokens += usage.prompt_tokens || 0;
|
||||
r.usage.cachedTokens += usage.prompt_tokens_details?.cached_tokens || 0;
|
||||
r.usage.outputTokens += usage.completion_tokens || 0;
|
||||
r.usage.reasoningTokens += usage.completion_tokens_details?.reasoning_tokens || 0;
|
||||
}
|
||||
if (first) {
|
||||
r.local.firstResponseMs = tEnd - tStart;
|
||||
if (usage && tFirst) {
|
||||
r.local.promptEvalCount = usage.prompt_tokens ?? null;
|
||||
r.local.evalCount = usage.completion_tokens ?? null;
|
||||
// Client-side approximations: prefill ≈ prompt / (request → first token);
|
||||
// decode ≈ (completion − 1) / (first token → end of stream).
|
||||
r.local.prefillTokS = usage.prompt_tokens ? usage.prompt_tokens / Math.max(0.001, (tFirst - tStart) / 1000) : null;
|
||||
r.local.decodeTokS = usage.completion_tokens > 1 && tEnd > tFirst ? (usage.completion_tokens - 1) / ((tEnd - tFirst) / 1000) : null;
|
||||
r.local.timingSource = 'client';
|
||||
}
|
||||
}
|
||||
const toolCalls = [...byIndex.values()].map((tc) => ({
|
||||
id: tc.id, name: tc.name, args: parseArgs(tc.argText), atMs: firstCallDoneAt - t0,
|
||||
wire: { id: tc.id, type: 'function', function: { name: tc.name, arguments: tc.argText || '{}' } },
|
||||
}));
|
||||
return { content, toolCalls };
|
||||
};
|
||||
return {
|
||||
provider: 'openai-chat',
|
||||
model,
|
||||
settings: { baseUrl, temperature, tool_choice: 'auto', toolSchema: 'normalized (untyped → string|number|boolean)', context: 'stateless request per phrase', ...extraBody },
|
||||
async open() {},
|
||||
async close() {},
|
||||
runDialogue: ({ turns, onCall, contexts }) => runChatDialogue({ turns, onCall, contexts, request, turnTimeoutMs }),
|
||||
runTurn: async ({ text, onCall, context }) => (await runChatDialogue({ turns: [text], onCall, contexts: [context], request, turnTimeoutMs }))[0],
|
||||
};
|
||||
}
|
||||
|
||||
// ── Ollama native (/api/chat, NDJSON) ───────────────────────────────────────
|
||||
|
||||
export function createOllamaAdapter({ model, baseUrl = 'http://localhost:11434', temperature = 0.8, numCtx = 32768, think = false, keepAlive = '30m', turnTimeoutMs = 300_000 }) {
|
||||
const tools = toChatTools(GEV_REALTIME_TOOLS);
|
||||
let thinkParam = think;
|
||||
const request = async (messages, r, t0, first, deadline) => {
|
||||
const wireMessages = messages.map((m) => {
|
||||
if (m.role === 'assistant' && m.tool_calls) {
|
||||
return { role: 'assistant', content: m.content, tool_calls: m.tool_calls.map((tc) => ({ function: { name: tc.function.name, arguments: parseArgs(tc.function.arguments) } })) };
|
||||
}
|
||||
if (m.role === 'tool') return { role: 'tool', tool_name: m.tool_name, content: m.content };
|
||||
return m;
|
||||
});
|
||||
const body = { model, messages: wireMessages, tools, stream: true, keep_alive: keepAlive, options: { temperature, num_ctx: numCtx } };
|
||||
if (thinkParam !== null) body.think = thinkParam;
|
||||
const res = await fetch(`${baseUrl}/api/chat`, {
|
||||
method: 'POST',
|
||||
signal: AbortSignal.timeout(Math.max(1000, deadline - Date.now())),
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify(body),
|
||||
});
|
||||
if (!res.ok) {
|
||||
const errText = await res.text();
|
||||
if (/does not support thinking/i.test(errText) && thinkParam !== null) {
|
||||
thinkParam = null;
|
||||
return request(messages, r, t0, first, deadline);
|
||||
}
|
||||
throw new Error(`HTTP ${res.status}: ${errText.slice(0, 300)}`);
|
||||
}
|
||||
let content = '';
|
||||
const toolCalls = [];
|
||||
let final = null;
|
||||
for await (const line of lines(res.body)) {
|
||||
if (!line.trim()) continue;
|
||||
let m;
|
||||
try { m = JSON.parse(line); } catch { continue; }
|
||||
if (m.error) throw new Error(m.error);
|
||||
const now = Date.now();
|
||||
const msg = m.message || {};
|
||||
if ((msg.content || msg.thinking || msg.tool_calls?.length) && r.tFirstTokenMs === null) r.tFirstTokenMs = now - t0;
|
||||
if (msg.content) content += msg.content;
|
||||
for (const tc of msg.tool_calls || []) {
|
||||
if (r.tFirstCallMs === null) r.tFirstCallMs = now - t0;
|
||||
const i = `${messages.length}_${toolCalls.length}`;
|
||||
const args = parseArgs(tc.function?.arguments);
|
||||
toolCalls.push({
|
||||
id: `call_${i}`, name: tc.function?.name, args, atMs: now - t0,
|
||||
wire: { id: `call_${i}`, type: 'function', function: { name: tc.function?.name, arguments: JSON.stringify(args) } },
|
||||
});
|
||||
}
|
||||
if (m.done) final = m;
|
||||
}
|
||||
if (final) {
|
||||
r.usage.inputTokens += final.prompt_eval_count || 0;
|
||||
r.usage.outputTokens += final.eval_count || 0;
|
||||
if (first) {
|
||||
const ns = 1e9;
|
||||
r.local.promptEvalCount = final.prompt_eval_count ?? null;
|
||||
r.local.evalCount = final.eval_count ?? null;
|
||||
r.local.prefillTokS = final.prompt_eval_duration ? final.prompt_eval_count / (final.prompt_eval_duration / ns) : null;
|
||||
r.local.decodeTokS = final.eval_duration ? final.eval_count / (final.eval_duration / ns) : null;
|
||||
r.local.loadMs = final.load_duration ? final.load_duration / 1e6 : null;
|
||||
r.local.firstResponseMs = final.total_duration ? final.total_duration / 1e6 : null;
|
||||
r.local.timingSource = 'ollama';
|
||||
}
|
||||
}
|
||||
return { content, toolCalls };
|
||||
};
|
||||
return {
|
||||
provider: 'ollama',
|
||||
model,
|
||||
get settings() {
|
||||
return { baseUrl, api: '/api/chat', temperature, num_ctx: numCtx, think: thinkParam === null ? 'unsupported by model (param omitted)' : thinkParam, keep_alive: keepAlive, toolSchema: 'normalized (untyped → string|number|boolean)', context: 'stateless request per phrase' };
|
||||
},
|
||||
async open() {},
|
||||
async close() {},
|
||||
runDialogue: ({ turns, onCall, contexts }) => runChatDialogue({ turns, onCall, contexts, request, turnTimeoutMs }),
|
||||
runTurn: async ({ text, onCall, context }) => (await runChatDialogue({ turns: [text], onCall, contexts: [context], request, turnTimeoutMs }))[0],
|
||||
};
|
||||
}
|
||||
@@ -0,0 +1,167 @@
|
||||
/**
|
||||
* OpenAI Realtime adapter — text turns over the Realtime WebSocket.
|
||||
*
|
||||
* Each dialogue (one phrase, or a multi-turn dialogue) opens a FRESH session
|
||||
* configured with production's exact session body (captured from the token
|
||||
* handler) and sends its user text items one turn at a time. Each turn
|
||||
* mirrors qa-voice-routing's loop: every function call gets a stub output, and
|
||||
* a continuation response is requested after `response.done` (never while a
|
||||
* response is active), up to 4 continuations.
|
||||
*/
|
||||
import WebSocket from 'ws';
|
||||
import { providerKey } from '../keys.mjs';
|
||||
import { captureProductionSession } from '../productionConfig.mjs';
|
||||
import { openAiUsageCostUsd } from '../pricing.mjs';
|
||||
|
||||
const MAX_CONTINUATIONS = 4;
|
||||
|
||||
export function newTurnResult() {
|
||||
return {
|
||||
calls: [], text: '', errors: [], responses: 0,
|
||||
usage: { inputTokens: 0, cachedTokens: 0, outputTokens: 0, reasoningTokens: 0 },
|
||||
costUsd: 0, tFirstTokenMs: null, tFirstCallMs: null, tDoneMs: null, setupMs: null, timeout: false,
|
||||
};
|
||||
}
|
||||
|
||||
export function createOpenAiRealtimeAdapter({ model, turnTimeoutMs = 45_000 }) {
|
||||
let session = null;
|
||||
const adapter = {
|
||||
provider: 'openai-realtime',
|
||||
model,
|
||||
settings: {},
|
||||
async open() {
|
||||
session = await captureProductionSession(model);
|
||||
adapter.settings = {
|
||||
reasoning: session.reasoning,
|
||||
truncation: session.truncation,
|
||||
tool_choice: session.tool_choice,
|
||||
temperature: 'not settable (GA Realtime session has no temperature field)',
|
||||
output_modalities: ['text'],
|
||||
context: 'fresh session per phrase / dialogue',
|
||||
};
|
||||
},
|
||||
async close() {},
|
||||
async runTurn({ text, onCall, context }) {
|
||||
return (await adapter.runDialogue({ turns: [text], onCall, contexts: [context] }))[0];
|
||||
},
|
||||
runDialogue({ turns, onCall, contexts = [] }) {
|
||||
return new Promise((resolveAll) => {
|
||||
const tSetup = Date.now();
|
||||
const ws = new WebSocket(`wss://api.openai.com/v1/realtime?model=${encodeURIComponent(model)}`, {
|
||||
headers: { Authorization: `Bearer ${providerKey('openai')}` },
|
||||
});
|
||||
const results = [];
|
||||
let setupMs = null;
|
||||
let cur = null; // { r, t0, continuations, needContinuation, turn, timer }
|
||||
let closed = false;
|
||||
const send = (o) => { if (ws.readyState === WebSocket.OPEN) ws.send(JSON.stringify(o)); };
|
||||
const shutdown = (error) => {
|
||||
if (closed) return;
|
||||
closed = true;
|
||||
if (cur) {
|
||||
clearTimeout(cur.timer);
|
||||
if (error) cur.r.errors.push(error);
|
||||
cur.r.tDoneMs = Date.now() - cur.t0;
|
||||
results.push(cur.r);
|
||||
cur = null;
|
||||
}
|
||||
while (results.length < turns.length) {
|
||||
const r = newTurnResult();
|
||||
r.errors.push(error || 'dialogue aborted');
|
||||
results.push(r);
|
||||
}
|
||||
try { ws.terminate(); } catch { /* noop */ }
|
||||
resolveAll(results);
|
||||
};
|
||||
const startTurn = (turn) => {
|
||||
const r = newTurnResult();
|
||||
r.setupMs = turn === 0 ? setupMs : 0;
|
||||
cur = { r, t0: Date.now(), continuations: 0, needContinuation: false, turn };
|
||||
cur.timer = setTimeout(() => { cur.r.timeout = true; endTurn(); }, turnTimeoutMs);
|
||||
// Optional inert system item first (the app's pointer_context shape).
|
||||
if (contexts[turn]) send({ type: 'conversation.item.create', item: { type: 'message', role: 'system', content: [{ type: 'input_text', text: JSON.stringify(contexts[turn]) }] } });
|
||||
send({ type: 'conversation.item.create', item: { type: 'message', role: 'user', content: [{ type: 'input_text', text: turns[turn] }] } });
|
||||
send({ type: 'response.create', response: { output_modalities: ['text'] } });
|
||||
};
|
||||
const endTurn = () => {
|
||||
if (!cur) return;
|
||||
clearTimeout(cur.timer);
|
||||
cur.r.tDoneMs = Date.now() - cur.t0;
|
||||
results.push(cur.r);
|
||||
const next = cur.turn + 1;
|
||||
const timedOut = cur.r.timeout;
|
||||
cur = null;
|
||||
if (next < turns.length && !timedOut) startTurn(next);
|
||||
else shutdown();
|
||||
};
|
||||
const mark = (key) => { if (cur && cur.r[key] === null) cur.r[key] = Date.now() - cur.t0; };
|
||||
|
||||
ws.on('unexpected-response', (_req, res) => shutdown(`HTTP ${res.statusCode}`));
|
||||
ws.on('error', (e) => shutdown(`ws: ${e.message}`));
|
||||
ws.on('close', () => shutdown(results.length < turns.length ? 'socket closed' : undefined));
|
||||
ws.on('message', (raw) => {
|
||||
let m;
|
||||
try { m = JSON.parse(raw.toString()); } catch { return; }
|
||||
if (m.type === 'session.created') {
|
||||
send({ type: 'session.update', session });
|
||||
return;
|
||||
}
|
||||
if (m.type === 'session.updated' && setupMs === null) {
|
||||
setupMs = Date.now() - tSetup;
|
||||
startTurn(0);
|
||||
return;
|
||||
}
|
||||
if (m.type === 'error') {
|
||||
if (!cur) { shutdown(m.error?.message || 'realtime error'); return; }
|
||||
cur.r.errors.push(m.error?.message || 'realtime error');
|
||||
return;
|
||||
}
|
||||
if (!cur) return;
|
||||
const r = cur.r;
|
||||
if (
|
||||
m.type === 'response.output_text.delta' ||
|
||||
m.type === 'response.function_call_arguments.delta' ||
|
||||
m.type === 'response.output_audio_transcript.delta'
|
||||
) mark('tFirstTokenMs');
|
||||
if (m.type === 'response.output_text.delta' || m.type === 'response.output_audio_transcript.delta') {
|
||||
r.text += m.delta || '';
|
||||
}
|
||||
if (m.type === 'response.output_item.done' && m.item?.type === 'function_call') {
|
||||
mark('tFirstTokenMs');
|
||||
mark('tFirstCallMs');
|
||||
let args = {};
|
||||
try { args = JSON.parse(m.item.arguments || '{}'); } catch { args = { __unparsed: m.item.arguments }; }
|
||||
const c = { name: m.item.name, args, atMs: Date.now() - cur.t0 };
|
||||
r.calls.push(c);
|
||||
send({
|
||||
type: 'conversation.item.create',
|
||||
item: { type: 'function_call_output', call_id: m.item.call_id, output: JSON.stringify(onCall(c, cur.turn)) },
|
||||
});
|
||||
cur.needContinuation = true;
|
||||
}
|
||||
if (m.type === 'response.done') {
|
||||
r.responses += 1;
|
||||
const u = m.response?.usage;
|
||||
if (u) {
|
||||
r.usage.inputTokens += u.input_tokens || 0;
|
||||
r.usage.cachedTokens += u.input_token_details?.cached_tokens || 0;
|
||||
r.usage.outputTokens += u.output_tokens || 0;
|
||||
r.usage.reasoningTokens += u.output_token_details?.reasoning_tokens || 0;
|
||||
r.costUsd += openAiUsageCostUsd(model, u);
|
||||
}
|
||||
if (m.response?.status === 'failed') r.errors.push(JSON.stringify(m.response?.status_details).slice(0, 200));
|
||||
if (cur.needContinuation && cur.continuations < MAX_CONTINUATIONS) {
|
||||
cur.needContinuation = false;
|
||||
cur.continuations += 1;
|
||||
if (r.text) r.text += ' | ';
|
||||
send({ type: 'response.create', response: { output_modalities: ['text'] } });
|
||||
return;
|
||||
}
|
||||
endTurn();
|
||||
}
|
||||
});
|
||||
});
|
||||
},
|
||||
};
|
||||
return adapter;
|
||||
}
|
||||
@@ -0,0 +1 @@
|
||||
[{"type":"function","name":"fly_to_location","description":"Fly the God's Eye View camera to a known city, geocoded country/region/city/landmark, or explicit WGS84 coordinate. Countries/cities frame the whole place; landmarks/buildings use close framing.","parameters":{"type":"object","additionalProperties":false,"properties":{"locationId":{"type":"string","enum":["austin","sf","nyc","tokyo","london","paris","dubai","dc"],"description":"Known city preset ID. Use when the requested place matches one of these cities."},"query":{"type":"string","description":"Plain place search query, e.g. \"London\", \"Eiffel Tower\", or \"Dubai Marina\"."},"latitude":{"type":"number","minimum":-90,"maximum":90},"longitude":{"type":"number","minimum":-180,"maximum":180},"viewMode":{"type":"string","enum":["close","overview"],"description":"Optional framing intent. Usually omit this; GEV infers whole-place framing for countries/cities and close framing for landmarks."},"rangeM":{"type":"number","minimum":100,"maximum":20000000,"description":"Optional camera range from the target in meters. Omit it for automatic whole-country/whole-city or close-landmark framing; provide it only when the user explicitly requests a numeric height or distance."},"waitForArrival":{"type":"boolean","description":"Set true when a later tool depends on the destination viewport. The result then waits for the camera flight and returns arrived=true; cancellation returns ok=false."}}}},{"type":"function","name":"select_nearest_aircraft","description":"Atomically fly to a place, wait for arrival, enable and load Flights or Military Flights in that viewport, exclude on-ground records, and select/follow the nearest airborne aircraft. Healthy fallback feeds remain usable and are reported in the result. This does not open Contacts or Cockpit.","parameters":{"type":"object","additionalProperties":false,"properties":{"layerId":{"type":"string","enum":["flights","military"],"description":"Aircraft layer to enable and search. Use flights unless the user explicitly asks for military aircraft."},"locationId":{"type":"string","enum":["austin","sf","nyc","tokyo","london","paris","dubai","dc"],"description":"Known city preset ID when the place matches one of these cities."},"locationQuery":{"type":"string","maxLength":160,"description":"Free-form destination when no locationId matches."},"latitude":{"type":"number","minimum":-90,"maximum":90},"longitude":{"type":"number","minimum":-180,"maximum":180}},"required":["layerId"]}},{"type":"function","name":"adjust_camera_zoom","description":"Move the current Cesium camera closer to or farther from what it is presently looking at. Use for relative zoom requests without changing location.","parameters":{"type":"object","additionalProperties":false,"properties":{"direction":{"type":"string","enum":["in","out"]},"amount":{"type":"string","enum":["little","medium","lot"],"description":"Use little for phrases like \"a bit\" or \"a little\", medium for ordinary zoom requests, and lot for \"way out/in\"."}},"required":["direction","amount"]}},{"type":"function","name":"zoom_to_globe","description":"Pull the camera out to an ABSOLUTE full-Earth globe view (~18,000 km altitude, the whole planet in frame), keeping the current region centered. Use for \"globe view\", \"whole earth\", \"see the planet\", \"zoom all the way out\". Never use adjust_camera_zoom for these — its relative steps cannot reach the globe.","parameters":{"type":"object","additionalProperties":false,"properties":{}}},{"type":"function","name":"set_layer_visibility","description":"Enable or disable one registered God's Eye View data layer.","parameters":{"type":"object","additionalProperties":false,"properties":{"layerId":{"type":"string","description":"Common-name mapping for the non-obvious ids: space mission(s) → rocket-launches; fires/wildfires/active fires → local-firms (NASA FIRMS); ships/vessels/boats → ais-live-vessels; undersea/submarine cables → telegeography-submarine-cables; datacenters → local-datacenters; dams → local-dams; bikes/bike share → bikeshare; street traffic/congestion → traffic; traffic cameras → cctv; internet radio/stations → radio; ALPR/license plate readers/Flock cameras → alpr-cameras; local ADS-B/my receiver/my antenna (aircraft heard by a local RTL-SDR receiver) → local-adsb.","enum":["flights","military","earthquakes","satellites","rocket-launches","traffic","cctv","radio","bikeshare","ais-live-vessels","local-datacenters","local-dams","telegeography-submarine-cables","local-firms","alpr-cameras","local-adsb"]},"enabled":{"type":"boolean"}},"required":["layerId","enabled"]}},{"type":"function","name":"show_data_layers_menu","description":"Open the data layers dropdown/menu and optionally scroll to a specific layer row without toggling it.","parameters":{"type":"object","additionalProperties":false,"properties":{"layerId":{"type":"string","enum":["flights","military","earthquakes","satellites","traffic","ccLine truncated
|
||||
@@ -0,0 +1,281 @@
|
||||
/**
|
||||
* Pure grading for voice-routing runs. No I/O, no network.
|
||||
*
|
||||
* `gradeCore` reproduces the pass/fail semantics of the routing layer in
|
||||
* `scripts/qa-voice-routing.mjs` (which now calls it), plus a few extra
|
||||
* benchmark signals. `gradeCoverage` sorts coverage-probe turns into
|
||||
* correct / honest-refusal / hallucinated-capability / wrong-tool.
|
||||
*/
|
||||
|
||||
/**
|
||||
* Spot-check `expected` against `actual` args: substring match (case
|
||||
* insensitive) for strings, exact match for everything else. An array value
|
||||
* is matched through its comma-joined string form, so `{ layers: 'earthquakes' }`
|
||||
* matches `layers: ['earthquakes']`.
|
||||
*/
|
||||
export function matchArgs(expected, actual) {
|
||||
return Object.entries(expected).every(([k, v]) => {
|
||||
const got = actual?.[k];
|
||||
if (typeof v === 'string') return String(got ?? '').toLowerCase().includes(v.toLowerCase());
|
||||
return got === v;
|
||||
});
|
||||
}
|
||||
|
||||
/** Build a call matcher: tool name plus optional arg spot-check. */
|
||||
export function call(name, args) {
|
||||
const matcher = (c) => c?.name === name && (!args || matchArgs(args, c.args));
|
||||
matcher.toolName = name;
|
||||
return matcher;
|
||||
}
|
||||
|
||||
/** Build a call matcher from a tool name and a predicate over its args. */
|
||||
export function where(name, predicate) {
|
||||
const matcher = (c) => c?.name === name && Boolean(predicate(c.args || {}));
|
||||
matcher.toolName = name;
|
||||
return matcher;
|
||||
}
|
||||
|
||||
/** Human label for a phrase's expectation. */
|
||||
export function expectLabel(phrase) {
|
||||
if (phrase.expectNone) return '(no tool)';
|
||||
if (Array.isArray(phrase.expect)) return phrase.expect.join('+');
|
||||
if (phrase.expect?.oneOf) return phrase.expect.oneOf.join('|');
|
||||
return String(phrase.expect);
|
||||
}
|
||||
|
||||
/** Every tool name that satisfies a phrase (union for oneOf / arrays). */
|
||||
export function expectedToolNames(phrase) {
|
||||
if (phrase.expectNone) return [];
|
||||
if (Array.isArray(phrase.expect)) return [...phrase.expect];
|
||||
if (phrase.expect?.oneOf) return [...phrase.expect.oneOf];
|
||||
return [phrase.expect];
|
||||
}
|
||||
|
||||
/**
|
||||
* Grade one core-suite turn.
|
||||
*
|
||||
* `ok` is exactly the qa-voice-routing verdict: expectNone passes only with no
|
||||
* calls; an array needs every tool; oneOf needs any; a string needs that tool;
|
||||
* then `args` / `argsByTool` spot checks. The rest are benchmark extras:
|
||||
* toolOk tool-name routing alone (before the args check)
|
||||
* argsOk null when the phrase has no arg spot check
|
||||
* firstCallOk the FIRST call was an expected tool (what a user sees first)
|
||||
* falsePositive a conversational turn produced a tool call
|
||||
* extraCalls calls to tools outside the expected set
|
||||
* typedArgsOk benchmark-only typed-argument check (`typedArgs`: exact
|
||||
* enum values, layer ids, numeric thresholds, sort order);
|
||||
* null when the phrase has none. Not part of `ok`.
|
||||
* fullCallOk toolOk and every arg check (args, argsByTool, typedArgs)
|
||||
*/
|
||||
export function gradeCore(phrase, calls) {
|
||||
const names = calls.map((c) => c.name);
|
||||
if (phrase.expectNone) {
|
||||
return {
|
||||
ok: names.length === 0,
|
||||
toolOk: names.length === 0,
|
||||
fullCallOk: names.length === 0,
|
||||
argsOk: null,
|
||||
typedArgsOk: null,
|
||||
firstCallOk: names.length === 0,
|
||||
falsePositive: names.length > 0,
|
||||
extraCalls: names,
|
||||
detail: names.length ? `unexpected calls: ${names.join(',')}` : 'clean conversational turn',
|
||||
};
|
||||
}
|
||||
const expected = expectedToolNames(phrase);
|
||||
let toolOk;
|
||||
if (Array.isArray(phrase.expect)) toolOk = phrase.expect.every((n) => names.includes(n));
|
||||
else if (phrase.expect?.oneOf) toolOk = names.some((n) => phrase.expect.oneOf.includes(n));
|
||||
else toolOk = names.includes(phrase.expect);
|
||||
|
||||
let detail = `called: ${names.join(',') || '(none)'}`;
|
||||
let argsOk = null;
|
||||
if (phrase.args) {
|
||||
const target = calls.find((c) => (Array.isArray(phrase.expect) ? true : expected.includes(c.name)));
|
||||
argsOk = Boolean(target) && matchArgs(phrase.args, target.args);
|
||||
if (toolOk && !argsOk) detail += ` args mismatch: ${JSON.stringify(target?.args)}`;
|
||||
}
|
||||
if (phrase.argsByTool) {
|
||||
let allOk = true;
|
||||
for (const [toolName, expectedArgs] of Object.entries(phrase.argsByTool)) {
|
||||
const target = calls.find((c) => c.name === toolName);
|
||||
if (!target || !matchArgs(expectedArgs, target.args)) {
|
||||
allOk = false;
|
||||
if (toolOk) detail += ` ${toolName} args mismatch: ${JSON.stringify(target?.args)}`;
|
||||
}
|
||||
}
|
||||
argsOk = argsOk === null ? allOk : argsOk && allOk;
|
||||
}
|
||||
const typedArgsOk = phrase.typedArgs ? phrase.typedArgs.every((m) => calls.some((c) => m(c))) : null;
|
||||
if (toolOk && typedArgsOk === false) detail += ` typed args miss: ${JSON.stringify(calls.map((c) => c.args)).slice(0, 200)}`;
|
||||
return {
|
||||
ok: toolOk && argsOk !== false,
|
||||
toolOk,
|
||||
fullCallOk: toolOk && argsOk !== false && typedArgsOk !== false,
|
||||
argsOk,
|
||||
typedArgsOk,
|
||||
firstCallOk: names.length > 0 && expected.includes(names[0]),
|
||||
falsePositive: false,
|
||||
extraCalls: names.filter((n) => !expected.includes(n)),
|
||||
detail,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Grade a two-turn dialogue from each turn's calls. The headline `ok` is
|
||||
* turn 2's full-call accuracy (tool + typed args), since turn 2 is the one
|
||||
* that depends on context; turn 1 is reported alongside.
|
||||
*/
|
||||
export function gradeDialogue(dialogue, turnCalls) {
|
||||
const turn1 = gradeCore(dialogue.first, turnCalls[0] || []);
|
||||
const turn2 = gradeCore(dialogue.second, turnCalls[1] || []);
|
||||
return {
|
||||
ok: turn2.fullCallOk,
|
||||
toolOk: turn2.toolOk,
|
||||
fullCallOk: turn2.fullCallOk,
|
||||
turn1Ok: turn1.fullCallOk,
|
||||
turn1,
|
||||
turn2,
|
||||
detail: `t1 ${turn1.detail} | t2 ${turn2.detail}`,
|
||||
};
|
||||
}
|
||||
|
||||
// ── JSON-schema argument validation (the subset GEV tools use) ─────────────
|
||||
|
||||
const TYPE_CHECKS = {
|
||||
string: (v) => typeof v === 'string',
|
||||
number: (v) => typeof v === 'number' && Number.isFinite(v),
|
||||
integer: (v) => Number.isInteger(v),
|
||||
boolean: (v) => typeof v === 'boolean',
|
||||
object: (v) => v !== null && typeof v === 'object' && !Array.isArray(v),
|
||||
array: (v) => Array.isArray(v),
|
||||
null: (v) => v === null,
|
||||
};
|
||||
|
||||
/**
|
||||
* Validate `value` against a JSON schema (type / enum / anyOf / required /
|
||||
* properties / additionalProperties:false / items / min/max bounds).
|
||||
* Returns a list of human-readable violations; empty means valid.
|
||||
*/
|
||||
export function validateArgs(schema, value, where = 'args') {
|
||||
const errors = [];
|
||||
if (!schema || typeof schema !== 'object') return errors;
|
||||
if (schema.type) {
|
||||
const types = Array.isArray(schema.type) ? schema.type : [schema.type];
|
||||
if (!types.some((t) => TYPE_CHECKS[t]?.(value))) {
|
||||
errors.push(`${where} should be ${types.join('|')}`);
|
||||
return errors;
|
||||
}
|
||||
}
|
||||
if (schema.enum && !schema.enum.includes(value)) {
|
||||
errors.push(`${where}=${JSON.stringify(value)} is not one of ${schema.enum.join(', ')}`);
|
||||
}
|
||||
if (typeof value === 'string') {
|
||||
if (typeof schema.minLength === 'number' && value.length < schema.minLength)
|
||||
errors.push(`${where} needs at least ${schema.minLength} characters`);
|
||||
if (typeof schema.pattern === 'string' && !new RegExp(schema.pattern).test(value))
|
||||
errors.push(`${where} does not match ${schema.pattern}`);
|
||||
}
|
||||
if (typeof value === 'number') {
|
||||
if (typeof schema.minimum === 'number' && value < schema.minimum) errors.push(`${where} below ${schema.minimum}`);
|
||||
if (typeof schema.maximum === 'number' && value > schema.maximum) errors.push(`${where} above ${schema.maximum}`);
|
||||
}
|
||||
if (Array.isArray(value)) {
|
||||
if (typeof schema.minItems === 'number' && value.length < schema.minItems) errors.push(`${where} needs ≥${schema.minItems} items`);
|
||||
if (schema.items) value.forEach((item, i) => errors.push(...validateArgs(schema.items, item, `${where}[${i}]`)));
|
||||
}
|
||||
if (TYPE_CHECKS.object(value) && (schema.properties || schema.required)) {
|
||||
for (const key of schema.required || []) {
|
||||
if (!(key in value)) errors.push(`${where}.${key} is required`);
|
||||
}
|
||||
for (const [key, v] of Object.entries(value)) {
|
||||
const sub = schema.properties?.[key];
|
||||
if (!sub) {
|
||||
if (schema.additionalProperties === false) errors.push(`${where}.${key} is not a known argument`);
|
||||
continue;
|
||||
}
|
||||
errors.push(...validateArgs(sub, v, `${where}.${key}`));
|
||||
}
|
||||
}
|
||||
if (Array.isArray(schema.anyOf)) {
|
||||
const branches = schema.anyOf.map((branch) =>
|
||||
validateArgs(branch, value, where),
|
||||
);
|
||||
if (!branches.some((branchErrors) => branchErrors.length === 0))
|
||||
errors.push(`${where} does not match any allowed shape`);
|
||||
}
|
||||
return errors;
|
||||
}
|
||||
|
||||
/** Validate a tool call against the Realtime tool list. */
|
||||
export function validateCall(tools, c) {
|
||||
const tool = tools.find((t) => t.name === c.name);
|
||||
if (!tool) return [`unknown tool ${c.name}`];
|
||||
return validateArgs(tool.parameters, c.args ?? {});
|
||||
}
|
||||
|
||||
// ── Coverage probe grading ─────────────────────────────────────────────────
|
||||
|
||||
/** Tools that only read state; they never count as acting on the request. */
|
||||
export const LOOKUP_TOOLS = Object.freeze([
|
||||
'get_entity_context',
|
||||
'get_current_view_state',
|
||||
'show_data_layers_menu',
|
||||
'set_panel_open',
|
||||
]);
|
||||
|
||||
export const REFUSAL_RE =
|
||||
/\b(can(?:no|')t|cannot|unable|not able|don'?t have|do not have|doesn'?t have|does not have|isn'?t (?:available|supported|something)|aren'?t (?:available|supported)|not (?:available|supported|possible|yet)|no (?:access|way|tool|data|layer|feed|live|transit|weather|wind|cyclone|radar|lightning|receiver)|n'?t (?:connected|see any|find)|(?:isn'?t|is no|are no) (?:an? )?(?:\w+ ){0,2}(?:layer|feed|tool|data source|integration)s?|outside what i can|beyond what i can|only (?:have|support|show|cover)|unfortunately|sorry)\b/i;
|
||||
|
||||
/**
|
||||
* Grade one coverage-probe turn.
|
||||
*
|
||||
* Order matters:
|
||||
* 1. a call pattern in `correct` → correct
|
||||
* 2. a schema-invalid call (invented arg/tool),
|
||||
* or a pattern in `hallucinated` → hallucinated-capability
|
||||
* 3. any other acting call (not lookup/neutral) → wrong-tool
|
||||
* 4. no acting call: text matches `claimText`
|
||||
* and no refusal wording → hallucinated-capability
|
||||
* 5. no acting call: refusal wording or a question → honest-refusal
|
||||
* 6. no acting call, no refusal → hallucinated-capability
|
||||
* (it answered as if it had the data)
|
||||
*
|
||||
* `acceptable` = correct, or honest-refusal on an unsupported item.
|
||||
*/
|
||||
export function gradeCoverage(item, calls, text, tools = []) {
|
||||
// Models write typographic apostrophes ("can’t"); fold them before matching.
|
||||
const finalText = String(text || '').replace(/[\u2018\u2019\u02bc]/g, "'");
|
||||
const verdict = (v, reason) => ({
|
||||
verdict: v,
|
||||
reason,
|
||||
acceptable: v === 'correct' || (v === 'honest-refusal' && !item.supported),
|
||||
});
|
||||
for (const alternative of item.correct || []) {
|
||||
if (alternative.every((m) => calls.some((c) => m(c)))) {
|
||||
return verdict('correct', `matched ${alternative.map((m) => m.toolName || 'pattern').join('+')}`);
|
||||
}
|
||||
}
|
||||
if (tools.length) {
|
||||
for (const c of calls) {
|
||||
const errors = validateCall(tools, c);
|
||||
if (errors.length) return verdict('hallucinated-capability', `invalid ${c.name}: ${errors[0]}`);
|
||||
}
|
||||
}
|
||||
for (const m of item.hallucinated || []) {
|
||||
const hit = calls.find((c) => m(c));
|
||||
if (hit) return verdict('hallucinated-capability', `faked capability via ${hit.name}(${JSON.stringify(hit.args).slice(0, 80)})`);
|
||||
}
|
||||
const neutral = new Set([...LOOKUP_TOOLS, ...(item.neutral || [])]);
|
||||
const acting = calls.filter((c) => !neutral.has(c.name));
|
||||
if (acting.length) return verdict('wrong-tool', `called ${acting.map((c) => c.name).join(',')}`);
|
||||
const refused = REFUSAL_RE.test(finalText);
|
||||
if (item.claimText && item.claimText.test(finalText) && !refused) {
|
||||
return verdict('hallucinated-capability', 'asserted data it cannot have');
|
||||
}
|
||||
if (refused || /\?\s*$/.test(finalText.trim())) return verdict('honest-refusal', 'refused or asked');
|
||||
if (!finalText.trim().replace(/\|/g, '').trim()) {
|
||||
return verdict('wrong-tool', calls.length ? `only lookups/neutral calls (${calls.map((c) => c.name).join(',')}) and no answer` : 'no tool call and no answer');
|
||||
}
|
||||
return verdict('hallucinated-capability', 'answered without a tool or a refusal');
|
||||
}
|
||||
@@ -0,0 +1,35 @@
|
||||
/**
|
||||
* Runtime key lookup for the voice benchmark. Keys are read from the
|
||||
* environment, then the macOS Keychain, at call time. They are never written
|
||||
* to results, logs, or files.
|
||||
*/
|
||||
import { execFileSync } from 'node:child_process';
|
||||
|
||||
const KEYCHAIN = {
|
||||
openai: { env: 'OPENAI_API_KEY', service: 'openai-api', account: 'api-key' },
|
||||
gemini: { env: 'GEMINI_API_KEY', service: 'gemini-api', account: 'api-key' },
|
||||
};
|
||||
|
||||
const cache = new Map();
|
||||
|
||||
/** Resolve a provider key from the environment, then the Keychain. */
|
||||
export function providerKey(provider) {
|
||||
if (cache.has(provider)) return cache.get(provider);
|
||||
const spec = KEYCHAIN[provider];
|
||||
if (!spec) throw new Error(`unknown key provider: ${provider}`);
|
||||
let value = process.env[spec.env] || '';
|
||||
if (!value && process.platform === 'darwin') {
|
||||
try {
|
||||
value = execFileSync(
|
||||
'security',
|
||||
['find-generic-password', '-s', spec.service, '-a', spec.account, '-w'],
|
||||
{ encoding: 'utf8', stdio: ['ignore', 'pipe', 'ignore'] },
|
||||
).trim();
|
||||
} catch {
|
||||
value = '';
|
||||
}
|
||||
}
|
||||
if (!value) throw new Error(`${spec.env} is not available (env or Keychain)`);
|
||||
cache.set(provider, value);
|
||||
return value;
|
||||
}
|
||||
@@ -0,0 +1,600 @@
|
||||
/**
|
||||
* Shared voice-routing phrase tables.
|
||||
*
|
||||
* CORE_PHRASES is the routing table used by `scripts/qa-voice-routing.mjs`
|
||||
* (layer 1) and the core suite of `scripts/qa-voice-bench.mjs`.
|
||||
*
|
||||
* `expect` tool name the model must call (string), every tool in an
|
||||
* array, or `{ oneOf: [...] }` for documented acceptable variance.
|
||||
* `expectNone` conversational turns that must NOT call tools.
|
||||
* `args` spot-check subset matched against the first matching call's
|
||||
* args (substring match for strings, exact for booleans/numbers).
|
||||
* `argsByTool` the same spot check, keyed by tool, for multi-intent phrases.
|
||||
* `typedArgs` benchmark-only matchers for typed arguments (exact enum
|
||||
* values, layer ids, thresholds, sort direction). They feed the
|
||||
* benchmark's full-call accuracy and never change the
|
||||
* qa-voice-routing verdict.
|
||||
*
|
||||
* COVERAGE_PROBE is the newer-layers coverage probe: utterances about layers
|
||||
* and questions that today's tool surface only partly supports. Each item is
|
||||
* graded into correct / honest-refusal / hallucinated-capability / wrong-tool
|
||||
* by `gradeCoverage` in ./grade.mjs.
|
||||
*/
|
||||
|
||||
import { call, where } from './grade.mjs';
|
||||
|
||||
export const CORE_PHRASES = Object.freeze([
|
||||
// — navigation & framing —
|
||||
{ phrase: 'Take me to Tokyo', expect: 'fly_to_location', typedArgs: [where('fly_to_location', (a) => a.locationId === 'tokyo' || /tokyo/i.test(a.query || ''))] },
|
||||
{ phrase: 'Fly to the Golden Gate Bridge', expect: 'fly_to_location' },
|
||||
{ phrase: 'Go to Sixth Street in Austin', expect: 'fly_to_location' },
|
||||
{ phrase: 'Show me the Alps from above', expect: { oneOf: ['fly_to_location', 'frame_overhead'] } },
|
||||
{ phrase: 'Zoom in a bit', expect: 'adjust_camera_zoom', typedArgs: [where('adjust_camera_zoom', (a) => a.direction === 'in' && a.amount === 'little')] },
|
||||
{ phrase: 'Zoom out a little', expect: 'adjust_camera_zoom', typedArgs: [where('adjust_camera_zoom', (a) => a.direction === 'out' && a.amount === 'little')] },
|
||||
{ phrase: 'Zoom out to a globe view', expect: 'zoom_to_globe' },
|
||||
{ phrase: 'Show me the whole earth', expect: 'zoom_to_globe' },
|
||||
{ phrase: 'Frame the aircraft near us from overhead', expect: 'frame_overhead', typedArgs: [where('frame_overhead', (a) => a.target === 'flights')] },
|
||||
|
||||
// — the satellites trap: data layer, never basemap —
|
||||
{ phrase: 'Show me the satellites', expect: { oneOf: ['set_layer_visibility', 'frame_overhead'] } },
|
||||
{ phrase: 'Turn off the satellites', expect: 'set_layer_visibility', args: { layerId: 'satellites' }, typedArgs: [where('set_layer_visibility', (a) => a.layerId === 'satellites' && a.enabled === false)] },
|
||||
{ phrase: 'Switch to Bing aerial', expect: 'set_map_stack', typedArgs: [where('set_map_stack', (a) => a.stack === 'bing-aerial')] },
|
||||
{ phrase: 'Switch the basemap to OSM', expect: 'set_map_stack', typedArgs: [where('set_map_stack', (a) => a.stack === 'osm')] },
|
||||
|
||||
// — layers —
|
||||
{ phrase: 'Turn on the flights layer', expect: 'set_layer_visibility', args: { layerId: 'flights' }, typedArgs: [where('set_layer_visibility', (a) => a.layerId === 'flights' && a.enabled === true)] },
|
||||
{ phrase: 'Show me live vessels', expect: 'set_layer_visibility', typedArgs: [where('set_layer_visibility', (a) => a.layerId === 'ais-live-vessels' && a.enabled === true)] },
|
||||
{ phrase: 'Turn on the fires layer', expect: 'set_layer_visibility', typedArgs: [where('set_layer_visibility', (a) => a.layerId === 'local-firms' && a.enabled === true)] },
|
||||
{ phrase: 'Turn on street traffic', expect: 'set_layer_visibility', args: { layerId: 'traffic' }, typedArgs: [where('set_layer_visibility', (a) => a.layerId === 'traffic' && a.enabled === true)] },
|
||||
{ phrase: 'Open the data layers menu', expect: 'show_data_layers_menu' },
|
||||
{ phrase: 'Show me the datacenter layers', expect: 'show_data_layers_menu', typedArgs: [where('show_data_layers_menu', (a) => a.layerId === 'local-datacenters')] },
|
||||
{ phrase: 'Turn on the datacenters layer', expect: 'set_layer_visibility', typedArgs: [where('set_layer_visibility', (a) => a.layerId === 'local-datacenters' && a.enabled === true)] },
|
||||
|
||||
// — visual styles & post-fx —
|
||||
{ phrase: 'Give me night vision', expect: 'set_visual_style', typedArgs: [where('set_visual_style', (a) => a.style === 'surveillance')] },
|
||||
{ phrase: 'Switch to thermal view', expect: 'set_visual_style', typedArgs: [where('set_visual_style', (a) => a.style === 'thermal')] },
|
||||
{ phrase: 'Back to the normal look', expect: 'set_visual_style', typedArgs: [where('set_visual_style', (a) => a.style === 'normal')] },
|
||||
{ phrase: 'Turn on bloom', expect: 'set_post_processing', typedArgs: [where('set_post_processing', (a) => a.bloom?.enabled === true)] },
|
||||
{ phrase: 'Sharpen the image a touch', expect: 'set_post_processing', typedArgs: [where('set_post_processing', (a) => a.sharpen && a.sharpen.enabled !== false)] },
|
||||
|
||||
// — HUD / detection / panels —
|
||||
{ phrase: 'Turn the HUD off', expect: 'set_hud', typedArgs: [where('set_hud', (a) => a.visible === 'off')] },
|
||||
{ phrase: 'Switch to the tactical layout', expect: 'set_hud', typedArgs: [where('set_hud', (a) => a.layout === 'tactical')] },
|
||||
{ phrase: 'Turn on detection', expect: 'set_detection', typedArgs: [where('set_detection', (a) => a.enabled !== false)] },
|
||||
{ phrase: 'Set detection density to fifty percent', expect: 'set_detection', typedArgs: [where('set_detection', (a) => a.densityPct === 50 || a.mode === 'balanced')] },
|
||||
|
||||
// — context questions —
|
||||
{ phrase: 'What am I looking at right now?', expect: 'get_entity_context' },
|
||||
{ phrase: 'What city is this below us?', expect: 'get_entity_context' },
|
||||
{ phrase: 'Is there anything interesting in view?', expect: 'get_entity_context' },
|
||||
|
||||
// — tracking —
|
||||
// Context-free text turns may reasonably look before tracking; either
|
||||
// routing is correct (production sessions always carry screen context).
|
||||
{ phrase: 'Track that plane', expect: { oneOf: ['track_entity', 'get_entity_context'] } },
|
||||
// The session instructions route "nearest" through analyst_query first
|
||||
// (sortBy distance, limit 1); stubbed results make that first hop the whole
|
||||
// observable turn.
|
||||
{ phrase: 'Follow the nearest aircraft', expect: { oneOf: ['track_entity', 'get_entity_context', 'analyst_query'] } },
|
||||
{ phrase: 'Stop tracking', expect: 'stop_tracking' },
|
||||
|
||||
// — CCTV / scenes / ISS —
|
||||
{ phrase: 'Show me the nearest traffic camera', expect: 'control_cctv', typedArgs: [where('control_cctv', (a) => a.action === 'nearest')] },
|
||||
{ phrase: 'Turn on the camera viewsheds', expect: 'control_cctv', typedArgs: [where('control_cctv', (a) => a.action === 'viewshed' && a.enabled !== false)] },
|
||||
{ phrase: 'Play a news radio station near Austin', expect: 'control_radio', args: { action: 'select', category: 'news', locationId: 'austin' }, typedArgs: [where('control_radio', (a) => a.action === 'select' && a.category === 'news' && a.locationId === 'austin')] },
|
||||
{ phrase: 'Turn on the radio', expect: 'control_radio', args: { action: 'play' }, typedArgs: [where('control_radio', (a) => a.action === 'play')] },
|
||||
{ phrase: 'Set the radio volume to thirty percent', expect: 'control_radio', args: { action: 'volume', volumePct: 30 } },
|
||||
{ phrase: 'Pause the radio', expect: 'control_radio', args: { action: 'pause' }, typedArgs: [where('control_radio', (a) => a.action === 'pause')] },
|
||||
{ phrase: 'Stop the radio', expect: 'control_radio', args: { action: 'stop' }, typedArgs: [where('control_radio', (a) => a.action === 'stop')] },
|
||||
{ phrase: 'When does the ISS pass over next?', expect: 'next_iss_pass' },
|
||||
|
||||
// — annotations —
|
||||
{ phrase: 'Annotate the Texas State Capitol and its grounds', expect: 'annotate_map', typedArgs: [where('annotate_map', (a) => (a.annotations || []).some((x) => x.type === 'area'))] },
|
||||
{ phrase: 'Outline the state of Texas', expect: 'annotate_map', typedArgs: [where('annotate_map', (a) => (a.annotations || []).some((x) => x.type === 'area'))] },
|
||||
{ phrase: 'Outline Lady Bird Lake', expect: 'annotate_map', typedArgs: [where('annotate_map', (a) => (a.annotations || []).some((x) => x.type === 'area'))] },
|
||||
{ phrase: 'Draw the walking route from the Capitol to Zilker Park', expect: 'annotate_map', typedArgs: [where('annotate_map', (a) => (a.annotations || []).some((x) => x.type === 'route' && (x.mode ?? 'walking') === 'walking'))] },
|
||||
{ phrase: 'How far is the Eiffel Tower from the Louvre?', expect: 'annotate_map', typedArgs: [where('annotate_map', (a) => (a.annotations || []).some((x) => x.type === 'arrow'))] },
|
||||
{ phrase: 'Clear the map', expect: 'clear_annotations' },
|
||||
|
||||
// — multi-intent (assert ALL tools fire before speech) —
|
||||
{
|
||||
phrase: 'Switch to night vision and turn on the flights layer',
|
||||
expect: ['set_visual_style', 'set_layer_visibility'],
|
||||
typedArgs: [where('set_visual_style', (a) => a.style === 'surveillance'), where('set_layer_visibility', (a) => a.layerId === 'flights' && a.enabled === true)],
|
||||
},
|
||||
{
|
||||
phrase: 'Turn off the HUD and take me to Paris',
|
||||
expect: ['set_hud', 'fly_to_location'],
|
||||
typedArgs: [where('set_hud', (a) => a.visible === 'off'), where('fly_to_location', (a) => a.locationId === 'paris' || /paris/i.test(a.query || ''))],
|
||||
},
|
||||
{
|
||||
phrase: 'Go to full planet view and then turn on the radio',
|
||||
expect: ['zoom_to_globe', 'control_radio'],
|
||||
argsByTool: { control_radio: { action: 'play' } },
|
||||
},
|
||||
|
||||
// — camera verbs (tools #23/#24) —
|
||||
{ phrase: 'Orbit around this area slowly', expect: 'move_camera', typedArgs: [where('move_camera', (a) => a.motion === 'orbit' && a.speed === 'slow')] },
|
||||
{ phrase: 'Pan left a bit', expect: 'move_camera', typedArgs: [where('move_camera', (a) => a.motion === 'pan' && a.direction === 'left')] },
|
||||
{ phrase: 'Stop moving the camera', expect: 'move_camera', typedArgs: [where('move_camera', (a) => a.motion === 'stop')] },
|
||||
{ phrase: 'Fly the route we just drew', expect: 'fly_route' },
|
||||
|
||||
// — analyst queries (tool #22) —
|
||||
{ phrase: 'How many flights are over Texas right now?', expect: 'analyst_query', typedArgs: [where('analyst_query', (a) => (a.layers || []).includes('flights') && a.scope?.kind === 'region' && /texas/i.test(a.scope?.name || ''))] },
|
||||
{ phrase: 'Which ships are headed to Oakland?', expect: 'analyst_query', typedArgs: [where('analyst_query', (a) => (a.layers || []).includes('ais-live-vessels') && (a.filters || []).some((f) => f.field === 'destination' && /oakland/i.test(String(f.value))))] },
|
||||
{ phrase: 'What is the biggest fire near Los Angeles?', expect: 'analyst_query', typedArgs: [where('analyst_query', (a) => (a.layers || []).includes('local-firms') && ['frp', 'acres'].includes(a.sortBy) && a.sortDir !== 'asc')] },
|
||||
{ phrase: 'Is anything flying above forty thousand feet?', expect: 'analyst_query', typedArgs: [where('analyst_query', (a) => (a.layers || []).some((l) => l === 'flights' || l === 'military') && (a.filters || []).some((f) => f.field === 'altitudeM' && (f.op === 'gt' || f.op === 'gte') && Math.abs(Number(f.value) - 12192) <= 50))] },
|
||||
|
||||
// — negative controls: conversation must NOT tool-call —
|
||||
{ phrase: 'How is your evening going?', expectNone: true },
|
||||
{ phrase: 'Tell me a fun fact about maps', expectNone: true },
|
||||
]);
|
||||
|
||||
// ── Coverage probe ──────────────────────────────────────────────────────────
|
||||
//
|
||||
// Text-mode port of the voice-coverage routing probe. The live probe ran each
|
||||
// utterance against a real app page (setup steps, real tool results); this
|
||||
// benchmark has no app, so every item states the scene it assumes in `scene`
|
||||
// and the model sees only stub tool results. Grading therefore judges the
|
||||
// ROUTING DECISION and the honesty of the final words, not app behaviour.
|
||||
//
|
||||
// supported true when today's tools can satisfy the ask (a refusal is a
|
||||
// miss); false when the best possible outcome is an honest
|
||||
// refusal, optionally after a neutral helper call.
|
||||
// correct call patterns that satisfy the ask (any alternative).
|
||||
// neutral extra tools that neither help nor hurt (navigation to the
|
||||
// named place, enabling a related layer that exists).
|
||||
// hallucinated call patterns that fake the capability (e.g. toggling the
|
||||
// satellites DATA layer for "recent satellite imagery").
|
||||
// claimText final-text pattern that asserts data the model cannot have.
|
||||
//
|
||||
// Any call whose arguments violate the tool schema (an invented layer id) is
|
||||
// graded hallucinated-capability regardless of the item.
|
||||
//
|
||||
// Two rubrics, so a before/after comparison stays honest:
|
||||
// v1 the original rubric, graded against the tool schema of 4b56d0e9
|
||||
// (baseline-tools-4b56d0e9.json) — "by the old yardstick".
|
||||
// v2 `item.v2` overrides (supported / correct / hallucinated / neutral),
|
||||
// graded against the CURRENT production tools after the voice layer
|
||||
// manifest and point-and-ask landed. Use
|
||||
// coverageRubric(item, 'v2') to get the merged item.
|
||||
|
||||
const hasRoute = (c) =>
|
||||
c.name === 'annotate_map' &&
|
||||
Array.isArray(c.args?.annotations) &&
|
||||
c.args.annotations.some((a) => a?.type === 'route');
|
||||
|
||||
/** The coverage item as graded under rubric 'v1' (original) or 'v2' (current tools). */
|
||||
export function coverageRubric(item, version = 'v1') {
|
||||
if (version !== 'v2' || !item.v2) return item;
|
||||
return { ...item, ...item.v2 };
|
||||
}
|
||||
|
||||
export const COVERAGE_PROBE = Object.freeze([
|
||||
{
|
||||
id: 'quakes-m5-week',
|
||||
v2: { supported: true },
|
||||
utterance: 'Show me earthquakes above magnitude 5 this week',
|
||||
scene: 'globe, no layers on',
|
||||
expect: 'enable earthquakes + analyst_query magnitude>=5; must say feed covers last 24h only (no week)',
|
||||
supported: true,
|
||||
correct: [
|
||||
[call('analyst_query', { layers: 'earthquakes' })],
|
||||
[call('set_layer_visibility', { layerId: 'earthquakes', enabled: true })],
|
||||
],
|
||||
neutral: ['zoom_to_globe'],
|
||||
},
|
||||
{
|
||||
id: 'trains-london',
|
||||
v2: { supported: true, correct: [[call('analyst_query', { layers: 'transit' })], [call('set_layer_visibility', { layerId: 'transit', enabled: true })]], hallucinated: [] },
|
||||
utterance: 'How many trains are moving in London right now?',
|
||||
scene: 'London',
|
||||
expect: 'honest "no transit data for London / transit not queryable" — no invented count',
|
||||
supported: false,
|
||||
neutral: ['fly_to_location'],
|
||||
hallucinated: [call('analyst_query')],
|
||||
claimText: /\b\d+\s+(trains?|services)\b/i,
|
||||
},
|
||||
{
|
||||
id: 'transit-on',
|
||||
v2: { supported: true, correct: [[call('set_layer_visibility', { layerId: 'transit', enabled: true })]] },
|
||||
utterance: 'Turn on transit',
|
||||
scene: 'Austin',
|
||||
expect: 'set_layer_visibility transit (no transit id in the voice enum today) — honest otherwise',
|
||||
supported: false,
|
||||
hallucinated: [call('set_layer_visibility', { layerId: 'traffic', enabled: true })],
|
||||
},
|
||||
{
|
||||
id: 'ship-fastest-singapore',
|
||||
v2: { supported: true },
|
||||
utterance: 'Which ship near Singapore is fastest?',
|
||||
scene: 'Singapore Strait, vessels layer on',
|
||||
expect: 'analyst_query ais-live-vessels near Singapore sortBy speedKts desc',
|
||||
supported: true,
|
||||
correct: [[call('analyst_query', { layers: 'ais-live-vessels' })]],
|
||||
neutral: ['fly_to_location', 'set_layer_visibility'],
|
||||
},
|
||||
{
|
||||
id: 'recent-imagery',
|
||||
v2: { supported: true, correct: [[call('set_layer_visibility', { layerId: 'recent-imagery', enabled: true })]] },
|
||||
utterance: 'Show recent satellite imagery of this area',
|
||||
scene: 'Austin',
|
||||
expect: 'enable recent-imagery for current view (no tool today) — must NOT toggle satellites data layer or silently switch basemap',
|
||||
supported: false,
|
||||
hallucinated: [
|
||||
call('set_layer_visibility', { layerId: 'satellites' }),
|
||||
call('set_map_stack'),
|
||||
call('frame_overhead', { target: 'satellites' }),
|
||||
],
|
||||
},
|
||||
{
|
||||
id: 'wind-here',
|
||||
v2: { supported: true, correct: [[call('set_layer_visibility', { layerId: 'wind', enabled: true })]] },
|
||||
utterance: "What's the wind doing here?",
|
||||
scene: 'Austin',
|
||||
expect: 'enable/inspect wind layer at view centre (no tool today) — honest otherwise',
|
||||
supported: false,
|
||||
claimText: /\b\d+\s*(mph|km\/?h|knots?|kts|m\/s|meters per second|miles per hour)\b/i,
|
||||
},
|
||||
{
|
||||
id: 'cyclones-active',
|
||||
v2: { supported: true, correct: [[call('analyst_query', { layers: 'weather-cyclones' })], [call('set_layer_visibility', { layerId: 'weather-cyclones', enabled: true })]] },
|
||||
utterance: 'Are there any cyclones active right now?',
|
||||
scene: 'globe',
|
||||
expect: 'enable cyclones + list active storms (no tool today) — honest otherwise',
|
||||
supported: false,
|
||||
claimText: /\b(there (are|is)|currently)\b[^.?!]*\b(cyclones?|storms?|hurricanes?|typhoons?)\b/i,
|
||||
},
|
||||
{
|
||||
id: 'fires-in-drawn-area',
|
||||
// There is no drawn-area scope yet; the prompt says to say so. Any count
|
||||
// presented as inside the drawn area is a faked capability.
|
||||
v2: {
|
||||
supported: false,
|
||||
correct: [],
|
||||
hallucinated: [where('analyst_query', (a) => (a.layers || []).includes('local-firms'))],
|
||||
},
|
||||
utterance: 'Count the fires inside the area I drew',
|
||||
scene: 'Los Angeles, fires layer on, an area drawn around Angeles National Forest',
|
||||
expect: 'analyst_query local-firms scoped to the drawn polygon (no polygon scope today)',
|
||||
supported: true,
|
||||
correct: [[call('analyst_query', { layers: 'local-firms' })]],
|
||||
},
|
||||
{
|
||||
id: 'local-adsb-on',
|
||||
v2: { supported: true },
|
||||
utterance: 'Turn on the local ADS-B layer',
|
||||
scene: 'globe',
|
||||
expect: 'set_layer_visibility local-adsb; narrate receiver state honestly',
|
||||
supported: true,
|
||||
correct: [[call('set_layer_visibility', { layerId: 'local-adsb', enabled: true })]],
|
||||
},
|
||||
{
|
||||
id: 'bikes-near-me',
|
||||
v2: { supported: true, correct: [[call('analyst_query', { layers: 'bikeshare' })]] },
|
||||
utterance: 'How many bikes are available near me?',
|
||||
scene: 'New York, bikeshare layer on',
|
||||
expect: 'count available bikes at nearby stations (bikeshare not in analyst today) — honest otherwise',
|
||||
supported: false,
|
||||
neutral: ['set_layer_visibility'],
|
||||
claimText: /\b\d+\s+(available\s+)?(bikes?|bicycles?)\b/i,
|
||||
},
|
||||
{
|
||||
id: 'rocket-launch-track',
|
||||
v2: { supported: true, correct: [[call('set_layer_visibility', { layerId: 'rocket-launches', enabled: true })], [call('set_context_mode', { mode: 'space-missions' })], [call('set_context_mode', { mode: 'missions' })], [call('analyst_query', { layers: 'rocket-launches' })]] },
|
||||
utterance: 'Track the next rocket launch',
|
||||
scene: 'globe',
|
||||
expect: 'enable rocket-launches / space-missions and select next launch (track_entity cannot) — honest otherwise',
|
||||
supported: true,
|
||||
correct: [
|
||||
[call('set_layer_visibility', { layerId: 'rocket-launches', enabled: true })],
|
||||
[call('set_context_mode', { mode: 'space-missions' })],
|
||||
[call('set_context_mode', { mode: 'missions' })],
|
||||
],
|
||||
hallucinated: [call('track_entity')],
|
||||
},
|
||||
{
|
||||
id: 'weather-radar-on',
|
||||
v2: { supported: true, correct: [[call('set_layer_visibility', { layerId: 'weather-radar', enabled: true })]] },
|
||||
utterance: 'Turn on the weather radar',
|
||||
scene: 'Austin',
|
||||
expect: 'set_layer_visibility weather-radar (not in the voice enum today) — honest otherwise',
|
||||
supported: false,
|
||||
hallucinated: [call('set_map_stack'), call('set_visual_style')],
|
||||
},
|
||||
{
|
||||
id: 'lightning',
|
||||
v2: { supported: true, correct: [[call('set_layer_visibility', { layerId: 'weather-lightning', enabled: true })]] },
|
||||
utterance: 'Show me lightning strikes',
|
||||
scene: 'globe',
|
||||
expect: 'set_layer_visibility weather-lightning (not in the voice enum today) — honest otherwise',
|
||||
supported: false,
|
||||
},
|
||||
{
|
||||
id: 'military-bases',
|
||||
v2: { supported: true, correct: [[call('set_layer_visibility', { layerId: 'military-installations', enabled: true })], [call('analyst_query', { layers: 'military-installations' })]], hallucinated: [] },
|
||||
utterance: 'Show military bases near here',
|
||||
scene: 'San Diego',
|
||||
expect: 'set_layer_visibility military-installations (not in the voice enum today) — honest otherwise',
|
||||
supported: false,
|
||||
hallucinated: [call('annotate_map')],
|
||||
},
|
||||
{
|
||||
id: 'driving-directions',
|
||||
v2: { supported: true, correct: [[hasRoute], [call('set_layer_visibility', { layerId: 'directions', enabled: true })]] },
|
||||
utterance: 'Give me driving directions from the Texas Capitol to the Austin airport',
|
||||
scene: 'Austin',
|
||||
expect: 'annotate_map route driving (or directions layer)',
|
||||
supported: true,
|
||||
correct: [[hasRoute]],
|
||||
neutral: ['fly_to_location'],
|
||||
},
|
||||
{
|
||||
id: 'satellites-overhead-count',
|
||||
v2: { supported: true },
|
||||
utterance: 'How many satellites are overhead right now?',
|
||||
scene: 'globe, satellites layer on',
|
||||
expect: 'analyst_query satellites (control: supported)',
|
||||
supported: true,
|
||||
correct: [
|
||||
[call('analyst_query', { layers: 'satellites' })],
|
||||
[call('frame_overhead', { target: 'satellites' })],
|
||||
],
|
||||
neutral: ['set_layer_visibility'],
|
||||
},
|
||||
{
|
||||
id: 'biggest-quake-today',
|
||||
v2: { supported: true },
|
||||
utterance: "What's the biggest earthquake today?",
|
||||
scene: 'globe, earthquakes layer on',
|
||||
expect: 'analyst_query earthquakes sortBy magnitude desc (control: supported)',
|
||||
supported: true,
|
||||
correct: [[call('analyst_query', { layers: 'earthquakes' })]],
|
||||
neutral: ['set_layer_visibility'],
|
||||
},
|
||||
{
|
||||
id: 'flock-on',
|
||||
v2: { supported: true },
|
||||
utterance: 'Turn on the Flock cameras',
|
||||
scene: 'Austin',
|
||||
expect: 'set_layer_visibility alpr-cameras (control: supported)',
|
||||
supported: true,
|
||||
correct: [[call('set_layer_visibility', { layerId: 'alpr-cameras', enabled: true })]],
|
||||
hallucinated: [call('control_cctv')],
|
||||
},
|
||||
{
|
||||
id: 'alpr-count',
|
||||
v2: { supported: true, correct: [[call('analyst_query', { layers: 'alpr-cameras' })]] },
|
||||
utterance: 'How many license plate cameras are around here?',
|
||||
scene: 'Austin, ALPR layer on',
|
||||
expect: 'count alpr-cameras in view (not in analyst today) — honest otherwise',
|
||||
supported: false,
|
||||
neutral: ['set_layer_visibility'],
|
||||
claimText: /\b\d+\s+(license|plate|alpr|flock|cameras?)\b/i,
|
||||
},
|
||||
{
|
||||
id: 'nearest-bus',
|
||||
v2: { supported: true, correct: [[call('analyst_query', { layers: 'transit' })]] },
|
||||
utterance: 'Which bus is closest to the center of the screen?',
|
||||
scene: 'Austin, transit layer on',
|
||||
expect: 'nearest transit vehicle (no transit query/context today) — honest otherwise',
|
||||
supported: false,
|
||||
claimText: /\b(route|bus|line)\s+#?\d+\b/i,
|
||||
},
|
||||
{
|
||||
id: 'weather-replay',
|
||||
v2: { supported: false, hallucinated: [call('control_scene')] },
|
||||
utterance: 'Replay the last hour of weather radar',
|
||||
scene: 'globe',
|
||||
expect: 'weather clock replay (no time tool today) — honest refusal',
|
||||
supported: false,
|
||||
},
|
||||
{
|
||||
id: 'receiver-count',
|
||||
v2: { supported: true, correct: [[call('analyst_query', { layers: 'local-adsb' })]], hallucinated: [call('frame_overhead')] },
|
||||
utterance: 'How many aircraft is my receiver picking up?',
|
||||
scene: 'globe, no receiver connected',
|
||||
expect: 'count local-adsb (not in analyst) — honest: no receiver connected',
|
||||
supported: false,
|
||||
neutral: ['set_layer_visibility'],
|
||||
hallucinated: [call('analyst_query'), call('frame_overhead')],
|
||||
claimText: /\b\d+\s+(aircraft|planes?|contacts?)\b/i,
|
||||
},
|
||||
{
|
||||
id: 'nearest-traffic-camera',
|
||||
v2: { supported: true },
|
||||
utterance: 'Show me the nearest traffic camera',
|
||||
scene: 'Austin',
|
||||
expect: 'control_cctv nearest (control: supported)',
|
||||
supported: true,
|
||||
correct: [[call('control_cctv', { action: 'nearest' })]],
|
||||
neutral: ['set_layer_visibility'],
|
||||
},
|
||||
{
|
||||
id: 'hurricane-strongest',
|
||||
v2: { supported: true, correct: [[call('set_layer_visibility', { layerId: 'weather-cyclones', enabled: true }), call('analyst_query', { layers: 'weather-cyclones' })], [call('analyst_query', { layers: 'weather-cyclones' })]], hallucinated: [] },
|
||||
utterance: 'Turn on the hurricane tracks and fly me to the strongest storm',
|
||||
scene: 'globe',
|
||||
expect: 'enable cyclones + fly to strongest (no tool today) — honest otherwise',
|
||||
supported: false,
|
||||
hallucinated: [call('fly_to_location')],
|
||||
},
|
||||
]);
|
||||
|
||||
// ── Multi-turn dialogues ───────────────────────────────────────────────────
|
||||
//
|
||||
// Two-turn dialogues where turn 2 only makes sense given turn 1. Both turns
|
||||
// run in ONE session/context. Turn 1's tools get the plausible canned results
|
||||
// in `results` (anything not listed gets the neutral stub), so turn 2 has
|
||||
// something real to refer back to. `first` and `second` use the CORE_PHRASES
|
||||
// grading shape (expect / oneOf / expectNone / typedArgs); the headline score
|
||||
// is turn 2's full-call accuracy.
|
||||
|
||||
const flightsTexas = {
|
||||
ok: true,
|
||||
count: 212,
|
||||
scopeLabel: '212 flights over Texas',
|
||||
feedProvenance: { overall: 'nominal' },
|
||||
examples: [
|
||||
{ callsign: 'AAL1402', operator: 'American Airlines', altitudeM: 10972, military: false },
|
||||
{ callsign: 'SWA2210', operator: 'Southwest Airlines', altitudeM: 11278, military: false },
|
||||
{ callsign: 'RCH451', operator: 'US Air Force', altitudeM: 9144, military: true },
|
||||
],
|
||||
coverage: 'Counts cover data loaded by enabled layers.',
|
||||
};
|
||||
const shipsOakland = {
|
||||
ok: true,
|
||||
count: 4,
|
||||
scopeLabel: '4 ships headed to Oakland',
|
||||
feedProvenance: { overall: 'nominal' },
|
||||
examples: [
|
||||
{ name: 'MAERSK KENSINGTON', destination: 'OAKLAND', speedKts: 14.2, distanceKm: 180 },
|
||||
{ name: 'EVER LOVELY', destination: 'OAKLAND', speedKts: 16.8, distanceKm: 410 },
|
||||
{ name: 'APL SINGAPURA', destination: 'OAKLAND', speedKts: 12.1, distanceKm: 62 },
|
||||
],
|
||||
};
|
||||
|
||||
export const DIALOGUES = Object.freeze([
|
||||
{
|
||||
id: 'flights-then-military',
|
||||
turns: ['How many flights are over Texas right now?', 'Now just the military ones'],
|
||||
results: { analyst_query: [flightsTexas, { ...flightsTexas, count: 9, scopeLabel: '9 military aircraft over Texas', examples: [flightsTexas.examples[2]] }] },
|
||||
first: { expect: 'analyst_query' },
|
||||
second: {
|
||||
expect: 'analyst_query',
|
||||
typedArgs: [where('analyst_query', (a) => a.followUp === true || (a.layers || []).includes('military') || (a.filters || []).some((f) => f.field === 'military'))],
|
||||
},
|
||||
},
|
||||
{
|
||||
id: 'quakes-then-biggest',
|
||||
turns: ['Turn on the earthquakes layer', 'Take me to the biggest one'],
|
||||
results: {
|
||||
set_layer_visibility: [{ ok: true, layerId: 'earthquakes', enabled: true, feedState: 'nominal', count: 143 }],
|
||||
analyst_query: [null, { ok: true, count: 143, results: [{ id: 'us7000abcd', place: 'Kermadec Islands', magnitude: 6.4, latitude: -29.9, longitude: -177.8 }] }],
|
||||
},
|
||||
first: { expect: 'set_layer_visibility', typedArgs: [where('set_layer_visibility', (a) => a.layerId === 'earthquakes' && a.enabled === true)] },
|
||||
second: {
|
||||
expect: { oneOf: ['track_entity', 'analyst_query'] },
|
||||
typedArgs: [(c) =>
|
||||
(c.name === 'track_entity' && /quake|biggest|largest|strongest|kermadec/i.test(c.args?.query || '')) ||
|
||||
(c.name === 'analyst_query' && (c.args?.layers || []).includes('earthquakes') && c.args?.sortBy === 'magnitude' && c.args?.sortDir !== 'asc')],
|
||||
},
|
||||
},
|
||||
{
|
||||
id: 'track-then-stop',
|
||||
turns: ['Track UAL428', 'Stop'],
|
||||
results: {
|
||||
track_entity: [{ ok: true, tracking: 'UAL428', layerId: 'flights' }],
|
||||
move_camera: [null, { ok: true, stopped: false, tracking: true, note: 'No camera motion was active; an entity is being tracked.' }],
|
||||
},
|
||||
first: { expect: 'track_entity', typedArgs: [where('track_entity', (a) => /UAL\s*428/i.test(a.query || ''))] },
|
||||
second: { expect: 'stop_tracking' },
|
||||
},
|
||||
{
|
||||
id: 'paris-then-closer',
|
||||
turns: ['Take me to Paris', 'Get a little closer'],
|
||||
results: { fly_to_location: [{ ok: true, arrived: true, place: 'Paris, France' }] },
|
||||
first: { expect: 'fly_to_location' },
|
||||
second: { expect: 'adjust_camera_zoom', typedArgs: [where('adjust_camera_zoom', (a) => a.direction === 'in')] },
|
||||
},
|
||||
{
|
||||
id: 'thermal-then-back',
|
||||
turns: ['Switch to thermal view', 'Actually, put it back the way it was'],
|
||||
results: { set_visual_style: [{ ok: true, style: 'thermal', previousStyle: 'normal' }] },
|
||||
first: { expect: 'set_visual_style', typedArgs: [where('set_visual_style', (a) => a.style === 'thermal')] },
|
||||
second: { expect: 'set_visual_style', typedArgs: [where('set_visual_style', (a) => a.style === 'normal')] },
|
||||
},
|
||||
{
|
||||
id: 'radio-then-volume',
|
||||
turns: ['Turn on the radio', 'Turn it down to twenty percent'],
|
||||
results: { control_radio: [{ ok: true, prepared: true, action: 'play', station: 'KUT 90.5 Austin' }] },
|
||||
first: { expect: 'control_radio', typedArgs: [where('control_radio', (a) => a.action === 'play')] },
|
||||
second: { expect: 'control_radio', typedArgs: [where('control_radio', (a) => a.action === 'volume' && a.volumePct === 20)] },
|
||||
},
|
||||
{
|
||||
id: 'mark-then-route',
|
||||
turns: ['Mark the Texas Capitol on the map', 'Now draw the walking route from there to Zilker Park'],
|
||||
results: { annotate_map: [{ ok: true, placed: [{ label: 'Texas State Capitol', type: 'pin' }] }] },
|
||||
first: { expect: 'annotate_map' },
|
||||
second: {
|
||||
expect: 'annotate_map',
|
||||
typedArgs: [where('annotate_map', (a) => (a.annotations || []).some((x) =>
|
||||
x.type === 'route' && (x.mode ?? 'walking') === 'walking' && /capitol/i.test(x.points?.[0]?.target || '')))],
|
||||
},
|
||||
},
|
||||
{
|
||||
id: 'ships-then-closest',
|
||||
turns: ['Which ships are headed to Oakland?', 'Which of those is closest?'],
|
||||
results: { analyst_query: [shipsOakland, { ...shipsOakland, count: 1, examples: [shipsOakland.examples[2]] }] },
|
||||
first: { expect: 'analyst_query' },
|
||||
second: { expect: 'analyst_query', typedArgs: [where('analyst_query', (a) => a.followUp === true && a.sortBy === 'distance')] },
|
||||
},
|
||||
{
|
||||
id: 'flights-then-thanks',
|
||||
turns: ['Turn on the flights layer', "Thanks, that's perfect"],
|
||||
results: { set_layer_visibility: [{ ok: true, layerId: 'flights', enabled: true, feedState: 'nominal', count: 1840 }] },
|
||||
first: { expect: 'set_layer_visibility' },
|
||||
second: { expectNone: true },
|
||||
},
|
||||
]);
|
||||
|
||||
/** Canned result for a dialogue tool call, or null for the neutral stub. */
|
||||
export function dialogueResult(dialogue, toolName, turn) {
|
||||
const byTurn = dialogue.results?.[toolName];
|
||||
if (!byTurn) return null;
|
||||
return byTurn[turn] ?? null;
|
||||
}
|
||||
|
||||
// ── New capabilities (point-and-ask, layer reach) ─────────────────────────
|
||||
//
|
||||
// ROUTING_PHRASES is the qa-voice-routing table, so that harness runs exactly
|
||||
// its own table. The benchmark runs CAPABILITY_EXTRAS as the separate
|
||||
// `capability` suite so the core suite stays comparable across tool versions.
|
||||
|
||||
/** The full qa-voice-routing table. */
|
||||
export const ROUTING_PHRASES = CORE_PHRASES;
|
||||
|
||||
// Pointer context items in the app's own format (src/voice/pointerContext.js
|
||||
// output), sent as an inert system item just before the user's words.
|
||||
const AIRCRAFT_POINTER = {
|
||||
type: 'pointer_context', at: 'speech_start',
|
||||
target: { kind: 'aircraft', layer: 'flights', label: 'UPS793' },
|
||||
ground: { lat: 30.3121, lon: -97.6013 }, screen: { x: 0.608, y: 0.408 },
|
||||
};
|
||||
const GROUND_POINTER = {
|
||||
type: 'pointer_context', at: 'speech_start',
|
||||
ground: { lat: 30.2849, lon: -97.7341 }, screen: { x: 0.427, y: 0.547 },
|
||||
};
|
||||
const isPointer = (v) => String(v || '').toLowerCase() === 'pointer';
|
||||
const flightsInView = {
|
||||
ok: true,
|
||||
count: 3,
|
||||
scopeLabel: 'in view',
|
||||
items: [
|
||||
{ n: 1, id: 'a1b2c3', callsign: 'UAL428', layerKey: 'flights', lat: 30.2, lon: -97.7 },
|
||||
{ n: 2, id: 'a4b5c6', callsign: 'SWA1201', layerKey: 'flights', lat: 30.3, lon: -97.6 },
|
||||
{ n: 3, id: 'a7b8c9', callsign: 'AAL88', layerKey: 'flights', lat: 30.1, lon: -97.8 },
|
||||
],
|
||||
};
|
||||
|
||||
/**
|
||||
* Extra single-turn phrases (with an optional pointer `context`) and two-turn
|
||||
* dialogues for the new capabilities. Items with `turns` are graded like
|
||||
* DIALOGUES (turn 2 full-call); the rest like CORE_PHRASES.
|
||||
*/
|
||||
export const CAPABILITY_EXTRAS = Object.freeze([
|
||||
{ phrase: 'Tell me about this', context: AIRCRAFT_POINTER, expect: 'get_entity_context', typedArgs: [where('get_entity_context', (a) => isPointer(a.scope))] },
|
||||
{ phrase: 'How many flights are around here?', context: GROUND_POINTER, expect: 'analyst_query', typedArgs: [where('analyst_query', (a) => isPointer(a.scope?.kind))] },
|
||||
{ phrase: 'Take me there', context: GROUND_POINTER, expect: 'fly_to_location', typedArgs: [where('fly_to_location', (a) => isPointer(a.query))] },
|
||||
{ phrase: 'Track that one', context: AIRCRAFT_POINTER, expect: 'track_entity', typedArgs: [where('track_entity', (a) => isPointer(a.query))] },
|
||||
{ phrase: 'Turn on the weather radar', expect: 'set_layer_visibility', typedArgs: [where('set_layer_visibility', (a) => a.layerId === 'weather-radar' && a.enabled === true)] },
|
||||
{ phrase: 'Are there any hurricanes right now?', expect: { oneOf: ['analyst_query', 'set_layer_visibility'] }, typedArgs: [(c) => (c.args?.layers || []).includes('weather-cyclones') || c.args?.layerId === 'weather-cyclones'] },
|
||||
{ phrase: 'Which bus is closest to the middle of the screen?', expect: 'analyst_query', typedArgs: [where('analyst_query', (a) => (a.layers || []).includes('transit'))] },
|
||||
{
|
||||
id: 'flights-then-second',
|
||||
turns: ['How many flights are in view?', 'Track the second one'],
|
||||
results: { analyst_query: [flightsInView] },
|
||||
first: { expect: 'analyst_query' },
|
||||
second: { expect: 'track_entity', typedArgs: [where('track_entity', (a) => Number(a.referent) === 2 || /SWA\s*1201/i.test(a.query || ''))] },
|
||||
},
|
||||
]);
|
||||
@@ -0,0 +1,58 @@
|
||||
/**
|
||||
* Cost estimation for benchmark turns (USD). Pure.
|
||||
*
|
||||
* OpenAI Realtime: the app's own rate table (`src/voice/voiceCost.js`).
|
||||
* `gpt-realtime-2.1` has no registry entry, so it resolves to the priciest
|
||||
* known table (gpt-realtime-2) — which matches OpenAI's published 2.1 rates
|
||||
* as read on 2026-09-23 ($4 / $0.40 cached / $24 per 1M text tokens).
|
||||
*
|
||||
* Gemini Live: https://ai.google.dev/gemini-api/docs/pricing, read 2026-09-23,
|
||||
* gemini-3.8-live paid tier, USD per 1M tokens. Thinking tokens bill as text
|
||||
* output. The native-audio Live model rejects TEXT output, so the benchmark
|
||||
* requests AUDIO and the spoken confirmation bills at the audio-output rate.
|
||||
*/
|
||||
import { estimateUsageCostUsd, resolveVoiceModelById } from '../../src/voice/voiceCost.js';
|
||||
|
||||
export const GEMINI_RATES = Object.freeze({
|
||||
'gemini-3.8-live': Object.freeze({
|
||||
textInput: 0.75,
|
||||
audioInput: 3.0,
|
||||
textOutput: 4.5,
|
||||
audioOutput: 12.0,
|
||||
verifiedOn: '2026-09-23',
|
||||
}),
|
||||
});
|
||||
|
||||
/** Cost of one Realtime `response.usage` for `model`. */
|
||||
export function openAiUsageCostUsd(model, usage) {
|
||||
return estimateUsageCostUsd(usage, resolveVoiceModelById(model).rates);
|
||||
}
|
||||
|
||||
const modalityCount = (details, modality) =>
|
||||
(Array.isArray(details) ? details : [])
|
||||
.filter((d) => String(d?.modality || '').toUpperCase() === modality)
|
||||
.reduce((sum, d) => sum + (Number(d?.tokenCount) || 0), 0);
|
||||
|
||||
/**
|
||||
* Cost of one Gemini Live `usageMetadata`. Unattributed prompt tokens bill
|
||||
* as audio input and unattributed response tokens as audio output — the
|
||||
* conservative direction for a spend cap.
|
||||
*/
|
||||
export function geminiUsageCostUsd(model, usage) {
|
||||
const rates = GEMINI_RATES[model] || GEMINI_RATES['gemini-3.8-live'];
|
||||
if (!usage) return 0;
|
||||
const prompt = Number(usage.promptTokenCount) || 0;
|
||||
const response = Number(usage.responseTokenCount) || 0;
|
||||
const thoughts = Number(usage.thoughtsTokenCount) || 0;
|
||||
const textIn = modalityCount(usage.promptTokensDetails, 'TEXT');
|
||||
const audioIn = Math.max(0, prompt - textIn);
|
||||
const textOut = modalityCount(usage.responseTokensDetails, 'TEXT');
|
||||
const audioOut = Math.max(0, response - textOut);
|
||||
const usd =
|
||||
(textIn * rates.textInput +
|
||||
audioIn * rates.audioInput +
|
||||
(textOut + thoughts) * rates.textOutput +
|
||||
audioOut * rates.audioOutput) /
|
||||
1_000_000;
|
||||
return Number.isFinite(usd) && usd > 0 ? usd : 0;
|
||||
}
|
||||
@@ -0,0 +1,36 @@
|
||||
/**
|
||||
* Capture the exact Realtime session config production mints.
|
||||
*
|
||||
* Rather than re-typing the session fields, this drives the production token
|
||||
* handler (`server/providers/openai/realtime.js`) with a fake upstream fetch
|
||||
* and returns the body it would have POSTed to OpenAI. Instructions, tools,
|
||||
* tool_choice, reasoning, and truncation therefore cannot drift from what a
|
||||
* real GEV mic session receives.
|
||||
*/
|
||||
import { createRealtimeTokenHandler } from '../../server/providers/openai/realtime.js';
|
||||
import { realtimeInstructions } from '../../server/providers/openai/instructions.js';
|
||||
import { GEV_REALTIME_TOOLS } from '../../server/providers/openai/tools.js';
|
||||
|
||||
export { realtimeInstructions, GEV_REALTIME_TOOLS };
|
||||
|
||||
/** Return production's `{ session: {...} }` client-secret body for `model`. */
|
||||
export async function captureProductionSession(model) {
|
||||
let captured = null;
|
||||
const handler = createRealtimeTokenHandler({
|
||||
resolveApiKey: () => 'bench-placeholder',
|
||||
models: { standard: model, mini: model },
|
||||
fetchImpl: async (_url, init) => {
|
||||
captured = JSON.parse(init.body);
|
||||
return new Response('{}', { status: 200, headers: { 'content-type': 'application/json' } });
|
||||
},
|
||||
});
|
||||
const res = {
|
||||
statusCode: 0,
|
||||
headers: {},
|
||||
setHeader(k, v) { this.headers[k] = v; },
|
||||
end() {},
|
||||
};
|
||||
await handler({ method: 'POST', url: '/api/realtime/token', headers: {}, socket: {} }, res);
|
||||
if (!captured?.session) throw new Error('production token handler did not produce a session config');
|
||||
return captured.session;
|
||||
}
|
||||
@@ -0,0 +1,272 @@
|
||||
/**
|
||||
* Pure aggregation of benchmark rows into summary.json / summary.md.
|
||||
* Rows: `{ type:'run', ... }` run metadata and `{ type:'turn', ... }` turns.
|
||||
*/
|
||||
|
||||
export function percentile(values, p) {
|
||||
const xs = values.filter((v) => typeof v === 'number' && Number.isFinite(v)).sort((a, b) => a - b);
|
||||
if (!xs.length) return null;
|
||||
const idx = Math.min(xs.length - 1, Math.max(0, Math.ceil((p / 100) * xs.length) - 1));
|
||||
return xs[idx];
|
||||
}
|
||||
|
||||
const pct = (n, d) => (d ? Math.round((1000 * n) / d) / 10 : null);
|
||||
const sum = (xs) => xs.reduce((a, b) => a + (Number(b) || 0), 0);
|
||||
const mean = (xs) => {
|
||||
const v = xs.filter((x) => typeof x === 'number' && Number.isFinite(x));
|
||||
return v.length ? sum(v) / v.length : null;
|
||||
};
|
||||
const round = (x, d = 0) => (x === null || x === undefined ? null : Math.round(x * 10 ** d) / 10 ** d);
|
||||
|
||||
export const COVERAGE_VERDICTS = Object.freeze(['correct', 'honest-refusal', 'hallucinated-capability', 'wrong-tool']);
|
||||
|
||||
export function modelKey(row) {
|
||||
return `${row.provider}:${row.model}${row.variant ? `#${row.variant}` : ''}`;
|
||||
}
|
||||
|
||||
/** Aggregate turn rows into one summary entry per provider×model(×variant). */
|
||||
export function summarize(rows) {
|
||||
const runs = rows.filter((r) => r.type === 'run');
|
||||
const turns = rows.filter((r) => r.type === 'turn');
|
||||
const keys = [...new Set(turns.map(modelKey))];
|
||||
const models = keys.map((key) => {
|
||||
const mine = turns.filter((t) => modelKey(t) === key);
|
||||
const core = mine.filter((t) => t.suite === 'core');
|
||||
const cov = mine.filter((t) => t.suite === 'coverage');
|
||||
const dia = mine.filter((t) => t.suite === 'dialogue');
|
||||
const capAll = mine.filter((t) => t.suite === 'capability');
|
||||
// Latency percentiles stay on the core + coverage phrase sets so runs of
|
||||
// different builds remain comparable.
|
||||
const single = mine.filter((t) => t.suite === 'core' || t.suite === 'coverage');
|
||||
const covV2 = cov.filter((t) => t.gradeV2);
|
||||
const reps = [...new Set(core.map((t) => t.rep))].sort((a, b) => a - b);
|
||||
const withCall = single.filter((t) => t.tFirstCallMs !== null && t.tFirstCallMs !== undefined);
|
||||
const typed = core.filter((t) => t.grade?.typedArgsOk !== null && t.grade?.typedArgsOk !== undefined);
|
||||
const diaFailures = dia.filter((t) => !t.grade?.ok).map((t) => ({
|
||||
id: t.phraseId, rep: t.rep, expect: t.expect,
|
||||
turn1: (t.turn1?.calls || []).map((c) => `${c.name}(${JSON.stringify(c.args).slice(0, 80)})`).join(', ') || '(none)',
|
||||
turn2: (t.calls || []).map((c) => `${c.name}(${JSON.stringify(c.args).slice(0, 80)})`).join(', ') || '(none)',
|
||||
}));
|
||||
const coverageV2Counts = Object.fromEntries(COVERAGE_VERDICTS.map((v) => [v, covV2.filter((t) => t.gradeV2?.verdict === v).length]));
|
||||
const capFailures = {};
|
||||
for (const t of capAll.filter((x) => !x.grade?.fullCallOk)) {
|
||||
const k = t.phraseId;
|
||||
capFailures[k] = capFailures[k] || { id: k, expect: t.expect, fails: 0, of: capAll.filter((x) => x.phraseId === k).length, called: [] };
|
||||
capFailures[k].fails += 1;
|
||||
capFailures[k].called.push((t.calls || []).map((c) => `${c.name}(${JSON.stringify(c.args).slice(0, 70)})`).join(', ') || '(none)');
|
||||
}
|
||||
const coverageCounts = Object.fromEntries(COVERAGE_VERDICTS.map((v) => [v, cov.filter((t) => t.grade?.verdict === v).length]));
|
||||
const coreFailures = {};
|
||||
for (const t of core.filter((x) => !x.grade?.ok)) {
|
||||
const k = t.phrase;
|
||||
coreFailures[k] = coreFailures[k] || { phrase: k, expect: t.expect, fails: 0, of: core.filter((x) => x.phrase === k).length, called: [] };
|
||||
coreFailures[k].fails += 1;
|
||||
coreFailures[k].called.push((t.calls || []).map((c) => c.name).join(',') || '(none)');
|
||||
}
|
||||
const covByItem = {};
|
||||
for (const t of cov) {
|
||||
covByItem[t.phraseId] = covByItem[t.phraseId] || { id: t.phraseId, utterance: t.phrase, supported: t.supported, verdicts: [] };
|
||||
covByItem[t.phraseId].verdicts.push(t.grade?.verdict);
|
||||
}
|
||||
const run = runs.filter((r) => modelKey(r) === key).at(-1) || {};
|
||||
const local = mine.filter((t) => t.local);
|
||||
return {
|
||||
key,
|
||||
provider: mine[0].provider,
|
||||
model: mine[0].model,
|
||||
variant: mine[0].variant || null,
|
||||
settings: run.settings || null,
|
||||
machine: run.machine || null,
|
||||
warmup: run.warmup || null,
|
||||
turns: mine.length,
|
||||
core: {
|
||||
n: core.length,
|
||||
reps: reps.length,
|
||||
accuracyPct: pct(core.filter((t) => t.grade?.ok).length, core.length),
|
||||
toolAccuracyPct: pct(core.filter((t) => t.grade?.toolOk).length, core.length),
|
||||
fullCallAccuracyPct: pct(core.filter((t) => t.grade?.fullCallOk).length, core.length),
|
||||
typedArgsAccuracyPct: pct(typed.filter((t) => t.grade?.typedArgsOk).length, typed.length),
|
||||
typedArgsN: typed.length,
|
||||
argsAccuracyPct: pct(core.filter((t) => t.grade?.argsOk === true).length, core.filter((t) => t.grade?.argsOk !== null && t.grade?.argsOk !== undefined).length),
|
||||
firstCallAccuracyPct: pct(core.filter((t) => t.grade?.firstCallOk).length, core.length),
|
||||
falsePositives: core.filter((t) => t.grade?.falsePositive).length,
|
||||
negativeTurns: core.filter((t) => t.expectNone).length,
|
||||
perRepAccuracyPct: reps.map((rep) => {
|
||||
const r = core.filter((t) => t.rep === rep);
|
||||
return pct(r.filter((t) => t.grade?.ok).length, r.length);
|
||||
}),
|
||||
failures: Object.values(coreFailures).sort((a, b) => b.fails - a.fails),
|
||||
},
|
||||
coverage: {
|
||||
n: cov.length,
|
||||
counts: coverageCounts,
|
||||
acceptablePct: pct(cov.filter((t) => t.grade?.acceptable).length, cov.length),
|
||||
supportedCorrectPct: pct(cov.filter((t) => t.supported && t.grade?.verdict === 'correct').length, cov.filter((t) => t.supported).length),
|
||||
unsupportedHonestPct: pct(cov.filter((t) => !t.supported && t.grade?.acceptable).length, cov.filter((t) => !t.supported).length),
|
||||
items: Object.values(covByItem),
|
||||
},
|
||||
coverageV2: {
|
||||
n: covV2.length,
|
||||
counts: coverageV2Counts,
|
||||
acceptablePct: pct(covV2.filter((t) => t.gradeV2?.acceptable).length, covV2.length),
|
||||
},
|
||||
capability: {
|
||||
n: capAll.length,
|
||||
toolPct: pct(capAll.filter((t) => t.grade?.toolOk).length, capAll.length),
|
||||
fullPct: pct(capAll.filter((t) => t.grade?.fullCallOk).length, capAll.length),
|
||||
failures: Object.values(capFailures).sort((a, b) => b.fails - a.fails),
|
||||
},
|
||||
prompt: run.prompt || null,
|
||||
dialogue: {
|
||||
n: dia.length,
|
||||
turn2FullPct: pct(dia.filter((t) => t.grade?.fullCallOk).length, dia.length),
|
||||
turn2ToolPct: pct(dia.filter((t) => t.grade?.toolOk).length, dia.length),
|
||||
turn1FullPct: pct(dia.filter((t) => t.grade?.turn1Ok).length, dia.length),
|
||||
turn2ToolCallP50: percentile(dia.map((t) => t.tFirstCallMs), 50),
|
||||
failures: diaFailures,
|
||||
},
|
||||
latencyMs: {
|
||||
scope: 'single-turn phrases (core + coverage); send → event, text mode',
|
||||
firstTokenP50: percentile(single.map((t) => t.tFirstTokenMs), 50),
|
||||
firstTokenP90: percentile(single.map((t) => t.tFirstTokenMs), 90),
|
||||
toolCallP50: percentile(withCall.map((t) => t.tFirstCallMs), 50),
|
||||
toolCallP90: percentile(withCall.map((t) => t.tFirstCallMs), 90),
|
||||
turnDoneP50: percentile(single.map((t) => t.tDoneMs), 50),
|
||||
turnDoneP90: percentile(single.map((t) => t.tDoneMs), 90),
|
||||
setupP50: percentile(single.map((t) => t.setupMs), 50),
|
||||
},
|
||||
tokens: {
|
||||
input: sum(mine.map((t) => t.usage?.inputTokens)),
|
||||
cached: sum(mine.map((t) => t.usage?.cachedTokens)),
|
||||
output: sum(mine.map((t) => t.usage?.outputTokens)),
|
||||
reasoning: sum(mine.map((t) => t.usage?.reasoningTokens)),
|
||||
inputPerTurnMean: round(mean(mine.map((t) => t.usage?.inputTokens))),
|
||||
inputPerRequestMean: round(mean(single.filter((t) => t.responses > 0 && t.usage?.inputTokens).map((t) => t.usage.inputTokens / t.responses))),
|
||||
},
|
||||
costUsd: round(sum(mine.map((t) => t.costUsd)), 4),
|
||||
costPerTurnUsd: mine.length ? round(sum(mine.map((t) => t.costUsd)) / mine.length, 5) : null,
|
||||
errors: mine.filter((t) => t.errors?.length).length,
|
||||
timeouts: mine.filter((t) => t.timeout).length,
|
||||
local: local.length
|
||||
? {
|
||||
prefillTokSMean: round(mean(local.map((t) => t.local.prefillTokS)), 1),
|
||||
prefillTokSP50: round(percentile(local.map((t) => t.local.prefillTokS), 50), 0),
|
||||
decodeTokSMean: round(mean(local.map((t) => t.local.decodeTokS)), 1),
|
||||
decodeTokSP50: round(percentile(local.map((t) => t.local.decodeTokS), 50), 1),
|
||||
decodeTokSP90: round(percentile(local.map((t) => t.local.decodeTokS), 90), 1),
|
||||
}
|
||||
: null,
|
||||
};
|
||||
});
|
||||
return {
|
||||
generatedAt: new Date().toISOString(),
|
||||
cloudSpendUsd: round(sum(turns.filter((t) => t.cloud).map((t) => t.costUsd)), 4),
|
||||
models,
|
||||
};
|
||||
}
|
||||
|
||||
const machineBrief = (machine) => {
|
||||
if (!machine) return '—';
|
||||
const load1 = String(machine.loadavg || '').split(' ')[0] || '?';
|
||||
const busy = (machine.busyProcesses || []).filter((p) => Number(p.match(/([\d.]+)%cpu/)?.[1]) >= 100)
|
||||
.map((p) => p.replace(/ Helper \(GPU\)/, ' GPU').replace(/Google Chrome for Testing/, 'Chrome-for-Testing'));
|
||||
return `load ${load1}${busy.length ? `; ${busy.join(', ')}` : ''}`;
|
||||
};
|
||||
const fmtCounts = (block) => (block?.n
|
||||
? `${block.counts.correct} / ${block.counts['honest-refusal']} / ${block.counts['hallucinated-capability']} / ${block.counts['wrong-tool']} (n=${block.n})`
|
||||
: '—');
|
||||
const fmtMs = (v) => (v === null || v === undefined ? '—' : v >= 10_000 ? `${(v / 1000).toFixed(1)}s` : `${Math.round(v)}ms`);
|
||||
const fmtPct = (v) => (v === null || v === undefined ? '—' : `${v}%`);
|
||||
|
||||
/** Render summary.md from a summarize() result plus free-form notes. */
|
||||
export function renderMarkdown(summary, { title = 'GEV voice-routing benchmark', notes = [], payload = [] } = {}) {
|
||||
const out = [`# ${title}`, '', `Generated ${summary.generatedAt}. Cloud spend so far: **$${summary.cloudSpendUsd?.toFixed(2)}**.`, ''];
|
||||
out.push('> **What this measures.** Tool CHOICE for text turns sent over each provider\'s API with production\'s instructions and tools, with stubbed (always-successful) tool results — the same method as `qa-voice-routing.mjs` layer 1. It is **not** end-to-end spoken accuracy (no speech recognition, no audio, no screenshot/scene context) and **not** spoken latency: every latency below is "send text → event", before any audio would be synthesized or played.', '');
|
||||
out.push('## Routing accuracy', '');
|
||||
out.push('| model | core tool-name | core full-call (tool + all arg checks) | typed args (n) | harness verdict (legacy) | first call right | false +ve | per-rep harness verdict | dialogue turn-2 full-call | dialogue turn-1 | coverage v1 C / HR / HC / WT | coverage v1 acceptable | coverage v2 C / HR / HC / WT | coverage v2 acceptable | capability tool / full-call |');
|
||||
out.push('|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|');
|
||||
for (const m of summary.models) {
|
||||
const c = m.coverage.counts;
|
||||
out.push(`| ${m.key} | ${fmtPct(m.core.toolAccuracyPct)} | ${fmtPct(m.core.fullCallAccuracyPct)} | ${fmtPct(m.core.typedArgsAccuracyPct)} (${m.core.typedArgsN}) | ${fmtPct(m.core.accuracyPct)} (n=${m.core.n}) | ${fmtPct(m.core.firstCallAccuracyPct)} | ${m.core.falsePositives}/${m.core.negativeTurns} | ${m.core.perRepAccuracyPct.map(fmtPct).join(', ')} | ${fmtPct(m.dialogue.turn2FullPct)} (n=${m.dialogue.n}) | ${fmtPct(m.dialogue.turn1FullPct)} | ${c.correct} / ${c['honest-refusal']} / ${c['hallucinated-capability']} / ${c['wrong-tool']} (n=${m.coverage.n}) | ${fmtPct(m.coverage.acceptablePct)} | ${fmtCounts(m.coverageV2)} | ${fmtPct(m.coverageV2.acceptablePct)} | ${m.capability.n ? `${fmtPct(m.capability.toolPct)} / ${fmtPct(m.capability.fullPct)} (n=${m.capability.n})` : '—'} |`);
|
||||
}
|
||||
out.push('', 'Core tool-name = right tool(s) called. Full-call = right tool(s) AND every argument check (the legacy `args` spot checks plus the benchmark\'s typed checks: exact layer ids, enum values, numeric thresholds, sort direction). Legacy harness verdict = exactly what `qa-voice-routing.mjs` would report. Coverage: C = correct, HR = honest refusal, HC = hallucinated capability, WT = wrong tool; "acceptable" = correct, or an honest refusal where today\'s tools cannot do it.', '');
|
||||
out.push('## Latency and cost', '');
|
||||
out.push('| model | p50 send → first token (text) | p50 send → tool call (text) | p90 send → tool call (text) | p50 send → turn done (text) | dialogue turn-2 p50 → tool call | session setup p50 | input tok/turn | $ total | $/turn |');
|
||||
out.push('|---|---|---|---|---|---|---|---|---|---|');
|
||||
for (const m of summary.models) {
|
||||
out.push(`| ${m.key} | ${fmtMs(m.latencyMs.firstTokenP50)} | ${fmtMs(m.latencyMs.toolCallP50)} | ${fmtMs(m.latencyMs.toolCallP90)} | ${fmtMs(m.latencyMs.turnDoneP50)} | ${fmtMs(m.dialogue.turn2ToolCallP50)} | ${fmtMs(m.latencyMs.setupP50)} | ${m.tokens.inputPerTurnMean ?? '—'} | ${m.costUsd?.toFixed(3)} | ${m.costPerTurnUsd ?? '—'} |`);
|
||||
}
|
||||
out.push('', 'Latencies are over single-turn phrases (core + coverage). "Turn done" includes the continuation response after the stubbed tool result. Cloud session setup (WebSocket + session config) is excluded from every latency and shown separately.', '');
|
||||
const locals = summary.models.filter((m) => m.local);
|
||||
if (locals.length) {
|
||||
out.push('## Local runtime (cold vs warm)', '');
|
||||
out.push('| model | load (wall) | COLD: first request after load → tool call | cold prefill tok/s (full prompt) | prompt tokens | WARM p50 → tool call | WARM p90 → tool call | warm per-turn prefill tok/s p50 | decode tok/s p50 / p90 | machine at start |');
|
||||
out.push('|---|---|---|---|---|---|---|---|---|---|');
|
||||
for (const m of locals) {
|
||||
const w = m.warmup || {};
|
||||
out.push(`| ${m.key} | ${fmtMs(w.loadWallMs ?? w.loadMs)} | ${fmtMs(w.coldToolCallMs ?? w.warmupTurnMs)}${w.coldToolCallMs == null ? ' (turn, no tool)' : ''} | ${w.prefillTokS ? Math.round(w.prefillTokS) : '—'} | ${w.promptTokens ?? '—'} | ${fmtMs(m.latencyMs.toolCallP50)} | ${fmtMs(m.latencyMs.toolCallP90)} | ${m.local.prefillTokSP50 ?? '—'} | ${m.local.decodeTokSP50 ?? '—'} / ${m.local.decodeTokSP90 ?? '—'} | ${machineBrief(m.machine)} |`);
|
||||
}
|
||||
out.push('', 'COLD = the first request after the model is loaded (pays the full ~11k-token prompt prefill). WARM = every later turn. "Warm per-turn prefill tok/s" is prompt tokens ÷ prompt-eval time as the runtime reports it: a value in the tens of thousands means the cached prompt prefix was reused and only the new user text was prefilled; a value near the cold rate means the WHOLE prompt was re-prefilled on every turn. A real voice session pays the cold cost once per model load. "Machine at start" = 1-minute load average and processes above 100% CPU when the run began (other GPU/CPU users on the machine inflate local timings).', '');
|
||||
}
|
||||
if (payload.length) {
|
||||
out.push('## Payload changes per provider', '');
|
||||
for (const p of payload) out.push(`- ${p}`);
|
||||
out.push('');
|
||||
}
|
||||
out.push('## Failures by model', '');
|
||||
for (const m of summary.models) {
|
||||
out.push(`### ${m.key}`, '');
|
||||
out.push(`Settings: \`${JSON.stringify(m.settings)}\``, '');
|
||||
if (m.errors || m.timeouts) out.push(`Errors: ${m.errors} turns, timeouts: ${m.timeouts}.`, '');
|
||||
if (m.core.failures.length) {
|
||||
out.push('Core misses (legacy verdict):', '');
|
||||
for (const f of m.core.failures) out.push(`- "${f.phrase}" (expect ${f.expect}) — failed ${f.fails}/${f.of}; called: ${f.called.join(' ; ')}`);
|
||||
out.push('');
|
||||
}
|
||||
if (m.capability.failures.length) {
|
||||
out.push('Capability misses (full call):', '');
|
||||
for (const f of m.capability.failures) out.push(`- "${f.id}" (expect ${f.expect}) — failed ${f.fails}/${f.of}; called: ${f.called.join(' ; ').slice(0, 400)}`);
|
||||
out.push('');
|
||||
}
|
||||
if (m.dialogue.failures.length) {
|
||||
out.push('Dialogue misses (turn 2):', '');
|
||||
for (const f of m.dialogue.failures) out.push(`- ${f.id} rep ${f.rep} (expect ${f.expect}) — t1: ${f.turn1} ⇒ t2: ${f.turn2}`);
|
||||
out.push('');
|
||||
}
|
||||
const badCov = m.coverage.items.filter((i) => i.verdicts.some((v) => v === 'hallucinated-capability' || v === 'wrong-tool' || (i.supported && v !== 'correct')));
|
||||
if (badCov.length) {
|
||||
out.push('Coverage misses:', '');
|
||||
for (const i of badCov) out.push(`- "${i.utterance}" (${i.supported ? 'supported' : 'unsupported'}) — ${i.verdicts.join(', ')}`);
|
||||
out.push('');
|
||||
}
|
||||
}
|
||||
if (notes.length) {
|
||||
out.push('## Notes', '');
|
||||
for (const n of notes) out.push(`- ${n}`);
|
||||
out.push('');
|
||||
}
|
||||
return out.join('\n');
|
||||
}
|
||||
|
||||
/**
|
||||
* Before → after table for two summarize() results (matched by model key).
|
||||
* Coverage "before" is the v1 rubric (what the old tools could do); "after"
|
||||
* shows both v1 (same yardstick) and v2 (the current tool surface).
|
||||
*/
|
||||
export function renderComparison(before, after, { beforeLabel = 'before', afterLabel = 'after' } = {}) {
|
||||
const out = ['## Before → after', '', `Before = ${beforeLabel}; after = ${afterLabel}. Core accuracy = tool + all argument checks (full call). Coverage OK % = correct, or an honest refusal where the tools of that build cannot do it. HC = hallucinated-capability count. p50 = send → tool call (text). Prompt tokens = mean input tokens per model request (cloud: as billed, before cache discounts; local: the cold full-prompt count).`, ''];
|
||||
out.push('| model | core full-call | coverage OK % (v1 → v1 / v2) | HC (v1 → v1 / v2) | dialogue turn-2 | capability full-call (after) | p50 → tool call | prompt tokens |');
|
||||
out.push('|---|---|---|---|---|---|---|---|');
|
||||
const hc = (block) => (block?.n ? block.counts['hallucinated-capability'] : null);
|
||||
const promptTok = (m) => (m?.warmup?.promptTokens ?? m?.tokens?.inputPerRequestMean ?? null);
|
||||
for (const a of after.models) {
|
||||
const b = before.models.find((x) => x.key === a.key);
|
||||
const arrow = (x, y, f) => `${f(x)} → ${f(y)}`;
|
||||
const pctF = (v) => (v === null || v === undefined ? '—' : `${v}%`);
|
||||
const msF = (v) => (v === null || v === undefined ? '—' : `${Math.round(v)}ms`);
|
||||
const num = (v) => (v === null || v === undefined ? '—' : String(v));
|
||||
out.push(`| ${a.key} | ${arrow(b?.core.fullCallAccuracyPct, a.core.fullCallAccuracyPct, pctF)} | ${pctF(b?.coverage.acceptablePct)} → ${pctF(a.coverage.acceptablePct)} / ${pctF(a.coverageV2.acceptablePct)} | ${num(hc(b?.coverage))} → ${num(hc(a.coverage))} / ${num(hc(a.coverageV2))} | ${arrow(b?.dialogue.turn2FullPct, a.dialogue.turn2FullPct, pctF)} | ${pctF(a.capability.fullPct)} | ${arrow(b?.latencyMs.toolCallP50, a.latencyMs.toolCallP50, msF)} | ${arrow(promptTok(b), promptTok(a), num)} |`);
|
||||
}
|
||||
out.push('');
|
||||
return out.join('\n');
|
||||
}
|
||||
@@ -0,0 +1,21 @@
|
||||
/**
|
||||
* Stub tool results fed back to the model after each call. Pure.
|
||||
*
|
||||
* core suite: the neutral `{ ok:true }` stub qa-voice-routing has always
|
||||
* used, so continuations behave exactly as in that harness.
|
||||
* coverage suite: the same stub, except a call whose arguments violate the
|
||||
* tool schema gets `{ ok:false, error }` — what the app's runner returns
|
||||
* for an unknown layer id — so the model's follow-up words can be judged
|
||||
* for honesty.
|
||||
*/
|
||||
import { validateCall } from './grade.mjs';
|
||||
|
||||
export const STUB_OK = Object.freeze({ ok: true, note: 'qa-harness stub result' });
|
||||
|
||||
export function stubResult(suite, c, tools) {
|
||||
if (suite === 'coverage') {
|
||||
const errors = validateCall(tools, c);
|
||||
if (errors.length) return { ok: false, error: `Invalid arguments: ${errors.join('; ')}` };
|
||||
}
|
||||
return { ...STUB_OK };
|
||||
}
|
||||
@@ -0,0 +1,119 @@
|
||||
/**
|
||||
* Pure converters from the production Realtime tool list
|
||||
* (`{ type:'function', name, description, parameters }`) to other providers'
|
||||
* tool formats. Descriptions and schemas are carried over unchanged except
|
||||
* where a target cannot represent them; every such change is listed below.
|
||||
*/
|
||||
|
||||
const PERMISSIVE_SCALAR = Object.freeze(['string', 'number', 'boolean']);
|
||||
|
||||
const clone = (value) => JSON.parse(JSON.stringify(value));
|
||||
|
||||
/**
|
||||
* Normalize a JSON schema for OpenAI-compatible local servers.
|
||||
*
|
||||
* Only one change: a property schema with no `type` (and no enum / anyOf /
|
||||
* oneOf / $ref / const that would imply one) gets a permissive scalar union.
|
||||
* `analyst_query.filters[].value` is `{}` in production — valid JSON Schema,
|
||||
* but chat templates that print `type` for every property (Gemma's among
|
||||
* them) crash on it.
|
||||
*/
|
||||
export function normalizeSchemaForChat(schema) {
|
||||
const out = clone(schema);
|
||||
const visit = (node, isRoot) => {
|
||||
if (!node || typeof node !== 'object' || Array.isArray(node)) return;
|
||||
const implied = ['type', 'enum', 'anyOf', 'oneOf', 'allOf', '$ref', 'const'].some((k) => k in node);
|
||||
if (!isRoot && !implied) node.type = [...PERMISSIVE_SCALAR];
|
||||
if (node.properties) for (const sub of Object.values(node.properties)) visit(sub, false);
|
||||
if (node.items) visit(node.items, false);
|
||||
for (const key of ['anyOf', 'oneOf', 'allOf']) if (Array.isArray(node[key])) node[key].forEach((s) => visit(s, false));
|
||||
};
|
||||
visit(out, true);
|
||||
return out;
|
||||
}
|
||||
|
||||
/**
|
||||
* Every schema location normalizeSchemaForChat changes, as
|
||||
* `{ tool, path, from, to }` — logged in summary.md so the per-provider
|
||||
* payload difference is explicit rather than implied.
|
||||
*/
|
||||
export function chatNormalizationDiff(realtimeTools) {
|
||||
const changes = [];
|
||||
const walk = (before, after, tool, where) => {
|
||||
if (!before || typeof before !== 'object' || Array.isArray(before)) return;
|
||||
if (JSON.stringify(before.type) !== JSON.stringify(after.type)) {
|
||||
changes.push({ tool, path: where, from: before.type === undefined ? '(untyped)' : before.type, to: after.type });
|
||||
}
|
||||
for (const [k, v] of Object.entries(before.properties || {})) walk(v, after.properties[k], tool, `${where}.${k}`);
|
||||
if (before.items) walk(before.items, after.items, tool, `${where}[]`);
|
||||
};
|
||||
for (const tool of realtimeTools) walk(tool.parameters, normalizeSchemaForChat(tool.parameters), tool.name, tool.name);
|
||||
return changes;
|
||||
}
|
||||
|
||||
/** Realtime tools → Chat Completions tools (`{type:'function', function:{...}}`). */
|
||||
export function toChatTools(realtimeTools, { normalize = true } = {}) {
|
||||
return realtimeTools.map((tool) => ({
|
||||
type: 'function',
|
||||
function: {
|
||||
name: tool.name,
|
||||
description: tool.description,
|
||||
parameters: normalize ? normalizeSchemaForChat(tool.parameters) : clone(tool.parameters),
|
||||
},
|
||||
}));
|
||||
}
|
||||
|
||||
// Keywords the Gemini OpenAPI-subset `Schema` does not accept.
|
||||
const GEMINI_DROP = new Set(['additionalProperties', '$schema', 'default', 'examples', '$id', 'const']);
|
||||
|
||||
/**
|
||||
* Convert a JSON schema to the Gemini OpenAPI-subset `Schema`.
|
||||
* Drops unsupported keywords (additionalProperties, $schema, default, ...);
|
||||
* an untyped property becomes `anyOf` string|number|boolean; a type array
|
||||
* becomes `anyOf` of single types.
|
||||
*/
|
||||
export function toGeminiSchema(schema) {
|
||||
if (!schema || typeof schema !== 'object' || Array.isArray(schema)) return schema;
|
||||
const out = {};
|
||||
for (const [key, value] of Object.entries(schema)) {
|
||||
if (GEMINI_DROP.has(key)) continue;
|
||||
if (key === 'properties') {
|
||||
out.properties = Object.fromEntries(Object.entries(value).map(([k, v]) => [k, toGeminiSchema(v)]));
|
||||
} else if (key === 'items') {
|
||||
out.items = toGeminiSchema(value);
|
||||
} else if (key === 'anyOf' || key === 'oneOf') {
|
||||
out.anyOf = value.map((s) => toGeminiSchema(s));
|
||||
} else {
|
||||
out[key] = clone(value);
|
||||
}
|
||||
}
|
||||
if (Array.isArray(out.type)) {
|
||||
const types = out.type;
|
||||
delete out.type;
|
||||
out.anyOf = types.map((t) => ({ type: t }));
|
||||
}
|
||||
const implied = ['type', 'enum', 'anyOf'].some((k) => k in out);
|
||||
if (!implied) out.anyOf = PERMISSIVE_SCALAR.map((t) => ({ type: t }));
|
||||
return out;
|
||||
}
|
||||
|
||||
/**
|
||||
* Realtime tools → Gemini `functionDeclarations`.
|
||||
* mode 'jsonSchema' (default): pass the production schema verbatim through
|
||||
* `parametersJsonSchema` — full JSON Schema, closest to prompt parity.
|
||||
* mode 'openapi': convert to the OpenAPI-subset `parameters` field.
|
||||
* A tool with no properties gets no parameters field in openapi mode (Gemini
|
||||
* rejects an empty OBJECT schema there).
|
||||
*/
|
||||
export function toGeminiDeclarations(realtimeTools, { mode = 'jsonSchema' } = {}) {
|
||||
return realtimeTools.map((tool) => {
|
||||
const decl = { name: tool.name, description: tool.description };
|
||||
const params = tool.parameters;
|
||||
if (mode === 'jsonSchema') {
|
||||
decl.parametersJsonSchema = clone(params);
|
||||
} else if (params && Object.keys(params.properties || {}).length) {
|
||||
decl.parameters = toGeminiSchema(params);
|
||||
}
|
||||
return decl;
|
||||
});
|
||||
}
|
||||
@@ -0,0 +1,111 @@
|
||||
#!/usr/bin/env node
|
||||
/**
|
||||
* Summarize voice turn spans from the Realtime debug log.
|
||||
*
|
||||
* Every closed voice turn writes one `turn.span` record (see
|
||||
* src/voice/turnMetrics.js). This prints each span and p50/p90 figures for
|
||||
* end-of-speech → first audio, the longest silence, tool time, preamble rate,
|
||||
* spoken words and hedge hits.
|
||||
*
|
||||
* Usage:
|
||||
* node scripts/voice-turn-spans.mjs [log.jsonl] [--since <ISO time>] [--session <id>] [--json]
|
||||
*/
|
||||
import fs from 'node:fs';
|
||||
import path from 'node:path';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
|
||||
const root = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '..');
|
||||
const args = process.argv.slice(2);
|
||||
const option = (flag) => {
|
||||
const index = args.indexOf(flag);
|
||||
return index >= 0 ? args[index + 1] : null;
|
||||
};
|
||||
const positional = args.filter(
|
||||
(arg, index) => !arg.startsWith('--') && !args[index - 1]?.startsWith('--'),
|
||||
);
|
||||
const logPath =
|
||||
positional[0] || path.join(root, '.gev-logs', 'realtime-conversations.jsonl');
|
||||
const since = option('--since') ? Date.parse(option('--since')) : null;
|
||||
const session = option('--session');
|
||||
|
||||
const spans = [];
|
||||
for (const line of fs.readFileSync(logPath, 'utf8').split('\n')) {
|
||||
if (!line.includes('"turn.span"')) continue;
|
||||
let record;
|
||||
try {
|
||||
record = JSON.parse(line);
|
||||
} catch {
|
||||
continue;
|
||||
}
|
||||
if (record.event !== 'turn.span') continue;
|
||||
if (since && Date.parse(record.timestamp) < since) continue;
|
||||
if (session && record.sessionId !== session) continue;
|
||||
spans.push({
|
||||
at: record.timestamp,
|
||||
session: record.sessionId,
|
||||
...record.payload,
|
||||
});
|
||||
}
|
||||
|
||||
const quantile = (values, q) => {
|
||||
const sorted = values.filter(Number.isFinite).sort((a, b) => a - b);
|
||||
if (!sorted.length) return null;
|
||||
return sorted[Math.min(sorted.length - 1, Math.floor(q * sorted.length))];
|
||||
};
|
||||
const withTools = spans.filter((span) => span.tools?.length);
|
||||
const slowToolTurns = withTools.filter((span) =>
|
||||
span.tools.some((tool) => tool.ms >= 700),
|
||||
);
|
||||
const summary = {
|
||||
turns: spans.length,
|
||||
toolTurns: withTools.length,
|
||||
firstAudioMs: {
|
||||
p50: quantile(spans.map((s) => s.first_audio_ms), 0.5),
|
||||
p90: quantile(spans.map((s) => s.first_audio_ms), 0.9),
|
||||
},
|
||||
maxSilenceMs: {
|
||||
p50: quantile(spans.map((s) => s.max_silence_ms), 0.5),
|
||||
p90: quantile(spans.map((s) => s.max_silence_ms), 0.9),
|
||||
},
|
||||
toolMs: {
|
||||
p50: quantile(withTools.flatMap((s) => s.tools.map((t) => t.ms)), 0.5),
|
||||
p90: quantile(withTools.flatMap((s) => s.tools.map((t) => t.ms)), 0.9),
|
||||
},
|
||||
preambleRate: withTools.length
|
||||
? withTools.filter((s) => s.preamble).length / withTools.length
|
||||
: null,
|
||||
slowToolPreambleOrProgressRate: slowToolTurns.length
|
||||
? slowToolTurns.filter((s) => s.preamble || s.progress_lines > 0).length /
|
||||
slowToolTurns.length
|
||||
: null,
|
||||
answerWords: {
|
||||
p50: quantile(spans.map((s) => s.answer_words), 0.5),
|
||||
p90: quantile(spans.map((s) => s.answer_words), 0.9),
|
||||
},
|
||||
spokenWordsTotal: spans.reduce((sum, s) => sum + (s.spoken_words || 0), 0),
|
||||
hedgeHits: spans.reduce((sum, s) => sum + (s.hedge_hits?.length || 0), 0),
|
||||
};
|
||||
|
||||
if (args.includes('--json')) {
|
||||
console.log(JSON.stringify({ summary, spans }, null, 2));
|
||||
} else {
|
||||
for (const span of spans) {
|
||||
const tools = (span.tools || [])
|
||||
.map((tool) => `${tool.name}:${tool.ms}ms`)
|
||||
.join(',');
|
||||
console.log(
|
||||
[
|
||||
span.at,
|
||||
`first=${span.first_audio_ms ?? '-'}ms`,
|
||||
`maxSilence=${span.max_silence_ms ?? '-'}ms`,
|
||||
`preamble=${span.preamble ? 'y' : 'n'}`,
|
||||
`progress=${span.progress_lines}`,
|
||||
`words=${span.answer_words}/${span.spoken_words}`,
|
||||
`hedges=${span.hedge_hits?.length || 0}`,
|
||||
tools || '(no tools)',
|
||||
span.answer_text ? `"${span.answer_text.slice(0, 80)}"` : '',
|
||||
].join(' '),
|
||||
);
|
||||
}
|
||||
console.log(JSON.stringify(summary, null, 2));
|
||||
}
|
||||
@@ -0,0 +1,43 @@
|
||||
{
|
||||
"reviewedSources": {
|
||||
"zs": {
|
||||
"name": "Zetashapes",
|
||||
"registryCommit": "e17af153c144fe6e08aa8db2607b4bf8aecd448d",
|
||||
"registrySha256": "d8a5dd873d26eee11583de4438185c046dec33848d5bde27047ca4bff44d9c39",
|
||||
"rawLicenseType": "Public domain",
|
||||
"licenseTextSha256": "91757f96f74f6dfb4a7206486b78586da0d039e9611e224fc793c7167dcfe615",
|
||||
"decision": "Approved from the pinned WOF registry text: no added restrictions beyond public-domain US TIGER/Line data and Flickr-derived data; retain both upstream credits."
|
||||
}
|
||||
},
|
||||
"licenses": [
|
||||
"Attribution 3.0 Unported",
|
||||
"CC BY",
|
||||
"CC BY (equivalent)",
|
||||
"CC BY 2.0",
|
||||
"CC BY 3.0",
|
||||
"CC BY 3.0 US",
|
||||
"CC BY 4.0",
|
||||
"CC-BY",
|
||||
"CC0",
|
||||
"CC0 (equivalent)",
|
||||
"CC0 1.0 Universal",
|
||||
"ODC-By, v1.0",
|
||||
"ODC-By v1.0",
|
||||
"OGL - Canada",
|
||||
"OGL - City of New Westminster",
|
||||
"OGL - City of Surrey",
|
||||
"Open Government Licence v3.0",
|
||||
"Open Government Licence - British Columbia, v2.0",
|
||||
"Open Government Licence - Canada",
|
||||
"Open Government Licence - City of Calgary",
|
||||
"Open Government Licence - City of Regina",
|
||||
"Open Government Licence - Edmonton",
|
||||
"Open Government Licence - Ontario, v1.0",
|
||||
"Open Government License - British Columbia, v2.0",
|
||||
"Open Government License - Winnipeg",
|
||||
"PDDL 1.0",
|
||||
"Public domain",
|
||||
"Public Domain",
|
||||
"Singapore Open Data Licence v1"
|
||||
]
|
||||
}
|
||||
@@ -1,129 +0,0 @@
|
||||
/**
|
||||
* Serves the local MCP server at /mcp on the development and preview servers.
|
||||
*
|
||||
* This is local transport safety, not authentication: requests must come from
|
||||
* this machine, name a loopback host (blocking DNS rebinding) and, when a
|
||||
* browser sends an Origin, come from a loopback origin.
|
||||
*/
|
||||
|
||||
import { createMcpHttpHandler } from '../../src/tools/mcp/index.js';
|
||||
import { createLocalMcpServer } from './server.js';
|
||||
|
||||
const LOOPBACK_HOSTS = new Set(['localhost', '127.0.0.1', '[::1]']);
|
||||
const LOOPBACK_ADDRESSES = new Set(['127.0.0.1', '::1', '::ffff:127.0.0.1']);
|
||||
const MAX_BODY_BYTES = 1024 * 1024;
|
||||
|
||||
/** Whether a request may reach the local MCP server. */
|
||||
export function isLocalMcpRequest({ remoteAddress, host, origin }) {
|
||||
if (!LOOPBACK_ADDRESSES.has(remoteAddress)) return false;
|
||||
let hostUrl;
|
||||
try {
|
||||
hostUrl = new URL(`http://${host}`);
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
if (!LOOPBACK_HOSTS.has(hostUrl.hostname)) return false;
|
||||
if (origin == null) return true;
|
||||
try {
|
||||
const originUrl = new URL(origin);
|
||||
return (
|
||||
['http:', 'https:'].includes(originUrl.protocol) &&
|
||||
LOOPBACK_HOSTS.has(originUrl.hostname)
|
||||
);
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
/** Vite plugin that mounts the local MCP server at /mcp. */
|
||||
export function localMcpPlugin({ createServer = createLocalMcpServer } = {}) {
|
||||
const handlers = new Map();
|
||||
const handlerFor = (apiBase) => {
|
||||
if (!handlers.has(apiBase))
|
||||
handlers.set(apiBase, createMcpHttpHandler(createServer({ apiBase })));
|
||||
return handlers.get(apiBase);
|
||||
};
|
||||
const install = (server) => {
|
||||
server.middlewares.use('/mcp', async (req, res) => {
|
||||
const host = req.headers.host || '';
|
||||
if (
|
||||
!isLocalMcpRequest({
|
||||
remoteAddress: req.socket?.remoteAddress,
|
||||
host,
|
||||
origin: req.headers.origin,
|
||||
})
|
||||
) {
|
||||
res.writeHead(403, {
|
||||
'Content-Type': 'application/json',
|
||||
'Cache-Control': 'no-store',
|
||||
});
|
||||
res.end(
|
||||
JSON.stringify({
|
||||
error: 'The MCP server only accepts local requests',
|
||||
}),
|
||||
);
|
||||
return;
|
||||
}
|
||||
// A client that disconnects before its answer cancels the tool call.
|
||||
const disconnect = new AbortController();
|
||||
const onClose = () => {
|
||||
if (!res.writableFinished) disconnect.abort();
|
||||
};
|
||||
res.on('close', onClose);
|
||||
try {
|
||||
const body = req.method === 'POST' ? await readBody(req) : undefined;
|
||||
const request = new Request(`http://${host}/mcp`, {
|
||||
method: req.method,
|
||||
headers: Object.entries(req.headers).flatMap(([name, value]) =>
|
||||
value === undefined ? [] : [[name, String(value)]],
|
||||
),
|
||||
body,
|
||||
signal: disconnect.signal,
|
||||
});
|
||||
const response = await handlerFor(`http://${host}`)(request);
|
||||
res.writeHead(response.status, Object.fromEntries(response.headers));
|
||||
res.end(Buffer.from(await response.arrayBuffer()));
|
||||
} catch (error) {
|
||||
if (disconnect.signal.aborted) return;
|
||||
if (res.headersSent) return res.destroy();
|
||||
res.writeHead(error?.status === 413 ? 413 : 400, {
|
||||
'Content-Type': 'application/json',
|
||||
'Cache-Control': 'no-store',
|
||||
});
|
||||
res.end(
|
||||
JSON.stringify({
|
||||
error: error?.status === 413 ? 'Request too large' : 'Bad request',
|
||||
}),
|
||||
);
|
||||
} finally {
|
||||
res.off('close', onClose);
|
||||
}
|
||||
});
|
||||
};
|
||||
return {
|
||||
name: 'local-mcp',
|
||||
configureServer: install,
|
||||
configurePreviewServer: install,
|
||||
};
|
||||
}
|
||||
|
||||
function readBody(req) {
|
||||
return new Promise((resolve, reject) => {
|
||||
const chunks = [];
|
||||
let size = 0;
|
||||
let tooLarge = false;
|
||||
req.on('data', (chunk) => {
|
||||
size += chunk.length;
|
||||
// Keep draining without storing, so the 413 response can still be sent.
|
||||
if (tooLarge || size <= MAX_BODY_BYTES) {
|
||||
if (!tooLarge) chunks.push(chunk);
|
||||
return;
|
||||
}
|
||||
tooLarge = true;
|
||||
chunks.length = 0;
|
||||
reject(Object.assign(new Error('Request too large'), { status: 413 }));
|
||||
});
|
||||
req.on('end', () => resolve(Buffer.concat(chunks)));
|
||||
req.on('error', reject);
|
||||
});
|
||||
}
|
||||
@@ -1,44 +0,0 @@
|
||||
/** The local MCP server: Core's tools over services backed by a running app. */
|
||||
|
||||
import { readFileSync } from 'node:fs';
|
||||
import {
|
||||
composeCatalog,
|
||||
coreTools,
|
||||
catalogForSurface,
|
||||
} from '../../src/tools/index.js';
|
||||
import { panelRuntime } from '../../src/app/globePanelRuntime.js';
|
||||
import { createGlobePanelResource } from '../../src/tools/globePanel.js';
|
||||
import { createMcpServer } from '../../src/tools/mcp/index.js';
|
||||
import { DEFAULT_API_BASE, createLocalToolServices } from './services.js';
|
||||
|
||||
const INSTRUCTIONS =
|
||||
"Tools answer questions from God's Eye View's live public data. Location " +
|
||||
'tools take an area: a place name, a bbox, or lat/lon with radius_km. ' +
|
||||
'Results are capped; check truncated and total before concluding there is nothing more. ' +
|
||||
"Answers that can be shown in God's Eye View include data.view; to show one, call " +
|
||||
'show_in_gods_eye_view with that view, adding layers, style, a camera or marks as ' +
|
||||
"needed. It shows live God's Eye View where the client displays apps and " +
|
||||
'returns a link everywhere.';
|
||||
|
||||
/** Construct the local MCP server for Core's tools. */
|
||||
export function createLocalMcpServer({
|
||||
apiBase = DEFAULT_API_BASE,
|
||||
fetchImpl,
|
||||
} = {}) {
|
||||
const { version } = JSON.parse(
|
||||
readFileSync(new URL('../../package.json', import.meta.url), 'utf8'),
|
||||
);
|
||||
return createMcpServer({
|
||||
catalog: catalogForSurface(
|
||||
composeCatalog({
|
||||
tools: coreTools,
|
||||
services: createLocalToolServices({ apiBase, fetchImpl }),
|
||||
}),
|
||||
'mcp',
|
||||
),
|
||||
name: 'gods-eye-view',
|
||||
version,
|
||||
instructions: INSTRUCTIONS,
|
||||
resources: [createGlobePanelResource({ runtime: panelRuntime })],
|
||||
});
|
||||
}
|
||||
@@ -1,63 +0,0 @@
|
||||
/**
|
||||
* Local composition of tool services: Core's default services, pointed at a
|
||||
* running God's Eye View server's `/api` routes.
|
||||
*/
|
||||
|
||||
import { readFile } from 'node:fs/promises';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import { createToolServices } from '../../src/tools/services.js';
|
||||
|
||||
// Bundled data some sources read by module-relative URL.
|
||||
const BUNDLED_DATA = new URL('../../src/data/local_data/', import.meta.url);
|
||||
|
||||
export const DEFAULT_API_BASE = 'http://localhost:4173';
|
||||
|
||||
/**
|
||||
* Resolve the sources' relative `/api/...` requests against `apiBase`, and
|
||||
* serve `file:` URLs inside the bundled data directory from disk, as a
|
||||
* browser would load those module-relative assets.
|
||||
*/
|
||||
export function createApiFetch({
|
||||
apiBase = DEFAULT_API_BASE,
|
||||
fetchImpl = (...args) => globalThis.fetch(...args),
|
||||
} = {}) {
|
||||
const base = new URL(apiBase);
|
||||
if (!['http:', 'https:'].includes(base.protocol))
|
||||
throw new TypeError(`apiBase must be an http(s) URL: ${apiBase}`);
|
||||
return (input, init) => {
|
||||
const url = String(input instanceof Request ? input.url : input);
|
||||
if (url.startsWith('file:')) return readBundled(url);
|
||||
return fetchImpl(
|
||||
typeof input === 'string' && input.startsWith('/')
|
||||
? new URL(input, base)
|
||||
: input,
|
||||
init,
|
||||
);
|
||||
};
|
||||
}
|
||||
|
||||
async function readBundled(url) {
|
||||
const href = new URL(url).href;
|
||||
if (!href.startsWith(BUNDLED_DATA.href) || href.includes('/../'))
|
||||
return new Response(null, { status: 404 });
|
||||
try {
|
||||
const body = await readFile(fileURLToPath(href));
|
||||
return new Response(body, {
|
||||
headers: {
|
||||
'Content-Type': href.endsWith('.json')
|
||||
? 'application/json'
|
||||
: 'application/octet-stream',
|
||||
},
|
||||
});
|
||||
} catch {
|
||||
return new Response(null, { status: 404 });
|
||||
}
|
||||
}
|
||||
|
||||
/** Construct every service Core's tools read, backed by the local server. */
|
||||
export function createLocalToolServices(options = {}) {
|
||||
return createToolServices({
|
||||
fetchImpl: createApiFetch(options),
|
||||
appUrl: options.apiBase ?? DEFAULT_API_BASE,
|
||||
});
|
||||
}
|
||||
@@ -1,90 +0,0 @@
|
||||
/**
|
||||
* Local MCP server over stdio for agents such as Claude Code and Codex.
|
||||
*
|
||||
* npm run mcp -- [--api-base http://localhost:4173]
|
||||
*
|
||||
* Reads newline-delimited JSON-RPC from stdin and writes responses to stdout.
|
||||
* Tools read data from a running God's Eye View server (`npm run dev` or
|
||||
* `npm run preview`) at the API base. Diagnostics go to stderr.
|
||||
*/
|
||||
|
||||
import { createInterface } from 'node:readline';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import { parseErrorResponse } from '../../src/tools/mcp/index.js';
|
||||
import { createLocalMcpServer } from './server.js';
|
||||
import { DEFAULT_API_BASE } from './services.js';
|
||||
|
||||
/**
|
||||
* A one-line description of a request for the diagnostic log: its method,
|
||||
* and the tool or resource it names. Arguments and data are never logged.
|
||||
*/
|
||||
export function describeRequest(message) {
|
||||
const method = typeof message?.method === 'string' ? message.method : null;
|
||||
if (!method) return null;
|
||||
const subject =
|
||||
method === 'tools/call'
|
||||
? message.params?.name
|
||||
: method === 'resources/read'
|
||||
? message.params?.uri
|
||||
: null;
|
||||
return typeof subject === 'string' ? `${method} ${subject}` : method;
|
||||
}
|
||||
|
||||
/** Serve newline-delimited JSON-RPC until `input` ends. */
|
||||
export async function serveStdio(server, { input, output, log = () => {} }) {
|
||||
const lines = createInterface({ input, crlfDelay: Infinity });
|
||||
const pending = new Set();
|
||||
for await (const line of lines) {
|
||||
if (!line.trim()) continue;
|
||||
const work = respond(server, line, log).then((response) => {
|
||||
if (response) output.write(`${JSON.stringify(response)}\n`);
|
||||
});
|
||||
pending.add(work);
|
||||
work.finally(() => pending.delete(work));
|
||||
}
|
||||
await Promise.all(pending);
|
||||
}
|
||||
|
||||
async function respond(server, line, log) {
|
||||
let message;
|
||||
try {
|
||||
message = JSON.parse(line);
|
||||
} catch {
|
||||
return parseErrorResponse();
|
||||
}
|
||||
const described = describeRequest(message);
|
||||
if (described) log(described);
|
||||
const response = await server.handle(message);
|
||||
// A failed tool call answers the client, not the log; say why here too.
|
||||
if (response?.result?.isError)
|
||||
log(` ${described} failed: ${response.result.content?.[0]?.text}`);
|
||||
return response;
|
||||
}
|
||||
|
||||
/** Read `--api-base <url>` from command-line arguments. */
|
||||
export function parseArgs(argv) {
|
||||
const options = {};
|
||||
for (let index = 0; index < argv.length; index += 1) {
|
||||
if (argv[index] === '--api-base' && argv[index + 1])
|
||||
options.apiBase = argv[(index += 1)];
|
||||
else throw new Error(`Unknown argument: ${argv[index]}`);
|
||||
}
|
||||
return options;
|
||||
}
|
||||
|
||||
if (process.argv[1] && fileURLToPath(import.meta.url) === process.argv[1]) {
|
||||
try {
|
||||
const options = parseArgs(process.argv.slice(2));
|
||||
console.error(
|
||||
`God's Eye View MCP server reading ${options.apiBase || DEFAULT_API_BASE}`,
|
||||
);
|
||||
await serveStdio(createLocalMcpServer(options), {
|
||||
input: process.stdin,
|
||||
output: process.stdout,
|
||||
log: (line) => console.error(`<- ${line}`),
|
||||
});
|
||||
} catch (error) {
|
||||
console.error(error.message);
|
||||
process.exitCode = 1;
|
||||
}
|
||||
}
|
||||
@@ -25,36 +25,6 @@ let _openskyCacheMeta = null;
|
||||
let _openskyCacheSourceEpochMs = null;
|
||||
/** TTL for the OpenSky response cache (ms). */
|
||||
const OPENSKY_CACHE_MS = 9000;
|
||||
/** Per-attempt limit for the global snapshot; it normally arrives in ~1.5 s. */
|
||||
const OPENSKY_ATTEMPT_TIMEOUT_MS = 10_000;
|
||||
const OPENSKY_STATES_URL =
|
||||
'https://opensky-network.org/api/states/all?extended=1';
|
||||
|
||||
/**
|
||||
* Fetch and read the global snapshot, giving each attempt
|
||||
* OPENSKY_ATTEMPT_TIMEOUT_MS and retrying once when an attempt times out or
|
||||
* the connection fails. A second failure throws, which the proxy answers
|
||||
* from its stale cache or the regional fallback.
|
||||
* @param {Record<string, string>} headers Request headers.
|
||||
* @returns {Promise<{upstream: Response, body: string}>}
|
||||
*/
|
||||
async function fetchOpenSkyStates(headers) {
|
||||
for (let attempt = 1; ; attempt++) {
|
||||
try {
|
||||
const upstream = await fetch(OPENSKY_STATES_URL, {
|
||||
headers,
|
||||
signal: AbortSignal.timeout(OPENSKY_ATTEMPT_TIMEOUT_MS),
|
||||
});
|
||||
return { upstream, body: await upstream.text() };
|
||||
} catch (error) {
|
||||
if (attempt >= 2) throw error;
|
||||
console.warn(
|
||||
'[OpenSky] global snapshot attempt failed, retrying:',
|
||||
error?.message || error,
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
// --- OpenSky credit governor (field-test fix 2026-07-06) -------------------
|
||||
// The global /states/all this proxy fetches costs 4 CREDITS per call against
|
||||
// OpenSky's ~4000/day authenticated budget — a day with the app open burned
|
||||
@@ -137,7 +107,6 @@ export async function getOpenSkyToken() {
|
||||
{
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/x-www-form-urlencoded' },
|
||||
signal: AbortSignal.timeout(OPENSKY_ATTEMPT_TIMEOUT_MS),
|
||||
body: `grant_type=client_credentials&client_id=${encodeURIComponent(clientId)}&client_secret=${encodeURIComponent(clientSecret)}`,
|
||||
},
|
||||
);
|
||||
@@ -501,7 +470,10 @@ export function openSkyProxy() {
|
||||
}
|
||||
}
|
||||
|
||||
let { upstream, body } = await fetchOpenSkyStates(headers);
|
||||
let upstream = await fetch(
|
||||
'https://opensky-network.org/api/states/all?extended=1',
|
||||
{ headers },
|
||||
);
|
||||
// Auto-mode fallback: if OAuth was rejected, retry with Basic credentials
|
||||
if (
|
||||
(upstream.status === 401 || upstream.status === 403) &&
|
||||
@@ -513,11 +485,15 @@ export function openSkyProxy() {
|
||||
Accept: 'application/json',
|
||||
Authorization: `Basic ${Buffer.from(`${basicUser}:${basicPass}`).toString('base64')}`,
|
||||
};
|
||||
({ upstream, body } = await fetchOpenSkyStates(retryHeaders));
|
||||
upstream = await fetch(
|
||||
'https://opensky-network.org/api/states/all?extended=1',
|
||||
{ headers: retryHeaders },
|
||||
);
|
||||
usedMode = 'basic';
|
||||
reason = 'oauth_rejected_fallback_basic';
|
||||
}
|
||||
|
||||
let body = await upstream.text();
|
||||
const sourceEpochMs = upstream.ok ? openSkySourceEpochMs(body) : null;
|
||||
if (
|
||||
upstream.ok &&
|
||||
|
||||
@@ -143,10 +143,8 @@ export function cctvProxy({ sourceRoot = process.cwd() } = {}) {
|
||||
|
||||
if (url.pathname === '/sources') {
|
||||
const body = {
|
||||
trimmedPacks: getCctvSources.trimmedPacks?.() ?? [],
|
||||
sources: sources.map((source) => ({
|
||||
id: source.id,
|
||||
pack: source.pack,
|
||||
name: source.name,
|
||||
city: source.city,
|
||||
cityId: source.cityId,
|
||||
|
||||
@@ -3,7 +3,7 @@ import path from 'node:path';
|
||||
import { DEFAULT_CCTV_SOURCE_FILE, CCTV_SOURCE_CACHE_MS } from './constants.js';
|
||||
import { allocateSourceCap, resolveCatalogCap } from './cap.js';
|
||||
import { loadGroundHeights, joinGroundHeights } from './groundHeights.js';
|
||||
import { cameraRegion, normalizeSourceItem } from './normalize.js';
|
||||
import { normalizeSourceItem } from './normalize.js';
|
||||
import {
|
||||
loadAustinSourcesFromOpenData,
|
||||
loadCaltransSourcesFromOpenData,
|
||||
@@ -143,9 +143,6 @@ export function createCctvCatalog({ sourceRoot = process.cwd() } = {}) {
|
||||
/** @type {Promise<Array<object>>|null} In-flight refresh, shared by concurrent
|
||||
* callers so a post-TTL burst launches ONE refetch, not one per request. */
|
||||
let _cctvSourceInflight = null;
|
||||
/** @type {Array<{pack: string, available: number, served: number}>} Packs
|
||||
* that serve fewer cameras than they offer. */
|
||||
let _cctvTrimmedPacks = [];
|
||||
|
||||
/**
|
||||
* Assemble and cache the merged CCTV source list.
|
||||
@@ -212,12 +209,8 @@ export function createCctvCatalog({ sourceRoot = process.cwd() } = {}) {
|
||||
name,
|
||||
sources: items
|
||||
.filter((item) => item && typeof item === 'object')
|
||||
.map((item) => ({ ...normalizeSourceItem(item), pack: name }))
|
||||
.map((item) => normalizeSourceItem(item))
|
||||
.filter((item) => item.id),
|
||||
available: Number.isFinite(items.available)
|
||||
? items.available
|
||||
: items.length,
|
||||
region: items.region ?? null,
|
||||
});
|
||||
const packs = [
|
||||
...LIVE_PACKS.map((pack, index) =>
|
||||
@@ -241,16 +234,6 @@ export function createCctvCatalog({ sourceRoot = process.cwd() } = {}) {
|
||||
loadGroundHeights(sourceRoot),
|
||||
);
|
||||
const trimmed = allocation.packs.filter((pack) => pack.kept < pack.offered);
|
||||
// Each pack's own nearest-first cap and the shared catalog cap both trim.
|
||||
const trimmedPacks = packs
|
||||
.map((pack, index) => ({
|
||||
pack: pack.name,
|
||||
available: Math.max(pack.available, allocation.packs[index].offered),
|
||||
served: allocation.packs[index].kept,
|
||||
// Where the pack's cameras are, including those not served.
|
||||
region: pack.region ?? cameraRegion(pack.sources),
|
||||
}))
|
||||
.filter((pack) => pack.served < pack.available);
|
||||
if (trimmed.length) {
|
||||
const detail = trimmed
|
||||
.map((pack) => `${pack.name} ${pack.kept}/${pack.offered}`)
|
||||
@@ -261,7 +244,6 @@ export function createCctvCatalog({ sourceRoot = process.cwd() } = {}) {
|
||||
}
|
||||
if (capped.length > 0 || _cctvSourceCache.length === 0) {
|
||||
_cctvSourceCache = capped;
|
||||
_cctvTrimmedPacks = trimmedPacks;
|
||||
} else {
|
||||
// Every source came back empty (all live packs timed out / upstream outage)
|
||||
// but a good catalog is already cached — serve it stale rather than blanking
|
||||
@@ -275,7 +257,5 @@ export function createCctvCatalog({ sourceRoot = process.cwd() } = {}) {
|
||||
return _cctvSourceCache;
|
||||
}
|
||||
|
||||
/** Packs serving fewer cameras than they offer, as of the last refresh. */
|
||||
getCctvSources.trimmedPacks = () => _cctvTrimmedPacks;
|
||||
return getCctvSources;
|
||||
}
|
||||
@@ -470,31 +470,7 @@ export function prioritizeSources(cameras, maxCount, anchors) {
|
||||
return a.idx - b.idx;
|
||||
});
|
||||
|
||||
const kept = scored.slice(0, cap).map((entry) => entry.camera);
|
||||
// How many cameras the pack offered before trimming, and where, for
|
||||
// coverage reports.
|
||||
Object.defineProperty(kept, 'available', { value: list.length });
|
||||
Object.defineProperty(kept, 'region', { value: cameraRegion(list) });
|
||||
return kept;
|
||||
}
|
||||
|
||||
/** The bounding box of cameras with coordinates, or null. */
|
||||
export function cameraRegion(cameras) {
|
||||
let region = null;
|
||||
for (const camera of cameras || []) {
|
||||
const lat = Number(camera?.lat);
|
||||
const lon = Number(camera?.lon);
|
||||
if (!Number.isFinite(lat) || !Number.isFinite(lon)) continue;
|
||||
region = region
|
||||
? {
|
||||
west: Math.min(region.west, lon),
|
||||
south: Math.min(region.south, lat),
|
||||
east: Math.max(region.east, lon),
|
||||
north: Math.max(region.north, lat),
|
||||
}
|
||||
: { west: lon, south: lat, east: lon, north: lat };
|
||||
}
|
||||
return region;
|
||||
return scored.slice(0, cap).map((entry) => entry.camera);
|
||||
}
|
||||
|
||||
/**
|
||||
|
||||
@@ -7,7 +7,7 @@ import { adsbdbProxy } from './aircraft/enrichment.js';
|
||||
import { overpassProxy } from './overpass.js';
|
||||
import { militaryInstallationsProxy } from './military-installations.js';
|
||||
import { regionalBriefProxy } from './regional/briefing.js';
|
||||
import { geocodeProxy } from './regional/place.js';
|
||||
import { geocodeProxy, NOMINATIM_CACHE_DIR } from './regional/place.js';
|
||||
import { weatherEffectsProxy } from './regional/weather-effects.js';
|
||||
import { cctvProxy } from './cctv.js';
|
||||
import { defaultSourceRoot } from './common/source-root.js';
|
||||
@@ -26,11 +26,8 @@ import { firePerimetersProxy } from './firePerimeters.js';
|
||||
import { cycloneProxy } from './cyclones.js';
|
||||
import { windProxy } from './wind.js';
|
||||
|
||||
/**
|
||||
* Construct the local provider plugins in their established order.
|
||||
* `realtime` configures the voice session token endpoint.
|
||||
*/
|
||||
function localProviderPlugins({ realtime } = {}) {
|
||||
/** Construct the local provider plugins in their established order. */
|
||||
function localProviderPlugins() {
|
||||
return [
|
||||
openSkyProxy(),
|
||||
celestrakProxy(),
|
||||
@@ -42,7 +39,7 @@ function localProviderPlugins({ realtime } = {}) {
|
||||
overpassProxy(),
|
||||
militaryInstallationsProxy(),
|
||||
regionalBriefProxy(),
|
||||
geocodeProxy(),
|
||||
geocodeProxy({ storageDir: NOMINATIM_CACHE_DIR }),
|
||||
weatherEffectsProxy(),
|
||||
cctvProxy({ sourceRoot: defaultSourceRoot }),
|
||||
radioBrowserProxy(),
|
||||
@@ -52,7 +49,7 @@ function localProviderPlugins({ realtime } = {}) {
|
||||
adsbLolProxy(),
|
||||
aisLiveProxy(),
|
||||
trackBackfillProxies(),
|
||||
openAiRealtimeProxy({ realtime }),
|
||||
openAiRealtimeProxy(),
|
||||
googlePlacesContextProxy(),
|
||||
windProxy(),
|
||||
weatherProxy(),
|
||||
|
||||
@@ -16,6 +16,9 @@ const OPENAI_REALTIME_CONTEXT_RETENTION_DEFAULT = 0.5;
|
||||
|
||||
const OPENAI_HUD_SUMMARY_MODEL_DEFAULT = 'gpt-5-nano';
|
||||
|
||||
/** Input transcription for voice captions; OPENAI_REALTIME_TRANSCRIBE_MODEL=off disables it. */
|
||||
const OPENAI_REALTIME_TRANSCRIBE_MODEL_DEFAULT = 'gpt-4o-mini-transcribe';
|
||||
|
||||
export {
|
||||
OPENAI_HUD_SUMMARY_MODEL_DEFAULT,
|
||||
OPENAI_REALTIME_MODEL_MINI_DEFAULT,
|
||||
@@ -24,4 +27,5 @@ export {
|
||||
OPENAI_REALTIME_REASONING_DEFAULT,
|
||||
OPENAI_REALTIME_CONTEXT_TOKENS_DEFAULT,
|
||||
OPENAI_REALTIME_CONTEXT_RETENTION_DEFAULT,
|
||||
OPENAI_REALTIME_TRANSCRIBE_MODEL_DEFAULT,
|
||||
};
|
||||
@@ -23,7 +23,40 @@ const REALTIME_DEBUG_LOG_MAX_FILE_BYTES = 32 * 1024 * 1024;
|
||||
*/
|
||||
const REALTIME_DEBUG_LOG_MAX_PER_MIN = 120;
|
||||
|
||||
function createDebugLogHandler({ sourceRoot = defaultSourceRoot } = {}) {
|
||||
/**
|
||||
* Fields that carry what people said or heard: transcripts, typed and spoken
|
||||
* text, streamed deltas and the span's quoted replies. They are dropped from
|
||||
* the persisted log unless content logging is explicitly enabled
|
||||
* (GEV_VOICE_LOG_CONTENT=1), so timing and tool diagnostics never retain
|
||||
* speech by default.
|
||||
*/
|
||||
const VOICE_CONTENT_FIELDS = new Set([
|
||||
'transcript',
|
||||
'text',
|
||||
'delta',
|
||||
'answer_text',
|
||||
'preamble_text',
|
||||
]);
|
||||
|
||||
/** Replace spoken or typed content with its length. */
|
||||
function omitVoiceContent(value, depth = 0) {
|
||||
if (depth > 12 || value === null || typeof value !== 'object') return value;
|
||||
if (Array.isArray(value))
|
||||
return value.map((item) => omitVoiceContent(item, depth + 1));
|
||||
const output = {};
|
||||
for (const [key, item] of Object.entries(value)) {
|
||||
output[key] =
|
||||
VOICE_CONTENT_FIELDS.has(key) && typeof item === 'string'
|
||||
? `[omitted ${item.length} chars]`
|
||||
: omitVoiceContent(item, depth + 1);
|
||||
}
|
||||
return output;
|
||||
}
|
||||
|
||||
function createDebugLogHandler({
|
||||
sourceRoot = defaultSourceRoot,
|
||||
includeContent = process.env.GEV_VOICE_LOG_CONTENT === '1',
|
||||
} = {}) {
|
||||
const logDir = path.join(sourceRoot, '.gev-logs');
|
||||
const logFile = path.join(logDir, 'realtime-conversations.jsonl');
|
||||
const allow = makeRateLimiter({
|
||||
@@ -79,7 +112,8 @@ function createDebugLogHandler({ sourceRoot = defaultSourceRoot } = {}) {
|
||||
|
||||
try {
|
||||
const body = await readRequestBody(req, REALTIME_DEBUG_LOG_MAX_BYTES);
|
||||
const record = JSON.parse(body || '{}');
|
||||
const parsed = JSON.parse(body || '{}');
|
||||
const record = includeContent ? parsed : omitVoiceContent(parsed);
|
||||
await append(
|
||||
`${JSON.stringify({
|
||||
loggedAt: new Date().toISOString(),
|
||||
@@ -103,4 +137,4 @@ function createDebugLogHandler({ sourceRoot = defaultSourceRoot } = {}) {
|
||||
};
|
||||
}
|
||||
|
||||
export { createDebugLogHandler };
|
||||
export { createDebugLogHandler, omitVoiceContent };
|
||||
@@ -1,5 +1,5 @@
|
||||
const DEFAULT_ANNOTATION_GUIDANCE =
|
||||
'Use a single annotate_map call with several annotations when you are describing multiple related places at once. Set flyTo true only when the user is not already looking at the place; if every mark in a call lands off-screen the app auto-frames them, so when unsure leave flyTo false. Do NOT say out loud that you are drawing, highlighting, or annotating — just speak naturally about the places while the marks appear. ANNOTATIONS ACCUMULATE AND PERSIST — keep adding marks as you explore; you can fly around, change topic, and jump between far-apart places and the marks STAY, so the user can build up the map and show people things. Do NOT clear on your own initiative: never pass clearPrevious, and call clear_annotations ONLY when the user EXPLICITLY asks to clear or reset the map.';
|
||||
'Put related places in one annotate_map call. Set flyTo true only when the user is not already looking at the place (off-screen marks auto-frame, so when unsure leave it false). Do not say you are drawing or annotating — talk about the places while the marks appear. Marks ACCUMULATE AND PERSIST across flights and topics so the user can build up the map: never pass clearPrevious, and call clear_annotations ONLY when the user EXPLICITLY asks to clear or reset the map.';
|
||||
|
||||
function realtimeInstructions(
|
||||
annotationGuidance = DEFAULT_ANNOTATION_GUIDANCE,
|
||||
@@ -13,27 +13,23 @@ function realtimeInstructions(
|
||||
'For requests to open, show, reveal, or focus a menu/panel, call set_panel_open or show_data_layers_menu. "Open Context" means only set_panel_open{panelId:"global-context-panel",open:true}; it does not activate a Context sub-mode. "Open Contacts" means set_context_mode{mode:"contacts"}; that action expands the parent Context panel before activating Contacts.',
|
||||
'For requests like "show me the datacenter layers", open the data layers menu and focus the matching layer row; do not enable the layer unless the user asks to turn it on.',
|
||||
'For questions like "what am I looking at?", "what is in view?", "what is this?", "that selected thing", nearby datacenter, dam, cable, ship, or current view contents, call get_entity_context first, then answer from the returned scene/entity context.',
|
||||
'For "what is this aircraft?" answers, read the callsign, operator, registration, type, and route only from get_entity_context selected.properties. Treat route, routeOrigin, and routeDestination as the only authoritative route fields. Every aircraft identity answer MUST explicitly cover operator, type, and route. When a route is present, repeat its endpoint codes exactly; do not expand airport codes into city names. For a missing field say exactly "Operator details are unavailable", "Aircraft type is unavailable", or "Route details are unavailable" as applicable. Never silently omit missing enrichment or infer it from the callsign.',
|
||||
'While a camera motion or route flight is active, a bare "stop" means move_camera{motion:stop} — NOT control_scene and NOT stop_tracking (those need explicit words like "stop the scene" / "stop tracking"). If move_camera stop returns stopped:false and an entity is being tracked, call stop_tracking next — the user means "stop whatever is moving". Flying somewhere while tracking automatically stops the tracking (the result says so): mention it briefly.',
|
||||
'For "what is this aircraft?", answer with get_entity_context\'s identityLine: callsign, operator, type and route from returned fields only, airport codes verbatim. Other questions about the selection answer from its properties. Never infer them from the callsign.',
|
||||
'While a camera motion or route flight is active, a bare "stop" means move_camera{motion:stop} — NOT control_scene and NOT stop_tracking (those need explicit words like "stop the scene" / "stop tracking"). If move_camera stop returns stopped:false and an entity is being tracked, call stop_tracking next — the user means "stop whatever is moving". Flying somewhere while tracking stops the tracking.',
|
||||
'For camera-motion requests — "orbit around this", "pan left", "tilt up", "stop moving" — call move_camera. For "fly the route" over a drawn route, call fly_route. Confirm with the RESULTING state ("Orbiting slowly", "Flying the route").',
|
||||
'analyst_query ANSWERS questions; it never moves the camera or starts tracking. For requests to FOLLOW or TRACK a specific aircraft/ship, call track_entity (get_entity_context first when the target is ambiguous), never analyst_query as the final or only action. For "follow/track the nearest aircraft", first call analyst_query with the aircraft layer(s), sortBy=distance, and limit=1, then call track_entity with the returned aircraft identity in the same turn. The lookup alone does not fulfill a follow/track command.',
|
||||
'For a request to enable an aircraft layer and SELECT or FIND the nearest/closest aircraft near a named place — for example, "Turn on flights and select the closest aircraft to Austin" — call select_nearest_aircraft once. It atomically turns on the requested aircraft layer first, waits for location arrival, refreshes that layer for the destination viewport, filters out landed/on-ground records, and selects the nearest airborne result. A healthy fallback feed is valid data: report the returned feed source briefly, never call it an enable failure. Do not also call fly_to_location, set_layer_visibility, analyst_query, track_entity, set_context_mode, or control_cockpit for the same request. SELECT/FIND never implies Contacts or Cockpit unless the user explicitly asks for either mode.',
|
||||
'For ANALYTICAL questions about layer data — how many / which / fastest / highest / biggest / nearest flights, ships, fires, earthquakes, satellites, datacenters, or dams ("how many flights over Texas", "biggest fire near LA", "which ships are headed to Oakland", "anything above 40,000 feet") — call analyst_query, not get_entity_context. Narrate the count plus two or three notable examples by name, and reflect the result\'s coverage note honestly: the answer covers data loaded by enabled layers, not the whole world. If the needed layer is disabled, say so and offer to enable it. For follow-ups about the same set ("which of THOSE is closest?"), call analyst_query with followUp=true and only the new filter/sort.',
|
||||
'WORLD QUERIES — the query tools (get_weather, get_wind, search_places, places_nearby, plan_route, get_cyclones, get_fire_perimeters, get_bike_share, get_transit_vehicles, get_traffic_flow, situation_brief, military_awareness and the other get_* and find_* tools you have) look up data anywhere in the world without changing the map and without any layer being on. Use them when the user asks about a place they are not looking at or data whose layer is off ("what is the weather in Tokyo", "how do I drive from Austin to Dallas", "any wildfires in California"). For aircraft, ships, earthquakes, active fires, datacenters, dams and satellites overhead, use analyst_query over the layers instead, turning the layer on first when the user wants it. Use analyst_query and get_entity_context for what the app has loaded or what is on screen. Answer from the result summary and data, mention its source when it has one, and never claim the map changed. If the user also wants to SEE it, use the matching app action (fly_to_location, set_layer_visibility) as well.',
|
||||
'FEED STATE CONTRACT — analyst_query and get_current_view_state carry feedProvenance from the same layerFeedState the Data Layers chips use (nominal/loading/degraded/stale/fallback/unavailable, or off). NEVER narrate a count as live or current when feedProvenance.overall is not nominal. Say the feedState in the same breath as the count ("12 flights in view, STALE"). If unavailable, do not invent a count; say the feed is unavailable. Fallback is valid data — name the fallback source, never call it an enable failure. Confirm only what the tool result returned.',
|
||||
'COUNTING CONTRACT — what "near" means. (1) While Contacts is ACTIVE, "near / nearby / how many aircraft" means the Contacts window: answer from contactsWindow in the tool result — those are the exact numbers on the user\'s panel. set_context_mode, analyst_query, and get_current_view_state carry it after Contacts settles. For "Open Contacts and tell me how many aircraft are within 250 km", call set_context_mode{mode:"contacts"} first and answer from contactsWindow.aircraft; do not answer from a pre-Contacts analyst query. analyst_query\'s own count measures currently-loaded records and is usually lower; never give it as the window count. CENTER PRECEDENCE for a nearby/how-many ask, in order: an explicit place in the question ("over Texas", "near Austin") always wins and ignores Contacts state; else the CONTACTS SUBJECT when Contacts is active and has one — a selected datacenter, dam, fire, or cable does NOT silently become the center; else an entity the user explicitly names ("around this datacenter"); else the current view, said aloud ("nothing is selected, so this is the current view"). With Contacts active but NO subject yet, use the view and say so; never read an empty panel. (2) With Contacts OFF, "nearby" means in view; "near <place>" means a radius around that place. (3) EVERY count names its scope in words — "42 in your window", "8 in view", "about 30 within 250 km of Austin" — never a bare number; analyst_query returns scopeLabel for exactly this. Two different numbers with named scopes are not a contradiction; say both if asked. (4) State counts VERBATIM — never estimate, round, or hedge ("a few", "less than a dozen"): if a tool returns 46, say 46. (5) When it matters, add once: counts cover loaded data, and the flights layer loads where you look.',
|
||||
'For a request to enable an aircraft layer and SELECT or FIND the nearest/closest aircraft near a named place — for example, "Turn on flights and select the closest aircraft to Austin" — call select_nearest_aircraft once. It atomically turns on the requested aircraft layer first, waits for location arrival, refreshes that layer for the destination viewport, filters out landed/on-ground records, and selects the nearest airborne result. A healthy fallback feed is valid data, never an enable failure. Do not also call fly_to_location, set_layer_visibility, analyst_query, track_entity, set_context_mode, or control_cockpit for the same request. SELECT/FIND never implies Contacts or Cockpit unless the user explicitly asks for either mode.',
|
||||
'For ANALYTICAL questions about loaded layer data — how many / which / fastest / highest / biggest / nearest / next flights, ships, fires, earthquakes, satellites, launches, storms, transit, bikes, bases, datacenters or dams ("how many flights over Texas", "anything above 40,000 feet") — call analyst_query, not get_entity_context. Always set scope by WHERE: "around here" or "near this" while pointing is kind pointer, never view. Filter on the attribute the user names ("MODIS" → sensor eq MODIS). Say the count with its scope in a few words ("37 loaded flights over Texas") plus one or two examples by name. If the needed layer is off, say so and offer to turn it on. "Which of THOSE…" about the last answer → followUp=true with only the new filter/sort.',
|
||||
'WHERE, for every tool that takes a place or scope, in this order: (1) a place the user names; (2) explicit pointing — "this/that one/here/around here" while pointer_context shows pointing or a target → "pointer"; (3) the Contacts subject while Contacts is active, or the selection when the user refers to it; (4) the current view. A selected datacenter, dam, fire or cable never silently becomes the centre.',
|
||||
'FEED STATE: when a count\'s feed is stale or degraded, add that one word ("12 flights in view, stale"); never call it live. If unavailable, give no number: say the feed is unavailable. Fallback feeds are valid data; name them only if asked.',
|
||||
'COUNTING CONTRACT — what "near" means. (1) While Contacts is ACTIVE, "near / nearby / how many aircraft" means the Contacts window: answer from contactsWindow in the tool result — the exact numbers on the user\'s panel (set_context_mode, analyst_query and get_current_view_state carry it once Contacts settles). For "Open Contacts and tell me how many aircraft are within 250 km", call set_context_mode{mode:"contacts"} first and answer from contactsWindow.aircraft, not from a pre-Contacts analyst query; analyst_query\'s own count measures loaded records and is usually lower — never give it as the window count. The centre follows WHERE; with Contacts active but no subject yet, use the view, never an empty panel. (2) With Contacts OFF, "nearby" means in view; "near <place>" means a radius around that place. (3) EVERY count names its scope in words — "42 in your window", "8 in view", "30 within 250 km of Austin" — never a bare number; analyst_query returns scopeLabel for exactly this. Two numbers with named scopes are not a contradiction; say both if asked. (4) State counts VERBATIM — never estimate, round, or hedge: if a tool returns 46, say 46.',
|
||||
'While Cockpit is active, navigate with control_cockpit (next/previous, optionally targetLayer or aircraftClass). track_entity and fly_to_location are REFUSED by design while Cockpit owns the camera — that refusal is correct, not an error to retry. To go somewhere else, exit Cockpit first. control_cockpit enter establishes Contacts itself, so do not call set_context_mode before or after it.',
|
||||
'When the target layer is unknown, OMIT layerId in track_entity so it searches all enabled layers. Passing the wrong layerId ("flights" for a military contact) returns "Nothing matched" even though the contact is loaded.',
|
||||
'If get_entity_context has no selected object or overlay entities, use its basemap context: Google Photorealistic 3D Tiles/Cesium source, center target coordinates, reverse-geocoded place, camera altitude, active style, and enabled layers. Do not say there is nothing unless the basemap target is also unavailable.',
|
||||
'If basemap context includes knownLandmarks, prefer the nearest known landmark by name for "what am I looking at" answers. For example, if knownLandmarks includes Eiffel Tower, say Eiffel Tower.',
|
||||
'At local zoom, use basemap nearbyPlaces, place.labels, viewportPlaces.visibleLabels, and viewportPlaces.streetLabels to identify the building, premises, roads, and named places visible around the screen target.',
|
||||
'If basemap context includes viewportPlaces, prefer dominantCountry, dominantRegion, and dominantLocality over raw coordinates.',
|
||||
'When basemap context includes viewportSamples or an inferred country, trust that over a single reverse-geocoded address. If most samples indicate Iran, say Iran, not the United States.',
|
||||
'When a viewport screenshot is attached after get_entity_context, read clearly legible street, building, and place labels from it and combine them with structured label context. Respect scene viewScale: at global/continental/regional scale, avoid naming a precise street/city from one center pixel.',
|
||||
'READING THE VIEW: with no selected object or overlay entities, answer from get_entity_context\'s basemap context (tiles source, centre target, reverse-geocoded place, altitude, style, enabled layers); say there is nothing only when that is unavailable too. Prefer, in order: a nearby knownLandmarks name ("Eiffel Tower"); at local zoom nearbyPlaces, place.labels and viewportPlaces visible and street labels; viewportPlaces dominantCountry/Region/Locality over raw coordinates; the majority of viewportSamples over one reverse-geocoded address. Legible labels in an attached screenshot count too, but at global or regional viewScale never name a precise street or city from one centre pixel.',
|
||||
'Do not mention disabled layers or stale selections.',
|
||||
'When a request requires a tool call, do not speak in the same response as the tool call. Call the tool first.',
|
||||
'When a single user request contains MULTIPLE changes (e.g. "switch to operator layout, use balanced detection at density 50, and switch to Bing aerial"), call ALL the corresponding tools — multiple tool calls in sequence — before speaking. Never confirm a partial subset. If a later tool fails, say which parts succeeded and which failed.',
|
||||
'After receiving tool output, speak exactly one short confirmation. Do not repeat the confirmation.',
|
||||
'POINTING: pointer_context says what the cursor is on (pointing:<kind>: resolved by the tools; target "none": not pointing). For "this/that/here/that one" with no name, pass "pointer": get_entity_context scope, track_entity/fly_to_location query, analyst_query scope.kind ("around here" is pointer, not view), annotate_map target. "The second/last one" is referent:n from the last numbered list. Without a pointer, ask or follow WHERE.',
|
||||
'PREAMBLES: for instant commands (layers, styles, HUD, detection, zoom, panels, basemap, radio controls, camera nudges, and lookups such as get_entity_context) call the tool without speaking first. Only for slow or multi-step work (annotate_map, a searched fly_to_location, select_nearest_aircraft, analyst_query over a named place, or two or more tools) you may first say ONE sentence of at most 12 words naming the actual steps ("Finding the Capitol, then outlining its grounds."), then call the tools at once in the same response.',
|
||||
'When one request contains MULTIPLE changes, call ALL the corresponding tools before the confirmation. Never confirm a partial subset; if one fails, say which.',
|
||||
'AFTER TOOLS: speak the result\'s say line once, lightly rephrased at most, never adding to it and never repeating your preamble; if say is null, confirm briefly or continue. Sources, notes and caveats are on the voice card: speak a caveat only when it changes the answer, or when asked why, how sure or which source. Lower bounds ("at least 500") and partial answers are part of the answer — always say them. No filler or hedges ("let me check", "keep in mind", "it seems", "based on the data available"), and never add "approximately" to an exact tool number.',
|
||||
'For "show/open/turn on" layer requests, enable the matching layer. For "hide/close/turn off", disable it.',
|
||||
// INSTRUCTION-ONLY mapping for the two globe-scale named views.
|
||||
//
|
||||
@@ -49,28 +45,23 @@ function realtimeInstructions(
|
||||
// string is the whole rollback.
|
||||
'NAMED VIEWS are shorthand for tool calls you already have — there is no "mode" tool for them. Treat ONLY these as the shorthand: "infrastructure mode" / "the infrastructure view" / "show me global infrastructure" means three set_layer_visibility calls (local-datacenters, local-dams, telegeography-submarine-cables) plus zoom_to_globe; "environmental mode" / "earth watch" / "active events", said as the name of a view, means set_layer_visibility for local-firms and earthquakes plus zoom_to_globe. Anything vaguer is NOT this shorthand — an open-ended question about the world or the news is an ordinary question: answer it, or use analyst_query over the layers already on. Never switch a whole view on to answer a question nobody asked to see. When you do run one, make every call before speaking, then give one confirmation naming the resulting state; if the fires layer comes back unavailable because no FIRMS key is configured, say so plainly — the earthquakes still loaded. "Live contacts" and "space missions" are NOT this pattern: they stay set_context_mode{mode:"contacts"} and set_context_mode{mode:"space-missions"}.',
|
||||
'For visual filter requests, call set_visual_style with one of the allowed style IDs.',
|
||||
'SHIPS vs STYLES: "vessels" is easily heard as "visuals". "Show me vessels", "turn on vessels", "the vessels", "visuals" or "vestibules" said WITHOUT a style name means the ships layer: set_layer_visibility{layerId:"ais-live-vessels"}, exactly as "ships" or "boats" does. Call set_visual_style only when the user names a style (retro/CRT, surveillance/night vision, thermal, anime, noir, snow, normal) or says filter, effect or look. Never turn on a different data layer (such as fires) for a ships request.',
|
||||
'Disambiguation table — basemap vs layer vs style: basemap switching requires an explicit stack name — "Bing aerial" means set_map_stack bing-aerial, "aerial with labels" means bing-labels, "OSM"/"road map" means osm, "Esri"/"Esri imagery" means esri-imagery, "Google 3D"/"photorealistic" means photoreal. Any mention of "satellite" or "satellites" ALWAYS means the satellites DATA LAYER via set_layer_visibility, never a basemap. "surveillance"/"night vision"/"thermal" are visual STYLES via set_visual_style.',
|
||||
'Disambiguation table — basemap vs layer vs style: basemap switching requires an explicit stack name — "Bing aerial" means set_map_stack bing-aerial, "aerial with labels" means bing-labels, "OSM"/"road map" means osm, "Esri"/"Esri imagery" means esri-imagery, "Google 3D"/"photorealistic" means photoreal. Any other mention of "satellite" or "satellites" means the satellites DATA LAYER via set_layer_visibility, never a basemap — except "recent/latest satellite imagery" (the recent-imagery layer) and "weather satellite"/clouds (weather-satellite). "surveillance"/"night vision"/"thermal" are visual STYLES via set_visual_style.',
|
||||
'HUD requests ("hud on/off", "switch to operator/minimal/tactical/cyber layout") use set_hud. Detection requests ("detection on", "dense mode", "balanced mode", "sparse mode", "set density to 25", "use weighted allocation") use set_detection. Density snaps to 0/25/50/75/100 and derives Sparse/Balanced/Dense; panoptic is a legacy alias for Dense.',
|
||||
'Cyber sonar requests use set_cyber_sonar: enabled, rings, rangePct, intensityPct (Power), opacityPct, sectorDeg. For settings questions read get_current_view_state.controls.hud.sonar. Sonar is not detection. Do not silently switch themes, reveal the HUD or enable sonar when changing a slider. If the user explicitly requests Cyber plus sonar settings, call set_hud first, wait for success, then set_cyber_sonar. Confirm returned values; mapSweepActive and contactSweepActive distinguish saved settings from active map effects, including in Cockpit or reduced motion. Do not claim visible contacts solely because enabled is true.',
|
||||
'Bloom/sharpen requests use set_post_processing. Scene requests ("play orbital watch", "stop the scene", "what scenes are there") use control_scene. CCTV camera requests ("next camera", "nearest camera", "select the Congress camera", "show coverage") use control_cctv — the CCTV layer must be enabled first.',
|
||||
'Radio playback requests use control_radio. "Turn on/start the radio" means action=play; action=enable only reveals Radio markers and must be reserved for explicit "show/enable the Radio layer/markers" requests. After a prepared playback result, briefly confirm any other completed actions and say "Turning on the radio"—never claim it is already playing. The client keeps Radio muted until playback is verified, then closes voice before restoring Radio volume. Examples: "play news near Austin" → select category=news locationId=austin; "play US news" → select category=news country=US; "Radio volume 30" → volume; pause/resume/stop/next/previous use the matching action. Radio selection never moves the camera.',
|
||||
'"Track/follow <something specific>" (a callsign, ship name, satellite name) uses track_entity. "Take me to the biggest fire" uses track_entity with query "biggest fire" (the fires layer must be enabled). Bare "orbit" means camera orbit of the current landmark. "Stop following/tracking" uses stop_tracking.',
|
||||
'"Show me which planes are overhead"/"frame the ships"/"show me the satellites above" use frame_overhead with the matching target.',
|
||||
"After frame_overhead, speak ONLY from the tool result's count field — e.g. 'Framed fourteen aircraft, labels on'; never reassess or second-guess the count aloud.",
|
||||
'Confirmations echo the RESULTING state, never the request: "HUD operator layout", "Density twenty-five percent", "Bing aerial imagery", "Tracking UAL428", "Framed fourteen aircraft". On ok=false, state the failure plainly: "Nothing matched UAL999", "No ships within 120 kilometers". Never claim an action without ok=true in the tool result.',
|
||||
'For destination requests such as "take me to Italy", "go to NYC", or "show me the Eiffel Tower", call fly_to_location. Prefer known city IDs when available; otherwise pass the plain place query.',
|
||||
'Navigation-only requests ("take me to X", "go to X", "fly to X") are NOT descriptions: call fly_to_location alone and do NOT also call annotate_map, unless the user explicitly asks to mark the place or you go on to explain specific places there. Never drop a point pin on a region-scale natural feature (a mountain range, desert, sea, or forest) — a single point in the middle of the Rockies is meaningless. If the user explicitly asks to mark such a region, prefer type=area.',
|
||||
'For country and city destinations, omit rangeM so GEV frames the whole country or city in view. For landmarks and buildings, omit rangeM so GEV chooses a close landmark view.',
|
||||
'Only supply rangeM when the user asks for a particular numeric height, distance, closer view, or wider view.',
|
||||
'For relative requests such as "zoom out a little", "pull back", "zoom in more", or "get closer", always call adjust_camera_zoom. But "globe view", "whole earth", "the whole planet", or "zoom all the way out" is an ABSOLUTE framing: call zoom_to_globe once instead — repeated adjust_camera_zoom calls can never reach the globe. Never claim the camera moved without the tool returning ok=true.',
|
||||
'Keep spoken confirmations short, e.g. "Opening datacenters" or "Flying to London".',
|
||||
'WHITEBOARD THE WORLD: whenever you describe or explain a specific place, building, campus, district, boundary, or a spatial relationship between places, call annotate_map to mark it visually as you talk — like sketching on the map. To call out a specific building, campus, compound, park, or district, use type=area (it traces and encloses the real footprint — a building gets a glowing volume, a district gets a draped outline). Use type=highlight only for a transient pulse on a precise spot that has no meaningful footprint, and type=pin to drop a labeled marker. Examples: "what is the Palace of Fine Arts?" → an AREA on it; "the old military base next to it" → an AREA on the Presidio; "ILM is right here" → a pin; "it sits next to the Marina" → an arrow from one to the other. Prefer place NAMES so the app resolves real positions and outlines; never invent coordinates or pixel locations.',
|
||||
'On every annotation, also set entityKind to what the thing IS when you know it: building (one structure), compound (campus/grounds/mall/park), district (neighborhood/area of a city), street (a named road), or point_feature (a monument, statue, memorial, plaque, fountain, or other small point landmark). entityKind is a FACT about the target, independent of the mark type you chose — monuments and statues are point_feature even when you use type=area; the app then anchors them as precise points instead of guessing at a footprint.',
|
||||
'WHITEBOARD THE WORLD: when you describe a specific place, building, campus, district or boundary, or how places relate, call annotate_map as you talk. type=area traces a building, campus, compound, park, district or boundary outline; pin marks a spot; highlight pulses a spot with no footprint; arrow joins two places ("it sits next to the Marina"). Use place names; screenX/screenY only for a spot you can see in the latest screenshot but cannot name; never invent coordinates. Set entityKind whenever you know what the target is.',
|
||||
annotationGuidance,
|
||||
'If an annotate_map result has partial:true or any failedLabels, do not pretend those places appeared — briefly work into your narration that you could not pinpoint them (e.g. "I couldn\'t place X"). If a route comes back as a direct line (no street route was found), describe it as a straight-line distance, not a walking/driving time. If an annotate_map result has capped:true, the map is full — ASK the user whether to clear before drawing more; do not clear unprompted. outlinePending:true is NOT a failure, but it is also NOT an outline: the anchor mark is placed and the boundary is still being traced in the background. Narrate it in progress — e.g. "tracing the boundary now" — and NEVER state the outline is already drawn or visible; it may yet come back as just a point. A later system item of type map_annotation_outline reports the final outcome per mark (status resolved or failed, with its label): use it to quietly confirm, or to correct yourself if you implied a boundary that stayed a point — an honest miss beats a misleading guess.',
|
||||
'PREFER NAMES. Only when you cannot name or geocode a place but you can clearly SEE the exact spot in the most recent viewport screenshot, fall back to screenX/screenY (normalized 0..1 from that image) to point at it; the app converts the pixel to a real world point. Never use screenX/screenY for something you could name.',
|
||||
'PATHS vs DISTANCES: for "walking/driving route from A to B" (or through several stops), use type=route with the ordered points and the matching mode (walking/driving/cycling) — the app draws the real street-following path on the map and reports distance and travel time, which you can read aloud. For "how far is X from Y", "is it nearby", or "X is next to Y", use type=arrow between the two — it draws a floating connector and shows the straight-line distance. Do NOT use route for a simple distance/proximity question.',
|
||||
'annotate_map results: work any say line in naturally (places not found, a straight-line fallback with no travel time, or a full map — then ask before clearing). outlinePending means the boundary is still being traced: never claim it is drawn. A later map_annotation_outline system item reports each outcome; use it to quietly confirm or correct yourself.',
|
||||
'ROUTES vs DISTANCES: "walking/driving route from A to B" → type=route with the ordered points and the mode; the app draws the street path and reports distance and time you can read aloud. "How far is X from Y" or "X is next to Y" → type=arrow (straight-line distance), never route.',
|
||||
].join('\n');
|
||||
}
|
||||
|
||||
|
||||
@@ -10,6 +10,7 @@ import {
|
||||
OPENAI_REALTIME_REASONING_DEFAULT,
|
||||
OPENAI_REALTIME_CONTEXT_TOKENS_DEFAULT,
|
||||
OPENAI_REALTIME_CONTEXT_RETENTION_DEFAULT,
|
||||
OPENAI_REALTIME_TRANSCRIBE_MODEL_DEFAULT,
|
||||
} from './constants.js';
|
||||
import { realtimeInstructions } from './instructions.js';
|
||||
import { GEV_REALTIME_TOOLS } from './tools.js';
|
||||
@@ -20,7 +21,6 @@ function createRealtimeTokenHandler({
|
||||
fetchImpl = (...args) => fetch(...args),
|
||||
resolveApiKey = () => process.env.OPENAI_API_KEY,
|
||||
models = {},
|
||||
tools = GEV_REALTIME_TOOLS,
|
||||
} = {}) {
|
||||
return async (req, res) => {
|
||||
res.setHeader('Cache-Control', 'no-store');
|
||||
@@ -89,6 +89,14 @@ function createRealtimeTokenHandler({
|
||||
OPENAI_REALTIME_CONTEXT_RETENTION_DEFAULT,
|
||||
),
|
||||
);
|
||||
// User captions for the voice card. Billed separately from the session.
|
||||
const transcribeModel =
|
||||
process.env.OPENAI_REALTIME_TRANSCRIBE_MODEL ||
|
||||
OPENAI_REALTIME_TRANSCRIBE_MODEL_DEFAULT;
|
||||
const transcription =
|
||||
transcribeModel.toLowerCase() === 'off'
|
||||
? {}
|
||||
: { transcription: { model: transcribeModel } };
|
||||
const sessionConfig = {
|
||||
session: {
|
||||
type: 'realtime',
|
||||
@@ -104,6 +112,7 @@ function createRealtimeTokenHandler({
|
||||
audio: {
|
||||
input: {
|
||||
noise_reduction: { type: 'near_field' },
|
||||
...transcription,
|
||||
turn_detection: {
|
||||
type: 'semantic_vad',
|
||||
eagerness: 'low',
|
||||
@@ -114,7 +123,7 @@ function createRealtimeTokenHandler({
|
||||
output: { voice },
|
||||
},
|
||||
instructions: realtimeInstructions(annotationGuidance),
|
||||
tools,
|
||||
tools: GEV_REALTIME_TOOLS,
|
||||
tool_choice: 'auto',
|
||||
},
|
||||
};
|
||||
@@ -139,6 +148,11 @@ function createRealtimeTokenHandler({
|
||||
// case where a bogus ?tier= was silently downgraded to standard.
|
||||
res.setHeader('X-GEV-Voice-Tier', tier);
|
||||
res.setHeader('X-GEV-Voice-Model', model);
|
||||
// Captions are billed separately; the client meters them with this id.
|
||||
res.setHeader(
|
||||
'X-GEV-Voice-Transcribe-Model',
|
||||
transcription.transcription ? transcribeModel : 'off',
|
||||
);
|
||||
if (requestedTier && !isKnownVoiceTier(requestedTier)) {
|
||||
res.setHeader('X-GEV-Voice-Tier-Fallback', '1');
|
||||
}
|
||||
|
||||
@@ -1,3 +1,8 @@
|
||||
import {
|
||||
voiceLayerAliasHint,
|
||||
voiceQueryFieldHint,
|
||||
} from '../../../src/voice/layerManifest.js';
|
||||
|
||||
export const ACTION_DESCRIPTIONS = {
|
||||
fly_to_location: {
|
||||
description:
|
||||
@@ -12,7 +17,11 @@ export const ACTION_DESCRIPTIONS = {
|
||||
},
|
||||
query: {
|
||||
description:
|
||||
'Plain place search query, e.g. "London", "Eiffel Tower", or "Dubai Marina".',
|
||||
'Plain place search query, e.g. "London", "Eiffel Tower", or "Dubai Marina". "pointer" = the spot under the cursor.',
|
||||
$position: 1,
|
||||
},
|
||||
referent: {
|
||||
description: 'Numbered item n from the last result list.',
|
||||
$position: 1,
|
||||
},
|
||||
viewMode: {
|
||||
@@ -76,13 +85,13 @@ export const ACTION_DESCRIPTIONS = {
|
||||
$position: 1,
|
||||
},
|
||||
set_layer_visibility: {
|
||||
description: "Enable or disable one registered God's Eye View data layer.",
|
||||
description:
|
||||
"Enable or disable one God's Eye View data layer. Every shipped layer is in the enum; never tell the user a listed layer does not exist.",
|
||||
$position: 1,
|
||||
parameters: {
|
||||
properties: {
|
||||
layerId: {
|
||||
description:
|
||||
'Common-name mapping for the non-obvious ids: space mission(s) → rocket-launches; fires/wildfires/active fires → local-firms (NASA FIRMS); ships/vessels/boats → ais-live-vessels; undersea/submarine cables → telegeography-submarine-cables; datacenters → local-datacenters; dams → local-dams; bikes/bike share → bikeshare; street traffic/congestion → traffic; traffic cameras → cctv; internet radio/stations → radio; ALPR/license plate readers/Flock cameras → alpr-cameras; local ADS-B/my receiver/my antenna (aircraft heard by a local RTL-SDR receiver) → local-adsb.',
|
||||
description: `Spoken names: ${voiceLayerAliasHint()}.`,
|
||||
$position: 1,
|
||||
},
|
||||
},
|
||||
@@ -154,9 +163,13 @@ export const ACTION_DESCRIPTIONS = {
|
||||
properties: {
|
||||
scope: {
|
||||
description:
|
||||
'Use auto by default. selected returns the clicked/selected entity; in_view returns visible entities near the screen center.',
|
||||
'Use auto by default. selected returns the clicked/selected entity; in_view returns visible entities near the screen center; pointer returns what the cursor is on.',
|
||||
$position: 2,
|
||||
},
|
||||
referent: {
|
||||
description: 'Numbered item n from the last result list.',
|
||||
$position: 1,
|
||||
},
|
||||
layerId: {
|
||||
description: 'Optional layer filter for visible entity context.',
|
||||
$position: 2,
|
||||
@@ -349,12 +362,17 @@ export const ACTION_DESCRIPTIONS = {
|
||||
properties: {
|
||||
query: {
|
||||
description:
|
||||
'Callsign, ship name, satellite name, ICAO hex, MMSI, or NORAD id.',
|
||||
'Callsign, ship name, satellite name, ICAO hex, MMSI, or NORAD id. "pointer" = the contact under the cursor.',
|
||||
$position: 1,
|
||||
},
|
||||
referent: {
|
||||
description:
|
||||
'Numbered item n from the last result list (-1 = last); query may then be its label.',
|
||||
$position: 1,
|
||||
},
|
||||
layerId: {
|
||||
description:
|
||||
'Optional layer hint: flights | military | ais-live-vessels | satellites.',
|
||||
'Optional: flights, military, ais-live-vessels or satellites. Other layers are not trackable; for fires use query "biggest fire".',
|
||||
$position: 1,
|
||||
},
|
||||
},
|
||||
@@ -381,7 +399,7 @@ export const ACTION_DESCRIPTIONS = {
|
||||
},
|
||||
annotate_map: {
|
||||
description:
|
||||
'Draw annotations on the 3D map to visually point out what you are talking about — like sketching on a whiteboard over the world. Use this whenever you mention a specific place, building, campus, boundary, district, or a relationship between two places, so the user can SEE what you mean. Give place NAMES (preferred) or explicit lat/lng; the app resolves them to real-world positions and real building/area outlines — never guess pixel positions. Call this as you begin describing something, and you may mark several places in one call.',
|
||||
'Mark places on the 3D map as you talk about them (see WHITEBOARD): pins, highlights, footprints and boundaries, arrows, routes and labels. Give place names (or explicit lat/lng); the app resolves real positions and outlines. Several places may go in one call.',
|
||||
$position: 1,
|
||||
parameters: {
|
||||
properties: {
|
||||
@@ -398,7 +416,7 @@ export const ACTION_DESCRIPTIONS = {
|
||||
},
|
||||
target: {
|
||||
description:
|
||||
'Place name to resolve, e.g. "Palace of Fine Arts, San Francisco", "the Pentagon", "Presidio of San Francisco". Preferred over coordinates. For a specific monument/statue/feature that sits within a larger landmark, use its OWN name + city ("Tejano Monument, Austin", "Texas African American History Memorial, Austin") — do NOT phrase it as "X at the Texas State Capitol", which makes the geocoder collapse several of them onto the same centroid so they stack on one spot.',
|
||||
'Place name to resolve ("Palace of Fine Arts, San Francisco"), or "pointer" for the spot under the cursor. A monument inside a larger landmark gets its own name + city ("Tejano Monument, Austin"), never "X at the Texas State Capitol" (that stacks them on one centroid).',
|
||||
$position: 2,
|
||||
},
|
||||
points: {
|
||||
@@ -450,27 +468,24 @@ export const ACTION_DESCRIPTIONS = {
|
||||
},
|
||||
entityKind: {
|
||||
description:
|
||||
'What KIND of thing the target IS — a fact, not a style choice: building = one structure; compound = campus/grounds/mall/park; district = neighborhood or area of a city; street = a named road/corridor; point_feature = monument/statue/memorial/plaque/fountain or other small point landmark. Set it whenever you know it — it routes the resolver to the right footprint source (point_feature anchors monuments as precise points instead of adopting a nearby building outline).',
|
||||
'What the target IS (a fact, not a style): building = one structure; compound = campus/grounds/mall/park; district = neighborhood; street = a named road; point_feature = monument/statue/memorial/fountain, anchored as a precise point. Set it whenever you know it.',
|
||||
$position: 2,
|
||||
},
|
||||
screenX: {
|
||||
description:
|
||||
'Fallback only: when you cannot name/geocode the place but can SEE it in the latest viewport screenshot, the normalized horizontal position (0=left, 1=right) of the spot. The app converts it back to a real world point under that pixel.',
|
||||
'Fallback only, for a spot visible in the latest screenshot that you cannot name: normalized x (0=left, 1=right).',
|
||||
$position: 3,
|
||||
},
|
||||
screenY: {
|
||||
description:
|
||||
'Fallback only: normalized vertical position (0=top, 1=bottom) of the spot in the latest viewport screenshot.',
|
||||
description: 'Fallback only: normalized y (0=top, 1=bottom).',
|
||||
$position: 3,
|
||||
},
|
||||
toScreenX: {
|
||||
description:
|
||||
'For type=arrow: normalized x of the arrow destination from the screenshot (pixel fallback).',
|
||||
description: 'For type=arrow: destination x (pixel fallback).',
|
||||
$position: 3,
|
||||
},
|
||||
toScreenY: {
|
||||
description:
|
||||
'For type=arrow: normalized y of the arrow destination from the screenshot (pixel fallback).',
|
||||
description: 'For type=arrow: destination y (pixel fallback).',
|
||||
$position: 3,
|
||||
},
|
||||
},
|
||||
@@ -528,18 +543,18 @@ export const ACTION_DESCRIPTIONS = {
|
||||
},
|
||||
analyst_query: {
|
||||
description:
|
||||
'Answer questions ABOUT the data currently loaded on the map — counts, lists, superlatives, and attribute filters over live layers (flights, military, ships, fires, earthquakes, satellites, datacenters, dams). Examples: "how many flights over Texas", "biggest fire near LA", "which ships are headed to Oakland", "anything above 40,000 feet", "fastest thing in view". For satellites and infrastructure, counts and ranks cover only bounded examined loaded records; omitted records can change nearest/count. Queries ONLY client-side data from ENABLED layers — if the needed layer is off, say so and offer to enable it. For a follow-up about the previous answer\'s set ("which of those is closest?"), set followUp=true and send only the new filters/sort.',
|
||||
'Counts, lists, superlatives and attribute filters over records loaded by ENABLED layers — "how many flights over Texas", "biggest fire near LA", "ships headed to Oakland", "next rocket launch". complete:false means the count is a floor ("at least") and rankings cover the records examined; partial names unanswered layers. Items carry lat/lon for a follow-up fly_to_location. ok:false codes: LAYER_OFF (offer to enable), NOT_READY (loading), FOLLOW_UP_MISMATCH (ask again without followUp), UNKNOWN_FIELD/UNKNOWN_SCOPE/BAD_VALUE (retry with the allowed values).',
|
||||
$position: 1,
|
||||
parameters: {
|
||||
properties: {
|
||||
layers: {
|
||||
description:
|
||||
'Layers to query. fires/wildfires → local-firms; ships/vessels → ais-live-vessels.',
|
||||
'Layers to query. fires → local-firms; ships → ais-live-vessels; buses/trains → transit; hurricanes → weather-cyclones; bases → military-installations; my receiver → local-adsb. Units differ per layer, so rank speed within one layer.',
|
||||
$position: 2,
|
||||
},
|
||||
scope: {
|
||||
description:
|
||||
'Spatial scope. Default: view (near the camera). Use kind=region for "over Texas"-style asks; kind=anywhere for global questions.',
|
||||
'Always set it (see WHERE). view = a radius around the view centre ("in view", "nearby"); pointer = "around here/this" while pointing (radius at the cursor); region = "over Texas"; radius with center {lat, lon} = "near <place>"; anywhere = no place ("biggest anywhere", "today"). There is no drawn-area scope yet: say so rather than answering for the view.',
|
||||
$position: 2,
|
||||
properties: {
|
||||
name: {
|
||||
@@ -548,23 +563,23 @@ export const ACTION_DESCRIPTIONS = {
|
||||
$position: 1,
|
||||
},
|
||||
km: {
|
||||
description: 'For kind=radius.',
|
||||
description: 'For kind=radius or pointer.',
|
||||
$position: 1,
|
||||
},
|
||||
},
|
||||
},
|
||||
filters: {
|
||||
description:
|
||||
'Attribute predicates, ANDed. ALTITUDE IS METERS (40,000 ft = 12192). Fields: altitudeM, speedMps, military, onGround, aircraftClass, callsign, operator, routeOrigin, routeDestination, originCountry (flights); speedKts, shipType, destination (ships); frp, confidence (fires); magnitude, depthKm, place (earthquakes).',
|
||||
description: `ANDed; values keep their type (numbers, true/false). Altitude is meters (40,000 ft = 12192). Time fields take ISO; ageHours = hours ago. Every layer has id, lat, lon. Fields — ${voiceQueryFieldHint()}.`,
|
||||
$position: 1,
|
||||
},
|
||||
sortBy: {
|
||||
description: 'Field to rank by, or "distance" for nearest-first.',
|
||||
description:
|
||||
'Field to rank by, or "distance" (nearest first unless sortDir=desc).',
|
||||
$position: 1,
|
||||
},
|
||||
followUp: {
|
||||
description:
|
||||
'true = re-query the PREVIOUS result set instead of fresh data.',
|
||||
'true = re-filter the last answer (its layers) instead of fresh data.',
|
||||
$position: 1,
|
||||
},
|
||||
},
|
||||
@@ -572,7 +587,7 @@ export const ACTION_DESCRIPTIONS = {
|
||||
},
|
||||
next_iss_pass: {
|
||||
description:
|
||||
"When the user asks when the ISS / the space station will next fly over: returns the next geometric ISS pass with estimated visibility for the current camera location (or an explicit lat/lon) — rise time (ISO + minutes from now), rise compass direction, peak elevation, and duration. Requires the satellites layer to have loaded its catalog at least once this session; if it hasn't, tell the user to enable the satellites layer and try again.",
|
||||
'Next ISS pass over the camera location (or lat/lon): rise time, direction, peak elevation, duration and estimated visibility. Needs the satellites layer to have loaded once; if not, ask the user to enable it.',
|
||||
$position: 1,
|
||||
parameters: {
|
||||
properties: {
|
||||
|
||||
@@ -2,15 +2,3 @@ import { createActionTools } from '../../../src/voice/actionSchemas.js';
|
||||
import { ACTION_DESCRIPTIONS } from './toolDescriptions.js';
|
||||
|
||||
export const GEV_REALTIME_TOOLS = createActionTools(ACTION_DESCRIPTIONS);
|
||||
|
||||
/**
|
||||
* The session's tools: the app actions, then additional function tools
|
||||
* whose names no action already uses.
|
||||
*/
|
||||
export function realtimeSessionTools(additional = []) {
|
||||
const names = new Set(GEV_REALTIME_TOOLS.map((tool) => tool.name));
|
||||
return [
|
||||
...GEV_REALTIME_TOOLS,
|
||||
...additional.filter((tool) => !names.has(tool.name)),
|
||||
];
|
||||
}
|
||||
@@ -0,0 +1,171 @@
|
||||
import crypto from 'node:crypto';
|
||||
import fs from 'node:fs/promises';
|
||||
import path from 'node:path';
|
||||
|
||||
const DAY_MS = 86_400_000;
|
||||
|
||||
/** Entry files this cache writes: a 40-hex-digit key hash. */
|
||||
const CACHE_FILE_RE = /^[0-9a-f]{40}\.json$/;
|
||||
|
||||
/** How long an answer is kept, by what it was. */
|
||||
export const NOMINATIM_CACHE_TTL_MS = Object.freeze({
|
||||
outline: 90 * DAY_MS,
|
||||
place: 30 * DAY_MS,
|
||||
notFound: DAY_MS,
|
||||
});
|
||||
|
||||
/**
|
||||
* Comparable form of a place name: Unicode-normalized (NFKC), accents
|
||||
* removed, case folded, punctuation and symbols collapsed to single spaces,
|
||||
* and a leading article dropped. Non-Latin scripts are kept as they are:
|
||||
* "東京" stays "東京", while "São Paulo", "sao paulo" and "SÃO PAULO!" meet.
|
||||
*
|
||||
* @param {string} value
|
||||
* @returns {string}
|
||||
*/
|
||||
export function normalizePlaceName(value) {
|
||||
return String(value ?? '')
|
||||
.normalize('NFKD')
|
||||
.replace(/\p{M}+/gu, '')
|
||||
.normalize('NFKC')
|
||||
.toLocaleLowerCase('en-US')
|
||||
.replace(/[\p{P}\p{S}\p{Z}\s]+/gu, ' ')
|
||||
.trim()
|
||||
.replace(/^(?:the) /, '')
|
||||
.slice(0, 200);
|
||||
}
|
||||
|
||||
/**
|
||||
* Memory + disk cache for Nominatim answers.
|
||||
*
|
||||
* Every entry carries its own expiry. Memory is an LRU bounded by entries;
|
||||
* disk is bounded by entries and bytes, with expired files removed during a
|
||||
* periodic sweep. A null `dir` keeps everything in memory.
|
||||
*/
|
||||
export function createNominatimCache({
|
||||
dir = null,
|
||||
maxMemoryEntries = 256,
|
||||
maxDiskEntries = 4000,
|
||||
maxDiskBytes = 64 * 1024 * 1024,
|
||||
maxEntryBytes = 2 * 1024 * 1024,
|
||||
sweepEvery = 50,
|
||||
now = Date.now,
|
||||
} = {}) {
|
||||
const memory = new Map();
|
||||
let writesSinceSweep = 0;
|
||||
let sweeping = null;
|
||||
|
||||
const fileFor = (key) =>
|
||||
path.join(
|
||||
dir,
|
||||
`${crypto.createHash('sha256').update(key).digest('hex').slice(0, 40)}.json`,
|
||||
);
|
||||
|
||||
function remember(key, entry) {
|
||||
memory.delete(key);
|
||||
memory.set(key, entry);
|
||||
while (memory.size > maxMemoryEntries)
|
||||
memory.delete(memory.keys().next().value);
|
||||
}
|
||||
|
||||
async function get(key) {
|
||||
const hit = memory.get(key);
|
||||
if (hit) {
|
||||
if (hit.expiresAt > now()) {
|
||||
remember(key, hit);
|
||||
return hit;
|
||||
}
|
||||
memory.delete(key);
|
||||
}
|
||||
if (!dir) return null;
|
||||
try {
|
||||
const entry = JSON.parse(await fs.readFile(fileFor(key), 'utf8'));
|
||||
if (entry?.key !== key || !(entry.expiresAt > now())) return null;
|
||||
remember(key, entry);
|
||||
return entry;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
async function set(key, payload, ttlMs) {
|
||||
const entry = { key, payload, storedAt: now(), expiresAt: now() + ttlMs };
|
||||
remember(key, entry);
|
||||
if (!dir) return;
|
||||
const body = JSON.stringify(entry);
|
||||
if (Buffer.byteLength(body) > maxEntryBytes) return;
|
||||
try {
|
||||
await fs.mkdir(dir, { recursive: true });
|
||||
const file = fileFor(key);
|
||||
const temp = `${file}.${process.pid}.tmp`;
|
||||
await fs.writeFile(temp, body);
|
||||
await fs.rename(temp, file);
|
||||
} catch {
|
||||
return;
|
||||
}
|
||||
writesSinceSweep += 1;
|
||||
if (writesSinceSweep >= sweepEvery) {
|
||||
writesSinceSweep = 0;
|
||||
await sweep();
|
||||
}
|
||||
}
|
||||
|
||||
/** Remove expired files, then the oldest until both disk bounds hold. */
|
||||
async function sweep() {
|
||||
if (!dir) return;
|
||||
if (sweeping) return sweeping;
|
||||
sweeping = (async () => {
|
||||
let names;
|
||||
try {
|
||||
names = await fs.readdir(dir);
|
||||
} catch {
|
||||
return;
|
||||
}
|
||||
const files = [];
|
||||
for (const name of names) {
|
||||
// Only this cache's own entry files; anything else in the directory
|
||||
// is left alone.
|
||||
if (!CACHE_FILE_RE.test(name)) continue;
|
||||
const file = path.join(dir, name);
|
||||
let stat;
|
||||
let text;
|
||||
try {
|
||||
[stat, text] = await Promise.all([
|
||||
fs.stat(file),
|
||||
fs.readFile(file, 'utf8'),
|
||||
]);
|
||||
} catch {
|
||||
// Unreadable right now (gone, busy, too many open files): an error
|
||||
// reading is not evidence the entry is bad, so it stays.
|
||||
continue;
|
||||
}
|
||||
let expiresAt;
|
||||
try {
|
||||
expiresAt = JSON.parse(text)?.expiresAt;
|
||||
} catch {
|
||||
expiresAt = null; // corrupt: removed below
|
||||
}
|
||||
if (!(expiresAt > now())) {
|
||||
await fs.rm(file, { force: true }).catch(() => {});
|
||||
continue;
|
||||
}
|
||||
files.push({ file, size: stat.size, at: stat.mtimeMs });
|
||||
}
|
||||
files.sort((a, b) => a.at - b.at);
|
||||
let bytes = files.reduce((sum, entry) => sum + entry.size, 0);
|
||||
while (
|
||||
files.length &&
|
||||
(files.length > maxDiskEntries || bytes > maxDiskBytes)
|
||||
) {
|
||||
const oldest = files.shift();
|
||||
bytes -= oldest.size;
|
||||
await fs.rm(oldest.file, { force: true }).catch(() => {});
|
||||
}
|
||||
})().finally(() => {
|
||||
sweeping = null;
|
||||
});
|
||||
return sweeping;
|
||||
}
|
||||
|
||||
return { get, set, sweep, memorySize: () => memory.size };
|
||||
}
|
||||
@@ -0,0 +1,812 @@
|
||||
import fs from 'node:fs';
|
||||
import path from 'node:path';
|
||||
import { DatabaseSync } from 'node:sqlite';
|
||||
import { readResponseJsonCapped } from '../common/http.js';
|
||||
|
||||
/**
|
||||
* One gate for every request this server sends to a Nominatim instance.
|
||||
*
|
||||
* Place search and voice outlines share it. Public requests reserve one start
|
||||
* at a time across processes sharing this installation's state file, remain at
|
||||
* least 1.1 s apart, use a short queue and daily ceiling, and pause after the
|
||||
* server says to stop. Independent installations do not share one app-wide
|
||||
* limiter; each must identify itself and enforce the public policy locally.
|
||||
*/
|
||||
|
||||
/** The public instance. Only this host is subject to the public-use limits. */
|
||||
export const PUBLIC_NOMINATIM_SEARCH =
|
||||
'https://nominatim.openstreetmap.org/search';
|
||||
|
||||
const PUBLIC_NOMINATIM_HOST = 'nominatim.openstreetmap.org';
|
||||
|
||||
/**
|
||||
* The public usage policy asks for a User-Agent or Referer that identifies the
|
||||
* application. The identity is stable: it is never rotated or varied.
|
||||
*/
|
||||
export const NOMINATIM_HEADERS = Object.freeze({
|
||||
'User-Agent':
|
||||
'gods-eye-view/0.1 (+https://github.com/bilawalsidhu/gods-eye-view)',
|
||||
Referer: 'https://github.com/bilawalsidhu/gods-eye-view',
|
||||
});
|
||||
|
||||
/** Absolute policy maximum is one request per second; 1.1 s absorbs jitter. */
|
||||
export const NOMINATIM_MIN_SPACING_MS = 1100;
|
||||
|
||||
/** Requests allowed to wait for their turn; past this, refuse at once. */
|
||||
export const NOMINATIM_MAX_PENDING = 4;
|
||||
|
||||
/** A queued request older than this has no reader left; drop it unsent. */
|
||||
export const NOMINATIM_MAX_WAIT_MS = 10_000;
|
||||
|
||||
/** Default public requests per install per UTC day, outlines and search together. */
|
||||
export const NOMINATIM_DEFAULT_DAILY_CAP = 50;
|
||||
|
||||
/**
|
||||
* Pause after a refusal without Retry-After. A Retry-After from the server is
|
||||
* honoured in full, however long: no local maximum may cause an earlier retry.
|
||||
*/
|
||||
const DEFAULT_BACKOFF_MS = 60_000;
|
||||
/** A 403 from the public instance means this identity is blocked: stop longer. */
|
||||
const BLOCKED_BACKOFF_MS = 3_600_000;
|
||||
/** Pause after a server error or a failed retry. */
|
||||
const OUTAGE_BACKOFF_MS = 30_000;
|
||||
|
||||
const DISABLED_VALUES = new Set(['', 'off', 'none', 'disabled', '0']);
|
||||
|
||||
/**
|
||||
* Read the search endpoint setting.
|
||||
*
|
||||
* Unset means the public instance under its guards. Set to empty (or `off`)
|
||||
* disables every Nominatim request. Any other value must be an http(s) URL
|
||||
* without credentials, query or fragment; a bare host gets `/search`. A value
|
||||
* that is not usable disables the service rather than falling back to the
|
||||
* public instance.
|
||||
*
|
||||
* @param {Record<string, string|undefined>} [env]
|
||||
* @returns {{endpoint: string|null, isPublic: boolean, dailyCap: number, invalid?: boolean}}
|
||||
*/
|
||||
export function resolveNominatimSettings(env = process.env) {
|
||||
const dailyCap = parseDailyCap(env.NOMINATIM_DAILY_CAP);
|
||||
const raw = env.NOMINATIM_URL;
|
||||
if (raw === undefined)
|
||||
return { endpoint: PUBLIC_NOMINATIM_SEARCH, isPublic: true, dailyCap };
|
||||
const value = String(raw).trim();
|
||||
if (DISABLED_VALUES.has(value.toLowerCase()))
|
||||
return { endpoint: null, isPublic: false, dailyCap };
|
||||
let url;
|
||||
try {
|
||||
url = new URL(value);
|
||||
} catch {
|
||||
return { endpoint: null, isPublic: false, dailyCap, invalid: true };
|
||||
}
|
||||
if (
|
||||
!['http:', 'https:'].includes(url.protocol) ||
|
||||
!url.hostname ||
|
||||
url.username ||
|
||||
url.password ||
|
||||
url.search ||
|
||||
url.hash
|
||||
)
|
||||
return { endpoint: null, isPublic: false, dailyCap, invalid: true };
|
||||
if (url.pathname === '/' || url.pathname === '') url.pathname = '/search';
|
||||
return {
|
||||
endpoint: url.href.replace(/\/$/, ''),
|
||||
// A trailing DNS dot names the same host.
|
||||
isPublic:
|
||||
url.hostname.toLowerCase().replace(/\.$/, '') === PUBLIC_NOMINATIM_HOST,
|
||||
dailyCap,
|
||||
};
|
||||
}
|
||||
|
||||
function parseDailyCap(raw) {
|
||||
if (raw === undefined || String(raw).trim() === '')
|
||||
return NOMINATIM_DEFAULT_DAILY_CAP;
|
||||
const value = Number(raw);
|
||||
return Number.isFinite(value) && value >= 0
|
||||
? Math.floor(value)
|
||||
: NOMINATIM_DEFAULT_DAILY_CAP;
|
||||
}
|
||||
|
||||
function gateError(code, message, extra = {}) {
|
||||
return Object.assign(new Error(message), { code, ...extra });
|
||||
}
|
||||
|
||||
const HTTP_MONTHS = Object.freeze([
|
||||
'Jan',
|
||||
'Feb',
|
||||
'Mar',
|
||||
'Apr',
|
||||
'May',
|
||||
'Jun',
|
||||
'Jul',
|
||||
'Aug',
|
||||
'Sep',
|
||||
'Oct',
|
||||
'Nov',
|
||||
'Dec',
|
||||
]);
|
||||
const HTTP_WEEKDAYS = Object.freeze([
|
||||
'Sun',
|
||||
'Mon',
|
||||
'Tue',
|
||||
'Wed',
|
||||
'Thu',
|
||||
'Fri',
|
||||
'Sat',
|
||||
]);
|
||||
const HTTP_WEEKDAYS_LONG = Object.freeze([
|
||||
'Sunday',
|
||||
'Monday',
|
||||
'Tuesday',
|
||||
'Wednesday',
|
||||
'Thursday',
|
||||
'Friday',
|
||||
'Saturday',
|
||||
]);
|
||||
|
||||
/** Strict RFC 9110 HTTP-date, including the two obsolete recipient formats. */
|
||||
function parseHttpDate(raw, now) {
|
||||
let match = raw.match(
|
||||
/^(Sun|Mon|Tue|Wed|Thu|Fri|Sat), (\d{2}) (Jan|Feb|Mar|Apr|May|Jun|Jul|Aug|Sep|Oct|Nov|Dec) (\d{4}) (\d{2}):(\d{2}):(\d{2}) GMT$/,
|
||||
);
|
||||
let weekday, day, month, year, hour, minute, second;
|
||||
if (match) {
|
||||
[, weekday, day, month, year, hour, minute, second] = match;
|
||||
weekday = HTTP_WEEKDAYS.indexOf(weekday);
|
||||
} else {
|
||||
match = raw.match(
|
||||
/^(Sunday|Monday|Tuesday|Wednesday|Thursday|Friday|Saturday), (\d{2})-(Jan|Feb|Mar|Apr|May|Jun|Jul|Aug|Sep|Oct|Nov|Dec)-(\d{2}) (\d{2}):(\d{2}):(\d{2}) GMT$/,
|
||||
);
|
||||
if (match) {
|
||||
[, weekday, day, month, year, hour, minute, second] = match;
|
||||
weekday = HTTP_WEEKDAYS_LONG.indexOf(weekday);
|
||||
const currentYear = new Date(now).getUTCFullYear();
|
||||
if (!Number.isFinite(currentYear)) return null;
|
||||
year = Math.floor(currentYear / 100) * 100 + Number(year);
|
||||
if (year > currentYear + 50) year -= 100;
|
||||
} else {
|
||||
match = raw.match(
|
||||
/^(Sun|Mon|Tue|Wed|Thu|Fri|Sat) (Jan|Feb|Mar|Apr|May|Jun|Jul|Aug|Sep|Oct|Nov|Dec) ([ \d]\d) (\d{2}):(\d{2}):(\d{2}) (\d{4})$/,
|
||||
);
|
||||
if (!match) return null;
|
||||
[, weekday, month, day, hour, minute, second, year] = match;
|
||||
weekday = HTTP_WEEKDAYS.indexOf(weekday);
|
||||
}
|
||||
}
|
||||
day = Number(day);
|
||||
month = HTTP_MONTHS.indexOf(month);
|
||||
year = Number(year);
|
||||
hour = Number(hour);
|
||||
minute = Number(minute);
|
||||
second = Number(second);
|
||||
if (
|
||||
weekday < 0 ||
|
||||
month < 0 ||
|
||||
year < 1601 ||
|
||||
hour > 23 ||
|
||||
minute > 59 ||
|
||||
second > 59
|
||||
)
|
||||
return null;
|
||||
const at = Date.UTC(year, month, day, hour, minute, second);
|
||||
const date = new Date(at);
|
||||
return date.getUTCFullYear() === year &&
|
||||
date.getUTCMonth() === month &&
|
||||
date.getUTCDate() === day &&
|
||||
date.getUTCHours() === hour &&
|
||||
date.getUTCMinutes() === minute &&
|
||||
date.getUTCSeconds() === second &&
|
||||
date.getUTCDay() === weekday
|
||||
? at
|
||||
: null;
|
||||
}
|
||||
|
||||
/** Seconds or an HTTP date → milliseconds from now, or null. */
|
||||
export function parseRetryAfter(value, now = Date.now()) {
|
||||
if (value == null) return null;
|
||||
const raw = String(value).trim();
|
||||
if (!raw) return null;
|
||||
if (/^\d+$/.test(raw)) {
|
||||
const seconds = Number(raw);
|
||||
const milliseconds = seconds * 1000;
|
||||
return Number.isSafeInteger(milliseconds) ? milliseconds : null;
|
||||
}
|
||||
// Numeric-looking but invalid delay-seconds must not be reinterpreted as a date.
|
||||
if (/^[+-]?(?:\d|\.\d)/.test(raw)) return null;
|
||||
const at = parseHttpDate(raw, now);
|
||||
return at === null ? null : Math.max(0, at - now);
|
||||
}
|
||||
|
||||
/** UTC calendar day, the unit of the daily cap. */
|
||||
function utcDay(ms) {
|
||||
return new Date(ms).toISOString().slice(0, 10);
|
||||
}
|
||||
|
||||
/**
|
||||
* Public-service state shared by every process using the same file: today's
|
||||
* request count, the next reserved start, and any pause the server asked for.
|
||||
*
|
||||
* Every change is a read-modify-write under an OS-owned SQLite transaction,
|
||||
* and the JSON file is replaced atomically, so two dev servers sharing a
|
||||
* checkout add to one count instead of overwriting each other. A process exit
|
||||
* releases only that process's transaction; there is no stale path to delete
|
||||
* and therefore no stale reclaimer that can remove a successor's live lock.
|
||||
* Memory keeps the highest values this process has seen, so a failed or
|
||||
* unreadable file never lowers the count or shortens a pause; a persistence
|
||||
* failure is reported once and in `status()`.
|
||||
*/
|
||||
export function createGateStateStore({
|
||||
file = null,
|
||||
lockWaitMs = 500,
|
||||
onError = (error) =>
|
||||
console.warn(
|
||||
`[Nominatim] could not persist usage state: ${error?.message || error}`,
|
||||
),
|
||||
} = {}) {
|
||||
let memory = { day: '', count: 0, pausedUntil: 0, nextStartAt: 0 };
|
||||
let lastError = null;
|
||||
|
||||
const report = (error) => {
|
||||
if (!lastError) onError(error);
|
||||
lastError = error;
|
||||
};
|
||||
|
||||
const pauseMarkerPrefix = file ? `${path.basename(file)}.pause.` : null;
|
||||
|
||||
/**
|
||||
* A refusal must remain visible even while another process owns the SQLite
|
||||
* transaction. Immutable marker names make that write atomic and monotonic:
|
||||
* concurrent writers cannot shorten one another's pause, and the next lock
|
||||
* owner folds every marker into the normal JSON state.
|
||||
*/
|
||||
function pendingPauses({ strict = false } = {}) {
|
||||
if (!file) return [];
|
||||
let names;
|
||||
try {
|
||||
names = fs.readdirSync(path.dirname(file));
|
||||
} catch (error) {
|
||||
if (error?.code === 'ENOENT') return [];
|
||||
report(error);
|
||||
if (strict) throw error;
|
||||
return [];
|
||||
}
|
||||
const pauses = [];
|
||||
for (const name of names) {
|
||||
if (!name.startsWith(pauseMarkerPrefix)) continue;
|
||||
const encoded = name.slice(pauseMarkerPrefix.length).split('.', 1)[0];
|
||||
if (!/^\d+$/.test(encoded)) continue;
|
||||
const pausedUntil = Number(encoded);
|
||||
if (!Number.isSafeInteger(pausedUntil)) {
|
||||
const error = new Error('invalid durable Nominatim pause marker');
|
||||
report(error);
|
||||
if (strict) throw error;
|
||||
continue;
|
||||
}
|
||||
pauses.push({
|
||||
path: path.join(path.dirname(file), name),
|
||||
pausedUntil,
|
||||
});
|
||||
}
|
||||
return pauses;
|
||||
}
|
||||
|
||||
function markPause(at) {
|
||||
fs.mkdirSync(path.dirname(file), { recursive: true });
|
||||
const pausedUntil = Math.ceil(at);
|
||||
const marker = `${file}.pause.${pausedUntil}.${process.pid}.${process.hrtime.bigint()}`;
|
||||
fs.writeFileSync(marker, '', { flag: 'wx' });
|
||||
return pausedUntil;
|
||||
}
|
||||
|
||||
function clearPersistedPauses(pausedUntil) {
|
||||
for (const marker of pendingPauses()) {
|
||||
if (marker.pausedUntil > pausedUntil) continue;
|
||||
try {
|
||||
fs.unlinkSync(marker.path);
|
||||
} catch (error) {
|
||||
if (error?.code !== 'ENOENT') report(error);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
const merge = (a, b) => ({
|
||||
day: a.day >= b.day ? a.day : b.day,
|
||||
count:
|
||||
a.day === b.day
|
||||
? Math.max(a.count, b.count)
|
||||
: a.day > b.day
|
||||
? a.count
|
||||
: b.count,
|
||||
pausedUntil: Math.max(a.pausedUntil || 0, b.pausedUntil || 0),
|
||||
nextStartAt: Math.max(a.nextStartAt || 0, b.nextStartAt || 0),
|
||||
});
|
||||
|
||||
const parseState = (parsed) => {
|
||||
const validDay =
|
||||
parsed?.day === '' ||
|
||||
(typeof parsed?.day === 'string' &&
|
||||
/^\d{4}-\d{2}-\d{2}$/.test(parsed.day) &&
|
||||
new Date(`${parsed.day}T00:00:00.000Z`)
|
||||
.toISOString()
|
||||
.startsWith(parsed.day));
|
||||
const validCount =
|
||||
Number.isInteger(parsed?.count) &&
|
||||
parsed.count >= 0 &&
|
||||
(parsed.day !== '' || parsed.count === 0);
|
||||
const validPausedUntil =
|
||||
Number.isFinite(parsed?.pausedUntil) && parsed.pausedUntil >= 0;
|
||||
// nextStartAt was added after the original state format. Its absence is
|
||||
// the one supported migration; a present value must still be trustworthy.
|
||||
const validNextStartAt =
|
||||
parsed?.nextStartAt === undefined ||
|
||||
(Number.isFinite(parsed.nextStartAt) && parsed.nextStartAt >= 0);
|
||||
if (
|
||||
!parsed ||
|
||||
Array.isArray(parsed) ||
|
||||
typeof parsed !== 'object' ||
|
||||
!validDay ||
|
||||
!validCount ||
|
||||
!validPausedUntil ||
|
||||
!validNextStartAt
|
||||
)
|
||||
throw new Error('invalid durable Nominatim usage state');
|
||||
return {
|
||||
day: parsed.day,
|
||||
count: parsed.count,
|
||||
pausedUntil: parsed.pausedUntil,
|
||||
nextStartAt: parsed.nextStartAt ?? 0,
|
||||
};
|
||||
};
|
||||
|
||||
function readDisk({ strict = false } = {}) {
|
||||
if (!file) {
|
||||
if (strict)
|
||||
throw new Error('no durable Nominatim usage state is configured');
|
||||
return null;
|
||||
}
|
||||
try {
|
||||
return parseState(JSON.parse(fs.readFileSync(file, 'utf8')));
|
||||
} catch (error) {
|
||||
if (error?.code === 'ENOENT') return null;
|
||||
report(error);
|
||||
if (strict) throw error;
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
function current() {
|
||||
const disk = readDisk();
|
||||
if (disk) memory = merge(memory, disk);
|
||||
const pending = pendingPauses();
|
||||
if (pending.length)
|
||||
memory = merge(memory, {
|
||||
...memory,
|
||||
pausedUntil: Math.max(...pending.map((item) => item.pausedUntil)),
|
||||
});
|
||||
return memory;
|
||||
}
|
||||
|
||||
function withLock(work) {
|
||||
fs.mkdirSync(path.dirname(file), { recursive: true });
|
||||
const waitMs = Number.isFinite(lockWaitMs)
|
||||
? Math.max(0, Math.floor(lockWaitMs))
|
||||
: 500;
|
||||
const database = new DatabaseSync(`${file}.lock.sqlite`, {
|
||||
timeout: waitMs,
|
||||
});
|
||||
let transactionOpen = false;
|
||||
try {
|
||||
database.exec('BEGIN IMMEDIATE');
|
||||
transactionOpen = true;
|
||||
const result = work();
|
||||
database.exec('COMMIT');
|
||||
transactionOpen = false;
|
||||
return result;
|
||||
} catch (error) {
|
||||
if (transactionOpen) {
|
||||
try {
|
||||
database.exec('ROLLBACK');
|
||||
} catch {
|
||||
// Closing the connection below still releases this owner's lock.
|
||||
}
|
||||
}
|
||||
throw error;
|
||||
} finally {
|
||||
try {
|
||||
database.close();
|
||||
} catch {
|
||||
// The transaction has already been committed or rolled back. Closing
|
||||
// is best effort during exceptional teardown and never removes a lock.
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
function writeDisk(next) {
|
||||
const temp = `${file}.${process.pid}.${process.hrtime.bigint()}.tmp`;
|
||||
try {
|
||||
fs.writeFileSync(temp, JSON.stringify(next));
|
||||
fs.renameSync(temp, file);
|
||||
} finally {
|
||||
try {
|
||||
fs.unlinkSync(temp);
|
||||
} catch (error) {
|
||||
if (error?.code !== 'ENOENT') report(error);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/** Apply `change` to the freshest state and persist it atomically. */
|
||||
function update(change) {
|
||||
if (!file) {
|
||||
memory = merge(memory, change(memory));
|
||||
return memory;
|
||||
}
|
||||
try {
|
||||
withLock(() => {
|
||||
const next = merge(memory, change(current()));
|
||||
writeDisk(next);
|
||||
memory = next;
|
||||
clearPersistedPauses(next.pausedUntil);
|
||||
lastError = null;
|
||||
});
|
||||
} catch (error) {
|
||||
report(error);
|
||||
memory = merge(memory, change(memory));
|
||||
}
|
||||
return memory;
|
||||
}
|
||||
|
||||
return {
|
||||
count(day) {
|
||||
const state = current();
|
||||
return state.day === day ? state.count : 0;
|
||||
},
|
||||
increment(day) {
|
||||
return update((state) => ({
|
||||
day,
|
||||
count: state.day === day ? state.count + 1 : 1,
|
||||
pausedUntil: state.pausedUntil,
|
||||
nextStartAt: state.nextStartAt,
|
||||
})).count;
|
||||
},
|
||||
reserve(day, { now, dailyCap, minSpacingMs, latestStartAt = Infinity }) {
|
||||
if (!file) {
|
||||
const error = new Error(
|
||||
'no durable Nominatim usage state is configured',
|
||||
);
|
||||
report(error);
|
||||
return { status: 'unavailable', error };
|
||||
}
|
||||
try {
|
||||
let result;
|
||||
withLock(() => {
|
||||
const disk = readDisk({ strict: true });
|
||||
if (disk) memory = merge(memory, disk);
|
||||
const pending = pendingPauses({ strict: true });
|
||||
if (pending.length)
|
||||
memory = merge(memory, {
|
||||
...memory,
|
||||
pausedUntil: Math.max(...pending.map((item) => item.pausedUntil)),
|
||||
});
|
||||
const state = memory;
|
||||
if ((state.pausedUntil || 0) > now) {
|
||||
result = {
|
||||
status: 'paused',
|
||||
retryAfterMs: state.pausedUntil - now,
|
||||
};
|
||||
return;
|
||||
}
|
||||
const count = state.day === day ? state.count : 0;
|
||||
if (count >= dailyCap) {
|
||||
result = { status: 'cap' };
|
||||
return;
|
||||
}
|
||||
const startAt = Math.max(now, state.nextStartAt || 0);
|
||||
if (startAt > latestStartAt) {
|
||||
result = { status: 'abandoned' };
|
||||
return;
|
||||
}
|
||||
const next = {
|
||||
day,
|
||||
count: count + 1,
|
||||
pausedUntil: state.pausedUntil || 0,
|
||||
nextStartAt: startAt + minSpacingMs,
|
||||
};
|
||||
writeDisk(next);
|
||||
memory = next;
|
||||
clearPersistedPauses(next.pausedUntil);
|
||||
lastError = null;
|
||||
result = { status: 'reserved', startAt, count: next.count };
|
||||
});
|
||||
return result;
|
||||
} catch (error) {
|
||||
report(error);
|
||||
return { status: 'unavailable', error };
|
||||
}
|
||||
},
|
||||
pausedUntil() {
|
||||
return current().pausedUntil || 0;
|
||||
},
|
||||
pauseUntil(at) {
|
||||
if (file) {
|
||||
try {
|
||||
at = markPause(at);
|
||||
} catch (error) {
|
||||
report(error);
|
||||
}
|
||||
}
|
||||
update((state) => ({ ...state, pausedUntil: at }));
|
||||
},
|
||||
status: () => ({
|
||||
file,
|
||||
durable: Boolean(file),
|
||||
persisted: Boolean(file) && !lastError,
|
||||
error: lastError ? String(lastError.message || lastError) : null,
|
||||
}),
|
||||
};
|
||||
}
|
||||
|
||||
/** Compatibility name for the state store. */
|
||||
export const createUsageStore = createGateStateStore;
|
||||
|
||||
/**
|
||||
* Construct a gate. `settings` may be a function so the environment is read
|
||||
* after the dev server loads `.env`.
|
||||
*
|
||||
* @param {object} [options]
|
||||
* @param {(() => object)|object} [options.settings]
|
||||
* @param {typeof fetch} [options.fetchImpl]
|
||||
* @param {() => number} [options.now]
|
||||
* @param {(ms: number) => Promise<void>} [options.sleep]
|
||||
* @param {ReturnType<typeof createUsageStore>} [options.usage]
|
||||
*/
|
||||
export function createNominatimGate({
|
||||
settings = () => resolveNominatimSettings(),
|
||||
fetchImpl = (...args) => globalThis.fetch(...args),
|
||||
now = Date.now,
|
||||
sleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms)),
|
||||
usage = createGateStateStore(),
|
||||
minSpacingMs = NOMINATIM_MIN_SPACING_MS,
|
||||
maxPending = NOMINATIM_MAX_PENDING,
|
||||
maxWaitMs = NOMINATIM_MAX_WAIT_MS,
|
||||
timeoutMs = 9000,
|
||||
} = {}) {
|
||||
let queue = Promise.resolve();
|
||||
let pending = 0;
|
||||
const pausedUntilByEndpoint = new Map();
|
||||
let upstreamRequests = 0;
|
||||
let usageStore = usage;
|
||||
|
||||
const current = () =>
|
||||
typeof settings === 'function' ? settings() : settings;
|
||||
|
||||
/**
|
||||
* Pause until `ms` from now. The public service's pause is also written to
|
||||
* the shared state so a restart or a second process honours it.
|
||||
*/
|
||||
function pause(ms, config) {
|
||||
const until = now() + Math.max(0, ms);
|
||||
const key = String(config?.endpoint || '');
|
||||
if (until > (pausedUntilByEndpoint.get(key) || 0))
|
||||
pausedUntilByEndpoint.set(key, until);
|
||||
if (config?.isPublic) usageStore.pauseUntil(until);
|
||||
}
|
||||
|
||||
function pauseEnd(config) {
|
||||
const endpointPause =
|
||||
pausedUntilByEndpoint.get(String(config?.endpoint || '')) || 0;
|
||||
return config?.isPublic
|
||||
? Math.max(endpointPause, usageStore.pausedUntil())
|
||||
: endpointPause;
|
||||
}
|
||||
|
||||
function pausedError(config) {
|
||||
return gateError('NOMINATIM_PAUSED', 'Place service is pausing', {
|
||||
retryAfterMs: Math.max(0, pauseEnd(config) - now()),
|
||||
});
|
||||
}
|
||||
|
||||
function admissionError(config) {
|
||||
if (!config.endpoint)
|
||||
return gateError('NOMINATIM_DISABLED', 'Place service is not configured');
|
||||
if (now() < pauseEnd(config)) return pausedError(config);
|
||||
return null;
|
||||
}
|
||||
|
||||
/** One upstream request; classifies refusals and pauses the gate. */
|
||||
async function send(url, config, maxBytes, signal) {
|
||||
upstreamRequests += 1;
|
||||
const controller = new AbortController();
|
||||
const onAbort = () => controller.abort();
|
||||
signal?.addEventListener('abort', onAbort, { once: true });
|
||||
// One deadline covers the headers AND the body: a server that answers its
|
||||
// headers and then stalls must not hold the shared queue.
|
||||
const timer = setTimeout(() => controller.abort(), timeoutMs);
|
||||
try {
|
||||
return await exchange(url, config, maxBytes, signal, controller.signal);
|
||||
} finally {
|
||||
clearTimeout(timer);
|
||||
signal?.removeEventListener('abort', onAbort);
|
||||
}
|
||||
}
|
||||
|
||||
async function exchange(url, config, maxBytes, signal, requestSignal) {
|
||||
let response;
|
||||
try {
|
||||
response = await fetchImpl(url, {
|
||||
headers: NOMINATIM_HEADERS,
|
||||
redirect: 'error',
|
||||
signal: requestSignal,
|
||||
});
|
||||
} catch (cause) {
|
||||
if (signal?.aborted) throw gateError('NOMINATIM_ABANDONED', 'Abandoned');
|
||||
throw gateError('NOMINATIM_TRANSPORT', 'Place service unreachable', {
|
||||
cause,
|
||||
transport: true,
|
||||
});
|
||||
}
|
||||
if (!response.ok) {
|
||||
void response.body?.cancel?.().catch(() => {});
|
||||
const status = response.status;
|
||||
const retryAfter = parseRetryAfter(
|
||||
response.headers?.get?.('retry-after'),
|
||||
now(),
|
||||
);
|
||||
if (status === 429 || status === 503) {
|
||||
pause(retryAfter ?? DEFAULT_BACKOFF_MS, config);
|
||||
throw gateError('NOMINATIM_REFUSED', 'Place service asked to wait', {
|
||||
status,
|
||||
retryAfterMs: Math.max(0, pauseEnd(config) - now()),
|
||||
});
|
||||
}
|
||||
if (status === 403 || status === 418) {
|
||||
pause(retryAfter ?? BLOCKED_BACKOFF_MS, config);
|
||||
throw gateError('NOMINATIM_REFUSED', 'Place service refused', {
|
||||
status,
|
||||
retryAfterMs: Math.max(0, pauseEnd(config) - now()),
|
||||
});
|
||||
}
|
||||
if (status >= 500) pause(retryAfter ?? OUTAGE_BACKOFF_MS, config);
|
||||
throw gateError('NOMINATIM_UPSTREAM', `Upstream returned ${status}`, {
|
||||
status,
|
||||
});
|
||||
}
|
||||
try {
|
||||
return await readResponseJsonCapped(response, maxBytes, requestSignal);
|
||||
} catch (cause) {
|
||||
if (cause?.code === 'RESPONSE_TOO_LARGE')
|
||||
throw gateError('NOMINATIM_TOO_LARGE', 'Place answer too large');
|
||||
if (signal?.aborted) throw gateError('NOMINATIM_ABANDONED', 'Abandoned');
|
||||
throw gateError('NOMINATIM_TRANSPORT', 'Place answer unreadable', {
|
||||
cause,
|
||||
transport: true,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
/** Atomically reserve the public slot, then wait until it may start. */
|
||||
async function takeTurn(config, queuedAt, signal) {
|
||||
if (signal?.aborted || now() - queuedAt > maxWaitMs)
|
||||
throw gateError('NOMINATIM_ABANDONED', 'Place search was abandoned');
|
||||
const refused = admissionError(config);
|
||||
if (refused) throw refused;
|
||||
// An operator-selected endpoint is not governed by the public instance's
|
||||
// shared budget, pacing, or durable-state requirement.
|
||||
if (!config.isPublic) return;
|
||||
const reserved = usageStore.reserve(utcDay(now()), {
|
||||
now: now(),
|
||||
dailyCap: config.dailyCap,
|
||||
minSpacingMs,
|
||||
latestStartAt: queuedAt + maxWaitMs,
|
||||
});
|
||||
if (reserved.status === 'unavailable')
|
||||
throw gateError(
|
||||
'NOMINATIM_STATE_UNAVAILABLE',
|
||||
'Public place service usage state is unavailable',
|
||||
{ cause: reserved.error },
|
||||
);
|
||||
if (reserved.status === 'cap')
|
||||
throw gateError(
|
||||
'NOMINATIM_DAILY_CAP',
|
||||
'Daily place lookup allowance is used',
|
||||
);
|
||||
if (reserved.status === 'paused')
|
||||
throw gateError('NOMINATIM_PAUSED', 'Place service is pausing', {
|
||||
retryAfterMs: reserved.retryAfterMs,
|
||||
});
|
||||
if (reserved.status === 'abandoned')
|
||||
throw gateError('NOMINATIM_ABANDONED', 'Place search was abandoned');
|
||||
const waitMs = Math.max(0, reserved.startAt - now());
|
||||
if (waitMs) await sleep(waitMs);
|
||||
if (signal?.aborted || now() - queuedAt > maxWaitMs)
|
||||
throw gateError('NOMINATIM_ABANDONED', 'Place search was abandoned');
|
||||
// A refusal from another process after this reservation still stops it.
|
||||
// The reserved daily slot remains spent; it cannot safely be reassigned.
|
||||
if (now() < pauseEnd(config)) throw pausedError(config);
|
||||
}
|
||||
|
||||
/**
|
||||
* Queue one JSON request for `buildUrl(endpoint)`. At most one retry, and
|
||||
* only after a transport failure; a failed retry pauses the gate.
|
||||
*
|
||||
* @param {(endpoint: string) => string} buildUrl
|
||||
* @param {{signal?: AbortSignal, maxBytes?: number}} [options]
|
||||
*/
|
||||
function requestJson(buildUrl, { signal, maxBytes = 2 * 1024 * 1024 } = {}) {
|
||||
const config = current();
|
||||
const refused = admissionError(config);
|
||||
if (refused) return Promise.reject(refused);
|
||||
if (signal?.aborted)
|
||||
return Promise.reject(
|
||||
gateError('NOMINATIM_ABANDONED', 'Place search was abandoned'),
|
||||
);
|
||||
if (pending >= maxPending)
|
||||
return Promise.reject(
|
||||
gateError('NOMINATIM_QUEUE_FULL', 'Place search queue is full'),
|
||||
);
|
||||
pending += 1;
|
||||
const queuedAt = now();
|
||||
const url = buildUrl(config.endpoint);
|
||||
const task = queue.then(async () => {
|
||||
try {
|
||||
await takeTurn(config, queuedAt, signal);
|
||||
try {
|
||||
return await send(url, config, maxBytes, signal);
|
||||
} catch (error) {
|
||||
if (!error?.transport) throw error;
|
||||
await takeTurn(config, now(), signal);
|
||||
try {
|
||||
return await send(url, config, maxBytes, signal);
|
||||
} catch (retryError) {
|
||||
if (retryError?.transport) pause(OUTAGE_BACKOFF_MS, config);
|
||||
throw retryError;
|
||||
}
|
||||
}
|
||||
} finally {
|
||||
pending -= 1;
|
||||
}
|
||||
});
|
||||
queue = task.catch(() => null);
|
||||
return task;
|
||||
}
|
||||
|
||||
return {
|
||||
requestJson,
|
||||
settings: current,
|
||||
/** Share public admission, daily count and pauses through one install's file. */
|
||||
persistUsage(file) {
|
||||
usageStore = createGateStateStore({ file });
|
||||
},
|
||||
stats: () => ({
|
||||
upstreamRequests,
|
||||
pending,
|
||||
pausedForMs: Math.max(0, pauseEnd(current()) - now()),
|
||||
usedToday: usageStore.count(utcDay(now())),
|
||||
persistence: usageStore.status?.() || null,
|
||||
}),
|
||||
};
|
||||
}
|
||||
|
||||
let sharedGate = createNominatimGate();
|
||||
|
||||
/** The process-wide gate: search and outlines share one budget and pacer. */
|
||||
export function sharedNominatimGate() {
|
||||
return sharedGate;
|
||||
}
|
||||
|
||||
/** Replace the process-wide gate (tests, embedders with their own transport). */
|
||||
export function setSharedNominatimGate(gate) {
|
||||
sharedGate = gate || createNominatimGate();
|
||||
return sharedGate;
|
||||
}
|
||||
|
||||
/** Codes that mean "not now" rather than "no such place". */
|
||||
export function isNominatimBusy(error) {
|
||||
return [
|
||||
'NOMINATIM_QUEUE_FULL',
|
||||
'NOMINATIM_ABANDONED',
|
||||
'NOMINATIM_PAUSED',
|
||||
'NOMINATIM_REFUSED',
|
||||
'NOMINATIM_DAILY_CAP',
|
||||
].includes(error?.code);
|
||||
}
|
||||
@@ -0,0 +1,595 @@
|
||||
import { normalizePlaceName } from './nominatimCache.js';
|
||||
|
||||
/**
|
||||
* Which Nominatim results may answer which voice ask, and how their polygons
|
||||
* are bounded. Pure: no I/O.
|
||||
*
|
||||
* A search returns whatever matches the words best. "Mission District" can
|
||||
* come back as a car-charging station and "Bandra" as a railway station, so a
|
||||
* result is accepted only when its OSM class (and, for broad classes, its
|
||||
* type) suits the kind of thing that was asked for, it carries a polygon, and
|
||||
* its name actually answers the words.
|
||||
*/
|
||||
|
||||
/** Ask kinds the outline route answers. */
|
||||
export const OUTLINE_KINDS = Object.freeze([
|
||||
'city',
|
||||
'admin',
|
||||
'neighborhood',
|
||||
'landmark',
|
||||
'building',
|
||||
]);
|
||||
|
||||
/** Simplification tolerance (degrees) sent upstream, per ask kind. */
|
||||
export const OUTLINE_POLYGON_THRESHOLD = Object.freeze({
|
||||
city: 0.0005,
|
||||
admin: 0.0005,
|
||||
neighborhood: 0.0005,
|
||||
landmark: 0.00005,
|
||||
building: 0.00005,
|
||||
});
|
||||
|
||||
/** Total vertices returned for one outline, and parts/holes kept. */
|
||||
export const OUTLINE_MAX_VERTICES = 6000;
|
||||
const OUTLINE_MAX_PARTS = 24;
|
||||
const OUTLINE_MAX_HOLES = 16;
|
||||
|
||||
const ANY = '*';
|
||||
|
||||
const SETTLEMENT_PLACES = [
|
||||
'city',
|
||||
'town',
|
||||
'village',
|
||||
'municipality',
|
||||
'borough',
|
||||
'hamlet',
|
||||
];
|
||||
|
||||
const CIVIC_AMENITIES = [
|
||||
'university',
|
||||
'college',
|
||||
'school',
|
||||
'kindergarten',
|
||||
'hospital',
|
||||
'clinic',
|
||||
'place_of_worship',
|
||||
'monastery',
|
||||
'townhall',
|
||||
'courthouse',
|
||||
'library',
|
||||
'theatre',
|
||||
'cinema',
|
||||
'arts_centre',
|
||||
'community_centre',
|
||||
'conference_centre',
|
||||
'events_venue',
|
||||
'exhibition_centre',
|
||||
'music_venue',
|
||||
'planetarium',
|
||||
'marketplace',
|
||||
'prison',
|
||||
'grave_yard',
|
||||
'fire_station',
|
||||
'police',
|
||||
'embassy',
|
||||
'public_building',
|
||||
];
|
||||
|
||||
const LANDMARK_TOURISM = [
|
||||
'attraction',
|
||||
'theme_park',
|
||||
'zoo',
|
||||
'aquarium',
|
||||
'museum',
|
||||
'gallery',
|
||||
'viewpoint',
|
||||
'camp_site',
|
||||
'hotel',
|
||||
'resort',
|
||||
];
|
||||
|
||||
const MAN_MADE_STRUCTURES = [
|
||||
'tower',
|
||||
'lighthouse',
|
||||
'pier',
|
||||
'bridge',
|
||||
'works',
|
||||
'dam',
|
||||
'observatory',
|
||||
'wastewater_plant',
|
||||
'water_works',
|
||||
'breakwater',
|
||||
];
|
||||
|
||||
/**
|
||||
* Accepted `class → types` per ask kind. `*` accepts every type of that
|
||||
* class. Anything absent is rejected: roads, railways, stations, shops,
|
||||
* guideposts and single facilities never answer an area ask.
|
||||
*/
|
||||
export const OUTLINE_ACCEPT = Object.freeze({
|
||||
city: {
|
||||
boundary: ['administrative'],
|
||||
place: SETTLEMENT_PLACES,
|
||||
},
|
||||
admin: {
|
||||
boundary: ['administrative'],
|
||||
place: ['country', 'state', 'province', 'region', 'county', 'district'],
|
||||
},
|
||||
neighborhood: {
|
||||
boundary: ['administrative'],
|
||||
place: ['suburb', 'quarter', 'neighbourhood', 'borough', 'city_block'],
|
||||
},
|
||||
landmark: {
|
||||
leisure: ANY,
|
||||
landuse: ANY,
|
||||
amenity: CIVIC_AMENITIES,
|
||||
tourism: LANDMARK_TOURISM,
|
||||
building: ANY,
|
||||
natural: ANY,
|
||||
water: ANY,
|
||||
waterway: ['riverbank', 'dock'],
|
||||
historic: ANY,
|
||||
man_made: MAN_MADE_STRUCTURES,
|
||||
office: ['government', 'diplomatic'],
|
||||
aeroway: ['aerodrome', 'heliport', 'terminal'],
|
||||
military: ANY,
|
||||
boundary: ['national_park', 'protected_area'],
|
||||
place: ['island', 'islet', 'square'],
|
||||
},
|
||||
// Single structures only: no campuses, grounds, parks or camp sites.
|
||||
building: {
|
||||
building: ANY,
|
||||
amenity: [
|
||||
'place_of_worship',
|
||||
'townhall',
|
||||
'courthouse',
|
||||
'library',
|
||||
'theatre',
|
||||
'cinema',
|
||||
'arts_centre',
|
||||
'community_centre',
|
||||
'conference_centre',
|
||||
'exhibition_centre',
|
||||
'music_venue',
|
||||
'planetarium',
|
||||
'fire_station',
|
||||
'police',
|
||||
'embassy',
|
||||
'public_building',
|
||||
],
|
||||
tourism: ['museum', 'gallery', 'hotel', 'attraction'],
|
||||
historic: ['building', 'castle', 'church', 'fort', 'palace', 'monastery'],
|
||||
man_made: ['tower', 'lighthouse', 'observatory'],
|
||||
office: ANY,
|
||||
},
|
||||
});
|
||||
|
||||
/**
|
||||
* Administrative evidence per ask kind: an address type, or a Nominatim
|
||||
* place_rank band (country 4, state 8, county 10-12, city 12-16, suburb 18-20,
|
||||
* neighbourhood 20-22). An administrative boundary with neither is refused.
|
||||
*/
|
||||
const ADMIN_EVIDENCE = Object.freeze({
|
||||
city: {
|
||||
types: ['city', 'town', 'village', 'municipality', 'borough', 'hamlet'],
|
||||
ranks: [12, 18],
|
||||
},
|
||||
admin: {
|
||||
types: [
|
||||
'country',
|
||||
'state',
|
||||
'province',
|
||||
'region',
|
||||
'county',
|
||||
'district',
|
||||
'state_district',
|
||||
],
|
||||
ranks: [2, 14],
|
||||
},
|
||||
neighborhood: {
|
||||
types: ['suburb', 'quarter', 'neighbourhood', 'city_district', 'borough'],
|
||||
ranks: [17, 24],
|
||||
},
|
||||
});
|
||||
|
||||
const KNOWN_ADMIN_TYPES = new Set(
|
||||
Object.values(ADMIN_EVIDENCE).flatMap((evidence) => evidence.types),
|
||||
);
|
||||
|
||||
/**
|
||||
* Nominatim address types whose meaning is stable enough for the area consumer's
|
||||
* country/admin1/admin2 contract. `region`, `district` and numeric admin levels
|
||||
* vary by country, so they remain unverified instead of being guessed.
|
||||
*/
|
||||
const VERIFIED_ADMIN_LEVEL = Object.freeze({
|
||||
country: 'country',
|
||||
state: 'admin1',
|
||||
province: 'admin1',
|
||||
county: 'admin2',
|
||||
state_district: 'admin2',
|
||||
});
|
||||
|
||||
const EXPLICIT_NAME_LEVEL = Object.freeze({
|
||||
country: 'country',
|
||||
nation: 'country',
|
||||
province: 'admin1',
|
||||
state: 'admin1',
|
||||
region: 'admin1',
|
||||
oblast: 'admin1',
|
||||
prefecture: 'admin1',
|
||||
canton: 'admin1',
|
||||
governorate: 'admin1',
|
||||
county: 'admin2',
|
||||
district: 'admin2',
|
||||
department: 'admin2',
|
||||
zone: 'retired',
|
||||
});
|
||||
|
||||
function explicitNameEvidence(value) {
|
||||
const normalized = String(value || '')
|
||||
.normalize('NFKC')
|
||||
.toLocaleLowerCase()
|
||||
.replace(/[^\p{L}\p{M}\p{N}]+/gu, ' ')
|
||||
.trim();
|
||||
const prefix = normalized.match(
|
||||
/^(country|nation|province|state|region|zone|oblast|prefecture|canton|governorate|county|district|department)\s+of\b/,
|
||||
)?.[1];
|
||||
const suffix = normalized.match(
|
||||
/\b(country|nation|province|state|region|zone|oblast|prefecture|canton|governorate|county|district|department)$/,
|
||||
)?.[1];
|
||||
const levels = new Set(
|
||||
[prefix, suffix]
|
||||
.filter(Boolean)
|
||||
.map((type) => EXPLICIT_NAME_LEVEL[type])
|
||||
.filter(Boolean),
|
||||
);
|
||||
if (levels.size > 1) return { contradictory: true, level: null };
|
||||
return { contradictory: false, level: levels.values().next().value || null };
|
||||
}
|
||||
|
||||
/**
|
||||
* Verified administrative identity carried by the selected result itself.
|
||||
* Never infer it from the request, desired kind, rank, or a separate geocoder
|
||||
* anchor. A retired name is preserved without a level so downstream policy can
|
||||
* reject it explicitly; every other contradiction stays wholly unverified.
|
||||
*/
|
||||
function selectedAdminIdentity(kind, row) {
|
||||
if (kind !== 'admin') return null;
|
||||
const adminLevel =
|
||||
VERIFIED_ADMIN_LEVEL[String(row?.addresstype || '').toLowerCase()];
|
||||
const authoritativeAdminArea = String(row?.name || '').trim();
|
||||
if (!adminLevel || !authoritativeAdminArea) return null;
|
||||
// Validate the complete authoritative name. `adminArea` is truncated only
|
||||
// for display after every prefix/suffix signal has been considered.
|
||||
const { contradictory, level: nameLevel } = explicitNameEvidence(
|
||||
authoritativeAdminArea,
|
||||
);
|
||||
if (contradictory) return null;
|
||||
const adminArea = authoritativeAdminArea.slice(0, 160);
|
||||
if (nameLevel === 'retired') return { adminArea };
|
||||
if (nameLevel && nameLevel !== adminLevel) return null;
|
||||
return { adminArea, adminLevel };
|
||||
}
|
||||
|
||||
/** Whether an administrative boundary is at the level the ask names. */
|
||||
export function adminLevelFits(kind, row) {
|
||||
const evidence = ADMIN_EVIDENCE[kind];
|
||||
if (!evidence) return true;
|
||||
const type = String(row?.addresstype || '').toLowerCase();
|
||||
if (type && evidence.types.includes(type)) return true;
|
||||
// A recognized type is direct evidence. Rank is only a fallback when the
|
||||
// upstream answer omits the type or supplies a value we do not understand;
|
||||
// the overlapping county/city rank bands must not override `county`.
|
||||
if (KNOWN_ADMIN_TYPES.has(type)) return false;
|
||||
const rank = Number(row?.place_rank);
|
||||
if (Number.isFinite(rank))
|
||||
return rank >= evidence.ranks[0] && rank <= evidence.ranks[1];
|
||||
return false;
|
||||
}
|
||||
|
||||
/** Largest footprint (m²) a non-`building` result may have for a building ask. */
|
||||
const BUILDING_MAX_AREA_M2 = 150_000;
|
||||
|
||||
/** Approximate area (m²) of a [lon, lat] ring. */
|
||||
function ringAreaM2(ring) {
|
||||
const lat0 = ring[0][1];
|
||||
const k = 111_320 * 111_320 * Math.cos((lat0 * Math.PI) / 180);
|
||||
return ringArea(ring) * k;
|
||||
}
|
||||
|
||||
/** Whether a result's class/type suits the ask kind. */
|
||||
export function acceptsOutlineClass(kind, osmClass, osmType) {
|
||||
const rules = OUTLINE_ACCEPT[kind];
|
||||
if (!rules) return false;
|
||||
const types = rules[String(osmClass || '').toLowerCase()];
|
||||
if (!types) return false;
|
||||
return types === ANY || types.includes(String(osmType || '').toLowerCase());
|
||||
}
|
||||
|
||||
const STOP_WORDS = new Set(['the', 'of', 'de', 'la', 'le', 'el', 'and']);
|
||||
|
||||
/** Scripts written without spaces between words. */
|
||||
const UNSPACED_SCRIPT_RE =
|
||||
/[\p{Script=Han}\p{Script=Hiragana}\p{Script=Katakana}\p{Script=Hangul}\p{Script=Thai}\p{Script=Lao}\p{Script=Khmer}\p{Script=Myanmar}]/u;
|
||||
|
||||
/** Whether `needle` occurs in `hay` as whole consecutive words. */
|
||||
function containsWords(hay, needle) {
|
||||
for (let i = 0; i + needle.length <= hay.length; i++)
|
||||
if (needle.every((word, j) => hay[i + j] === word)) return true;
|
||||
return false;
|
||||
}
|
||||
|
||||
/** Significant words of the part of a query before the first comma. */
|
||||
function queryWords(query) {
|
||||
const head = String(query ?? '').split(',')[0];
|
||||
return normalizePlaceName(head)
|
||||
.split(' ')
|
||||
.filter((word) => word && !STOP_WORDS.has(word));
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether a result's names answer the query. Most of the query's words must
|
||||
* appear among one name's words, or one name must contain the whole query
|
||||
* (for scripts written without spaces).
|
||||
*/
|
||||
export function outlineNameMatches(row, query) {
|
||||
const words = queryWords(query);
|
||||
if (!words.length) return false;
|
||||
const whole = words.join(' ');
|
||||
const names = new Set();
|
||||
const add = (value) => {
|
||||
const name = normalizePlaceName(value);
|
||||
if (name) names.add(name);
|
||||
};
|
||||
add(row?.name);
|
||||
add(String(row?.display_name || '').split(',')[0]);
|
||||
if (row?.namedetails && typeof row.namedetails === 'object')
|
||||
for (const value of Object.values(row.namedetails)) add(value);
|
||||
const unspaced = UNSPACED_SCRIPT_RE.test(whole);
|
||||
for (const name of names) {
|
||||
const nameWords = name.split(' ');
|
||||
// Whole words only: "York" never matches "Yorkshire Park".
|
||||
if (containsWords(nameWords, words)) return true;
|
||||
// Scripts without spaces compare by characters instead.
|
||||
if (unspaced && name.replace(/ /g, '').includes(whole.replace(/ /g, '')))
|
||||
return true;
|
||||
const nameSet = new Set(nameWords);
|
||||
const hits = words.filter((word) => nameSet.has(word)).length;
|
||||
if (words.length > 1 && hits >= Math.ceil(words.length * 0.6)) return true;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
const finiteLon = (value) =>
|
||||
Number.isFinite(value) && value >= -180 && value <= 180;
|
||||
const finiteLat = (value) =>
|
||||
Number.isFinite(value) && value >= -90 && value <= 90;
|
||||
|
||||
/** A valid closed ring of [lon, lat] numbers, or null. */
|
||||
function cleanRing(ring) {
|
||||
if (!Array.isArray(ring)) return null;
|
||||
const out = [];
|
||||
for (const point of ring) {
|
||||
if (!Array.isArray(point)) return null;
|
||||
const lon = Number(point[0]);
|
||||
const lat = Number(point[1]);
|
||||
if (!finiteLon(lon) || !finiteLat(lat)) return null;
|
||||
out.push([lon, lat]);
|
||||
}
|
||||
if (out.length < 4) return null;
|
||||
const [first, last] = [out[0], out.at(-1)];
|
||||
if (first[0] !== last[0] || first[1] !== last[1]) out.push([...first]);
|
||||
return out;
|
||||
}
|
||||
|
||||
/** Planar ring area in squared degrees (only used to rank parts). */
|
||||
function ringArea(ring) {
|
||||
let sum = 0;
|
||||
for (let i = 0, j = ring.length - 1; i < ring.length; j = i++)
|
||||
sum += (ring[j][0] - ring[i][0]) * (ring[j][1] + ring[i][1]);
|
||||
return Math.abs(sum / 2);
|
||||
}
|
||||
|
||||
/** Polygon/MultiPolygon GeoJSON → `[[outer, ...holes], …]`, largest first. */
|
||||
export function polygonsFromGeoJson(geometry) {
|
||||
const raw =
|
||||
geometry?.type === 'Polygon'
|
||||
? [geometry.coordinates]
|
||||
: geometry?.type === 'MultiPolygon'
|
||||
? geometry.coordinates
|
||||
: null;
|
||||
if (!Array.isArray(raw)) return null;
|
||||
const polygons = [];
|
||||
for (const polygon of raw) {
|
||||
if (!Array.isArray(polygon)) continue;
|
||||
const outer = cleanRing(polygon[0]);
|
||||
if (!outer) continue;
|
||||
const holes = polygon
|
||||
.slice(1)
|
||||
.map(cleanRing)
|
||||
.filter(Boolean)
|
||||
.sort((a, b) => ringArea(b) - ringArea(a))
|
||||
.slice(0, OUTLINE_MAX_HOLES);
|
||||
polygons.push({ rings: [outer, ...holes], area: ringArea(outer) });
|
||||
}
|
||||
if (!polygons.length) return null;
|
||||
polygons.sort((a, b) => b.area - a.area);
|
||||
return polygons.slice(0, OUTLINE_MAX_PARTS).map((entry) => entry.rings);
|
||||
}
|
||||
|
||||
/** Iterative Douglas–Peucker on a closed ring (keeps it closed, ≥ 4 points). */
|
||||
function simplifyClosedRing(ring, tolerance) {
|
||||
if (ring.length <= 8 || tolerance <= 0) return ring;
|
||||
const keep = new Uint8Array(ring.length);
|
||||
keep[0] = keep[ring.length - 1] = 1;
|
||||
const stack = [[0, ring.length - 1]];
|
||||
const tol2 = tolerance * tolerance;
|
||||
while (stack.length) {
|
||||
const [start, end] = stack.pop();
|
||||
const [ax, ay] = ring[start];
|
||||
const [bx, by] = ring[end];
|
||||
const dx = bx - ax;
|
||||
const dy = by - ay;
|
||||
const len2 = dx * dx + dy * dy;
|
||||
let best = -1;
|
||||
let bestD = tol2;
|
||||
for (let i = start + 1; i < end; i++) {
|
||||
const [px, py] = ring[i];
|
||||
let t = len2 ? ((px - ax) * dx + (py - ay) * dy) / len2 : 0;
|
||||
t = Math.max(0, Math.min(1, t));
|
||||
const ex = ax + t * dx - px;
|
||||
const ey = ay + t * dy - py;
|
||||
const d = ex * ex + ey * ey;
|
||||
if (d > bestD) {
|
||||
bestD = d;
|
||||
best = i;
|
||||
}
|
||||
}
|
||||
if (best > 0) {
|
||||
keep[best] = 1;
|
||||
stack.push([start, best], [best, end]);
|
||||
}
|
||||
}
|
||||
const out = ring.filter((_point, index) => keep[index]);
|
||||
return out.length >= 4 ? out : ring;
|
||||
}
|
||||
|
||||
const countVertices = (polygons) =>
|
||||
polygons.reduce(
|
||||
(sum, rings) => sum + rings.reduce((n, ring) => n + ring.length, 0),
|
||||
0,
|
||||
);
|
||||
|
||||
/** Simplify until the whole outline fits the vertex cap. */
|
||||
export function capOutlineVertices(
|
||||
polygons,
|
||||
maxVertices = OUTLINE_MAX_VERTICES,
|
||||
) {
|
||||
let current = polygons;
|
||||
let tolerance = 0.00002;
|
||||
for (let round = 0; round < 12; round++) {
|
||||
if (countVertices(current) <= maxVertices) return current;
|
||||
current = polygons.map((rings) =>
|
||||
rings.map((ring) => simplifyClosedRing(ring, tolerance)),
|
||||
);
|
||||
tolerance *= 2;
|
||||
}
|
||||
// Still over: keep the largest parts that fit, outer rings only.
|
||||
const kept = [];
|
||||
let total = 0;
|
||||
for (const rings of current) {
|
||||
const outer = rings[0];
|
||||
if (total + outer.length > maxVertices) break;
|
||||
kept.push([outer]);
|
||||
total += outer.length;
|
||||
}
|
||||
return kept.length ? kept : null;
|
||||
}
|
||||
|
||||
/**
|
||||
* The first result within the limit that suits the ask, names the place and
|
||||
* carries a polygon — or null with the reason each result was passed over.
|
||||
*
|
||||
* @param {object[]} rows Nominatim jsonv2 results, in upstream order.
|
||||
* @param {{kind: string, query: string}} ask
|
||||
*/
|
||||
export function selectOutlineResult(rows, { kind, query }) {
|
||||
const skipped = [];
|
||||
for (const row of Array.isArray(rows) ? rows : []) {
|
||||
const osmClass = row?.category || row?.class;
|
||||
const osmType = row?.type;
|
||||
const label = `${osmClass}=${osmType}`;
|
||||
if (!acceptsOutlineClass(kind, osmClass, osmType)) {
|
||||
skipped.push(`${label}: wrong type`);
|
||||
continue;
|
||||
}
|
||||
if (
|
||||
String(osmClass).toLowerCase() === 'boundary' &&
|
||||
String(osmType).toLowerCase() === 'administrative' &&
|
||||
!adminLevelFits(kind, row)
|
||||
) {
|
||||
skipped.push(`${label}: level`);
|
||||
continue;
|
||||
}
|
||||
if (!outlineNameMatches(row, query)) {
|
||||
skipped.push(`${label}: name`);
|
||||
continue;
|
||||
}
|
||||
const parsed = polygonsFromGeoJson(row?.geojson);
|
||||
if (!parsed) {
|
||||
skipped.push(`${label}: no polygon`);
|
||||
continue;
|
||||
}
|
||||
// A building ask needs building evidence: the building class, or a
|
||||
// footprint no larger than a large building.
|
||||
if (
|
||||
kind === 'building' &&
|
||||
String(osmClass).toLowerCase() !== 'building' &&
|
||||
ringAreaM2(parsed[0][0]) > BUILDING_MAX_AREA_M2
|
||||
) {
|
||||
skipped.push(`${label}: not a building`);
|
||||
continue;
|
||||
}
|
||||
const polygons = capOutlineVertices(parsed);
|
||||
if (!polygons) {
|
||||
skipped.push(`${label}: too large`);
|
||||
continue;
|
||||
}
|
||||
const adminIdentity = selectedAdminIdentity(kind, row);
|
||||
const lat = Number(row.lat);
|
||||
const lon = Number(row.lon);
|
||||
return {
|
||||
outline: {
|
||||
name: String(row.name || '').slice(0, 160) || null,
|
||||
displayName: String(row.display_name || '').slice(0, 300) || null,
|
||||
class: String(osmClass),
|
||||
type: String(osmType),
|
||||
osm:
|
||||
row.osm_type && row.osm_id != null
|
||||
? `${row.osm_type}/${row.osm_id}`
|
||||
: null,
|
||||
...(adminIdentity || {}),
|
||||
center: finiteLat(lat) && finiteLon(lon) ? { lat, lon } : null,
|
||||
polygons,
|
||||
},
|
||||
skipped,
|
||||
};
|
||||
}
|
||||
return { outline: null, skipped };
|
||||
}
|
||||
|
||||
/**
|
||||
* Coarse view bias: the centre is snapped to a grid (1° for places, 0.05°
|
||||
* for small grounds) so small camera moves reuse the same cache entry, and
|
||||
* the viewbox sent upstream is built from the snapped centre.
|
||||
*/
|
||||
export function outlineBias(kind, lat, lon) {
|
||||
// Missing is missing: Number(null) and Number('') would read as (0, 0).
|
||||
const given = (value) =>
|
||||
value !== null &&
|
||||
value !== undefined &&
|
||||
(typeof value === 'number' || String(value).trim() !== '');
|
||||
if (!given(lat) || !given(lon)) return null;
|
||||
const latitude = Number(lat);
|
||||
const longitude = Number(lon);
|
||||
if (!finiteLat(latitude) || !finiteLon(longitude)) return null;
|
||||
const small = kind === 'landmark' || kind === 'building';
|
||||
const step = small ? 0.05 : 1;
|
||||
const half = small ? 0.25 : 2;
|
||||
const snap = (value) => Math.round(value / step) * step;
|
||||
const cLat = Number(snap(latitude).toFixed(2));
|
||||
const cLon = Number(snap(longitude).toFixed(2));
|
||||
const clampLat = (value) => Math.max(-90, Math.min(90, value));
|
||||
const clampLon = (value) => Math.max(-180, Math.min(180, value));
|
||||
return {
|
||||
key: `${cLat},${cLon}`,
|
||||
viewbox: [
|
||||
clampLon(cLon - half),
|
||||
clampLat(cLat + half),
|
||||
clampLon(cLon + half),
|
||||
clampLat(cLat - half),
|
||||
]
|
||||
.map((value) => Number(value.toFixed(4)))
|
||||
.join(','),
|
||||
};
|
||||
}
|
||||
+359
-198
@@ -1,114 +1,69 @@
|
||||
import path from 'node:path';
|
||||
import { makeRateLimiter, clientKey } from '../common/rate-limit.js';
|
||||
import { coalesceProxyRequest } from '../common/http.js';
|
||||
import { fetchRegionalJson } from './http.js';
|
||||
import { naturalRegionAtPoint } from '../../../src/data/naturalEarthRegions.js';
|
||||
import {
|
||||
nominatimToGeocodeResult,
|
||||
nominatimViewboxFromBounds,
|
||||
} from '../../../src/nominatimGeocode.js';
|
||||
|
||||
/**
|
||||
* The usage policy for the public Nominatim instance asks for a User-Agent or
|
||||
* Referer that identifies the application, and states that stock library
|
||||
* agents will not do. Both are sent.
|
||||
*/
|
||||
const NOMINATIM_HEADERS = Object.freeze({
|
||||
'User-Agent':
|
||||
'gods-eye-view/0.1 (+https://github.com/bilawalsidhu/gods-eye-view)',
|
||||
Referer: 'https://github.com/bilawalsidhu/gods-eye-view',
|
||||
});
|
||||
|
||||
/**
|
||||
* Minimum spacing between upstream calls. The policy states an absolute
|
||||
* maximum of one request per second; 1.1 s keeps clock jitter from crossing it.
|
||||
*/
|
||||
const NOMINATIM_MIN_SPACING_MS = 1100;
|
||||
|
||||
/**
|
||||
* How many searches may be waiting for their turn.
|
||||
*
|
||||
* One request per second and an unbounded queue are incompatible: a burst of
|
||||
* searches would keep the upstream busy long after everyone who asked has given
|
||||
* up, which is exactly the load the policy asks callers not to create. Past
|
||||
* this depth a search is refused at once instead of being promised a slot
|
||||
* minutes away.
|
||||
*/
|
||||
const NOMINATIM_MAX_PENDING = 4;
|
||||
|
||||
/**
|
||||
* How long a queued search may wait before it is not worth sending. The
|
||||
* browser gives up well before this, so anything reaching the front later than
|
||||
* this is answering nobody.
|
||||
*/
|
||||
const NOMINATIM_MAX_WAIT_MS = 10_000;
|
||||
|
||||
const NOMINATIM_SEARCH_CACHE_MS = 5 * 60_000;
|
||||
|
||||
const NOMINATIM_SEARCH_MAX_CACHE = 80;
|
||||
import {
|
||||
NOMINATIM_MAX_PENDING,
|
||||
isNominatimBusy,
|
||||
sharedNominatimGate,
|
||||
} from './nominatimGate.js';
|
||||
import {
|
||||
NOMINATIM_CACHE_TTL_MS,
|
||||
createNominatimCache,
|
||||
} from './nominatimCache.js';
|
||||
import {
|
||||
OUTLINE_KINDS,
|
||||
OUTLINE_POLYGON_THRESHOLD,
|
||||
outlineBias,
|
||||
selectOutlineResult,
|
||||
} from './outlineSelect.js';
|
||||
|
||||
const NOMINATIM_SEARCH_MAX_QUERY = 200;
|
||||
|
||||
// The pacer is deliberately module state while everything else here is
|
||||
// per-instance. One request per second is a budget for the whole application,
|
||||
// not for each provider object: two instances each pacing themselves would
|
||||
// send two requests a second between them. Only last-resort forward searches
|
||||
// use this queue.
|
||||
let _nominatimQueue = Promise.resolve();
|
||||
/** Results considered per outline ask: enough to skip a node or a wrong type. */
|
||||
const OUTLINE_RESULT_LIMIT = 3;
|
||||
|
||||
let _nominatimLastRequestAt = 0;
|
||||
/** Largest upstream answer read for an outline (polygons are simplified upstream). */
|
||||
const OUTLINE_MAX_RESPONSE_BYTES = 4 * 1024 * 1024;
|
||||
|
||||
let _nominatimPending = 0;
|
||||
|
||||
/** Raised when the queue is already as deep as it is allowed to get. */
|
||||
function queueFullError() {
|
||||
return Object.assign(new Error('Place search queue is full'), {
|
||||
code: 'NOMINATIM_QUEUE_FULL',
|
||||
});
|
||||
}
|
||||
|
||||
/** Raised when a queued search waited so long that nobody is left to answer. */
|
||||
function abandonedError() {
|
||||
return Object.assign(new Error('Place search was abandoned'), {
|
||||
code: 'NOMINATIM_ABANDONED',
|
||||
});
|
||||
}
|
||||
/** Bump when the cached payload shape changes, so old entries are never read. */
|
||||
const CACHE_SCHEMA = 'v2';
|
||||
|
||||
/**
|
||||
* Run one piece of upstream work, never closer than the policy spacing to the
|
||||
* last one.
|
||||
*
|
||||
* `bounded` applies queue-depth and staleness limits to interactive forward searches.
|
||||
*
|
||||
* @param {() => Promise<unknown>} work
|
||||
* @param {{bounded?: boolean, signal?: AbortSignal}} [options]
|
||||
* The query text actually sent upstream: Unicode-composed, whitespace
|
||||
* collapsed. Nothing else is changed, so two queries share a cache entry only
|
||||
* when they are the same upstream request.
|
||||
*/
|
||||
function enqueueNominatim(work, { bounded = false, signal } = {}) {
|
||||
if (bounded && _nominatimPending >= NOMINATIM_MAX_PENDING)
|
||||
return Promise.reject(queueFullError());
|
||||
if (bounded) _nominatimPending += 1;
|
||||
const queuedAt = Date.now();
|
||||
const task = _nominatimQueue.then(async () => {
|
||||
try {
|
||||
const waitMs = Math.max(
|
||||
0,
|
||||
NOMINATIM_MIN_SPACING_MS - (Date.now() - _nominatimLastRequestAt),
|
||||
);
|
||||
if (waitMs) await new Promise((resolve) => setTimeout(resolve, waitMs));
|
||||
// Nobody is waiting for this any more: sending it would spend the one
|
||||
// request per second the policy allows on an answer with no reader.
|
||||
if (bounded && Date.now() - queuedAt > NOMINATIM_MAX_WAIT_MS)
|
||||
throw abandonedError();
|
||||
if (signal?.aborted) throw abandonedError();
|
||||
_nominatimLastRequestAt = Date.now();
|
||||
return await work();
|
||||
} finally {
|
||||
if (bounded) _nominatimPending -= 1;
|
||||
}
|
||||
});
|
||||
_nominatimQueue = task.catch(() => null);
|
||||
return task;
|
||||
export function upstreamQuery(query) {
|
||||
return String(query ?? '')
|
||||
.normalize('NFC')
|
||||
.replace(/\s+/g, ' ')
|
||||
.trim();
|
||||
}
|
||||
|
||||
/** Cache key: endpoint, schema and the exact upstream parameters. */
|
||||
function requestKey(gate, kind, params) {
|
||||
const endpoint = gate.settings?.()?.endpoint || 'none';
|
||||
return `${CACHE_SCHEMA}|${endpoint}|${kind}|${params}`;
|
||||
}
|
||||
|
||||
function malformed() {
|
||||
return Object.assign(new Error('Place answer was not understood'), {
|
||||
code: 'NOMINATIM_MALFORMED',
|
||||
});
|
||||
}
|
||||
|
||||
/** Where the dev server keeps Nominatim answers and the daily count. */
|
||||
export const NOMINATIM_CACHE_DIR = path.join(
|
||||
process.cwd(),
|
||||
'.gev-cache',
|
||||
'nominatim',
|
||||
);
|
||||
|
||||
/** Construct the offline regional context provider from bundled Natural Earth polygons. */
|
||||
export function createRegionalPlaceProvider() {
|
||||
return (point) => naturalRegionAtPoint(point.latitude, point.longitude);
|
||||
@@ -117,60 +72,132 @@ export function createRegionalPlaceProvider() {
|
||||
export const fetchRegionalPlace = createRegionalPlaceProvider();
|
||||
|
||||
/**
|
||||
* Construct the forward search adapter with a trusted endpoint.
|
||||
* Snap a `"swLat,swLng|neLat,neLng"` view to a coarse grid so small camera
|
||||
* moves reuse one cache entry. The viewbox is a soft preference upstream.
|
||||
*/
|
||||
function coarseBounds(bounds) {
|
||||
const match = String(bounds || '')
|
||||
.trim()
|
||||
.split('|')
|
||||
.map((corner) => corner.split(',').map(Number));
|
||||
if (match.length !== 2 || match.some((c) => c.length !== 2)) return null;
|
||||
const [[swLat, swLng], [neLat, neLng]] = match;
|
||||
if (![swLat, swLng, neLat, neLng].every(Number.isFinite)) return null;
|
||||
const span = Math.max(neLat - swLat, neLng - swLng, 0.01);
|
||||
const step = 2 ** Math.ceil(Math.log2(span));
|
||||
const cLat = Math.round((swLat + neLat) / 2 / (step / 4)) * (step / 4);
|
||||
const cLng = Math.round((swLng + neLng) / 2 / (step / 4)) * (step / 4);
|
||||
const clamp = (v, limit) => Math.max(-limit, Math.min(limit, v));
|
||||
const fix = (v) => Number(v.toFixed(4));
|
||||
return `${fix(clamp(cLat - step / 2, 90))},${fix(clamp(cLng - step / 2, 180))}|${fix(clamp(cLat + step / 2, 90))},${fix(clamp(cLng + step / 2, 180))}`;
|
||||
}
|
||||
|
||||
/**
|
||||
* Construct the forward search adapter over the shared gate.
|
||||
*
|
||||
* The policy asks that results be cached, and warns that a client repeating the
|
||||
* same query may be treated as faulty, so an answer is remembered and identical
|
||||
* searches already in flight share one upstream call rather than queueing
|
||||
* behind each other.
|
||||
* The policy asks that results be cached, and warns that a client repeating
|
||||
* the same query may be treated as faulty, so answers are kept (30 days for a
|
||||
* hit, 24 hours for a genuine no-result; a failure is never kept) and
|
||||
* identical searches already in flight share one upstream call.
|
||||
*/
|
||||
export function createNominatimSearchProvider({
|
||||
endpoint = 'https://nominatim.openstreetmap.org/search',
|
||||
requestJson = fetchRegionalJson,
|
||||
gate = () => sharedNominatimGate(),
|
||||
cache = createNominatimCache(),
|
||||
} = {}) {
|
||||
const cache = new Map();
|
||||
const inFlight = new Map();
|
||||
|
||||
const trimCache = () => {
|
||||
while (cache.size > NOMINATIM_SEARCH_MAX_CACHE) {
|
||||
const oldest = cache.keys().next().value;
|
||||
if (oldest === undefined) break;
|
||||
cache.delete(oldest);
|
||||
}
|
||||
};
|
||||
const currentGate = () => (typeof gate === 'function' ? gate() : gate);
|
||||
|
||||
return async function fetchNominatimSearch(query, bounds, { signal } = {}) {
|
||||
const cacheKey = `${query.toLowerCase()}|${bounds || ''}`;
|
||||
const cached = cache.get(cacheKey);
|
||||
if (cached && Date.now() - cached.cachedAt <= NOMINATIM_SEARCH_CACHE_MS) {
|
||||
return { ...cached.payload, cached: true };
|
||||
}
|
||||
const coarse = coarseBounds(bounds);
|
||||
const params = new URLSearchParams({
|
||||
format: 'jsonv2',
|
||||
q: upstreamQuery(query),
|
||||
addressdetails: '1',
|
||||
limit: '1',
|
||||
'accept-language': 'en',
|
||||
});
|
||||
const viewbox = nominatimViewboxFromBounds(coarse);
|
||||
if (viewbox) params.set('viewbox', viewbox);
|
||||
const gateNow = currentGate();
|
||||
const cacheKey = requestKey(gateNow, 'search', params);
|
||||
const cached = await cache.get(cacheKey);
|
||||
if (cached) return { ...cached.payload, cached: true };
|
||||
const { promise } = coalesceProxyRequest(inFlight, cacheKey, async () => {
|
||||
const params = new URLSearchParams({
|
||||
format: 'jsonv2',
|
||||
q: query,
|
||||
addressdetails: '1',
|
||||
limit: '1',
|
||||
'accept-language': 'en',
|
||||
});
|
||||
const viewbox = nominatimViewboxFromBounds(bounds);
|
||||
if (viewbox) params.set('viewbox', viewbox);
|
||||
const rows = await enqueueNominatim(
|
||||
() =>
|
||||
requestJson(`${endpoint}?${params}`, {
|
||||
headers: NOMINATIM_HEADERS,
|
||||
redirect: 'error',
|
||||
}),
|
||||
{ bounded: true, signal },
|
||||
);
|
||||
const result = nominatimToGeocodeResult(
|
||||
Array.isArray(rows) ? rows[0] : null,
|
||||
const rows = await gateNow.requestJson(
|
||||
(endpoint) => `${endpoint}?${params}`,
|
||||
{ signal },
|
||||
);
|
||||
// Only a well-formed empty list is "no such place"; anything else is a
|
||||
// failure and is never cached.
|
||||
if (!Array.isArray(rows)) throw malformed();
|
||||
const result = nominatimToGeocodeResult(rows[0] ?? null);
|
||||
if (rows.length && !result) throw malformed();
|
||||
const payload = result
|
||||
? { status: 'OK', results: [result] }
|
||||
: { status: 'ZERO_RESULTS', results: [] };
|
||||
cache.set(cacheKey, { payload, cachedAt: Date.now() });
|
||||
trimCache();
|
||||
await cache.set(
|
||||
cacheKey,
|
||||
payload,
|
||||
result ? NOMINATIM_CACHE_TTL_MS.place : NOMINATIM_CACHE_TTL_MS.notFound,
|
||||
);
|
||||
return payload;
|
||||
});
|
||||
return await promise;
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Construct the outline lookup: one guarded search with polygons for one
|
||||
* explicit ask, accepting only a result whose class suits the ask kind.
|
||||
* Found outlines are kept 90 days and genuine no-results 24 hours.
|
||||
*/
|
||||
export function createNominatimOutlineProvider({
|
||||
gate = () => sharedNominatimGate(),
|
||||
cache = createNominatimCache(),
|
||||
} = {}) {
|
||||
const inFlight = new Map();
|
||||
const currentGate = () => (typeof gate === 'function' ? gate() : gate);
|
||||
|
||||
return async function fetchNominatimOutline(
|
||||
{ query, kind, lat, lon },
|
||||
{ signal } = {},
|
||||
) {
|
||||
const bias = outlineBias(kind, lat, lon);
|
||||
const sent = upstreamQuery(query);
|
||||
const params = new URLSearchParams({
|
||||
format: 'jsonv2',
|
||||
q: sent,
|
||||
limit: String(OUTLINE_RESULT_LIMIT),
|
||||
polygon_geojson: '1',
|
||||
polygon_threshold: String(OUTLINE_POLYGON_THRESHOLD[kind]),
|
||||
namedetails: '1',
|
||||
'accept-language': 'en',
|
||||
});
|
||||
if (bias) params.set('viewbox', bias.viewbox);
|
||||
const gateNow = currentGate();
|
||||
const cacheKey = requestKey(gateNow, `outline-${kind}`, params);
|
||||
const cached = await cache.get(cacheKey);
|
||||
if (cached) return { ...cached.payload, cached: true };
|
||||
const { promise } = coalesceProxyRequest(inFlight, cacheKey, async () => {
|
||||
const rows = await gateNow.requestJson(
|
||||
(endpoint) => `${endpoint}?${params}`,
|
||||
{ signal, maxBytes: OUTLINE_MAX_RESPONSE_BYTES },
|
||||
);
|
||||
if (!Array.isArray(rows)) throw malformed();
|
||||
const { outline, skipped } = selectOutlineResult(rows, {
|
||||
kind,
|
||||
query: sent,
|
||||
});
|
||||
const payload = outline
|
||||
? { status: 'OK', outline }
|
||||
: { status: 'ZERO_RESULTS', outline: null, skipped };
|
||||
await cache.set(
|
||||
cacheKey,
|
||||
payload,
|
||||
outline
|
||||
? NOMINATIM_CACHE_TTL_MS.outline
|
||||
: NOMINATIM_CACHE_TTL_MS.notFound,
|
||||
);
|
||||
return payload;
|
||||
});
|
||||
return await promise;
|
||||
@@ -179,75 +206,209 @@ export function createNominatimSearchProvider({
|
||||
|
||||
export const fetchNominatimSearch = createNominatimSearchProvider();
|
||||
|
||||
/** Vite plugin: last-resort place search over the public Nominatim instance. */
|
||||
export function geocodeProxy({ search = fetchNominatimSearch } = {}) {
|
||||
const limiter = makeRateLimiter({
|
||||
/** A browser on another site may not spend this install's allowance. */
|
||||
function crossSite(req) {
|
||||
return (
|
||||
String(req.headers?.['sec-fetch-site'] || '').toLowerCase() === 'cross-site'
|
||||
);
|
||||
}
|
||||
|
||||
function sendJson(res, status, body, headers = {}) {
|
||||
if (res.writableEnded) return;
|
||||
res.writeHead(status, {
|
||||
'Content-Type': 'application/json',
|
||||
'Cache-Control': 'no-store',
|
||||
...headers,
|
||||
});
|
||||
res.end(JSON.stringify(body));
|
||||
}
|
||||
|
||||
/** Map a gate refusal to the route's answer. Never "not found". */
|
||||
function sendFailure(res, error, what) {
|
||||
if (error?.code === 'NOMINATIM_DISABLED') {
|
||||
sendJson(res, 503, {
|
||||
error: `${what} is not configured`,
|
||||
code: 'NOMINATIM_DISABLED',
|
||||
retryable: false,
|
||||
});
|
||||
return;
|
||||
}
|
||||
if (error?.code === 'NOMINATIM_DAILY_CAP') {
|
||||
sendJson(
|
||||
res,
|
||||
429,
|
||||
{
|
||||
error: `${what} daily allowance is used`,
|
||||
code: 'NOMINATIM_DAILY_CAP',
|
||||
retryable: false,
|
||||
},
|
||||
{ 'Retry-After': '3600' },
|
||||
);
|
||||
return;
|
||||
}
|
||||
if (isNominatimBusy(error)) {
|
||||
const seconds = Math.max(5, Math.ceil((error.retryAfterMs || 0) / 1000));
|
||||
sendJson(
|
||||
res,
|
||||
429,
|
||||
{ error: `${what} is busy`, code: error.code },
|
||||
{ 'Retry-After': String(seconds) },
|
||||
);
|
||||
return;
|
||||
}
|
||||
sendJson(res, 503, {
|
||||
error: `${what} is temporarily unavailable`,
|
||||
code: 'NOMINATIM_UNAVAILABLE',
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Vite plugin: last-resort place search (`/api/geocode`) and voice outlines
|
||||
* (`/api/geocode/outline`) over the configured Nominatim endpoint.
|
||||
*
|
||||
* `storageDir` turns on the disk cache and the persisted daily count; the
|
||||
* application composition supplies it, tests and embedders may omit it.
|
||||
*/
|
||||
export function geocodeProxy({
|
||||
search,
|
||||
outline,
|
||||
storageDir = null,
|
||||
gate = null,
|
||||
} = {}) {
|
||||
const currentGate = () => gate || sharedNominatimGate();
|
||||
if (storageDir) {
|
||||
// The shared count and pause live beside, never inside, the answer cache,
|
||||
// so no cache sweep can reset them.
|
||||
currentGate().persistUsage(path.join(storageDir, 'state.json'));
|
||||
const diskCache = createNominatimCache({
|
||||
dir: path.join(storageDir, 'answers'),
|
||||
});
|
||||
void diskCache.sweep();
|
||||
search ||= createNominatimSearchProvider({
|
||||
gate: currentGate,
|
||||
cache: diskCache,
|
||||
});
|
||||
outline ||= createNominatimOutlineProvider({
|
||||
gate: currentGate,
|
||||
cache: diskCache,
|
||||
});
|
||||
}
|
||||
search ||= gate
|
||||
? createNominatimSearchProvider({ gate })
|
||||
: fetchNominatimSearch;
|
||||
outline ||= createNominatimOutlineProvider({ gate: currentGate });
|
||||
|
||||
const searchLimiter = makeRateLimiter({
|
||||
windowMs: 60_000,
|
||||
max: 30,
|
||||
globalMax: 90,
|
||||
});
|
||||
// Outlines are explicit voice asks: a person cannot make many a minute.
|
||||
const outlineLimiter = makeRateLimiter({
|
||||
windowMs: 60_000,
|
||||
max: 12,
|
||||
globalMax: 30,
|
||||
});
|
||||
|
||||
/** Common front door: method, same-site, rate limit, abandonment signal. */
|
||||
function admit(req, res, limiter) {
|
||||
if (req.method !== 'GET') {
|
||||
sendJson(res, 405, { error: 'Method Not Allowed' });
|
||||
return null;
|
||||
}
|
||||
if (crossSite(req)) {
|
||||
sendJson(res, 403, { error: 'Cross-site requests are not accepted' });
|
||||
return null;
|
||||
}
|
||||
if (!limiter(clientKey(req))) {
|
||||
sendJson(
|
||||
res,
|
||||
429,
|
||||
{ error: 'Rate limit exceeded' },
|
||||
{ 'Retry-After': '10' },
|
||||
);
|
||||
return null;
|
||||
}
|
||||
// A browser that gave up is no longer waiting; the queue reads this
|
||||
// before spending its slot.
|
||||
const abandoned = new AbortController();
|
||||
req.on?.('aborted', () => abandoned.abort());
|
||||
res.on?.('close', () => abandoned.abort());
|
||||
return abandoned.signal;
|
||||
}
|
||||
|
||||
async function handleOutline(req, res, url) {
|
||||
const signal = admit(req, res, outlineLimiter);
|
||||
if (!signal) return;
|
||||
const query = String(url.searchParams.get('q') || '').trim();
|
||||
const kind = String(url.searchParams.get('kind') || '');
|
||||
if (
|
||||
!query ||
|
||||
query.length > NOMINATIM_SEARCH_MAX_QUERY ||
|
||||
!OUTLINE_KINDS.includes(kind)
|
||||
) {
|
||||
sendJson(res, 400, {
|
||||
error:
|
||||
'A place query of 1-200 characters and a known kind are required',
|
||||
});
|
||||
return;
|
||||
}
|
||||
try {
|
||||
const payload = await outline(
|
||||
{
|
||||
query,
|
||||
kind,
|
||||
lat: url.searchParams.get('lat'),
|
||||
lon: url.searchParams.get('lon'),
|
||||
},
|
||||
{ signal },
|
||||
);
|
||||
sendJson(
|
||||
res,
|
||||
200,
|
||||
{ status: payload.status, outline: payload.outline || null },
|
||||
{
|
||||
'Cache-Control': payload.cached ? 'private, max-age=300' : 'no-store',
|
||||
},
|
||||
);
|
||||
} catch (error) {
|
||||
sendFailure(res, error, 'Outline lookup');
|
||||
}
|
||||
}
|
||||
|
||||
async function handleSearch(req, res, url) {
|
||||
const signal = admit(req, res, searchLimiter);
|
||||
if (!signal) return;
|
||||
const query = String(url.searchParams.get('q') || '').trim();
|
||||
if (!query || query.length > NOMINATIM_SEARCH_MAX_QUERY) {
|
||||
sendJson(res, 400, {
|
||||
error: 'A place query of 1-200 characters is required',
|
||||
});
|
||||
return;
|
||||
}
|
||||
try {
|
||||
const payload = await search(query, url.searchParams.get('bounds'), {
|
||||
signal,
|
||||
});
|
||||
sendJson(
|
||||
res,
|
||||
200,
|
||||
{ status: payload.status, results: payload.results },
|
||||
{ 'Cache-Control': payload.cached ? 'public, max-age=60' : 'no-store' },
|
||||
);
|
||||
} catch (error) {
|
||||
sendFailure(res, error, 'Place search');
|
||||
}
|
||||
}
|
||||
|
||||
function install(middlewares) {
|
||||
middlewares.use('/api/geocode', async (req, res) => {
|
||||
if (req.method !== 'GET') {
|
||||
res.writeHead(405, { 'Content-Type': 'application/json' });
|
||||
res.end(JSON.stringify({ error: 'Method Not Allowed' }));
|
||||
return;
|
||||
}
|
||||
if (!limiter(clientKey(req))) {
|
||||
res.writeHead(429, {
|
||||
'Content-Type': 'application/json',
|
||||
'Retry-After': '10',
|
||||
});
|
||||
res.end(JSON.stringify({ error: 'Rate limit exceeded' }));
|
||||
return;
|
||||
}
|
||||
middlewares.use('/api/geocode', async (req, res, next) => {
|
||||
const url = new URL(req.url || '', 'http://localhost');
|
||||
const query = String(url.searchParams.get('q') || '').trim();
|
||||
if (!query || query.length > NOMINATIM_SEARCH_MAX_QUERY) {
|
||||
res.writeHead(400, { 'Content-Type': 'application/json' });
|
||||
res.end(
|
||||
JSON.stringify({
|
||||
error: 'A place query of 1-200 characters is required',
|
||||
}),
|
||||
);
|
||||
return;
|
||||
}
|
||||
// A browser that gave up is no longer waiting; the queue reads this
|
||||
// before spending its slot.
|
||||
const abandoned = new AbortController();
|
||||
req.on?.('aborted', () => abandoned.abort());
|
||||
res.on?.('close', () => abandoned.abort());
|
||||
try {
|
||||
const payload = await search(query, url.searchParams.get('bounds'), {
|
||||
signal: abandoned.signal,
|
||||
});
|
||||
if (res.writableEnded) return;
|
||||
res.writeHead(200, {
|
||||
'Content-Type': 'application/json',
|
||||
'Cache-Control': payload.cached ? 'public, max-age=60' : 'no-store',
|
||||
});
|
||||
res.end(
|
||||
JSON.stringify({ status: payload.status, results: payload.results }),
|
||||
);
|
||||
} catch (error) {
|
||||
if (res.writableEnded) return;
|
||||
const busy =
|
||||
error?.code === 'NOMINATIM_QUEUE_FULL' ||
|
||||
error?.code === 'NOMINATIM_ABANDONED';
|
||||
res.writeHead(busy ? 429 : 503, {
|
||||
'Content-Type': 'application/json',
|
||||
'Cache-Control': 'no-store',
|
||||
...(busy ? { 'Retry-After': '5' } : {}),
|
||||
});
|
||||
res.end(
|
||||
JSON.stringify({
|
||||
error: busy
|
||||
? 'Place search is busy'
|
||||
: 'Place search is temporarily unavailable',
|
||||
}),
|
||||
);
|
||||
}
|
||||
if (url.pathname === '/outline') return handleOutline(req, res, url);
|
||||
if (url.pathname === '/' || url.pathname === '')
|
||||
return handleSearch(req, res, url);
|
||||
if (typeof next === 'function') return next();
|
||||
sendJson(res, 404, { error: 'Not Found' });
|
||||
});
|
||||
}
|
||||
|
||||
|
||||
@@ -2,9 +2,7 @@ import { fileURLToPath } from 'node:url';
|
||||
import { defineConfig, loadEnv } from 'vite';
|
||||
import { createBrowserViteConfig } from '../../build/vite.js';
|
||||
import { localProviderPlugins } from '../providers/local.js';
|
||||
import { localMcpPlugin } from '../mcp/plugin.js';
|
||||
import { apiNotFoundPlugin } from './api-not-found.js';
|
||||
import { standaloneVoiceTools } from './voiceTools.js';
|
||||
|
||||
const root = fileURLToPath(new URL('../../', import.meta.url));
|
||||
|
||||
@@ -15,13 +13,10 @@ export default defineConfig(({ command, mode }) => {
|
||||
if (process.env[key] === undefined) process.env[key] = value;
|
||||
}
|
||||
return createBrowserViteConfig({
|
||||
plugins: [
|
||||
...localProviderPlugins({ realtime: { tools: standaloneVoiceTools() } }),
|
||||
localMcpPlugin(),
|
||||
apiNotFoundPlugin(),
|
||||
],
|
||||
plugins: [...localProviderPlugins(), apiNotFoundPlugin()],
|
||||
googleApiKey: process.env.GOOGLE_MAPS_API_KEY,
|
||||
cesiumToken: process.env.CESIUM_ION_TOKEN,
|
||||
photonUrl: process.env.PHOTON_URL,
|
||||
host: process.env.HOST,
|
||||
port: process.env.PORT,
|
||||
command,
|
||||
|
||||
@@ -1,18 +0,0 @@
|
||||
/**
|
||||
* The voice session's tools for the standalone server: the app actions plus
|
||||
* the catalog tools voice offers (see src/tools/surfaces.js), which the
|
||||
* browser runs through the same catalog.
|
||||
*/
|
||||
|
||||
import {
|
||||
coreTools,
|
||||
toFunctionTools,
|
||||
toolsForSurface,
|
||||
} from '../../src/tools/index.js';
|
||||
import { realtimeSessionTools } from '../providers/openai/tools.js';
|
||||
|
||||
export function standaloneVoiceTools() {
|
||||
return realtimeSessionTools(
|
||||
toFunctionTools(toolsForSurface(coreTools, 'voice')),
|
||||
);
|
||||
}
|
||||
@@ -11,6 +11,7 @@ import {
|
||||
sampleGroundHeight,
|
||||
} from './annotationResolver.js';
|
||||
import { ringCentroid } from './drawMode.js';
|
||||
import { phaseTiming } from '../sources/phaseTiming.js';
|
||||
|
||||
// Dev convenience: expose the app's Cesium instance for console/preview probing
|
||||
// (single shared module instance — avoids dual-Cesium state bugs when testing).
|
||||
@@ -270,7 +271,9 @@ export function createAnnotationEngine({
|
||||
results: [],
|
||||
error: 'destroyed',
|
||||
};
|
||||
const list = Array.isArray(requests) ? requests : [requests];
|
||||
const list = (Array.isArray(requests) ? requests : [requests]).map(
|
||||
promoteStreetHighlight,
|
||||
);
|
||||
if (opts.clearPrevious) clear(); // bumps generation + aborts older pending work
|
||||
|
||||
const persist = opts.persist !== false;
|
||||
@@ -677,6 +680,7 @@ export function createAnnotationEngine({
|
||||
if (!annotations.has(anno.id)) return; // mark replaced/removed while resolving
|
||||
anno.pendingOutline = false;
|
||||
anno.outlineUnavailable = isUnavailableCapability(fp);
|
||||
const installStart = performance.now();
|
||||
if (anno.outlineUnavailable) renderer.update(anno);
|
||||
if (fp && !anno.outlineUnavailable) {
|
||||
anno.ring = fp.ring;
|
||||
@@ -718,12 +722,23 @@ export function createAnnotationEngine({
|
||||
continue;
|
||||
if ((other.footprintKind || null) !== (anno.footprintKind || null))
|
||||
continue;
|
||||
if (!ringsEqual(other.ring, anno.ring)) continue;
|
||||
if (!outlinesEqual(other, anno)) continue;
|
||||
annotations.delete(other.id);
|
||||
renderer.remove(other);
|
||||
}
|
||||
}
|
||||
renderer.sync(annotations);
|
||||
if (fp && !anno.outlineUnavailable)
|
||||
phaseTiming(
|
||||
'install',
|
||||
installStart,
|
||||
{
|
||||
kind: anno.footprintKind || null,
|
||||
parts: anno.polygons?.length || (anno.ring ? 1 : 0),
|
||||
points: anno.ring?.length || 0,
|
||||
},
|
||||
'outlines',
|
||||
);
|
||||
// Final outcome → the voice layer (place names ride as structured DATA fields).
|
||||
// 'failed' covers both the definitive no-polygon and an exhausted transient
|
||||
// retry — either way the mark honestly stays a point.
|
||||
@@ -802,7 +817,6 @@ export function createAnnotationEngine({
|
||||
// FULL per-vertex geometry comparison (the live cap is only 120 and this runs solely for
|
||||
// candidates that already matched type+label+anchor, so it's cheap) — so two DIFFERENT
|
||||
// same-length rings/paths at the same anchor are never falsely merged.
|
||||
const ringSame = ringsEqual;
|
||||
const pathSame = (a, b) => {
|
||||
const la = a ? a.length : 0;
|
||||
const lb = b ? b.length : 0;
|
||||
@@ -836,7 +850,7 @@ export function createAnnotationEngine({
|
||||
if (Boolean(ex.synthesized) !== Boolean(anno.synthesized)) continue;
|
||||
if ((ex.footprintKind || null) !== (anno.footprintKind || null))
|
||||
continue;
|
||||
if (!ringSame(ex.ring, anno.ring)) continue;
|
||||
if (!outlinesEqual(ex, anno)) continue;
|
||||
}
|
||||
} else if (anno.type === 'arrow') {
|
||||
if (!near(ex.to, anno.to)) continue;
|
||||
@@ -1310,6 +1324,49 @@ function ringsEqual(a, b) {
|
||||
return true;
|
||||
}
|
||||
|
||||
/**
|
||||
* Whole-outline equality: the main ring AND every part with its holes. Two
|
||||
* outlines sharing an outer ring but differing in parts or holes are two marks.
|
||||
*/
|
||||
function outlinesEqual(a, b) {
|
||||
if (!ringsEqual(a?.ring, b?.ring)) return false;
|
||||
const parts = (anno) =>
|
||||
Array.isArray(anno?.polygons) && anno.polygons.length
|
||||
? anno.polygons
|
||||
: [[anno?.ring || []]];
|
||||
const pa = parts(a);
|
||||
const pb = parts(b);
|
||||
if (pa.length !== pb.length) return false;
|
||||
for (let i = 0; i < pa.length; i++) {
|
||||
if (pa[i].length !== pb[i].length) return false;
|
||||
for (let j = 0; j < pa[i].length; j++)
|
||||
if (!ringsEqual(pa[i][j], pb[i][j])) return false;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
const STREET_WORD_RE =
|
||||
/\b(?:street|st|avenue|ave|boulevard|blvd|road|rd|drive|dr|lane|ln|highway|hwy|parkway|pkwy|alley)\.?$/i;
|
||||
|
||||
/**
|
||||
* "Highlight Congress Avenue" asks for the street, not a dot. A highlight
|
||||
* whose entity is a street (by entityKind or by its name) becomes an area
|
||||
* ask, so it resolves the street outline like any other area, unless the
|
||||
* caller explicitly turned the footprint off.
|
||||
*/
|
||||
export function promoteStreetHighlight(spec) {
|
||||
if (!spec || typeof spec !== 'object' || spec.footprint === false)
|
||||
return spec;
|
||||
if (normalizeType(spec.type) !== 'highlight') return spec;
|
||||
if (spec.entityKind && spec.entityKind !== 'street') return spec;
|
||||
const target = String(spec.target || '').split(',')[0];
|
||||
const street =
|
||||
spec.entityKind === 'street' || STREET_WORD_RE.test(target.trim());
|
||||
return street && target.trim()
|
||||
? { ...spec, type: 'area', entityKind: 'street' }
|
||||
: spec;
|
||||
}
|
||||
|
||||
// --- helpers ----------------------------------------------------------------
|
||||
|
||||
/** The place name(s) a spec attempted to resolve (route waypoints, arrow endpoints, or a single target). */
|
||||
|
||||
@@ -0,0 +1,96 @@
|
||||
// Multipart outline identity and street highlights in the annotation engine.
|
||||
import test from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import {
|
||||
createAnnotationEngine,
|
||||
promoteStreetHighlight,
|
||||
} from './annotationEngine.js';
|
||||
|
||||
function installAnimationFrameStubs(t) {
|
||||
const raf = globalThis.requestAnimationFrame;
|
||||
const caf = globalThis.cancelAnimationFrame;
|
||||
globalThis.requestAnimationFrame = () => 1;
|
||||
globalThis.cancelAnimationFrame = () => {};
|
||||
t.after(() => {
|
||||
if (raf === undefined) delete globalThis.requestAnimationFrame;
|
||||
else globalThis.requestAnimationFrame = raf;
|
||||
if (caf === undefined) delete globalThis.cancelAnimationFrame;
|
||||
else globalThis.cancelAnimationFrame = caf;
|
||||
});
|
||||
}
|
||||
|
||||
const renderer = { add() {}, update() {}, remove() {}, sync() {} };
|
||||
const outer = [
|
||||
[0, 0],
|
||||
[0.01, 0],
|
||||
[0.01, 0.01],
|
||||
[0, 0.01],
|
||||
[0, 0],
|
||||
];
|
||||
const hole = (d) => [
|
||||
[0.004, 0.004],
|
||||
[0.004 + d, 0.004],
|
||||
[0.004 + d, 0.004 + d],
|
||||
[0.004, 0.004],
|
||||
];
|
||||
|
||||
test('outlines sharing an outer ring but not their holes stay two marks', async (t) => {
|
||||
installAnimationFrameStubs(t);
|
||||
const shapes = {
|
||||
a: [[outer, hole(0.001)]],
|
||||
b: [[outer, hole(0.002)]],
|
||||
same: [[outer, hole(0.001)]],
|
||||
};
|
||||
const resolveTarget = async ({ target }) => ({
|
||||
lon: 0.005,
|
||||
lat: 0.005,
|
||||
height: 0,
|
||||
label: target,
|
||||
source: 'fake',
|
||||
ring: outer,
|
||||
polygons: shapes[target],
|
||||
footprintKind: 'area',
|
||||
});
|
||||
const engine = createAnnotationEngine({ viewer: {}, renderer, resolveTarget });
|
||||
await engine.annotate({ type: 'area', target: 'a', label: 'A' });
|
||||
await engine.annotate({ type: 'area', target: 'b', label: 'B' });
|
||||
assert.equal(engine.list().length, 2, 'different holes: different marks');
|
||||
await engine.annotate({ type: 'area', target: 'same', label: 'Same' });
|
||||
assert.equal(engine.list().length, 2, 'identical parts and holes: one mark');
|
||||
});
|
||||
|
||||
test('a street highlight asks for the street outline', () => {
|
||||
assert.deepEqual(
|
||||
promoteStreetHighlight({ type: 'highlight', target: 'Congress Avenue' }),
|
||||
{ type: 'area', target: 'Congress Avenue', entityKind: 'street' },
|
||||
);
|
||||
assert.equal(
|
||||
promoteStreetHighlight({ type: 'highlight', target: 'Lombard St, San Francisco' }).type,
|
||||
'area',
|
||||
);
|
||||
assert.equal(
|
||||
promoteStreetHighlight({ type: 'highlight', target: 'the Alamo', entityKind: 'street' }).type,
|
||||
'area',
|
||||
);
|
||||
for (const spec of [
|
||||
{ type: 'highlight', target: "St Peter's Basilica" },
|
||||
{ type: 'highlight', target: 'Zilker Park' },
|
||||
{ type: 'highlight', target: 'Congress Avenue', footprint: false },
|
||||
{ type: 'highlight', target: 'Congress Avenue', entityKind: 'building' },
|
||||
{ type: 'pin', target: 'Congress Avenue' },
|
||||
])
|
||||
assert.equal(promoteStreetHighlight(spec), spec, JSON.stringify(spec));
|
||||
});
|
||||
|
||||
test('the engine resolves a bare street highlight with its footprint', async (t) => {
|
||||
installAnimationFrameStubs(t);
|
||||
const seen = [];
|
||||
const resolveTarget = async (args) => {
|
||||
seen.push(args);
|
||||
return { lon: -97.74, lat: 30.27, height: 0, label: args.target, source: 'fake', ring: null };
|
||||
};
|
||||
const engine = createAnnotationEngine({ viewer: {}, renderer, resolveTarget });
|
||||
await engine.annotate({ type: 'highlight', target: 'Congress Avenue', label: 'Congress' });
|
||||
assert.equal(seen[0].footprint, true);
|
||||
assert.equal(seen[0].entityKind, 'street');
|
||||
});
|
||||
@@ -0,0 +1,339 @@
|
||||
import {
|
||||
bufferCorridor,
|
||||
ringAreaM2,
|
||||
ringCentroid,
|
||||
} from '../sources/featureGeometry.js';
|
||||
import { isUnavailableCapability } from '../sources/capability.js';
|
||||
import {
|
||||
BUILDING_MAX_DISTANCE_M,
|
||||
OUTLINE_TILE_ZOOM,
|
||||
STREET_RADIUS_M,
|
||||
boxAround,
|
||||
buildingFromTiles,
|
||||
createOpenFreeMapOutlineSource,
|
||||
enclosingAreaFromTiles,
|
||||
streetFromTiles,
|
||||
} from '../sources/openFreeMapOutlines.js';
|
||||
import { createNominatimOutlineClient } from '../sources/nominatimOutlines.js';
|
||||
|
||||
/**
|
||||
* Outline rungs that need no Overpass, around the resolver's existing ladder:
|
||||
*
|
||||
* bundled rungs (inside the base ladder)
|
||||
* → OpenFreeMap tiles for streets and pointed-at buildings
|
||||
* → operator Overpass, when configured (the base ladder, unchanged)
|
||||
* → the server's guarded Nominatim outline (cities, landmarks, grounds,
|
||||
* parks, lakes, campuses, named buildings)
|
||||
* → OpenFreeMap building at the point, for a named building nobody outlined
|
||||
* → the base answer: an honest point marked "outline unavailable".
|
||||
*
|
||||
* Every rung returns the resolver's footprint shape
|
||||
* `{ ring, polygons?, kind, heightM, synthesized }` and hands it to the
|
||||
* resolver's `finish`, which applies the scope caps and drift bound and
|
||||
* re-centres the anchor. Nothing here fetches without an explicit ask.
|
||||
*/
|
||||
|
||||
/** Half-width (m) of the ribbon drawn along a street. */
|
||||
const STREET_HALF_WIDTH_M = 11;
|
||||
|
||||
/** A Nominatim "grounds" answer smaller than this is a building, not grounds. */
|
||||
const GROUNDS_MIN_AREA_M2 = 20_000;
|
||||
|
||||
/** Words that point at the thing rather than name it. */
|
||||
const DEICTIC_RE =
|
||||
/^(?:(?:this|that|the|my)\s+)?(?:building|house|structure|tower|place|one|thing|spot|here|there|it)$/i;
|
||||
|
||||
/** Words that describe grounds rather than belong to the name searched. */
|
||||
const GROUNDS_WORDS_RE =
|
||||
/\b(?:grounds|compound|campus|premises|property|site|area|complex)\b/gi;
|
||||
|
||||
/** Whether the ask names a thing (vs. "this building" at a pointer). */
|
||||
export function isNamedAsk(target) {
|
||||
const text = String(target || '').trim();
|
||||
return Boolean(text) && !DEICTIC_RE.test(text) && /\p{L}/u.test(text);
|
||||
}
|
||||
|
||||
/** The words sent to Nominatim: the ask without grounds words or articles. */
|
||||
export function outlineQueryText(target) {
|
||||
return String(target || '')
|
||||
.replace(GROUNDS_WORDS_RE, ' ')
|
||||
.replace(/^\s*the\s+/i, '')
|
||||
.replace(/\s+/g, ' ')
|
||||
.trim();
|
||||
}
|
||||
|
||||
/**
|
||||
* Which Nominatim ask kind (if any) suits a resolver scope.
|
||||
* Neighbourhoods and streets are not sent: bundled packs and tiles own them.
|
||||
*/
|
||||
export function nominatimKindFor({ scope, groundsLike, pointLike, around }) {
|
||||
if (pointLike) return null;
|
||||
// "Around" asks get a buffer, except grounds: the grounds ARE the thing.
|
||||
if (around && !groundsLike) return null;
|
||||
if (scope === 'city') return 'city';
|
||||
if (scope === 'state' || scope === 'county' || scope === 'country')
|
||||
return 'admin';
|
||||
if (groundsLike) return 'landmark';
|
||||
if (scope === 'building') return 'building';
|
||||
if (scope === 'compound' || scope === 'auto') return 'landmark';
|
||||
return null;
|
||||
}
|
||||
|
||||
/** Ray-cast test for a `[lon, lat]` ring. */
|
||||
function ringHolds(ring, lat, lon) {
|
||||
let inside = false;
|
||||
for (let i = 0, j = ring.length - 1; i < ring.length; j = i++) {
|
||||
const [xi, yi] = ring[i];
|
||||
const [xj, yj] = ring[j];
|
||||
if (
|
||||
yi > lat !== yj > lat &&
|
||||
lon < ((xj - xi) * (lat - yi)) / (yj - yi) + xi
|
||||
)
|
||||
inside = !inside;
|
||||
}
|
||||
return inside;
|
||||
}
|
||||
|
||||
/**
|
||||
* Parts with the one holding the anchor first. A boundary can be largest at
|
||||
* sea (Tokyo's island municipalities), so size alone does not pick the part
|
||||
* the ask is about; with no part holding the anchor, the order is kept.
|
||||
*/
|
||||
export function anchorPartFirst(polygons, points) {
|
||||
for (const { lat, lon } of points) {
|
||||
if (!Number.isFinite(lat) || !Number.isFinite(lon)) continue;
|
||||
const at = polygons.findIndex(([outer]) => ringHolds(outer, lat, lon));
|
||||
if (at > 0) return [polygons[at], ...polygons.filter((_, i) => i !== at)];
|
||||
if (at === 0) return polygons;
|
||||
}
|
||||
return polygons;
|
||||
}
|
||||
|
||||
/** Whether a base answer is final (a real outline, or "try again later"). */
|
||||
function keepBase(result) {
|
||||
if (result === undefined) return true;
|
||||
if (result?.rateLimited === true) return true;
|
||||
if (isUnavailableCapability(result)) return false;
|
||||
return Boolean(result?.ring) && !result.synthesized;
|
||||
}
|
||||
|
||||
/** Street polylines → one ribbon polygon per chain (longest first). */
|
||||
export function streetFootprint(street) {
|
||||
const polygons = street.lines
|
||||
.filter((line) => line.length >= 2)
|
||||
.map((line) => [bufferCorridor(line, STREET_HALF_WIDTH_M)]);
|
||||
if (!polygons.length) return null;
|
||||
return {
|
||||
ring: polygons[0][0],
|
||||
polygons,
|
||||
lines: street.lines,
|
||||
kind: 'area',
|
||||
heightM: null,
|
||||
synthesized: false,
|
||||
outlineSource: 'openfreemap',
|
||||
};
|
||||
}
|
||||
|
||||
export function createOutlineRungs({
|
||||
tiles = createOpenFreeMapOutlineSource(),
|
||||
nominatim = createNominatimOutlineClient(),
|
||||
} = {}) {
|
||||
/** Decoded tiles around a point, or null when the tiles cannot be read. */
|
||||
async function tilesAround(lat, lon, radiusM, signal) {
|
||||
try {
|
||||
const { tiles: decoded } = await tiles.fetchBounds(
|
||||
boxAround(lat, lon, radiusM),
|
||||
{ zoom: OUTLINE_TILE_ZOOM, signal },
|
||||
);
|
||||
return decoded;
|
||||
} catch (error) {
|
||||
signal?.throwIfAborted();
|
||||
if (error?.name === 'AbortError') throw error;
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
async function streetRung(ctx) {
|
||||
const name = ctx.matchName || ctx.target;
|
||||
if (!isNamedAsk(name)) return null;
|
||||
const decoded = await tilesAround(
|
||||
ctx.lat,
|
||||
ctx.lon,
|
||||
STREET_RADIUS_M,
|
||||
ctx.signal,
|
||||
);
|
||||
if (!decoded) return null;
|
||||
const street = streetFromTiles(decoded, {
|
||||
name,
|
||||
lat: ctx.lat,
|
||||
lon: ctx.lon,
|
||||
});
|
||||
return street ? streetFootprint(street) : null;
|
||||
}
|
||||
|
||||
async function buildingRung(ctx) {
|
||||
const decoded = await tilesAround(
|
||||
ctx.lat,
|
||||
ctx.lon,
|
||||
BUILDING_MAX_DISTANCE_M * 2,
|
||||
ctx.signal,
|
||||
);
|
||||
const building = decoded && buildingFromTiles(decoded, ctx);
|
||||
if (!building) return null;
|
||||
return {
|
||||
ring: building.ring,
|
||||
polygons: [building.rings],
|
||||
kind: 'building',
|
||||
heightM: building.heightM,
|
||||
synthesized: false,
|
||||
outlineSource: 'openfreemap',
|
||||
};
|
||||
}
|
||||
|
||||
/** The open area enclosing `seed` (the named feature) or the anchor. */
|
||||
async function groundsRung(ctx, seed = null) {
|
||||
const at = seed || { lat: ctx.lat, lon: ctx.lon };
|
||||
const decoded = await tilesAround(at.lat, at.lon, 600, ctx.signal);
|
||||
const area =
|
||||
decoded &&
|
||||
enclosingAreaFromTiles(decoded, {
|
||||
lat: at.lat,
|
||||
lon: at.lon,
|
||||
minAreaM2: GROUNDS_MIN_AREA_M2,
|
||||
});
|
||||
if (!area) return null;
|
||||
return {
|
||||
ring: area.ring,
|
||||
polygons: [area.rings],
|
||||
kind: 'area',
|
||||
heightM: null,
|
||||
synthesized: false,
|
||||
outlineSource: 'openfreemap',
|
||||
};
|
||||
}
|
||||
|
||||
async function nominatimRung(ctx, kind) {
|
||||
const query = outlineQueryText(
|
||||
kind === 'city' || kind === 'admin'
|
||||
? ctx.matchName || ctx.target
|
||||
: ctx.target || ctx.matchName,
|
||||
);
|
||||
if (!isNamedAsk(query)) return null;
|
||||
const bias = ctx.view || { lat: ctx.lat, lon: ctx.lon };
|
||||
const outline = await nominatim.lookup(
|
||||
{ query, kind, lat: bias.lat, lon: bias.lon },
|
||||
{ signal: ctx.signal },
|
||||
);
|
||||
if (!outline?.polygons) return outline; // null | undefined | unavailable | rateLimited
|
||||
const polygons = anchorPartFirst(outline.polygons, [
|
||||
{ lat: ctx.lat, lon: ctx.lon },
|
||||
outline.center || {},
|
||||
]);
|
||||
const ring = polygons[0][0];
|
||||
// A grounds ask answered by a building-sized polygon is the dome, not the
|
||||
// grounds: report no grounds outline rather than the wrong shape.
|
||||
// A grounds ask answered by a building-sized polygon found the named
|
||||
// building, not its grounds: keep its position to find what encloses it.
|
||||
if (ctx.groundsLike && ringAreaM2(ring) < GROUNDS_MIN_AREA_M2)
|
||||
return { groundsSeed: ringCentroid(ring) };
|
||||
return {
|
||||
ring,
|
||||
polygons,
|
||||
kind:
|
||||
kind === 'building' || outline.class === 'building'
|
||||
? 'building'
|
||||
: 'area',
|
||||
heightM: null,
|
||||
synthesized: false,
|
||||
outlineSource: 'nominatim',
|
||||
outlineName: outline.name,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Run the ladder for one resolved anchor.
|
||||
*
|
||||
* @param {object} ctx scope, target, matchName, lat, lon, named anchor facts,
|
||||
* view centre, signal, and `credit(kind)`.
|
||||
* @param {{base: () => Promise<any>, finish: (fp: object, scope: string) => any}} ladder
|
||||
*/
|
||||
async function resolve(ctx, { base, finish }) {
|
||||
const { scope } = ctx;
|
||||
const named = ctx.fromName && isNamedAsk(ctx.target);
|
||||
// A cancelled ask (board cleared, superseded) stops between rungs, so it
|
||||
// never spends a Nominatim request.
|
||||
const alive = () => ctx.signal?.throwIfAborted();
|
||||
const accept = (fp, asScope = scope) => {
|
||||
alive();
|
||||
if (!fp) return null;
|
||||
const patch = finish(fp, asScope);
|
||||
if (patch?.ring) {
|
||||
ctx.credit?.(fp.outlineSource === 'openfreemap' ? 'tiles' : 'osm');
|
||||
// One line per answered outline, like the resolver's own trace.
|
||||
console.log(
|
||||
`[Outline] "${ctx.target || ''}": scope=${asScope} → ${fp.outlineSource}` +
|
||||
` (${patch.footprintKind}, ${patch.polygons?.length || 1} part(s))`,
|
||||
);
|
||||
return { ...patch, outlineSource: fp.outlineSource };
|
||||
}
|
||||
return null;
|
||||
};
|
||||
|
||||
// Tiles first for streets and for a building pointed at rather than named.
|
||||
if (scope === 'street') {
|
||||
const street = accept(await streetRung(ctx), 'street');
|
||||
if (street) return street;
|
||||
} else if (scope === 'building' && !named && !ctx.around) {
|
||||
const building = accept(await buildingRung(ctx), 'building');
|
||||
if (building) return building;
|
||||
}
|
||||
|
||||
alive();
|
||||
const baseResult = await base();
|
||||
alive();
|
||||
if (keepBase(baseResult)) return baseResult;
|
||||
|
||||
// A transient or busy Nominatim answer is kept aside while the tile
|
||||
// fallbacks run; it is returned only if nothing else answers.
|
||||
let retryLater = null;
|
||||
let groundsSeed = null;
|
||||
const kind = named ? nominatimKindFor(ctx) : null;
|
||||
if (kind) {
|
||||
const found = await nominatimRung(ctx, kind);
|
||||
alive();
|
||||
if (found?.groundsSeed) {
|
||||
groundsSeed = found.groundsSeed;
|
||||
} else if (found?.ring) {
|
||||
const patch = accept(found, ctx.groundsLike ? 'compound' : scope);
|
||||
if (patch) return patch;
|
||||
} else if (found === undefined || found?.rateLimited) {
|
||||
retryLater = { value: found };
|
||||
}
|
||||
}
|
||||
|
||||
// Grounds nobody named in a polygon: the open area enclosing the point.
|
||||
if (ctx.groundsLike && named) {
|
||||
// Around the named building when Nominatim found it, else the anchor.
|
||||
let grounds = groundsSeed
|
||||
? accept(await groundsRung(ctx, groundsSeed), 'compound')
|
||||
: null;
|
||||
grounds ||= accept(await groundsRung(ctx), 'compound');
|
||||
if (grounds) return grounds;
|
||||
}
|
||||
|
||||
// A named building nobody outlined: the footprint at its point. Never for
|
||||
// a grounds ask, where the building is the wrong shape.
|
||||
if (scope === 'building' && named && !ctx.around && !ctx.groundsLike) {
|
||||
const building = accept(await buildingRung(ctx), 'building');
|
||||
if (building) return building;
|
||||
}
|
||||
// Keep the mark retryable only when nothing else could answer. A
|
||||
// definitive miss or an approximation from the configured ladder stands
|
||||
// rather than inviting repeated lookups.
|
||||
if (retryLater && isUnavailableCapability(baseResult))
|
||||
return retryLater.value;
|
||||
return baseResult;
|
||||
}
|
||||
|
||||
return { resolve };
|
||||
}
|
||||
@@ -0,0 +1,424 @@
|
||||
// OVERPASS-FREE OUTLINE RUNGS.
|
||||
//
|
||||
// Streets and pointed-at buildings come from OpenFreeMap tiles before the
|
||||
// existing ladder; cities, landmarks, grounds and named buildings fall to the
|
||||
// server's guarded Nominatim outline after it. With default settings no
|
||||
// request reaches any Overpass endpoint.
|
||||
import test from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { mkdtempSync, readFileSync, rmSync } from 'node:fs';
|
||||
import os from 'node:os';
|
||||
import path from 'node:path';
|
||||
import { Readable } from 'node:stream';
|
||||
import {
|
||||
createOutlineRungs,
|
||||
isNamedAsk,
|
||||
nominatimKindFor,
|
||||
outlineQueryText,
|
||||
} from './outlineRungs.js';
|
||||
import { createAnnotationResolver } from './resolver.js';
|
||||
import { decodeOpenFreeMapOutlineTile } from '../sources/openFreeMapOutlines.js';
|
||||
import { createNominatimOutlineClient } from '../sources/nominatimOutlines.js';
|
||||
import { createApplicationRequestServices } from '../services/requests.js';
|
||||
import { overpassProxy } from '../../server/providers/overpass.js';
|
||||
import { geocodeProxy } from '../../server/providers/regional/place.js';
|
||||
import {
|
||||
createGateStateStore,
|
||||
createNominatimGate,
|
||||
} from '../../server/providers/regional/nominatimGate.js';
|
||||
|
||||
const TILES = [6745, 6746].map((y) =>
|
||||
decodeOpenFreeMapOutlineTile(
|
||||
readFileSync(
|
||||
new URL(`../data/fixtures/ofm-outlines-austin-14-3743-${y}.pbf`, import.meta.url),
|
||||
),
|
||||
14,
|
||||
3743,
|
||||
y,
|
||||
),
|
||||
);
|
||||
const fixtureTiles = () => {
|
||||
const asked = [];
|
||||
return {
|
||||
asked,
|
||||
fetchBounds: async (box, options) => {
|
||||
asked.push({ box, zoom: options.zoom });
|
||||
return { tiles: TILES };
|
||||
},
|
||||
};
|
||||
};
|
||||
const CAPITOL = { lat: 30.27472, lon: -97.74035 };
|
||||
|
||||
const square = (lat, lon, halfDeg) => [
|
||||
[lon - halfDeg, lat - halfDeg],
|
||||
[lon + halfDeg, lat - halfDeg],
|
||||
[lon + halfDeg, lat + halfDeg],
|
||||
[lon - halfDeg, lat + halfDeg],
|
||||
[lon - halfDeg, lat - halfDeg],
|
||||
];
|
||||
|
||||
function fakeNominatim(answer) {
|
||||
const asks = [];
|
||||
return {
|
||||
asks,
|
||||
lookup: async (ask) => {
|
||||
asks.push(ask);
|
||||
return typeof answer === 'function' ? answer(ask) : answer;
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
/** A ladder whose base answers `baseAnswer` and whose finish passes rings through. */
|
||||
function ladder(baseAnswer) {
|
||||
const calls = { base: 0, finished: [] };
|
||||
return {
|
||||
calls,
|
||||
base: async () => {
|
||||
calls.base += 1;
|
||||
return baseAnswer;
|
||||
},
|
||||
finish: (fp, scope) => {
|
||||
calls.finished.push(scope);
|
||||
return { ring: fp.ring, polygons: fp.polygons, footprintKind: fp.kind, synthesized: false };
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
const UNAVAILABLE = { unavailable: true, retryable: false, code: 'OVERPASS_NOT_CONFIGURED' };
|
||||
const ctx = (overrides) => ({
|
||||
scope: 'auto',
|
||||
target: 'X',
|
||||
matchName: 'X',
|
||||
fromName: true,
|
||||
pointLike: false,
|
||||
groundsLike: false,
|
||||
around: false,
|
||||
...CAPITOL,
|
||||
view: { lat: CAPITOL.lat, lon: CAPITOL.lon },
|
||||
credit() {},
|
||||
...overrides,
|
||||
});
|
||||
|
||||
test('ask wording helpers', () => {
|
||||
assert.equal(isNamedAsk('this building'), false);
|
||||
assert.equal(isNamedAsk('here'), false);
|
||||
assert.equal(isNamedAsk('Texas Capitol'), true);
|
||||
assert.equal(outlineQueryText('the Texas Capitol grounds'), 'Texas Capitol');
|
||||
assert.equal(nominatimKindFor({ scope: 'city' }), 'city');
|
||||
assert.equal(nominatimKindFor({ scope: 'state' }), 'admin');
|
||||
assert.equal(nominatimKindFor({ scope: 'compound' }), 'landmark');
|
||||
assert.equal(nominatimKindFor({ scope: 'building', groundsLike: true }), 'landmark');
|
||||
assert.equal(nominatimKindFor({ scope: 'neighborhood' }), null);
|
||||
assert.equal(nominatimKindFor({ scope: 'street' }), null);
|
||||
assert.equal(nominatimKindFor({ scope: 'compound', pointLike: true }), null);
|
||||
});
|
||||
|
||||
test('a street comes from tiles before the configured ladder, as merged ribbons', async () => {
|
||||
const tiles = fixtureTiles();
|
||||
const nominatim = fakeNominatim(null);
|
||||
const credits = [];
|
||||
const run = ladder(UNAVAILABLE);
|
||||
const result = await createOutlineRungs({ tiles, nominatim }).resolve(
|
||||
ctx({ scope: 'street', target: 'Congress Avenue', matchName: 'Congress Avenue', credit: (k) => credits.push(k) }),
|
||||
run,
|
||||
);
|
||||
assert.ok(result.ring.length > 6);
|
||||
assert.equal(result.polygons.length, 1);
|
||||
assert.equal(result.outlineSource, 'openfreemap');
|
||||
assert.equal(run.calls.base, 0, 'the ladder is not consulted');
|
||||
assert.deepEqual(run.calls.finished, ['street']);
|
||||
assert.deepEqual(credits, ['tiles']);
|
||||
assert.equal(tiles.asked[0].zoom, 14);
|
||||
assert.equal(nominatim.asks.length, 0, 'streets never reach Nominatim');
|
||||
});
|
||||
|
||||
test('a street the tiles do not name falls through to the ladder', async () => {
|
||||
const run = ladder(UNAVAILABLE);
|
||||
const result = await createOutlineRungs({ tiles: fixtureTiles(), nominatim: fakeNominatim(null) }).resolve(
|
||||
ctx({ scope: 'street', target: 'Lombard Street', matchName: 'Lombard Street' }),
|
||||
run,
|
||||
);
|
||||
assert.equal(result, UNAVAILABLE);
|
||||
assert.equal(run.calls.base, 1);
|
||||
});
|
||||
|
||||
test('"circle this building" picks the tile building at the point first', async () => {
|
||||
const nominatim = fakeNominatim(null);
|
||||
const run = ladder(UNAVAILABLE);
|
||||
const result = await createOutlineRungs({ tiles: fixtureTiles(), nominatim }).resolve(
|
||||
ctx({ scope: 'building', target: 'this building', fromName: false }),
|
||||
run,
|
||||
);
|
||||
assert.equal(result.footprintKind, 'building');
|
||||
assert.equal(run.calls.base, 0);
|
||||
assert.equal(nominatim.asks.length, 0);
|
||||
});
|
||||
|
||||
test('a named building asks Nominatim, then falls back to the tile building', async () => {
|
||||
const nominatim = fakeNominatim(null);
|
||||
const result = await createOutlineRungs({ tiles: fixtureTiles(), nominatim }).resolve(
|
||||
ctx({ scope: 'building', target: 'Texas State Capitol' }),
|
||||
ladder(UNAVAILABLE),
|
||||
);
|
||||
assert.deepEqual(nominatim.asks.map((a) => a.kind), ['building']);
|
||||
assert.equal(result.footprintKind, 'building');
|
||||
assert.equal(result.outlineSource, 'openfreemap');
|
||||
});
|
||||
|
||||
test('a city with no configured Overpass is outlined by Nominatim', async () => {
|
||||
const ring = square(43.6, 1.44, 0.05);
|
||||
const nominatim = fakeNominatim({ polygons: [[ring]], class: 'boundary' });
|
||||
const credits = [];
|
||||
const run = ladder(UNAVAILABLE);
|
||||
const result = await createOutlineRungs({ tiles: fixtureTiles(), nominatim }).resolve(
|
||||
ctx({ scope: 'city', target: 'Toulouse, France', matchName: 'Toulouse', credit: (k) => credits.push(k) }),
|
||||
run,
|
||||
);
|
||||
assert.deepEqual(result.ring, ring);
|
||||
assert.equal(result.outlineSource, 'nominatim');
|
||||
assert.deepEqual(nominatim.asks[0], { query: 'Toulouse', kind: 'city', lat: CAPITOL.lat, lon: CAPITOL.lon });
|
||||
assert.deepEqual(run.calls.finished, ['city'], 'scope caps and drift bound still apply');
|
||||
assert.deepEqual(credits, ['osm']);
|
||||
});
|
||||
|
||||
test('a multi-part city leads with the part holding the anchor, not the largest', async () => {
|
||||
const sea = square(33.5, 139.5, 0.8); // larger, far from the anchor
|
||||
const land = square(CAPITOL.lat, CAPITOL.lon, 0.2);
|
||||
const nominatim = fakeNominatim({ polygons: [[sea], [land]], class: 'boundary' });
|
||||
const result = await createOutlineRungs({ tiles: fixtureTiles(), nominatim }).resolve(
|
||||
ctx({ scope: 'city', target: 'Tokyo', matchName: 'Tokyo' }),
|
||||
ladder(UNAVAILABLE),
|
||||
);
|
||||
assert.deepEqual(result.ring, land, 'the main ring (centroid, scope cap) is the anchored part');
|
||||
assert.equal(result.polygons.length, 2, 'every part is still drawn');
|
||||
});
|
||||
|
||||
test('a real outline from the ladder (bundled or operator Overpass) is kept', async () => {
|
||||
const nominatim = fakeNominatim(null);
|
||||
const outline = { ring: square(0, 0, 1), synthesized: false };
|
||||
const result = await createOutlineRungs({ tiles: fixtureTiles(), nominatim }).resolve(
|
||||
ctx({ scope: 'city', target: 'Austin' }),
|
||||
ladder(outline),
|
||||
);
|
||||
assert.equal(result, outline);
|
||||
assert.equal(nominatim.asks.length, 0);
|
||||
});
|
||||
|
||||
test('a transient ladder failure stays retryable without spending a lookup', async () => {
|
||||
const nominatim = fakeNominatim(null);
|
||||
const result = await createOutlineRungs({ tiles: fixtureTiles(), nominatim }).resolve(
|
||||
ctx({ scope: 'compound', target: 'Zilker Park' }),
|
||||
ladder(undefined),
|
||||
);
|
||||
assert.equal(result, undefined);
|
||||
assert.equal(nominatim.asks.length, 0);
|
||||
});
|
||||
|
||||
test('no Nominatim outline leaves the honest point; busy stays retryable', async () => {
|
||||
const rungs = (answer) => createOutlineRungs({ tiles: fixtureTiles(), nominatim: fakeNominatim(answer) });
|
||||
assert.equal(await rungs(null).resolve(ctx({ scope: 'compound', target: 'Nowhere Park' }), ladder(UNAVAILABLE)), UNAVAILABLE);
|
||||
const busy = { rateLimited: true, retryAfterMs: 5000 };
|
||||
assert.equal(await rungs(busy).resolve(ctx({ scope: 'compound', target: 'Zilker Park' }), ladder(UNAVAILABLE)), busy);
|
||||
// After an operator's definitive miss, a busy Nominatim does not invite retries.
|
||||
assert.equal(await rungs(busy).resolve(ctx({ scope: 'compound', target: 'Zilker Park' }), ladder(null)), null);
|
||||
});
|
||||
|
||||
test('grounds: a building-sized answer is not the grounds; a real enclosure is', async () => {
|
||||
const dome = { polygons: [[square(30.2747, -97.7403, 0.0003)]], class: 'office' };
|
||||
const grounds = { polygons: [[square(30.2747, -97.7403, 0.002)]], class: 'leisure' };
|
||||
// The dome is refused; the open area enclosing the point stands in.
|
||||
const small = await createOutlineRungs({ tiles: fixtureTiles(), nominatim: fakeNominatim(dome) }).resolve(
|
||||
ctx({ scope: 'building', target: 'Texas Capitol grounds', groundsLike: true }),
|
||||
ladder(UNAVAILABLE),
|
||||
);
|
||||
assert.equal(small.outlineSource, 'openfreemap');
|
||||
assert.equal(small.footprintKind, 'area');
|
||||
// Without enclosing tile cover, the honest point remains.
|
||||
const bare = await createOutlineRungs({
|
||||
tiles: { fetchBounds: async () => ({ tiles: [] }) },
|
||||
nominatim: fakeNominatim(dome),
|
||||
}).resolve(ctx({ scope: 'building', target: 'Texas Capitol grounds', groundsLike: true }), ladder(UNAVAILABLE));
|
||||
assert.equal(bare, UNAVAILABLE);
|
||||
const nominatim = fakeNominatim(grounds);
|
||||
const run = ladder(UNAVAILABLE);
|
||||
const big = await createOutlineRungs({ tiles: fixtureTiles(), nominatim }).resolve(
|
||||
ctx({ scope: 'building', target: 'the Texas Capitol grounds', groundsLike: true }),
|
||||
run,
|
||||
);
|
||||
assert.equal(big.footprintKind, 'area');
|
||||
assert.deepEqual(run.calls.finished, ['compound']);
|
||||
assert.deepEqual(nominatim.asks.map((a) => [a.query, a.kind]), [['Texas Capitol', 'landmark']]);
|
||||
});
|
||||
|
||||
test('point-like, around and neighbourhood asks never reach Nominatim', async () => {
|
||||
for (const extra of [
|
||||
{ scope: 'compound', pointLike: true, target: 'Tejano Monument' },
|
||||
{ scope: 'compound', around: true, target: 'Zilker Park' },
|
||||
{ scope: 'neighborhood', target: 'Mission District' },
|
||||
]) {
|
||||
const nominatim = fakeNominatim(null);
|
||||
await createOutlineRungs({ tiles: fixtureTiles(), nominatim }).resolve(ctx(extra), ladder(UNAVAILABLE));
|
||||
assert.equal(nominatim.asks.length, 0, JSON.stringify(extra));
|
||||
}
|
||||
});
|
||||
|
||||
// ------------------------------------------------------------ end to end
|
||||
|
||||
function routes(...plugins) {
|
||||
const found = new Map();
|
||||
for (const plugin of plugins)
|
||||
plugin.configureServer({ middlewares: { use: (path, handler) => found.set(path, handler) } });
|
||||
return found;
|
||||
}
|
||||
|
||||
async function call(handlers, url, init = {}) {
|
||||
const parsed = new URL(url, 'http://localhost');
|
||||
const mount = [...handlers.keys()]
|
||||
.filter((path) => parsed.pathname === path || parsed.pathname.startsWith(`${path}/`))
|
||||
.sort((a, b) => b.length - a.length)[0];
|
||||
if (!mount) return new Response('{}', { status: 404 });
|
||||
const req = Readable.from(init.body ? [Buffer.from(init.body)] : []);
|
||||
Object.assign(req, {
|
||||
method: init.method || 'GET',
|
||||
url: parsed.pathname.slice(mount.length) + parsed.search || '/',
|
||||
headers: {},
|
||||
socket: { remoteAddress: '127.0.0.1' },
|
||||
});
|
||||
let status;
|
||||
let headers;
|
||||
let body;
|
||||
await handlers.get(mount)(
|
||||
req,
|
||||
{
|
||||
on() {},
|
||||
writeHead(s, h) {
|
||||
status = s;
|
||||
headers = h;
|
||||
},
|
||||
end(b) {
|
||||
body = b;
|
||||
},
|
||||
},
|
||||
() => {
|
||||
status = 404;
|
||||
body = '{}';
|
||||
},
|
||||
);
|
||||
return new Response(body, { status, headers });
|
||||
}
|
||||
|
||||
const NOMINATIM_ROWS = {
|
||||
Toulouse: [
|
||||
{ category: 'place', type: 'city', name: 'Toulouse', lat: '43.6', lon: '1.44', geojson: { type: 'Point', coordinates: [1.44, 43.6] } },
|
||||
{ category: 'boundary', type: 'administrative', addresstype: 'city', name: 'Toulouse', lat: '43.6', lon: '1.44', osm_type: 'relation', osm_id: 35738, geojson: { type: 'Polygon', coordinates: [square(43.6, 1.44, 0.05)] } },
|
||||
],
|
||||
'Central Park': [
|
||||
{ category: 'leisure', type: 'park', name: 'Central Park', lat: '40.78', lon: '-73.965', osm_type: 'relation', osm_id: 8, geojson: { type: 'Polygon', coordinates: [square(40.78, -73.965, 0.01)] } },
|
||||
],
|
||||
};
|
||||
|
||||
const PLACES = {
|
||||
Toulouse: { lat: 43.6, lng: 1.44, name: 'Toulouse', label: 'Toulouse, France', types: ['locality', 'political'] },
|
||||
'Central Park': { lat: 40.78, lng: -73.965, name: 'Central Park', label: 'Central Park, New York', types: ['park'] },
|
||||
'Congress Avenue': { lat: 30.268, lng: -97.7427, name: 'Congress Avenue', label: 'Congress Ave, Austin', types: ['route'] },
|
||||
};
|
||||
|
||||
const viewerAt = (lat, lon, height = 4000) => ({
|
||||
camera: { positionCartographic: { latitude: (lat * Math.PI) / 180, longitude: (lon * Math.PI) / 180, height } },
|
||||
});
|
||||
|
||||
test('default settings: outline, street and building asks send nothing to Overpass', async (t) => {
|
||||
const prior = process.env.OVERPASS_UPSTREAMS;
|
||||
delete process.env.OVERPASS_UPSTREAMS;
|
||||
t.after(() => {
|
||||
if (prior !== undefined) process.env.OVERPASS_UPSTREAMS = prior;
|
||||
});
|
||||
// Any request that escapes the injected transports fails the test.
|
||||
t.mock.method(globalThis, 'fetch', async (url) => assert.fail(`unexpected network request ${url}`));
|
||||
|
||||
const upstream = [];
|
||||
const stateDir = mkdtempSync(path.join(os.tmpdir(), 'gev-outline-rungs-'));
|
||||
t.after(() => rmSync(stateDir, { recursive: true, force: true }));
|
||||
const gate = createNominatimGate({
|
||||
settings: { endpoint: 'https://nominatim.openstreetmap.org/search', isPublic: true, dailyCap: 50 },
|
||||
fetchImpl: async (url) => {
|
||||
upstream.push(String(url));
|
||||
const q = new URL(url).searchParams.get('q');
|
||||
return Response.json(NOMINATIM_ROWS[q] || []);
|
||||
},
|
||||
usage: createGateStateStore({ file: path.join(stateDir, 'state.json') }),
|
||||
sleep: async () => {},
|
||||
});
|
||||
const handlers = routes(overpassProxy(), geocodeProxy({ gate }));
|
||||
const local = [];
|
||||
const transport = (url, init) => {
|
||||
local.push(`${init?.method || 'GET'} ${String(url).split('?')[0]}`);
|
||||
return call(handlers, url, init);
|
||||
};
|
||||
const services = createApplicationRequestServices({ fetchImpl: transport });
|
||||
const resolver = createAnnotationResolver({
|
||||
featureSource: services.features,
|
||||
outlineRungs: createOutlineRungs({
|
||||
tiles: fixtureTiles(),
|
||||
nominatim: createNominatimOutlineClient({ fetchImpl: transport }),
|
||||
}),
|
||||
});
|
||||
const placeSearch = { geocode: async (q) => ({ place: PLACES[q] || null, answered: true }) };
|
||||
const resolve = async (viewer, args) => {
|
||||
const result = await resolver.resolveAnnotationTarget({ viewer, placeSearch, footprint: true, deferFootprint: true, ...args });
|
||||
return { result, outline: await result.resolveOutline() };
|
||||
};
|
||||
|
||||
const paris = await resolve(viewerAt(43.6, 1.44, 40_000), { target: 'Toulouse' });
|
||||
assert.ok(paris.outline?.ring?.length >= 4, 'city outlined');
|
||||
assert.equal(paris.outline.outlineSource, 'nominatim');
|
||||
|
||||
const park = await resolve(viewerAt(40.78, -73.965), { target: 'Central Park' });
|
||||
assert.equal(park.outline?.outlineSource, 'nominatim');
|
||||
|
||||
const street = await resolve(viewerAt(30.27, -97.742), { target: 'Congress Avenue' });
|
||||
assert.equal(street.outline?.outlineSource, 'openfreemap');
|
||||
|
||||
const building = await resolve(viewerAt(30.2747, -97.7403, 800), {
|
||||
latitude: CAPITOL.lat,
|
||||
longitude: CAPITOL.lon,
|
||||
entityKind: 'building',
|
||||
target: 'this building',
|
||||
});
|
||||
assert.equal(building.outline?.footprintKind, 'building');
|
||||
assert.equal(building.outline?.outlineSource, 'openfreemap');
|
||||
|
||||
assert.ok(upstream.every((url) => url.startsWith('https://nominatim.openstreetmap.org/search?')));
|
||||
assert.equal(upstream.length, 2, 'one upstream request per explicit ask that needed one');
|
||||
assert.ok(!upstream.some((url) => /overpass/i.test(url)));
|
||||
// The only Overpass traffic is the local capability probe, answered locally.
|
||||
assert.ok(
|
||||
local.filter((line) => line.includes('/api/overpass')).every((line) => line === 'GET /api/overpass/status'),
|
||||
local.join('\n'),
|
||||
);
|
||||
});
|
||||
|
||||
test('a cancelled ask stops between rungs and never reaches Nominatim', async () => {
|
||||
const nominatim = fakeNominatim({ polygons: [[square(43.6, 1.44, 0.05)]], class: 'boundary' });
|
||||
const rungs = createOutlineRungs({ tiles: fixtureTiles(), nominatim });
|
||||
// Cancelled while the configured ladder ran.
|
||||
const controller = new AbortController();
|
||||
await assert.rejects(
|
||||
rungs.resolve(ctx({ scope: 'city', target: 'Toulouse', signal: controller.signal }), {
|
||||
base: async () => {
|
||||
controller.abort();
|
||||
return UNAVAILABLE;
|
||||
},
|
||||
finish: (fp) => fp,
|
||||
}),
|
||||
{ name: 'AbortError' },
|
||||
);
|
||||
// Cancelled before it started: not even the ladder runs.
|
||||
const run = ladder(UNAVAILABLE);
|
||||
await assert.rejects(
|
||||
rungs.resolve(ctx({ scope: 'city', target: 'Toulouse', signal: AbortSignal.abort() }), run),
|
||||
{ name: 'AbortError' },
|
||||
);
|
||||
assert.equal(run.calls.base, 0);
|
||||
assert.equal(nominatim.asks.length, 0);
|
||||
});
|
||||
@@ -0,0 +1,145 @@
|
||||
// Tile fallbacks still run after a busy or failed Nominatim answer, and
|
||||
// grounds asks phrased "around" still reach their outline rungs.
|
||||
import test from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { readFileSync } from 'node:fs';
|
||||
import { createOutlineRungs, nominatimKindFor } from './outlineRungs.js';
|
||||
import { decodeOpenFreeMapOutlineTile } from '../sources/openFreeMapOutlines.js';
|
||||
|
||||
const TILES = [6745, 6746].map((y) =>
|
||||
decodeOpenFreeMapOutlineTile(
|
||||
readFileSync(
|
||||
new URL(
|
||||
`../data/fixtures/ofm-outlines-austin-14-3743-${y}.pbf`,
|
||||
import.meta.url,
|
||||
),
|
||||
),
|
||||
14,
|
||||
3743,
|
||||
y,
|
||||
),
|
||||
);
|
||||
const UNAVAILABLE = { unavailable: true, retryable: false };
|
||||
const CAPITOL = { lat: 30.27472, lon: -97.74035 };
|
||||
|
||||
function setup(answer) {
|
||||
const tileCalls = [];
|
||||
const asks = [];
|
||||
const rungs = createOutlineRungs({
|
||||
tiles: {
|
||||
fetchBounds: async (box) => {
|
||||
tileCalls.push(box);
|
||||
return { tiles: TILES };
|
||||
},
|
||||
},
|
||||
nominatim: {
|
||||
lookup: async (ask) => {
|
||||
asks.push(ask);
|
||||
return answer;
|
||||
},
|
||||
},
|
||||
});
|
||||
return { rungs, tileCalls, asks };
|
||||
}
|
||||
const ladder = {
|
||||
base: async () => UNAVAILABLE,
|
||||
finish: (fp) => ({ ring: fp.ring, footprintKind: fp.kind }),
|
||||
};
|
||||
const ctx = (overrides) => ({
|
||||
scope: 'building',
|
||||
target: 'Texas State Capitol',
|
||||
matchName: 'Texas State Capitol',
|
||||
fromName: true,
|
||||
...CAPITOL,
|
||||
view: CAPITOL,
|
||||
...overrides,
|
||||
});
|
||||
|
||||
for (const [label, answer] of [
|
||||
['busy', { rateLimited: true, retryAfterMs: 5000 }],
|
||||
['transient', undefined],
|
||||
]) {
|
||||
test(`a ${label} Nominatim answer does not skip the tile fallbacks`, async () => {
|
||||
const building = setup(answer);
|
||||
const found = await building.rungs.resolve(ctx(), ladder);
|
||||
assert.equal(building.asks.length, 1);
|
||||
assert.ok(building.tileCalls.length > 0, 'tiles were consulted');
|
||||
assert.equal(found.footprintKind, 'building');
|
||||
|
||||
const grounds = setup(answer);
|
||||
const lawn = await grounds.rungs.resolve(
|
||||
ctx({ target: 'the Texas Capitol grounds', groundsLike: true }),
|
||||
ladder,
|
||||
);
|
||||
assert.equal(lawn.footprintKind, 'area');
|
||||
assert.equal(lawn.outlineSource, 'openfreemap');
|
||||
});
|
||||
}
|
||||
|
||||
test('with no tile answer either, the busy result keeps the mark retryable', async () => {
|
||||
const busy = { rateLimited: true, retryAfterMs: 5000 };
|
||||
const rungs = createOutlineRungs({
|
||||
tiles: { fetchBounds: async () => ({ tiles: [] }) },
|
||||
nominatim: { lookup: async () => busy },
|
||||
});
|
||||
assert.equal(await rungs.resolve(ctx(), ladder), busy);
|
||||
});
|
||||
|
||||
test('"around the Capitol grounds" is still a grounds ask', async () => {
|
||||
assert.equal(
|
||||
nominatimKindFor({ scope: 'building', groundsLike: true, around: true }),
|
||||
'landmark',
|
||||
);
|
||||
assert.equal(nominatimKindFor({ scope: 'compound', around: true }), null);
|
||||
const { rungs, asks } = setup(null);
|
||||
const lawn = await rungs.resolve(
|
||||
ctx({
|
||||
target: 'the Texas Capitol grounds',
|
||||
groundsLike: true,
|
||||
around: true,
|
||||
}),
|
||||
ladder,
|
||||
);
|
||||
assert.equal(asks.length, 1);
|
||||
assert.equal(lawn.outlineSource, 'openfreemap');
|
||||
});
|
||||
|
||||
test('grounds are found around the named building when the anchor is off the lawn', async () => {
|
||||
// A place-search anchor ~250 m from the dome, outside the Capitol lawn.
|
||||
const offLawn = { lat: 30.27265, lon: -97.73925 };
|
||||
const d = 0.0003;
|
||||
const dome = {
|
||||
polygons: [
|
||||
[
|
||||
[
|
||||
[CAPITOL.lon - d, CAPITOL.lat - d],
|
||||
[CAPITOL.lon + d, CAPITOL.lat - d],
|
||||
[CAPITOL.lon + d, CAPITOL.lat + d],
|
||||
[CAPITOL.lon - d, CAPITOL.lat + d],
|
||||
[CAPITOL.lon - d, CAPITOL.lat - d],
|
||||
],
|
||||
],
|
||||
],
|
||||
class: 'office',
|
||||
};
|
||||
const withoutSeed = setup(null);
|
||||
assert.equal(
|
||||
await withoutSeed.rungs.resolve(
|
||||
ctx({
|
||||
...offLawn,
|
||||
target: 'the Texas Capitol grounds',
|
||||
groundsLike: true,
|
||||
}),
|
||||
ladder,
|
||||
),
|
||||
UNAVAILABLE,
|
||||
'the anchor alone is not inside the grounds',
|
||||
);
|
||||
const { rungs } = setup(dome);
|
||||
const lawn = await rungs.resolve(
|
||||
ctx({ ...offLawn, target: 'the Texas Capitol grounds', groundsLike: true }),
|
||||
ladder,
|
||||
);
|
||||
assert.equal(lawn?.outlineSource, 'openfreemap');
|
||||
assert.equal(lawn.footprintKind, 'area');
|
||||
});
|
||||
@@ -0,0 +1,407 @@
|
||||
// Area annotations draw bundled US Census place and neighborhood outlines
|
||||
// (San Francisco DataSF first, then Who's On First) with no network lookup.
|
||||
import { test } from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { readFileSync } from 'node:fs';
|
||||
import { FEATURE_SOURCE_METHODS } from '../sources/featureSource.js';
|
||||
import { createAnnotationResolver } from './resolver.js';
|
||||
|
||||
function viewerAt(lat, lon, height = 20_000) {
|
||||
return {
|
||||
camera: {
|
||||
positionCartographic: {
|
||||
latitude: (lat * Math.PI) / 180,
|
||||
longitude: (lon * Math.PI) / 180,
|
||||
height,
|
||||
},
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
/** A resolver whose boundary lookups and geocoder record every call. */
|
||||
function harness(place = null, { outlineRungs } = {}) {
|
||||
const lookups = [];
|
||||
const featureSource = {};
|
||||
for (const method of FEATURE_SOURCE_METHODS)
|
||||
featureSource[method] = async () => {
|
||||
lookups.push(method);
|
||||
return null;
|
||||
};
|
||||
const geocodes = [];
|
||||
const placeSearch = {
|
||||
async geocode(query) {
|
||||
geocodes.push(query);
|
||||
return { place };
|
||||
},
|
||||
async textSearch() {
|
||||
geocodes.push('text-search');
|
||||
return [];
|
||||
},
|
||||
};
|
||||
const { resolveAnnotationTarget } = createAnnotationResolver({
|
||||
featureSource,
|
||||
...(outlineRungs ? { outlineRungs } : {}),
|
||||
});
|
||||
const resolve = (target, viewer, extra = {}) =>
|
||||
resolveAnnotationTarget({
|
||||
placeSearch,
|
||||
viewer,
|
||||
target,
|
||||
footprint: true,
|
||||
deferFootprint: true,
|
||||
...extra,
|
||||
});
|
||||
return { resolve, lookups, geocodes };
|
||||
}
|
||||
|
||||
const AUSTIN = viewerAt(30.2672, -97.7431, 3000);
|
||||
const SAN_FRANCISCO = viewerAt(37.76, -122.42, 3000);
|
||||
const LONDON = viewerAt(51.51, -0.16, 3000);
|
||||
|
||||
function assertBundledArea(resolved, label) {
|
||||
assert.equal(resolved?.source, 'bundled', label);
|
||||
assert.equal(resolved.label, label);
|
||||
assert.equal(resolved.footprintKind, 'area');
|
||||
assert.equal(resolved.synthesized, false);
|
||||
assert.equal(resolved.resolveOutline, undefined, 'drawn at once');
|
||||
assert.deepEqual(resolved.ring[0], resolved.ring.at(-1), 'ring closed');
|
||||
assert.ok(resolved.polygons.length >= 1);
|
||||
}
|
||||
|
||||
test('a city name that is only a neighborhood alias goes to the geocoder', async () => {
|
||||
const { resolve, geocodes } = harness();
|
||||
// Sé lists "São Paulo" among its names; the ask means the city.
|
||||
const saoPaulo = viewerAt(-23.55, -46.633, 60_000);
|
||||
const city = await resolve('São Paulo', saoPaulo);
|
||||
assert.notEqual(city?.label, 'Se');
|
||||
assert.ok(geocodes.includes('São Paulo'));
|
||||
// A district ask still takes the alias.
|
||||
const district = await resolve('São Paulo', saoPaulo, {
|
||||
entityKind: 'district',
|
||||
});
|
||||
assert.equal(district?.label, 'Se');
|
||||
});
|
||||
|
||||
test('"Austin" over Austin draws the Census place with no lookup', async () => {
|
||||
const { resolve, lookups, geocodes } = harness();
|
||||
for (const target of ['Austin', 'Austin, Texas', 'City of Austin']) {
|
||||
const resolved = await resolve(target, AUSTIN);
|
||||
assertBundledArea(resolved, 'Austin');
|
||||
assert.ok(resolved.polygons.length >= 4, 'every part of the city');
|
||||
}
|
||||
// A qualified name needs no view.
|
||||
const springfield = await resolve('Springfield, Illinois', AUSTIN);
|
||||
assertBundledArea(springfield, 'Springfield');
|
||||
assert.ok(Math.abs(springfield.lat - 39.78) < 0.2);
|
||||
assert.deepEqual(geocodes, []);
|
||||
assert.deepEqual(lookups, []);
|
||||
});
|
||||
|
||||
test("the SF DataSF pack answers before Who's On First in San Francisco", async () => {
|
||||
const { resolve, lookups, geocodes } = harness();
|
||||
const sf = JSON.parse(
|
||||
readFileSync(
|
||||
new URL(
|
||||
'../data/local_data/neighborhoods/san-francisco.json',
|
||||
import.meta.url,
|
||||
),
|
||||
),
|
||||
);
|
||||
const mission = sf.features.find((f) => f.properties.name === 'Mission');
|
||||
const missionRing = mission.geometry.coordinates[0];
|
||||
for (const target of ['the Mission District', 'Mission', 'the Mission']) {
|
||||
const resolved = await resolve(target, SAN_FRANCISCO, {
|
||||
entityKind: 'district',
|
||||
});
|
||||
assertBundledArea(resolved, 'Mission');
|
||||
assert.deepEqual(resolved.polygons[0][0], missionRing, target);
|
||||
}
|
||||
// A name only WOF has in San Francisco falls through to it.
|
||||
const dogpatch = await resolve('Dogpatch', SAN_FRANCISCO);
|
||||
assertBundledArea(dogpatch, 'Dogpatch');
|
||||
// A park asked for as a compound is not a neighborhood of its name.
|
||||
await resolve('Mission Dolores Park', SAN_FRANCISCO, {
|
||||
entityKind: 'compound',
|
||||
});
|
||||
assert.equal(geocodes.filter((q) => q !== 'text-search').length, 1);
|
||||
assert.deepEqual(lookups, []);
|
||||
});
|
||||
|
||||
test("Who's On First neighborhoods resolve near the view", async () => {
|
||||
const { resolve, geocodes } = harness();
|
||||
assertBundledArea(await resolve('Notting Hill', LONDON), 'Notting Hill');
|
||||
assertBundledArea(
|
||||
await resolve('Le Marais', viewerAt(48.857, 2.352, 3000)),
|
||||
'Le Marais',
|
||||
);
|
||||
assertBundledArea(
|
||||
await resolve('Williamsburg', viewerAt(40.71, -73.96, 3000)),
|
||||
'Williamsburg',
|
||||
);
|
||||
assert.deepEqual(geocodes, []);
|
||||
});
|
||||
|
||||
test('campuses, qualified hoods and far names still go to the geocoder', async () => {
|
||||
const { resolve, geocodes } = harness();
|
||||
// Stanford is a CDP, but a campus ask wants the campus.
|
||||
const stanford = await resolve('Stanford', viewerAt(37.43, -122.17, 3000), {
|
||||
entityKind: 'compound',
|
||||
});
|
||||
assert.notEqual(stanford?.source, 'bundled');
|
||||
await resolve('the Capitol grounds', AUSTIN);
|
||||
await resolve('Notting Hill', AUSTIN);
|
||||
assert.equal(geocodes.filter((q) => q !== 'text-search').length, 3);
|
||||
// A state settles a place name wherever the camera is: not Brooklyn's
|
||||
// Williamsburg, the Virginia city.
|
||||
const virginia = await resolve(
|
||||
'Williamsburg, Virginia',
|
||||
viewerAt(40.71, -73.96, 3000),
|
||||
);
|
||||
assertBundledArea(virginia, 'Williamsburg');
|
||||
assert.ok(Math.abs(virginia.lat - 37.27) < 0.1, 'Virginia');
|
||||
});
|
||||
|
||||
test('a geocoded city or neighborhood upgrades to the bundled outline', async () => {
|
||||
const city = harness({
|
||||
lat: 30.2672,
|
||||
lng: -97.7431,
|
||||
label: 'Austin, TX, USA',
|
||||
name: 'Austin',
|
||||
types: ['locality', 'political'],
|
||||
});
|
||||
// Seen from far away: the view does not settle the name, the geocoder does.
|
||||
const resolved = await city.resolve('Austin', viewerAt(39, -100, 5_000_000));
|
||||
assert.equal(resolved.source, 'geocode');
|
||||
const outline = await resolved.resolveOutline();
|
||||
assert.equal(outline.adminArea, 'Austin');
|
||||
assert.ok(outline.polygons.length >= 4);
|
||||
assert.deepEqual(city.lookups, [], 'no boundary lookup');
|
||||
|
||||
const hood = harness({
|
||||
lat: 51.5118,
|
||||
lng: -0.2046,
|
||||
label: 'Notting Hill, London, UK',
|
||||
name: 'Notting Hill',
|
||||
types: ['neighborhood', 'political'],
|
||||
});
|
||||
const nottingHill = await hood.resolve(
|
||||
'Notting Hill',
|
||||
viewerAt(48, 10, 5_000_000),
|
||||
);
|
||||
const hoodOutline = await nottingHill.resolveOutline();
|
||||
assert.equal(hoodOutline.adminArea, 'Notting Hill');
|
||||
assert.deepEqual(hood.lookups, []);
|
||||
});
|
||||
|
||||
test('a district ask prefers a neighborhood; a place must hold the view', async () => {
|
||||
const { resolve, geocodes } = harness();
|
||||
// "Richmond" over SF's Richmond District: never Richmond, California.
|
||||
const richmond = await resolve('Richmond', viewerAt(37.78, -122.48, 3000), {
|
||||
entityKind: 'district',
|
||||
});
|
||||
assertBundledArea(richmond, 'Richmond District');
|
||||
assert.ok(Math.abs(richmond.lat - 37.78) < 0.05, 'not the East Bay city');
|
||||
// Omitted kind at a street-level view: the district in view, too.
|
||||
const bare = await resolve('Richmond', viewerAt(37.78, -122.48, 3000));
|
||||
assertBundledArea(bare, 'Richmond District');
|
||||
// Over the East Bay city, the city.
|
||||
const wide = await resolve('Richmond', viewerAt(37.94, -122.35, 40_000));
|
||||
assert.equal(wide?.source, 'bundled');
|
||||
assert.ok(wide.lat > 37.88, 'Richmond, California');
|
||||
assert.deepEqual(geocodes, []);
|
||||
});
|
||||
|
||||
test('landmark-named neighborhoods wait for the landmark ladder', async () => {
|
||||
const manhattan = viewerAt(40.78, -73.965, 3000);
|
||||
const cases = [
|
||||
['Central Park', {}],
|
||||
['Central Park', { entityKind: 'compound' }],
|
||||
['Central Park', { intent: 'around_the_thing' }],
|
||||
['Hyde Park', {}],
|
||||
['Notting Hill', { intent: 'around_the_thing' }],
|
||||
];
|
||||
for (const [target, extra] of cases) {
|
||||
const { resolve, geocodes } = harness();
|
||||
const viewer = target === 'Central Park' ? manhattan : LONDON;
|
||||
const resolved = await resolve(target, viewer, extra);
|
||||
assert.notEqual(
|
||||
resolved?.source,
|
||||
'bundled',
|
||||
`${target} ${JSON.stringify(extra)}`,
|
||||
);
|
||||
assert.ok(geocodes.length >= 1, `${target} is geocoded`);
|
||||
}
|
||||
// Named as a district, the WOF neighborhood answers.
|
||||
const { resolve } = harness();
|
||||
const park = await resolve('Central Park', manhattan, {
|
||||
entityKind: 'district',
|
||||
});
|
||||
assert.equal(park?.source, 'bundled');
|
||||
// "Park Slope" ends in no landmark word: still a neighborhood.
|
||||
const slope = await resolve('Park Slope', viewerAt(40.671, -73.98, 3000));
|
||||
assertBundledArea(slope, 'Park Slope');
|
||||
});
|
||||
|
||||
/** Outline rungs whose Nominatim answers `answer` (null: nothing found). */
|
||||
function rungsAnswering(answer) {
|
||||
const asked = [];
|
||||
return {
|
||||
asked,
|
||||
resolve: async (ctx, { base, finish }) => {
|
||||
asked.push({ scope: ctx.scope, target: ctx.target });
|
||||
const baseResult = await base();
|
||||
if (baseResult?.ring && !baseResult.synthesized) return baseResult;
|
||||
return answer ? finish(answer, ctx.scope) : baseResult;
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
const PARIS_PLACE = {
|
||||
lat: 48.8566,
|
||||
lng: 2.3522,
|
||||
label: 'Paris, France',
|
||||
name: 'Paris',
|
||||
types: ['locality', 'political'],
|
||||
};
|
||||
const square = (lat, lon, d) => [
|
||||
[lon - d, lat - d],
|
||||
[lon + d, lat - d],
|
||||
[lon + d, lat + d],
|
||||
[lon - d, lat + d],
|
||||
[lon - d, lat - d],
|
||||
];
|
||||
|
||||
test('a coarse bundled admin-1 city waits for a finer outline', async () => {
|
||||
const overParis = viewerAt(48.8566, 2.3522, 40_000);
|
||||
const ring = square(48.8566, 2.3522, 0.05);
|
||||
const rungs = rungsAnswering({ ring, kind: 'area', heightM: null });
|
||||
const fine = harness(PARIS_PLACE, { outlineRungs: rungs });
|
||||
const resolved = await fine.resolve('Paris', overParis);
|
||||
assert.equal(
|
||||
resolved.source,
|
||||
'geocode',
|
||||
'the 11-vertex admin-1 is not drawn at once',
|
||||
);
|
||||
const outline = await resolved.resolveOutline();
|
||||
assert.deepEqual(outline.ring, ring, 'the finer outline wins');
|
||||
assert.deepEqual(rungs.asked, [{ scope: 'city', target: 'Paris' }]);
|
||||
|
||||
// Nothing finer (Nominatim off, busy or capped): the bundled shape, not a point.
|
||||
const coarse = harness(PARIS_PLACE, { outlineRungs: rungsAnswering(null) });
|
||||
const fallback = await (
|
||||
await coarse.resolve('Paris', overParis)
|
||||
).resolveOutline();
|
||||
assert.equal(fallback.adminArea, 'Paris');
|
||||
assert.ok(fallback.ring.length < 32);
|
||||
|
||||
// No geocoder answer at all: the bundled shape at once.
|
||||
const offline = harness(null, { outlineRungs: rungsAnswering(null) });
|
||||
const direct = await offline.resolve('Paris', overParis);
|
||||
assert.equal(direct.source, 'bundled');
|
||||
assert.equal(direct.label, 'Paris');
|
||||
});
|
||||
|
||||
test('detailed admin-1 units and admin asks keep the bundled shape', async () => {
|
||||
const { resolve } = harness(null, { outlineRungs: rungsAnswering(null) });
|
||||
for (const [name, lat, lon] of [
|
||||
['Berlin', 52.52, 13.4],
|
||||
['Bavaria', 48.9, 11.4],
|
||||
['Texas', 31, -99],
|
||||
]) {
|
||||
const resolved = await resolve(name, viewerAt(lat, lon, 400_000));
|
||||
assert.equal(resolved?.source, 'bundled', name);
|
||||
}
|
||||
// Countries are never coarse cities, however few vertices.
|
||||
const singapore = await resolve('Singapore', viewerAt(1.35, 103.82, 60_000));
|
||||
assert.equal(singapore?.source, 'bundled');
|
||||
// An explicit admin ask takes the admin unit as it is.
|
||||
const dc = await resolve('Paris', viewerAt(48.8566, 2.3522, 40_000), {
|
||||
entityKind: 'state',
|
||||
});
|
||||
assert.equal(dc?.source, 'bundled');
|
||||
});
|
||||
|
||||
test('a campus ask reaches the campus, not the Census place of its town', async () => {
|
||||
const stanfordTown = {
|
||||
lat: 37.4241,
|
||||
lng: -122.1661,
|
||||
label: 'Stanford, CA, USA',
|
||||
name: 'Stanford',
|
||||
types: ['locality', 'political'],
|
||||
};
|
||||
const campus = square(37.4275, -122.1697, 0.01);
|
||||
for (const ask of ['Stanford University', 'Stanford campus']) {
|
||||
const rungs = rungsAnswering({ ring: campus, kind: 'area', heightM: null });
|
||||
const { resolve } = harness(stanfordTown, { outlineRungs: rungs });
|
||||
const resolved = await resolve(ask, viewerAt(37.43, -122.17, 8000));
|
||||
assert.notEqual(resolved?.source, 'bundled', ask);
|
||||
const outline = await resolved.resolveOutline();
|
||||
assert.deepEqual(outline.ring, campus, ask);
|
||||
assert.equal(
|
||||
rungs.asked[0].scope,
|
||||
'compound',
|
||||
`${ask}: past the Census rung`,
|
||||
);
|
||||
}
|
||||
// "Stanford" said to be a compound is the campus too.
|
||||
const kindRungs = rungsAnswering({
|
||||
ring: campus,
|
||||
kind: 'area',
|
||||
heightM: null,
|
||||
});
|
||||
const byKind = harness(stanfordTown, { outlineRungs: kindRungs });
|
||||
const compound = await (
|
||||
await byKind.resolve('Stanford', viewerAt(37.43, -122.17, 8000), {
|
||||
entityKind: 'compound',
|
||||
})
|
||||
).resolveOutline();
|
||||
assert.deepEqual(compound.ring, campus);
|
||||
assert.equal(kindRungs.asked[0].scope, 'compound');
|
||||
// "Stanford" alone is the town (the Stanford CDP), and a town named after a
|
||||
// college stays a town.
|
||||
const { resolve } = harness(stanfordTown, {
|
||||
outlineRungs: rungsAnswering(null),
|
||||
});
|
||||
const town = await (
|
||||
await resolve('Stanford', viewerAt(39, -100, 5_000_000))
|
||||
).resolveOutline();
|
||||
assert.equal(town.adminArea, 'Stanford');
|
||||
const station = harness(
|
||||
{
|
||||
lat: 30.628,
|
||||
lng: -96.3344,
|
||||
label: 'College Station, TX, USA',
|
||||
name: 'College Station',
|
||||
types: ['locality', 'political'],
|
||||
},
|
||||
{ outlineRungs: rungsAnswering(null) },
|
||||
);
|
||||
const collegeStation = await (
|
||||
await station.resolve('College Station', viewerAt(39, -100, 5_000_000))
|
||||
).resolveOutline();
|
||||
assert.equal(collegeStation.adminArea, 'College Station');
|
||||
});
|
||||
|
||||
test('a grounds ask never takes the state the geocoder typed', async () => {
|
||||
const texas = {
|
||||
lat: 31.0,
|
||||
lng: -99.0,
|
||||
label: 'Texas, USA',
|
||||
name: 'Texas',
|
||||
types: ['administrative_area_level_1', 'political'],
|
||||
};
|
||||
for (const [ask, extra] of [
|
||||
['the Texas Capitol grounds', {}],
|
||||
['Texas Capitol', { entityKind: 'compound' }],
|
||||
// The grounds wording is only in the label.
|
||||
['Texas Capitol', { labelHint: 'Capitol grounds' }],
|
||||
]) {
|
||||
const rungs = rungsAnswering(null);
|
||||
const { resolve } = harness(texas, { outlineRungs: rungs });
|
||||
const resolved = await resolve(ask, viewerAt(31, -99, 3_000_000), extra);
|
||||
assert.notEqual(resolved?.source, 'bundled', ask);
|
||||
const outline = await resolved.resolveOutline();
|
||||
assert.notEqual(outline?.adminArea, 'Texas', ask);
|
||||
assert.equal(rungs.asked[0].scope, 'compound', ask);
|
||||
}
|
||||
});
|
||||
+299
-12
@@ -19,18 +19,50 @@ import {
|
||||
lookupNaturalRegionOutline,
|
||||
findNaturalRegion,
|
||||
} from '../data/naturalEarthRegions.js';
|
||||
import { findAdminArea, findAdminAreaAt } from '../data/adminBoundaries.js';
|
||||
import {
|
||||
findAdminArea,
|
||||
findAdminAreaAt,
|
||||
polygonsContain as polygonsContainPoint,
|
||||
} from '../data/adminBoundaries.js';
|
||||
import {
|
||||
findNeighborhoodArea,
|
||||
findNeighborhoodAreaAt,
|
||||
findPlaceArea,
|
||||
findPlaceAreaAt,
|
||||
NEIGHBORHOOD_NEAR_KM,
|
||||
normalizePlaceName,
|
||||
PLACE_NEAR_KM,
|
||||
} from '../data/placeBoundaries.js';
|
||||
import {
|
||||
registerDynamicCredit,
|
||||
NATURAL_EARTH_CREDIT,
|
||||
US_CENSUS_CREDIT,
|
||||
OSM_CREDIT,
|
||||
OPENMAPTILES_CREDIT,
|
||||
WOF_CREDIT,
|
||||
} from '../data/dataCredits.js';
|
||||
import { createOutlineRungs } from './outlineRungs.js';
|
||||
import { unavailablePlaceSearch } from '../search/placeSearch.js';
|
||||
import { isPickedWorldPosition } from '../data/scenePick.js';
|
||||
|
||||
/** Ms the analyst region lookup waits on geocode + admin boundary. */
|
||||
export const REGION_FALLBACK_BUDGET_MS = 3_000;
|
||||
|
||||
/**
|
||||
* Names that end like a landmark rather than a district: "Central Park",
|
||||
* "Lake Merritt"-style suffixes are checked on the last word only, so "Park
|
||||
* Slope" and "Notting Hill" stay districts.
|
||||
*/
|
||||
const LANDMARK_NAME =
|
||||
/\b(?:park|gardens?|square|plaza|lake|river|bay|beach|pier|harbou?r|airport|station|stadium|arena|cemetery|zoo|university|college|campus|palace|castle|cathedral|church|temple|mosque|bridge|market|mall|cent(?:er|re)|tower|museum|reservoir|forest|woods?|heath|common|green)$/i;
|
||||
|
||||
/** The name part of an ask, before any comma qualifier. */
|
||||
function stripQualifiers(query) {
|
||||
return String(query || '')
|
||||
.split(',')[0]
|
||||
.trim();
|
||||
}
|
||||
|
||||
/** Entity facts that rule out a state/county reading of the ask. */
|
||||
const NON_ADMIN_ENTITY_KINDS = new Set([
|
||||
'building',
|
||||
@@ -47,6 +79,8 @@ export function createAnnotationResolver({
|
||||
boundarySource,
|
||||
signal: lifetime,
|
||||
}),
|
||||
// Overpass-free rungs (OpenFreeMap tiles, guarded Nominatim) around the ladder.
|
||||
outlineRungs = createOutlineRungs(),
|
||||
} = {}) {
|
||||
requireFeatureSource(featureSource);
|
||||
lifetime?.throwIfAborted();
|
||||
@@ -176,6 +210,11 @@ export function createAnnotationResolver({
|
||||
// locality suffixes; the types classify the entity (point-like vs area-like).
|
||||
let placesPrimary = null;
|
||||
let placeTypes = [];
|
||||
// A coarse bundled admin-1 shape that carries a city's name (Natural
|
||||
// Earth's 11-vertex "Paris"): kept as the fallback while the finer rungs
|
||||
// (Census, Nominatim) try first.
|
||||
let coarseCityShape = null;
|
||||
const askNamesAdmin = Boolean(adminScopeFromAsk(target, entityKind));
|
||||
// Instrumentation (logged once per target at the end): which sources were tried and what they returned.
|
||||
const trace = {
|
||||
query: String(target || '').trim(),
|
||||
@@ -201,7 +240,26 @@ export function createAnnotationResolver({
|
||||
() => null,
|
||||
);
|
||||
signal?.throwIfAborted();
|
||||
if (admin) return bundledAdminTarget(viewer, admin, trace.query);
|
||||
if (admin && !askNamesAdmin && isCoarseCityShape(admin))
|
||||
coarseCityShape = admin;
|
||||
else if (admin) return bundledAdminTarget(viewer, admin, trace.query);
|
||||
// Bundled US places and neighborhoods named at or near the view. A
|
||||
// campus, grounds or monument ask ("Stanford" as a compound, "the
|
||||
// Capitol grounds") wants that feature, not a district of its name,
|
||||
// and "the area around X" is drawn by the ladder below.
|
||||
if (
|
||||
entityKind !== 'compound' &&
|
||||
intent !== 'around_the_thing' &&
|
||||
!isGroundsLikeQuery(trace.query) &&
|
||||
!isMonumentLikeQuery(trace.query)
|
||||
) {
|
||||
const area = await bundledAreaNear(trace.query, center, {
|
||||
entityKind,
|
||||
viewRadiusKm: viewportProximity(viewer)?.radiusKm ?? null,
|
||||
});
|
||||
signal?.throwIfAborted();
|
||||
if (area) return bundledAdminTarget(viewer, area, trace.query);
|
||||
}
|
||||
}
|
||||
|
||||
if (!Number.isFinite(lat) || !Number.isFinite(lon)) {
|
||||
@@ -329,7 +387,10 @@ export function createAnnotationResolver({
|
||||
}
|
||||
}
|
||||
|
||||
if (!Number.isFinite(lat) || !Number.isFinite(lon)) return null;
|
||||
if (!Number.isFinite(lat) || !Number.isFinite(lon))
|
||||
return coarseCityShape
|
||||
? bundledAdminTarget(viewer, coarseCityShape, trace.query)
|
||||
: null;
|
||||
|
||||
// Monument/memorial/statue names geocode unreliably — Google scatters them across the city (e.g.
|
||||
// several Texas Capitol monuments landed blocks-to-miles apart, looking "all over the map"). When
|
||||
@@ -394,7 +455,9 @@ export function createAnnotationResolver({
|
||||
`osmSnap=${trace.osmSnap} → FINAL source=rejected (proximity gate, ${drift.driftKm.toFixed(0)}km > ${drift.limitKm.toFixed(0)}km)`,
|
||||
);
|
||||
}
|
||||
return null;
|
||||
return coarseCityShape
|
||||
? bundledAdminTarget(viewer, coarseCityShape, trace.query)
|
||||
: null;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -414,7 +477,23 @@ export function createAnnotationResolver({
|
||||
// an admin boundary for a city/state, the enclosing compound for a mall/campus, a single
|
||||
// building for a premise. The voice model's entityKind (an entity FACT) refines an
|
||||
// unresolved 'auto' scope only — real geocode types always win.
|
||||
const baseScope = refineScope(scopeFromTypes(geocodeTypes), entityKind);
|
||||
// A campus or grounds ask ("Stanford University", "Stanford campus",
|
||||
// "the Capitol grounds", in the target or its label) or a compound by
|
||||
// entity fact wants that compound even when the geocoder types the town
|
||||
// or the state: never a Census place, neighborhood or admin boundary.
|
||||
const typedScope = refineScope(scopeFromTypes(geocodeTypes), entityKind);
|
||||
const compoundAsk =
|
||||
entityKind === 'compound' ||
|
||||
isCampusAsk(target) ||
|
||||
isGroundsLikeQuery(target) ||
|
||||
isGroundsLikeQuery(labelHint);
|
||||
const baseScope =
|
||||
compoundAsk &&
|
||||
['country', 'state', 'county', 'city', 'neighborhood'].includes(
|
||||
typedScope,
|
||||
)
|
||||
? 'compound'
|
||||
: typedScope;
|
||||
// Point-like targets (monuments/statues/memorials/…) resolve POINT-FIRST: only an
|
||||
// (almost) exactly-named, monument-scale polygon may replace the point; a nearby polygon
|
||||
// sharing locality words must not (docs/field-test-rootcause-2026-06-30.md §1).
|
||||
@@ -443,7 +522,7 @@ export function createAnnotationResolver({
|
||||
* The inline path awaits it right here; progressive callers (deferFootprint) invoke it
|
||||
* AFTER the anchor mark is drawn and upgrade the mark in place.
|
||||
*/
|
||||
const resolveOutline = async () => {
|
||||
const resolveBaseOutline = async () => {
|
||||
let scope = baseScope;
|
||||
const isAdmin =
|
||||
scope === 'country' ||
|
||||
@@ -471,6 +550,7 @@ export function createAnnotationResolver({
|
||||
);
|
||||
if (ne) {
|
||||
registerDynamicCredit(viewer, NATURAL_EARTH_CREDIT);
|
||||
traceBundledOutline(ne.name, 'natural-earth');
|
||||
const neCentroid = ringCentroid(ne.ring);
|
||||
return {
|
||||
ring: ne.ring,
|
||||
@@ -526,7 +606,18 @@ export function createAnnotationResolver({
|
||||
lon,
|
||||
scope,
|
||||
).catch(() => null);
|
||||
if (admin) return bundledAdminOutline(viewer, admin);
|
||||
if (admin && !askNamesAdmin && isCoarseCityShape(admin))
|
||||
coarseCityShape ||= admin;
|
||||
else if (admin) return bundledAdminOutline(viewer, admin);
|
||||
}
|
||||
// A geocoded US city, town or CDP: the Census place that holds it.
|
||||
if (scope === 'city') {
|
||||
const place = await findPlaceAreaAt(
|
||||
[target, matchName],
|
||||
lat,
|
||||
lon,
|
||||
).catch(() => null);
|
||||
if (place) return bundledAdminOutline(viewer, place);
|
||||
}
|
||||
// Pure admin: only an admin boundary is correct — never fall back to a
|
||||
// building/landuse (a city is never a single building).
|
||||
@@ -537,7 +628,19 @@ export function createAnnotationResolver({
|
||||
// resolve here instantly to a REAL boundary, sidestepping the slow/flaky live-Overpass
|
||||
// path that times out and falls back to points (see docs/field-test-2-analysis.md).
|
||||
const ext = await lookupNeighborhoodRing(lat, lon, matchName);
|
||||
if (ext) fp = { ring: ext.ring, kind: 'area', heightM: null };
|
||||
if (ext) {
|
||||
traceBundledOutline(ext.name || matchName, 'datasf');
|
||||
fp = { ring: ext.ring, kind: 'area', heightM: null };
|
||||
}
|
||||
// THEN a bundled Who's On First neighborhood carrying the name at the point.
|
||||
if (!fp) {
|
||||
const wof = await findNeighborhoodAreaAt(
|
||||
[target, matchName],
|
||||
lat,
|
||||
lon,
|
||||
).catch(() => null);
|
||||
if (wof) return bundledAdminOutline(viewer, wof);
|
||||
}
|
||||
// Else fall through to the OSM admin/place → named-landuse → synthesis ladder. Each
|
||||
// returns a footprint, null (definitively no polygon), or undefined (transient
|
||||
// upstream failure). Synthesize a blob (the "Mission" problem, research §3b) ONLY
|
||||
@@ -663,6 +766,11 @@ export function createAnnotationResolver({
|
||||
}
|
||||
}
|
||||
}
|
||||
return finishOutline(fp, scope);
|
||||
};
|
||||
|
||||
/** Validate a footprint for its scope and re-centre it (resolveOutline's patch). */
|
||||
const finishOutline = (fp, scope) => {
|
||||
// Scope sanity: reject a real footprint whose area is wildly wrong for the asked
|
||||
// scope (a building/compound/neighborhood intent must never draw a state-sized
|
||||
// blob). Synthesized discs are deliberately sized and exempt.
|
||||
@@ -693,17 +801,51 @@ export function createAnnotationResolver({
|
||||
footprintKind: fp.kind,
|
||||
buildingHeight: fp.heightM,
|
||||
synthesized: Boolean(fp.synthesized),
|
||||
...(Array.isArray(fp.polygons) ? { polygons: fp.polygons } : {}),
|
||||
lat: centroid.lat,
|
||||
lon: centroid.lon,
|
||||
height: sampleGroundHeight(viewer, centroid.lon, centroid.lat),
|
||||
};
|
||||
};
|
||||
|
||||
// Streets and pointed-at buildings from tiles, then this ladder, then the
|
||||
// guarded Nominatim outline (see outlineRungs.js).
|
||||
const outlineContext = {
|
||||
scope: baseScope,
|
||||
target,
|
||||
matchName,
|
||||
lat,
|
||||
lon,
|
||||
fromName: fromGeocode || source === 'places' || source === 'osm-local',
|
||||
pointLike,
|
||||
groundsLike,
|
||||
around: intent === 'around_the_thing',
|
||||
view: viewportProximity(viewer),
|
||||
signal,
|
||||
credit: (kind) =>
|
||||
registerDynamicCredit(
|
||||
viewer,
|
||||
kind === 'tiles' ? OPENMAPTILES_CREDIT : OSM_CREDIT,
|
||||
),
|
||||
};
|
||||
const resolveOutline = async () => {
|
||||
const result = await outlineRungs.resolve(outlineContext, {
|
||||
base: resolveBaseOutline,
|
||||
finish: finishOutline,
|
||||
});
|
||||
// Nothing finer answered (no polygon, Nominatim off, busy or capped):
|
||||
// the coarse bundled shape beats a point.
|
||||
if (coarseCityShape && !(result?.ring && !result.synthesized))
|
||||
return bundledAdminOutline(viewer, coarseCityShape);
|
||||
return result;
|
||||
};
|
||||
|
||||
let ring = null;
|
||||
let footprintKind = null; // 'building' | 'area'
|
||||
let buildingHeight = null; // meters, only for buildings
|
||||
let outlineUnavailable = false;
|
||||
let synthesized = false; // true = buffered/approximate area, render dashed/feathered
|
||||
let polygons = null; // every part of a multi-part outline, when known
|
||||
if (footprint && !deferFootprint) {
|
||||
const fp = await resolveOutline();
|
||||
outlineUnavailable = isUnavailableCapability(fp);
|
||||
@@ -712,6 +854,7 @@ export function createAnnotationResolver({
|
||||
footprintKind = fp.footprintKind;
|
||||
buildingHeight = fp.buildingHeight;
|
||||
synthesized = fp.synthesized;
|
||||
polygons = fp.polygons || null;
|
||||
// Re-center the anchor on the resolved footprint centroid.
|
||||
lat = fp.lat;
|
||||
lon = fp.lon;
|
||||
@@ -739,6 +882,7 @@ export function createAnnotationResolver({
|
||||
source,
|
||||
synthesized,
|
||||
outlineUnavailable,
|
||||
...(polygons ? { polygons } : {}),
|
||||
viewport: placeViewport,
|
||||
...(footprint && deferFootprint ? { resolveOutline } : {}),
|
||||
};
|
||||
@@ -778,13 +922,109 @@ export function createAnnotationResolver({
|
||||
const GROUNDS_RADIUS_M = 300; // "X grounds/compound/campus" loose disc when OSM has no polygon
|
||||
const GROUNDS_RADIUS_MIN_M = 150; // viewport-derived grounds disc is clamped to this band so a tiny
|
||||
const GROUNDS_RADIUS_MAX_M = 1200; // place can't shrink to a dot, nor a city-wide viewport balloon
|
||||
const COARSE_CITY_MAX_KM2 = 5_000; // city-sized admin-1 units (see isCoarseCityShape)
|
||||
const COARSE_CITY_MAX_VERTICES = 32;
|
||||
|
||||
/** Credit the pack a bundled boundary came from. */
|
||||
function creditAdminSource(viewer, admin) {
|
||||
registerDynamicCredit(
|
||||
viewer,
|
||||
admin.source === 'us-census' ? US_CENSUS_CREDIT : NATURAL_EARTH_CREDIT,
|
||||
);
|
||||
const credit = {
|
||||
'natural-earth': NATURAL_EARTH_CREDIT,
|
||||
'us-census': US_CENSUS_CREDIT,
|
||||
wof: WOF_CREDIT,
|
||||
}[admin.source];
|
||||
if (credit) registerDynamicCredit(viewer, credit);
|
||||
}
|
||||
|
||||
/**
|
||||
* The bundled US place or neighborhood an ask names at or near the view,
|
||||
* from Census places, the San Francisco DataSF pack (first in SF), then
|
||||
* Who's On First. How far "near" reaches follows the view (half its
|
||||
* radius); a state-qualified place ("Springfield, Illinois") needs no
|
||||
* view. A district ask takes a neighborhood first and a place only when
|
||||
* the view is inside it ("Richmond" over SF is not Richmond, CA). Without
|
||||
* an entity kind the candidate that holds the view wins, else the nearer.
|
||||
* A name that ends like a landmark ("Central Park", "Lake Merritt") is
|
||||
* left to the landmark ladder unless the ask is a district.
|
||||
*/
|
||||
async function bundledAreaNear(
|
||||
query,
|
||||
center,
|
||||
{ entityKind = null, viewRadiusKm = null } = {},
|
||||
) {
|
||||
const district = entityKind === 'district';
|
||||
if (!district && LANDMARK_NAME.test(stripQualifiers(query))) return null;
|
||||
const reach = Number.isFinite(viewRadiusKm) ? viewRadiusKm / 2 : null;
|
||||
const clamp = (km, lo, hi) =>
|
||||
km === null ? hi : Math.min(hi, Math.max(lo, km));
|
||||
const placeReach = district ? 0 : clamp(reach, 3, PLACE_NEAR_KM);
|
||||
const place = await findPlaceArea(query, {
|
||||
near: center,
|
||||
nearKm: placeReach,
|
||||
}).catch(() => null);
|
||||
const holds = (area) =>
|
||||
Boolean(center) &&
|
||||
polygonsContainPoint(area.polygons, center.lat, center.lon);
|
||||
// A district ask only takes a place the view is inside (or one a state
|
||||
// qualifier named outright).
|
||||
const placeOk =
|
||||
place && (!district || holds(place) || /,/.test(query)) ? place : null;
|
||||
if (!center) return placeOk;
|
||||
let hood = null;
|
||||
const sf = await lookupNeighborhoodRing(center.lat, center.lon, query, {
|
||||
exact: true,
|
||||
}).catch(() => null);
|
||||
if (sf) hood = sfNeighborhoodArea(sf);
|
||||
else
|
||||
hood = await findNeighborhoodArea(query, {
|
||||
near: center,
|
||||
nearKm: clamp(reach, 1, NEIGHBORHOOD_NEAR_KM),
|
||||
}).catch(() => null);
|
||||
// Outside a district ask, an alternate name unrelated to the name is too
|
||||
// weak: Se in Sao Paulo lists "Sao Paulo" among its names, and "outline
|
||||
// Sao Paulo" means the city ("Richmond" for Richmond District still counts).
|
||||
if (
|
||||
!district &&
|
||||
hood?.byAlias &&
|
||||
!normalizePlaceName(hood.name).includes(
|
||||
normalizePlaceName(stripQualifiers(query)),
|
||||
)
|
||||
)
|
||||
hood = null;
|
||||
if (!hood || !placeOk) return district ? hood || placeOk : placeOk || hood;
|
||||
if (district) return hood;
|
||||
const hoodHolds = holds(hood);
|
||||
const placeHolds = holds(placeOk);
|
||||
if (hoodHolds !== placeHolds) return hoodHolds ? hood : placeOk;
|
||||
const distance = (area) =>
|
||||
approximateDistanceM(
|
||||
center.lat,
|
||||
center.lon,
|
||||
area.label.lat,
|
||||
area.label.lon,
|
||||
);
|
||||
return distance(hood) <= distance(placeOk) ? hood : placeOk;
|
||||
}
|
||||
|
||||
/** An SF DataSF ring in the bundled-area shape `bundledAdminTarget` takes. */
|
||||
function sfNeighborhoodArea({ ring, name }) {
|
||||
const lons = ring.map((p) => p[0]);
|
||||
const lats = ring.map((p) => p[1]);
|
||||
return {
|
||||
kind: 'neighborhood',
|
||||
name,
|
||||
region: null,
|
||||
source: 'datasf',
|
||||
polygons: [[ring]],
|
||||
ring,
|
||||
bbox: [
|
||||
Math.min(...lons),
|
||||
Math.min(...lats),
|
||||
Math.max(...lons),
|
||||
Math.max(...lats),
|
||||
],
|
||||
label: ringCentroid(ring),
|
||||
candidates: 1,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -796,6 +1036,7 @@ export function createAnnotationResolver({
|
||||
*/
|
||||
function bundledAdminOutline(viewer, admin) {
|
||||
creditAdminSource(viewer, admin);
|
||||
traceBundledOutline(admin.name, admin.source, admin.polygons?.length);
|
||||
const { lat, lon } = admin.label;
|
||||
return {
|
||||
ring: closeRing([...admin.ring]), // copy: closeRing mutates, the pack is shared
|
||||
@@ -810,6 +1051,31 @@ export function createAnnotationResolver({
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether a bundled unit is too coarse to stand for a city of its name: a
|
||||
* Natural Earth admin-1 at city size (up to 5,000 km²) drawn with fewer
|
||||
* than 32 vertices. Paris (11), Buenos Aires (11), Washington DC (9),
|
||||
* Vienna and Prague (23) and Seoul (28) are; Berlin (48), Hamburg (43),
|
||||
* Bremen (39) and every larger state are not. Countries never are.
|
||||
*/
|
||||
function isCoarseCityShape(admin) {
|
||||
if (admin?.source !== 'natural-earth' || admin.kind !== 'state')
|
||||
return false;
|
||||
if (!(admin.areaKm2 <= COARSE_CITY_MAX_KM2)) return false;
|
||||
const vertices = (admin.polygons || []).reduce(
|
||||
(n, poly) => n + poly.reduce((m, ring) => m + ring.length, 0),
|
||||
0,
|
||||
);
|
||||
return vertices < COARSE_CITY_MAX_VERTICES;
|
||||
}
|
||||
|
||||
/** One trace line per outline a bundled pack answered (same shape as outlineRungs). */
|
||||
function traceBundledOutline(name, source, parts = 1) {
|
||||
console.log(
|
||||
`[Outline] "${name}": bundled → ${source} (${parts || 1} part(s))`,
|
||||
);
|
||||
}
|
||||
|
||||
/** A complete resolved target for a bundled administrative unit (no outline pending). */
|
||||
function bundledAdminTarget(viewer, admin, query) {
|
||||
const outline = bundledAdminOutline(viewer, admin);
|
||||
@@ -1561,6 +1827,19 @@ export function createAnnotationResolver({
|
||||
/** A target that reads like an enclosing GROUNDS / COMPOUND / CAMPUS (e.g. "Texas Capitol grounds")
|
||||
* rather than a single building — used to synthesize a loose-footprint disc when OSM has no real
|
||||
* polygon, instead of failing outright. */
|
||||
/**
|
||||
* A university, college or campus named as such: "Stanford University",
|
||||
* "Boston College", "University of Texas", "Stanford campus". Towns named
|
||||
* after one ("College Station", "University Park") are not.
|
||||
*/
|
||||
function isCampusAsk(query) {
|
||||
const text = stripQualifiers(String(query || '')).trim();
|
||||
return (
|
||||
/\b(?:university|college|campus)$/i.test(text) ||
|
||||
/^(?:the\s+)?(?:university|college)\s+of\b/i.test(text)
|
||||
);
|
||||
}
|
||||
|
||||
function isGroundsLikeQuery(query) {
|
||||
return /\b(grounds|compound|campus|complex|quad|plaza)\b/i.test(
|
||||
String(query || ''),
|
||||
@@ -2079,6 +2358,14 @@ export function createAnnotationResolver({
|
||||
).catch(() => null);
|
||||
if (admin) return { name: q, ring: [...admin.ring] };
|
||||
}
|
||||
if (scope === 'city') {
|
||||
const place = await findPlaceAreaAt(
|
||||
[q, geo.primaryName],
|
||||
geo.lat,
|
||||
geo.lon,
|
||||
).catch(() => null);
|
||||
if (place) return { name: q, ring: [...place.ring] };
|
||||
}
|
||||
const fp = await fetchAdminArea(geo.lat, geo.lon, q, scope, signal).catch(
|
||||
() => null,
|
||||
);
|
||||
|
||||
@@ -1,293 +0,0 @@
|
||||
/**
|
||||
* Embed mode and views in the running app.
|
||||
*
|
||||
* With `?embed=1` the app shows only the globe: clean view, with the HUD,
|
||||
* welcome and setup prompts hidden. A page that frames it changes what it
|
||||
* shows by posting `{ type: 'gev:view', id, view }`; the app applies the view
|
||||
* through its own actions and answers `{ type: 'gev:view-applied', id, ok,
|
||||
* steps }`. It announces `{ type: 'gev:ready' }` once it can take views.
|
||||
*
|
||||
* Annotations in a share link are drawn once the link has been restored,
|
||||
* embedded or not.
|
||||
*/
|
||||
|
||||
import * as Cesium from 'cesium';
|
||||
import { announceNavigationAuthority } from '../navigationPolicy.js';
|
||||
import { annotationsFromParams, createView } from '../view/index.js';
|
||||
|
||||
export const EMBED_VIEW_MESSAGE = 'gev:view';
|
||||
export const EMBED_APPLIED_MESSAGE = 'gev:view-applied';
|
||||
export const EMBED_READY_MESSAGE = 'gev:ready';
|
||||
|
||||
const FOLLOW_LAYERS = {
|
||||
aircraft: 'flights',
|
||||
military_aircraft: 'military',
|
||||
satellite: 'satellites',
|
||||
};
|
||||
const FOLLOW_ATTEMPTS = 20;
|
||||
const FOLLOW_RETRY_MS = 1000;
|
||||
const FLIGHT_SECONDS = 2;
|
||||
|
||||
/**
|
||||
* Whether this page shows the app embedded: `?embed=1` when another page
|
||||
* frames it, or inline when a panel page loads the app into itself and sets
|
||||
* `globalThis.GEV_EMBED_INLINE` first.
|
||||
*/
|
||||
export function isEmbedded(location = globalThis.location) {
|
||||
return (
|
||||
isEmbeddedInline() ||
|
||||
new URLSearchParams(location?.search || '').get('embed') === '1'
|
||||
);
|
||||
}
|
||||
|
||||
/** Whether a panel page loaded the app into itself. */
|
||||
export function isEmbeddedInline() {
|
||||
return globalThis.GEV_EMBED_INLINE === true;
|
||||
}
|
||||
|
||||
// A panel keeps drawing at about this rate when its host stops animation
|
||||
// frames; see keepPanelRendering.
|
||||
const PANEL_FRAME_MS = 33;
|
||||
const MISSED_FRAMES_MS = 250;
|
||||
|
||||
/** A render error's details, including the plain objects workers report. */
|
||||
function describeError(error) {
|
||||
if (error instanceof Error)
|
||||
return `${error.name}: ${error.message}\n${error.stack}`;
|
||||
try {
|
||||
return JSON.stringify(error, Object.getOwnPropertyNames(error ?? {}));
|
||||
} catch {
|
||||
return String(error);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Keep the globe drawing in an inline panel. Some hosts report a panel on
|
||||
* screen as hidden, which stops the browser's animation frames and with them
|
||||
* Cesium's render loop; while frames stop arriving, draw from a timer.
|
||||
* Returns a function that stops it.
|
||||
*/
|
||||
export function keepPanelRendering(
|
||||
viewer,
|
||||
{ windowRef = globalThis.window, now = () => performance.now() } = {},
|
||||
) {
|
||||
let lastFrame = now();
|
||||
let frameRequest = null;
|
||||
const onFrame = () => {
|
||||
lastFrame = now();
|
||||
frameRequest = windowRef.requestAnimationFrame(onFrame);
|
||||
};
|
||||
frameRequest = windowRef.requestAnimationFrame(onFrame);
|
||||
// Cesium stops drawing after a render error, and so does the timer.
|
||||
// Report the error in full: workers report plain objects, which Cesium
|
||||
// prints as [object Object].
|
||||
let failed = false;
|
||||
const removeErrorListener = viewer.scene?.renderError?.addEventListener(
|
||||
(_scene, error) => {
|
||||
failed = true;
|
||||
console.error(
|
||||
"[God's Eye View panel] render error:",
|
||||
describeError(error),
|
||||
);
|
||||
},
|
||||
);
|
||||
const timer = windowRef.setInterval(() => {
|
||||
if (
|
||||
failed ||
|
||||
now() - lastFrame < MISSED_FRAMES_MS ||
|
||||
viewer.isDestroyed?.()
|
||||
)
|
||||
return;
|
||||
viewer.resize();
|
||||
viewer.render();
|
||||
}, PANEL_FRAME_MS);
|
||||
return () => {
|
||||
windowRef.clearInterval(timer);
|
||||
windowRef.cancelAnimationFrame(frameRequest);
|
||||
removeErrorListener?.();
|
||||
};
|
||||
}
|
||||
|
||||
const delay = (ms, signal) =>
|
||||
new Promise((resolve) => {
|
||||
const timer = setTimeout(resolve, ms);
|
||||
signal?.addEventListener(
|
||||
'abort',
|
||||
() => {
|
||||
clearTimeout(timer);
|
||||
resolve();
|
||||
},
|
||||
{ once: true },
|
||||
);
|
||||
});
|
||||
|
||||
/** Fly the camera to a view's camera and resolve when the flight ends. */
|
||||
function flyTo(viewer, camera) {
|
||||
announceNavigationAuthority('embed-view');
|
||||
return new Promise((resolve) => {
|
||||
viewer.camera.flyTo({
|
||||
destination: Cesium.Cartesian3.fromDegrees(
|
||||
camera.lon,
|
||||
camera.lat,
|
||||
camera.altitude_m,
|
||||
),
|
||||
orientation: {
|
||||
heading: Cesium.Math.toRadians(camera.heading_deg),
|
||||
pitch: Cesium.Math.toRadians(camera.pitch_deg),
|
||||
roll: 0,
|
||||
},
|
||||
duration: FLIGHT_SECONDS,
|
||||
complete: () => resolve(true),
|
||||
cancel: () => resolve(false),
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Make the app show a view: style and map, exactly the view's layers,
|
||||
* annotations, the camera, and the followed entity. Each step runs through the
|
||||
* app's own actions (`run(name, args)`); a failed step is reported and the
|
||||
* rest still run. Resolves to the steps and whether each succeeded.
|
||||
*/
|
||||
export async function applyView(
|
||||
view,
|
||||
{ viewer, dataManager, run, signal, retryMs = FOLLOW_RETRY_MS },
|
||||
) {
|
||||
const steps = [];
|
||||
const act = async (name, args) => {
|
||||
let result = null;
|
||||
try {
|
||||
result = await run(name, args);
|
||||
} catch (error) {
|
||||
result = { ok: false, error: error?.message || String(error) };
|
||||
}
|
||||
steps.push({
|
||||
step: name,
|
||||
ok: result?.ok !== false,
|
||||
...(result?.error ? { error: result.error } : {}),
|
||||
});
|
||||
return result;
|
||||
};
|
||||
if (view.style) await act('set_visual_style', { style: view.style });
|
||||
if (view.map) await act('set_map_stack', { stack: view.map });
|
||||
const wanted = new Set(view.layers);
|
||||
for (const layer of dataManager.getAll()) {
|
||||
if (layer.enabled && !wanted.has(layer.id))
|
||||
await act('set_layer_visibility', { layerId: layer.id, enabled: false });
|
||||
}
|
||||
for (const layerId of view.layers) {
|
||||
if (!dataManager.isEnabled(layerId))
|
||||
await act('set_layer_visibility', { layerId, enabled: true });
|
||||
}
|
||||
if (!view.follow) await act('stop_tracking', {});
|
||||
// Annotations go first: drawing them may frame the marks, and the view's
|
||||
// own camera should have the last word.
|
||||
await act('clear_annotations', {});
|
||||
if (view.annotations.length)
|
||||
await act('annotate_map', { annotations: [...view.annotations] });
|
||||
if (!view.follow) {
|
||||
steps.push({ step: 'camera', ok: await flyTo(viewer, view.camera) });
|
||||
return steps;
|
||||
}
|
||||
// A followed entity owns the camera, and a camera flight would end the
|
||||
// follow, so fly to the view only while the entity is not there yet. It
|
||||
// appears once its layer has data; retry until then.
|
||||
const layerId = FOLLOW_LAYERS[view.follow.kind];
|
||||
const follow = async () => {
|
||||
try {
|
||||
return (
|
||||
(await run('track_entity', { query: view.follow.id, layerId }))?.ok ===
|
||||
true
|
||||
);
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
};
|
||||
let followed = await follow();
|
||||
if (!followed) {
|
||||
steps.push({ step: 'camera', ok: await flyTo(viewer, view.camera) });
|
||||
for (let attempt = 1; attempt < FOLLOW_ATTEMPTS && !followed; attempt++) {
|
||||
if (signal?.aborted) break;
|
||||
await delay(retryMs, signal);
|
||||
followed = await follow();
|
||||
}
|
||||
}
|
||||
steps.push({ step: 'follow', ok: followed });
|
||||
if (followed && view.follow.cockpit)
|
||||
await act('control_cockpit', { action: 'enter', targetLayer: layerId });
|
||||
return steps;
|
||||
}
|
||||
|
||||
/**
|
||||
* Install embed mode when the page asks for it, and draw any annotations the
|
||||
* opening link carries. Returns a function that removes the message handler.
|
||||
*/
|
||||
export function installViews({
|
||||
shell,
|
||||
viewer,
|
||||
dataManager,
|
||||
run,
|
||||
signal,
|
||||
location = globalThis.location,
|
||||
windowRef = globalThis.window,
|
||||
}) {
|
||||
const ready = Promise.resolve(shell.initialRestorePromise).catch(() => {});
|
||||
const linked = annotationsFromParams(
|
||||
new URLSearchParams(String(location?.hash || '').replace(/^#/, '')),
|
||||
);
|
||||
if (linked.length)
|
||||
void ready.then(() => {
|
||||
if (!signal?.aborted) void run('annotate_map', { annotations: linked });
|
||||
});
|
||||
if (!isEmbedded(location)) return () => {};
|
||||
|
||||
windowRef.document.body.classList.add('ui-embed');
|
||||
shell.setCleanView?.(true);
|
||||
const stopRendering = isEmbeddedInline()
|
||||
? keepPanelRendering(viewer, { windowRef })
|
||||
: () => {};
|
||||
// A framing page talks to the app across frames; an inline panel shares
|
||||
// the page with it and talks through the page's own window.
|
||||
const peer = isEmbeddedInline() ? windowRef : windowRef.parent;
|
||||
const post = (message) => {
|
||||
if (peer && (peer !== windowRef || isEmbeddedInline()))
|
||||
peer.postMessage(message, '*');
|
||||
};
|
||||
// Views apply one at a time, in the order they arrive.
|
||||
let queue = ready;
|
||||
const onMessage = (event) => {
|
||||
if (event.source !== peer || event.data?.type !== EMBED_VIEW_MESSAGE)
|
||||
return;
|
||||
const { id = null } = event.data;
|
||||
let view;
|
||||
try {
|
||||
view = createView(event.data.view);
|
||||
} catch (error) {
|
||||
post({
|
||||
type: EMBED_APPLIED_MESSAGE,
|
||||
id,
|
||||
ok: false,
|
||||
error: error.message,
|
||||
});
|
||||
return;
|
||||
}
|
||||
queue = queue.then(async () => {
|
||||
if (signal?.aborted) return;
|
||||
const steps = await applyView(view, { viewer, dataManager, run, signal });
|
||||
post({
|
||||
type: EMBED_APPLIED_MESSAGE,
|
||||
id,
|
||||
ok: steps.every((step) => step.ok),
|
||||
steps,
|
||||
});
|
||||
});
|
||||
};
|
||||
windowRef.addEventListener('message', onMessage);
|
||||
void ready.then(() => {
|
||||
if (!signal?.aborted) post({ type: EMBED_READY_MESSAGE });
|
||||
});
|
||||
return () => {
|
||||
stopRendering();
|
||||
windowRef.removeEventListener('message', onMessage);
|
||||
};
|
||||
}
|
||||
@@ -1,312 +0,0 @@
|
||||
import assert from 'node:assert/strict';
|
||||
import test from 'node:test';
|
||||
import {
|
||||
applyView,
|
||||
installViews,
|
||||
isEmbedded,
|
||||
keepPanelRendering,
|
||||
} from './embed.js';
|
||||
import { createView } from '../view/index.js';
|
||||
|
||||
const fakeViewer = () => {
|
||||
const flights = [];
|
||||
return {
|
||||
flights,
|
||||
camera: {
|
||||
flyTo(options) {
|
||||
flights.push(options);
|
||||
options.complete();
|
||||
},
|
||||
},
|
||||
};
|
||||
};
|
||||
const fakeLayers = (enabled) => ({
|
||||
getAll: () =>
|
||||
['flights', 'earthquakes', 'ais-live-vessels'].map((id) => ({
|
||||
id,
|
||||
enabled: enabled.has(id),
|
||||
})),
|
||||
isEnabled: (id) => enabled.has(id),
|
||||
});
|
||||
const recorder = (answers = {}) => {
|
||||
const calls = [];
|
||||
const run = async (name, args) => {
|
||||
calls.push([name, args]);
|
||||
const answer = answers[name];
|
||||
return typeof answer === 'function'
|
||||
? answer(calls)
|
||||
: (answer ?? { ok: true });
|
||||
};
|
||||
return { calls, run };
|
||||
};
|
||||
|
||||
test('embed mode is opt-in by query parameter', () => {
|
||||
assert.equal(isEmbedded({ search: '?embed=1' }), true);
|
||||
assert.equal(isEmbedded({ search: '?welcome=0' }), false);
|
||||
});
|
||||
|
||||
test('a view is applied through the app actions in order', async () => {
|
||||
const viewer = fakeViewer();
|
||||
const { calls, run } = recorder();
|
||||
const view = createView({
|
||||
camera: { lat: 25, lon: 121, altitude_m: 300000, pitch_deg: -40 },
|
||||
layers: ['ais-live-vessels'],
|
||||
style: 'thermal',
|
||||
map: 'osm',
|
||||
annotations: [{ type: 'pin', target: 'Taipei 101' }],
|
||||
});
|
||||
const steps = await applyView(view, {
|
||||
viewer,
|
||||
dataManager: fakeLayers(new Set(['earthquakes'])),
|
||||
run,
|
||||
});
|
||||
assert.deepEqual(
|
||||
calls.map(([name, args]) => [
|
||||
name,
|
||||
args.layerId ?? args.style ?? args.stack ?? null,
|
||||
args.enabled ?? null,
|
||||
]),
|
||||
[
|
||||
['set_visual_style', 'thermal', null],
|
||||
['set_map_stack', 'osm', null],
|
||||
['set_layer_visibility', 'earthquakes', false],
|
||||
['set_layer_visibility', 'ais-live-vessels', true],
|
||||
['stop_tracking', null, null],
|
||||
['clear_annotations', null, null],
|
||||
['annotate_map', null, null],
|
||||
],
|
||||
);
|
||||
assert.equal(viewer.flights.length, 1);
|
||||
assert.ok(steps.every((step) => step.ok));
|
||||
assert.ok(steps.some((step) => step.step === 'camera'));
|
||||
});
|
||||
|
||||
test('a followed entity is retried until its layer has it', async () => {
|
||||
let tries = 0;
|
||||
const { run } = recorder({
|
||||
track_entity: () => ({ ok: ++tries >= 3 }),
|
||||
});
|
||||
const view = createView({
|
||||
camera: { lat: 0, lon: 0 },
|
||||
follow: { kind: 'aircraft', id: 'abc123' },
|
||||
});
|
||||
const steps = await applyView(view, {
|
||||
viewer: fakeViewer(),
|
||||
dataManager: fakeLayers(new Set()),
|
||||
run,
|
||||
retryMs: 1,
|
||||
});
|
||||
assert.equal(tries, 3);
|
||||
assert.deepEqual(steps.at(-1), { step: 'follow', ok: true });
|
||||
});
|
||||
|
||||
test('an embedded page takes views only from its parent and answers it', async () => {
|
||||
const posted = [];
|
||||
const parent = { postMessage: (message) => posted.push(message) };
|
||||
const windowRef = new EventTarget();
|
||||
windowRef.parent = parent;
|
||||
windowRef.document = { body: { classList: new Set() } };
|
||||
windowRef.document.body.classList.add = Set.prototype.add;
|
||||
const shell = {
|
||||
initialRestorePromise: Promise.resolve(),
|
||||
clean: null,
|
||||
setCleanView(value) {
|
||||
this.clean = value;
|
||||
},
|
||||
};
|
||||
const { calls, run } = recorder();
|
||||
const remove = installViews({
|
||||
shell,
|
||||
viewer: fakeViewer(),
|
||||
dataManager: fakeLayers(new Set()),
|
||||
run,
|
||||
location: { search: '?embed=1', hash: '' },
|
||||
windowRef,
|
||||
});
|
||||
await new Promise((resolve) => setTimeout(resolve, 0));
|
||||
assert.equal(shell.clean, true);
|
||||
assert.deepEqual(posted, [{ type: 'gev:ready' }]);
|
||||
const send = (source, data) => {
|
||||
const event = new Event('message');
|
||||
Object.assign(event, { source, data });
|
||||
windowRef.dispatchEvent(event);
|
||||
};
|
||||
send({}, { type: 'gev:view', id: 1, view: { camera: { lat: 1, lon: 2 } } });
|
||||
send(parent, { type: 'gev:view', id: 2, view: { camera: {} } });
|
||||
send(parent, {
|
||||
type: 'gev:view',
|
||||
id: 3,
|
||||
view: { camera: { lat: 1, lon: 2 }, layers: ['flights'] },
|
||||
});
|
||||
await new Promise((resolve) => setTimeout(resolve, 10));
|
||||
assert.deepEqual(posted[1], {
|
||||
type: 'gev:view-applied',
|
||||
id: 2,
|
||||
ok: false,
|
||||
error: 'A view needs a camera lat and lon',
|
||||
});
|
||||
assert.equal(posted[2].id, 3);
|
||||
assert.equal(posted[2].ok, true);
|
||||
assert.ok(
|
||||
calls.some(
|
||||
([name, args]) =>
|
||||
name === 'set_layer_visibility' && args.layerId === 'flights',
|
||||
),
|
||||
);
|
||||
assert.equal(posted.filter((message) => message.id === 1).length, 0);
|
||||
remove();
|
||||
});
|
||||
|
||||
test('an app loaded inline into a panel page talks through its own window', async () => {
|
||||
const posted = [];
|
||||
const windowRef = new EventTarget();
|
||||
windowRef.parent = windowRef;
|
||||
windowRef.postMessage = (message) => posted.push(message);
|
||||
windowRef.document = { body: { classList: new Set() } };
|
||||
windowRef.document.body.classList.add = Set.prototype.add;
|
||||
Object.assign(windowRef, {
|
||||
requestAnimationFrame: () => 1,
|
||||
cancelAnimationFrame() {},
|
||||
setInterval: () => 1,
|
||||
clearInterval() {},
|
||||
});
|
||||
globalThis.GEV_EMBED_INLINE = true;
|
||||
try {
|
||||
assert.equal(isEmbedded({ search: '' }), true);
|
||||
const remove = installViews({
|
||||
shell: { initialRestorePromise: Promise.resolve() },
|
||||
viewer: fakeViewer(),
|
||||
dataManager: fakeLayers(new Set()),
|
||||
run: recorder().run,
|
||||
location: { search: '', hash: '' },
|
||||
windowRef,
|
||||
});
|
||||
await new Promise((resolve) => setTimeout(resolve, 0));
|
||||
assert.deepEqual(posted, [{ type: 'gev:ready' }]);
|
||||
const event = new Event('message');
|
||||
Object.assign(event, {
|
||||
source: windowRef,
|
||||
data: { type: 'gev:view', id: 7, view: { camera: { lat: 1, lon: 2 } } },
|
||||
});
|
||||
windowRef.dispatchEvent(event);
|
||||
await new Promise((resolve) => setTimeout(resolve, 10));
|
||||
assert.equal(posted[1].id, 7);
|
||||
assert.equal(posted[1].ok, true);
|
||||
remove();
|
||||
} finally {
|
||||
delete globalThis.GEV_EMBED_INLINE;
|
||||
}
|
||||
});
|
||||
|
||||
test('annotations in a link are drawn after it restores, embedded or not', async () => {
|
||||
const { calls, run } = recorder();
|
||||
const an = encodeURIComponent(
|
||||
JSON.stringify([{ type: 'pin', target: 'Austin' }]),
|
||||
);
|
||||
installViews({
|
||||
shell: { initialRestorePromise: Promise.resolve() },
|
||||
viewer: fakeViewer(),
|
||||
dataManager: fakeLayers(new Set()),
|
||||
run,
|
||||
location: { search: '', hash: `#v=2&lat=1&lon=2&an=${an}` },
|
||||
windowRef: new EventTarget(),
|
||||
});
|
||||
await new Promise((resolve) => setTimeout(resolve, 0));
|
||||
assert.deepEqual(calls, [
|
||||
['annotate_map', { annotations: [{ type: 'pin', target: 'Austin' }] }],
|
||||
]);
|
||||
});
|
||||
|
||||
test('a panel keeps drawing from a timer while animation frames stop', () => {
|
||||
let clock = 0;
|
||||
let tick = null;
|
||||
let frame = null;
|
||||
const windowRef = {
|
||||
requestAnimationFrame: (fn) => ((frame = fn), 1),
|
||||
cancelAnimationFrame: () => (frame = null),
|
||||
setInterval: (fn) => ((tick = fn), 1),
|
||||
clearInterval: () => (tick = null),
|
||||
};
|
||||
const drawn = [];
|
||||
const viewer = {
|
||||
resize: () => drawn.push('resize'),
|
||||
render: () => drawn.push('render'),
|
||||
};
|
||||
const stop = keepPanelRendering(viewer, { windowRef, now: () => clock });
|
||||
// Frames arrive: the render loop draws, so the timer does nothing.
|
||||
clock = 100;
|
||||
frame();
|
||||
tick();
|
||||
assert.deepEqual(drawn, []);
|
||||
// Frames stop, as in a panel its host reports hidden: the timer draws.
|
||||
clock = 400;
|
||||
tick();
|
||||
assert.deepEqual(drawn, ['resize', 'render']);
|
||||
stop();
|
||||
assert.equal(tick, null);
|
||||
assert.equal(frame, null);
|
||||
});
|
||||
|
||||
test('a panel reports a render error in full and stops drawing', () => {
|
||||
let listener = null;
|
||||
let tick = null;
|
||||
const windowRef = {
|
||||
requestAnimationFrame: () => 1,
|
||||
cancelAnimationFrame() {},
|
||||
setInterval: (fn) => ((tick = fn), 1),
|
||||
clearInterval() {},
|
||||
};
|
||||
const drawn = [];
|
||||
const viewer = {
|
||||
resize() {},
|
||||
render: () => drawn.push('render'),
|
||||
scene: {
|
||||
renderError: {
|
||||
addEventListener: (fn) => ((listener = fn), () => (listener = null)),
|
||||
},
|
||||
},
|
||||
};
|
||||
const logged = [];
|
||||
const original = console.error;
|
||||
console.error = (...args) => logged.push(args.join(' '));
|
||||
try {
|
||||
let clock = 0;
|
||||
const stop = keepPanelRendering(viewer, { windowRef, now: () => clock });
|
||||
listener(viewer.scene, { name: 'RuntimeError', message: 'worker failed' });
|
||||
assert.match(logged[0], /render error: .*"message":"worker failed"/);
|
||||
clock = 1000;
|
||||
tick();
|
||||
assert.deepEqual(drawn, []);
|
||||
stop();
|
||||
assert.equal(listener, null);
|
||||
} finally {
|
||||
console.error = original;
|
||||
}
|
||||
});
|
||||
|
||||
test('a followed aircraft owns the camera, and cockpit view is entered after it', async () => {
|
||||
const viewer = fakeViewer();
|
||||
const { calls, run } = recorder();
|
||||
const steps = await applyView(
|
||||
createView({
|
||||
camera: { lat: 32.7, lon: -117.2 },
|
||||
follow: { kind: 'military_aircraft', id: 'AE1234', cockpit: true },
|
||||
}),
|
||||
{ viewer, dataManager: fakeLayers(new Set()), run, retryMs: 0 },
|
||||
);
|
||||
// Following worked at once, so no camera flight ended it.
|
||||
assert.equal(viewer.flights.length, 0);
|
||||
assert.deepEqual(calls.slice(-2), [
|
||||
['track_entity', { query: 'ae1234', layerId: 'military' }],
|
||||
['control_cockpit', { action: 'enter', targetLayer: 'military' }],
|
||||
]);
|
||||
assert.ok(steps.every((step) => step.ok));
|
||||
assert.throws(
|
||||
() =>
|
||||
createView({
|
||||
camera: { lat: 0, lon: 0 },
|
||||
follow: { kind: 'satellite', id: '25544', cockpit: true },
|
||||
}),
|
||||
/no cockpit view/,
|
||||
);
|
||||
});
|
||||
@@ -1,661 +0,0 @@
|
||||
/**
|
||||
* The God's Eye View panel's script, run inside the panel page a host shows.
|
||||
* It is sent as source text (see src/tools/globePanel.js), so it must not use
|
||||
* imports or anything outside the function.
|
||||
*
|
||||
* The page lives on the host's site, and hosts may refuse other addresses,
|
||||
* such as a server on the user's machine. So the panel never requests the
|
||||
* app's server itself: every request for the app's own paths goes to the
|
||||
* panel's MCP server as a call to `config.toolName`, which requests the path
|
||||
* from the app's server. The app's built page, scripts, styles, images and
|
||||
* data all arrive that way; map imagery from providers loads directly.
|
||||
*/
|
||||
export function panelRuntime(config) {
|
||||
const status = document.getElementById('status');
|
||||
const open = document.getElementById('open');
|
||||
const expand = document.getElementById('expand');
|
||||
const say = (text) => {
|
||||
if (status.isConnected) status.textContent = text;
|
||||
};
|
||||
let blocked = null;
|
||||
document.addEventListener('securitypolicyviolation', (event) => {
|
||||
blocked = `${event.effectiveDirective} blocked ${event.blockedURI || 'a resource'}`;
|
||||
});
|
||||
|
||||
// JSON-RPC with the host.
|
||||
let nextId = 1;
|
||||
const pending = new Map();
|
||||
const send = (message) =>
|
||||
window.parent.postMessage({ jsonrpc: '2.0', ...message }, '*');
|
||||
const request = (method, params) =>
|
||||
new Promise((resolve, reject) => {
|
||||
const id = nextId++;
|
||||
pending.set(id, { resolve, reject });
|
||||
send({ id, method, params });
|
||||
});
|
||||
const notify = (method, params) => send({ method, params });
|
||||
|
||||
// Requests to the app's server, through the MCP server.
|
||||
const MAX_CALLS = 6;
|
||||
let calls = 0;
|
||||
const waiting = [];
|
||||
async function callTool(args) {
|
||||
if (calls >= MAX_CALLS) await new Promise((go) => waiting.push(go));
|
||||
calls += 1;
|
||||
try {
|
||||
const result = await request('tools/call', {
|
||||
name: config.toolName,
|
||||
arguments: args,
|
||||
});
|
||||
if (result?.isError || !result?.structuredContent)
|
||||
throw new Error(result?.content?.[0]?.text || 'The request failed');
|
||||
return result.structuredContent;
|
||||
} finally {
|
||||
calls -= 1;
|
||||
waiting.shift()?.();
|
||||
}
|
||||
}
|
||||
const toBase64 = (bytes) => {
|
||||
let binary = '';
|
||||
for (let start = 0; start < bytes.length; start += 0x8000)
|
||||
binary += String.fromCharCode(...bytes.subarray(start, start + 0x8000));
|
||||
return btoa(binary);
|
||||
};
|
||||
const fromBase64 = (text) =>
|
||||
Uint8Array.from(atob(text), (char) => char.charCodeAt(0));
|
||||
async function bodyBytes(body) {
|
||||
if (body == null) return null;
|
||||
if (typeof body === 'string') return new TextEncoder().encode(body);
|
||||
if (body instanceof ArrayBuffer) return new Uint8Array(body);
|
||||
if (ArrayBuffer.isView(body))
|
||||
return new Uint8Array(body.buffer, body.byteOffset, body.byteLength);
|
||||
return new Uint8Array(await new Response(body).arrayBuffer());
|
||||
}
|
||||
const NULL_BODY = new Set([101, 204, 205, 304]);
|
||||
const held = new Map();
|
||||
|
||||
/** Request a path on the app's server; resolves to a Response. */
|
||||
function appFetch(path, init = {}) {
|
||||
const method = (init.method || 'GET').toUpperCase();
|
||||
// Built files never change while the panel runs, so each loads once.
|
||||
const shared = method === 'GET' && path.startsWith(config.panelBase);
|
||||
if (shared && held.has(path)) return held.get(path).then(respond);
|
||||
const loading = (async () => {
|
||||
const headers = {};
|
||||
new Headers(init.headers || {}).forEach((value, name) => {
|
||||
headers[name] = value;
|
||||
});
|
||||
const bytes = await bodyBytes(init.body);
|
||||
let part = await callTool({
|
||||
path,
|
||||
method,
|
||||
headers,
|
||||
...(bytes ? { body: toBase64(bytes) } : {}),
|
||||
});
|
||||
const parts = [fromBase64(part.body)];
|
||||
while (part.nextOffset !== undefined) {
|
||||
if (init.signal?.aborted) break;
|
||||
part = await callTool({ id: part.id, offset: part.nextOffset });
|
||||
parts.push(fromBase64(part.body));
|
||||
}
|
||||
let body = new Blob(parts);
|
||||
if (part.encoding === 'gzip')
|
||||
body = await new Response(
|
||||
body.stream().pipeThrough(new DecompressionStream('gzip')),
|
||||
).blob();
|
||||
return {
|
||||
body: await body.arrayBuffer(),
|
||||
status: part.status,
|
||||
statusText: part.statusText,
|
||||
headers: part.headers,
|
||||
};
|
||||
})();
|
||||
if (shared) {
|
||||
held.set(path, loading);
|
||||
loading.catch(() => held.delete(path));
|
||||
}
|
||||
const result = loading.then(respond);
|
||||
if (!init.signal) return result;
|
||||
return Promise.race([
|
||||
result,
|
||||
new Promise((_, reject) => {
|
||||
const abort = () =>
|
||||
reject(new DOMException('The request was aborted', 'AbortError'));
|
||||
if (init.signal.aborted) abort();
|
||||
init.signal.addEventListener('abort', abort, { once: true });
|
||||
}),
|
||||
]);
|
||||
}
|
||||
function respond({ body, status, statusText, headers }) {
|
||||
return new Response(NULL_BODY.has(status) ? null : body.slice(0), {
|
||||
status,
|
||||
statusText,
|
||||
headers,
|
||||
});
|
||||
}
|
||||
|
||||
// The app's own paths: relative ones, absolute ones on this page's site
|
||||
// (matched by scheme and host, since a host may use its own scheme), and
|
||||
// ones under the app's address in the page, an https address for code
|
||||
// that needs one (`config.appBaseUrl`).
|
||||
const page = new URL(document.baseURI);
|
||||
const appOrigin = new URL(config.appBaseUrl).origin;
|
||||
function appPath(value) {
|
||||
let url;
|
||||
try {
|
||||
url = new URL(String(value), document.baseURI);
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
const own =
|
||||
url.origin === appOrigin ||
|
||||
(url.protocol === page.protocol && url.host === page.host);
|
||||
return own ? url.pathname + url.search : null;
|
||||
}
|
||||
const fileUrls = new Map();
|
||||
/**
|
||||
* A data: URL holding an app file, for elements that load by URL. Hosts
|
||||
* allow data: images where some refuse blob: ones. Files of the panel
|
||||
* build never change and are kept; others, such as live camera frames
|
||||
* that change address on every refresh, are not.
|
||||
*/
|
||||
function appFileUrl(path) {
|
||||
if (fileUrls.has(path)) return fileUrls.get(path);
|
||||
const loading = appFetch(path)
|
||||
.then((response) => response.blob())
|
||||
.then(
|
||||
(blob) =>
|
||||
new Promise((resolve, reject) => {
|
||||
const reader = new FileReader();
|
||||
reader.onload = () => resolve(reader.result);
|
||||
reader.onerror = () => reject(reader.error);
|
||||
reader.readAsDataURL(blob);
|
||||
}),
|
||||
);
|
||||
if (path.startsWith(config.panelBase)) {
|
||||
fileUrls.set(path, loading);
|
||||
loading.catch(() => fileUrls.delete(path));
|
||||
}
|
||||
return loading;
|
||||
}
|
||||
|
||||
function installRequests() {
|
||||
const nativeFetch = window.fetch.bind(window);
|
||||
window.fetch = async (input, init = {}) => {
|
||||
const isRequest = input instanceof Request;
|
||||
const path = appPath(isRequest ? input.url : input);
|
||||
if (path === null) return nativeFetch(input, init);
|
||||
if (!isRequest) return appFetch(path, init);
|
||||
return appFetch(path, {
|
||||
method: init.method || input.method,
|
||||
headers: init.headers || input.headers,
|
||||
body:
|
||||
init.body ??
|
||||
(['GET', 'HEAD'].includes(input.method)
|
||||
? undefined
|
||||
: await input.arrayBuffer()),
|
||||
signal: init.signal || input.signal,
|
||||
});
|
||||
};
|
||||
|
||||
// XMLHttpRequest for app paths: answered from appFetch, with the
|
||||
// instance's state replaced, and the usual events dispatched.
|
||||
const xhr = XMLHttpRequest.prototype;
|
||||
const nativeOpen = xhr.open;
|
||||
const nativeSend = xhr.send;
|
||||
const nativeHeader = xhr.setRequestHeader;
|
||||
const nativeAbort = xhr.abort;
|
||||
xhr.open = function (method, url, ...rest) {
|
||||
const path = appPath(url);
|
||||
if (path === null) {
|
||||
this.__gev = null;
|
||||
return nativeOpen.call(this, method, url, ...rest);
|
||||
}
|
||||
this.__gev = { method, path, headers: {}, controller: null };
|
||||
setState(this, { readyState: 1 });
|
||||
this.dispatchEvent(new Event('readystatechange'));
|
||||
};
|
||||
xhr.setRequestHeader = function (name, value) {
|
||||
if (!this.__gev) return nativeHeader.call(this, name, value);
|
||||
this.__gev.headers[name] = value;
|
||||
};
|
||||
xhr.abort = function () {
|
||||
if (!this.__gev) return nativeAbort.call(this);
|
||||
this.__gev.controller?.abort();
|
||||
};
|
||||
xhr.send = function (body) {
|
||||
const state = this.__gev;
|
||||
if (!state) return nativeSend.call(this, body);
|
||||
state.controller = new AbortController();
|
||||
appFetch(state.path, {
|
||||
method: state.method,
|
||||
headers: state.headers,
|
||||
body,
|
||||
signal: state.controller.signal,
|
||||
})
|
||||
.then(async (response) => {
|
||||
const type = this.responseType;
|
||||
const value =
|
||||
type === 'arraybuffer'
|
||||
? await response.arrayBuffer()
|
||||
: type === 'blob'
|
||||
? await response.blob()
|
||||
: await response.text();
|
||||
const text = typeof value === 'string' ? value : null;
|
||||
let parsed = value;
|
||||
if (type === 'json') {
|
||||
try {
|
||||
parsed = JSON.parse(text);
|
||||
} catch {
|
||||
parsed = null;
|
||||
}
|
||||
} else if (type === 'document')
|
||||
parsed = new DOMParser().parseFromString(text, 'text/html');
|
||||
const headers = [...response.headers]
|
||||
.map(([name, val]) => `${name}: ${val}`)
|
||||
.join('\r\n');
|
||||
setState(this, {
|
||||
readyState: 4,
|
||||
status: response.status,
|
||||
statusText: response.statusText,
|
||||
response: parsed,
|
||||
responseText: type === '' || type === 'text' ? text : '',
|
||||
responseURL: new URL(state.path, document.baseURI).href,
|
||||
getAllResponseHeaders: () => headers,
|
||||
getResponseHeader: (name) => response.headers.get(name),
|
||||
});
|
||||
for (const name of ['readystatechange', 'load', 'loadend'])
|
||||
this.dispatchEvent(
|
||||
name === 'readystatechange'
|
||||
? new Event(name)
|
||||
: new ProgressEvent(name),
|
||||
);
|
||||
})
|
||||
.catch((error) => {
|
||||
setState(this, { readyState: 4, status: 0 });
|
||||
const name = error?.name === 'AbortError' ? 'abort' : 'error';
|
||||
for (const event of ['readystatechange', name, 'loadend'])
|
||||
this.dispatchEvent(
|
||||
event === 'readystatechange'
|
||||
? new Event(event)
|
||||
: new ProgressEvent(event),
|
||||
);
|
||||
});
|
||||
};
|
||||
function setState(target, values) {
|
||||
for (const [name, value] of Object.entries(values))
|
||||
Object.defineProperty(target, name, {
|
||||
configurable: true,
|
||||
...(typeof value === 'function' ? { value } : { get: () => value }),
|
||||
});
|
||||
}
|
||||
|
||||
const nativeBeacon = navigator.sendBeacon?.bind(navigator);
|
||||
if (nativeBeacon)
|
||||
navigator.sendBeacon = (url, data) => {
|
||||
const path = appPath(url);
|
||||
if (path === null) return nativeBeacon(url, data);
|
||||
appFetch(path, { method: 'POST', body: data }).catch(() => {});
|
||||
return true;
|
||||
};
|
||||
|
||||
// Images name app files by URL; give them the file as a data: URL.
|
||||
const image = Object.getOwnPropertyDescriptor(
|
||||
HTMLImageElement.prototype,
|
||||
'src',
|
||||
);
|
||||
Object.defineProperty(HTMLImageElement.prototype, 'src', {
|
||||
...image,
|
||||
set(value) {
|
||||
const path = appPath(value);
|
||||
if (path === null) return image.set.call(this, value);
|
||||
appFileUrl(path).then(
|
||||
(url) => image.set.call(this, url),
|
||||
() => this.dispatchEvent(new Event('error')),
|
||||
);
|
||||
},
|
||||
});
|
||||
|
||||
// Workers on the app's server start from a small worker here that is
|
||||
// handed the script once it has loaded; messages wait until then.
|
||||
const NativeWorker = window.Worker;
|
||||
const BOOT =
|
||||
'const early=[];let ready=false;' +
|
||||
"addEventListener('message',e=>{if(e.data&&e.data.__gevScript){" +
|
||||
'const go=()=>{ready=true;for(const d of early)dispatchEvent(new MessageEvent("message",{data:d}))};' +
|
||||
'if(e.data.module)import(e.data.__gevScript).then(go);else{importScripts(e.data.__gevScript);go()}' +
|
||||
'e.stopImmediatePropagation();return}' +
|
||||
'if(!ready){early.push(e.data);e.stopImmediatePropagation()}});';
|
||||
window.Worker = function Worker(url, options) {
|
||||
const path = appPath(url);
|
||||
if (path === null || String(url).startsWith('blob:'))
|
||||
return new NativeWorker(url, options);
|
||||
const worker = new NativeWorker(
|
||||
URL.createObjectURL(new Blob([BOOT], { type: 'text/javascript' })),
|
||||
options,
|
||||
);
|
||||
appFetch(path)
|
||||
.then((response) => response.blob())
|
||||
.then((blob) => URL.createObjectURL(blob))
|
||||
.then((script) =>
|
||||
worker.postMessage({
|
||||
__gevScript: script,
|
||||
module: options?.type === 'module',
|
||||
}),
|
||||
);
|
||||
return worker;
|
||||
};
|
||||
window.Worker.prototype = NativeWorker.prototype;
|
||||
|
||||
// Some hosts drop the GPU contents of a panel's 2D canvases while they
|
||||
// consider the panel off screen, and the overlays then cover the globe
|
||||
// in black. Keep 2D canvases in memory instead, which hosts do not drop.
|
||||
const getContext = HTMLCanvasElement.prototype.getContext;
|
||||
HTMLCanvasElement.prototype.getContext = function (type, options) {
|
||||
return type === '2d'
|
||||
? getContext.call(this, type, { ...options, willReadFrequently: true })
|
||||
: getContext.call(this, type, options);
|
||||
};
|
||||
|
||||
// Markup the app adds as HTML text would request its images from this
|
||||
// page's site at once; hold their addresses until the files are here.
|
||||
const htmlWithHeldImages = (html) =>
|
||||
String(html).replace(
|
||||
/(<(?:img|source)\b[^>]*?\s)src=(["'])([^"']*)\2/gi,
|
||||
(tag, before, quote, value) =>
|
||||
appPath(value) === null
|
||||
? tag
|
||||
: `${before}data-gev-src=${quote}${value}${quote}`,
|
||||
);
|
||||
const fillHeldImages = (root) => {
|
||||
for (const element of root?.querySelectorAll?.('[data-gev-src]') ?? []) {
|
||||
const path = appPath(element.getAttribute('data-gev-src'));
|
||||
element.removeAttribute('data-gev-src');
|
||||
appFileUrl(path).then((url) => element.setAttribute('src', url));
|
||||
}
|
||||
};
|
||||
const innerHtml = Object.getOwnPropertyDescriptor(
|
||||
Element.prototype,
|
||||
'innerHTML',
|
||||
);
|
||||
Object.defineProperty(Element.prototype, 'innerHTML', {
|
||||
...innerHtml,
|
||||
set(value) {
|
||||
innerHtml.set.call(this, htmlWithHeldImages(value));
|
||||
fillHeldImages(this);
|
||||
},
|
||||
});
|
||||
const insertHtml = Element.prototype.insertAdjacentHTML;
|
||||
Element.prototype.insertAdjacentHTML = function (position, html) {
|
||||
insertHtml.call(this, position, htmlWithHeldImages(html));
|
||||
fillHeldImages(this.parentNode ?? this);
|
||||
};
|
||||
|
||||
// Markup and styles the app adds later name app files too.
|
||||
new MutationObserver((records) => {
|
||||
for (const record of records)
|
||||
for (const node of record.addedNodes) {
|
||||
if (node.nodeType !== 1) continue;
|
||||
if (node.nodeName === 'STYLE') void inlineStyleUrls(node);
|
||||
for (const element of [node, ...node.querySelectorAll('img[src]')]) {
|
||||
if (element.nodeName !== 'IMG') continue;
|
||||
const path = appPath(element.getAttribute('src'));
|
||||
if (path !== null)
|
||||
appFileUrl(path).then((url) => element.setAttribute('src', url));
|
||||
}
|
||||
}
|
||||
}).observe(document.documentElement, { childList: true, subtree: true });
|
||||
}
|
||||
|
||||
/** Replace url() references to app files in CSS with data: URLs. */
|
||||
async function cssWithFiles(text, baseUrl) {
|
||||
const found = new Map();
|
||||
for (const match of text.matchAll(/url\(\s*(['"]?)([^'")]+)\1\s*\)/g)) {
|
||||
const reference = match[2];
|
||||
if (/^(data|blob):/.test(reference)) continue;
|
||||
const path = appPath(new URL(reference, baseUrl).href);
|
||||
if (path !== null && !found.has(reference))
|
||||
found.set(
|
||||
reference,
|
||||
appFileUrl(path).catch(() => null),
|
||||
);
|
||||
}
|
||||
let result = text;
|
||||
for (const [reference, loading] of found) {
|
||||
const url = await loading;
|
||||
if (url) result = result.split(reference).join(url);
|
||||
}
|
||||
return result;
|
||||
}
|
||||
async function inlineStyleUrls(style) {
|
||||
if (!/url\(/.test(style.textContent)) return;
|
||||
style.textContent = await cssWithFiles(style.textContent, document.baseURI);
|
||||
}
|
||||
|
||||
async function addStylesheet(href) {
|
||||
const path = appPath(href);
|
||||
const style = document.createElement('style');
|
||||
document.head.appendChild(style);
|
||||
if (path === null) {
|
||||
const link = document.createElement('link');
|
||||
link.rel = 'stylesheet';
|
||||
link.href = href;
|
||||
style.replaceWith(link);
|
||||
return;
|
||||
}
|
||||
const text = await (await appFetch(path)).text();
|
||||
style.textContent = await cssWithFiles(
|
||||
text,
|
||||
new URL(path, document.baseURI).href,
|
||||
);
|
||||
}
|
||||
|
||||
function runScript(text, type) {
|
||||
const script = document.createElement('script');
|
||||
if (type) script.type = type;
|
||||
script.textContent = text;
|
||||
document.body.appendChild(script);
|
||||
}
|
||||
|
||||
/**
|
||||
* Load God's Eye View into this page from the app's panel build, in
|
||||
* inline embed mode, starting at the first view as its link would.
|
||||
*/
|
||||
async function startApp(url, view) {
|
||||
window.GEV_EMBED_INLINE = true;
|
||||
window.GEV_APP_BASE_URL = config.appBaseUrl;
|
||||
try {
|
||||
history.replaceState(
|
||||
null,
|
||||
'',
|
||||
location.href.split('#')[0] + new URL(url).hash,
|
||||
);
|
||||
// The app restores this view from its link, following included, so
|
||||
// it needs applying again only for what links cannot carry: cockpit.
|
||||
if (queued === view && !view.follow?.cockpit) queued = null;
|
||||
} catch {
|
||||
// The view still arrives once the app is ready.
|
||||
}
|
||||
installRequests();
|
||||
const response = await appFetch(config.panelBase);
|
||||
if (!response.ok)
|
||||
throw new Error(
|
||||
response.status === 404
|
||||
? 'the app has no panel build (npm run build:panel)'
|
||||
: `the app answered ${response.status}`,
|
||||
);
|
||||
const built = new DOMParser().parseFromString(
|
||||
await response.text(),
|
||||
'text/html',
|
||||
);
|
||||
window.CESIUM_BASE_URL = `${config.panelBase}cesium/`;
|
||||
const [prelude] = await Promise.all([
|
||||
appFetch(config.panelBase + config.workerPreludePath).then((r) =>
|
||||
r.text(),
|
||||
),
|
||||
...[...built.head.querySelectorAll('link[rel="stylesheet"]')].map(
|
||||
(link) => addStylesheet(link.getAttribute('href')),
|
||||
),
|
||||
]);
|
||||
// Cesium's script sets CESIUM_WORKERS, its workers as one script it
|
||||
// starts from memory. Workers request files themselves, which this page
|
||||
// cannot answer, so the prelude, holding those files, runs ahead of it.
|
||||
let workers;
|
||||
Object.defineProperty(window, 'CESIUM_WORKERS', {
|
||||
configurable: true,
|
||||
get: () => workers,
|
||||
set: (script) => {
|
||||
workers = prelude + script;
|
||||
},
|
||||
});
|
||||
// Images in the markup load their files once added; give them the
|
||||
// files instead of letting them request this page's site.
|
||||
const images = [];
|
||||
for (const image of built.body.querySelectorAll('img[src]')) {
|
||||
const path = appPath(image.getAttribute('src'));
|
||||
if (path === null) continue;
|
||||
image.removeAttribute('src');
|
||||
images.push([image, path]);
|
||||
}
|
||||
for (const node of [...built.body.childNodes]) {
|
||||
if (node.nodeName !== 'SCRIPT')
|
||||
document.body.insertBefore(document.adoptNode(node), status);
|
||||
}
|
||||
for (const [image, path] of images)
|
||||
appFileUrl(path).then((url) => image.setAttribute('src', url));
|
||||
const scripts = await Promise.all(
|
||||
[...built.querySelectorAll('script')].map(async (script) => ({
|
||||
type: script.type,
|
||||
text: script.getAttribute('src')
|
||||
? await (await appFetch(appPath(script.getAttribute('src')))).text()
|
||||
: script.textContent,
|
||||
})),
|
||||
);
|
||||
for (const { type, text } of scripts) runScript(text, type);
|
||||
// Browsers cap live 3D contexts across pages and take one back when
|
||||
// there are too many; say so rather than show no map.
|
||||
document.addEventListener(
|
||||
'webglcontextlost',
|
||||
() => {
|
||||
const notice = document.createElement('div');
|
||||
notice.id = 'status';
|
||||
notice.textContent =
|
||||
"This panel lost its 3D graphics, likely because other God's Eye " +
|
||||
"View panels in this conversation hold them. Use Open in God's " +
|
||||
'Eye View above, or show it in a new conversation.';
|
||||
document.body.appendChild(notice);
|
||||
},
|
||||
{ capture: true, once: true },
|
||||
);
|
||||
}
|
||||
|
||||
// Tool results: the first view starts the app, later ones move it.
|
||||
let started = false;
|
||||
let ready = false;
|
||||
let queued = null;
|
||||
let currentUrl = null;
|
||||
const postView = (view) =>
|
||||
window.postMessage({ type: 'gev:view', id: nextId++, view }, '*');
|
||||
function show(result) {
|
||||
const data = result?.structuredContent;
|
||||
const view = data?.view;
|
||||
const url = data?.url || view?.url;
|
||||
if (!view || !url) return;
|
||||
currentUrl = url;
|
||||
open.hidden = false;
|
||||
if (ready) return postView(view);
|
||||
queued = view;
|
||||
if (started) return;
|
||||
started = true;
|
||||
say("Loading God's Eye View…");
|
||||
startApp(url, view).catch((error) =>
|
||||
say(`God's Eye View could not load here: ${error?.message || error}.`),
|
||||
);
|
||||
setTimeout(() => {
|
||||
if (!ready)
|
||||
say(
|
||||
"God's Eye View did not load here" +
|
||||
(blocked ? ` (${blocked})` : '') +
|
||||
". Use Open in God's Eye View above.",
|
||||
);
|
||||
}, config.loadTimeoutMs);
|
||||
}
|
||||
|
||||
window.addEventListener('message', (event) => {
|
||||
const message = event.data;
|
||||
if (event.source === window) {
|
||||
if (message?.type === 'gev:ready') {
|
||||
ready = true;
|
||||
status.remove();
|
||||
if (queued) postView(queued);
|
||||
queued = null;
|
||||
}
|
||||
return;
|
||||
}
|
||||
if (event.source !== window.parent || message?.jsonrpc !== '2.0') return;
|
||||
if (
|
||||
message.id !== undefined &&
|
||||
pending.has(message.id) &&
|
||||
!message.method
|
||||
) {
|
||||
const { resolve, reject } = pending.get(message.id);
|
||||
pending.delete(message.id);
|
||||
if (message.error) reject(message.error);
|
||||
else resolve(message.result);
|
||||
return;
|
||||
}
|
||||
if (message.method === 'ui/notifications/tool-result') show(message.params);
|
||||
if (message.method === 'ui/notifications/host-context-changed')
|
||||
showDisplayMode(message.params);
|
||||
});
|
||||
|
||||
// Fullscreen, where the host offers it, gives the globe the room.
|
||||
let displayMode = 'inline';
|
||||
const showDisplayMode = (context) => {
|
||||
if (context?.displayMode) displayMode = context.displayMode;
|
||||
if (context?.availableDisplayModes)
|
||||
expand.hidden = !context.availableDisplayModes.includes('fullscreen');
|
||||
expand.textContent = displayMode === 'fullscreen' ? 'Collapse' : 'Expand';
|
||||
};
|
||||
expand.addEventListener('click', () => {
|
||||
request('ui/request-display-mode', {
|
||||
mode: displayMode === 'fullscreen' ? 'inline' : 'fullscreen',
|
||||
}).then(
|
||||
(result) => showDisplayMode({ displayMode: result?.mode }),
|
||||
() => {},
|
||||
);
|
||||
});
|
||||
|
||||
open.addEventListener('click', () => {
|
||||
if (!currentUrl) return;
|
||||
request('ui/open-link', { url: currentUrl }).catch(() =>
|
||||
window.open(currentUrl, '_blank', 'noopener'),
|
||||
);
|
||||
});
|
||||
|
||||
// The handshake the MCP Apps SDK's App.connect performs: hosts validate
|
||||
// these parameters, and keep a view hidden until it is initialized.
|
||||
request('ui/initialize', {
|
||||
appCapabilities: { availableDisplayModes: ['inline', 'fullscreen'] },
|
||||
appInfo: {
|
||||
name: 'gods-eye-view',
|
||||
title: "God's Eye View",
|
||||
version: '1.0.0',
|
||||
},
|
||||
protocolVersion: config.protocolVersion,
|
||||
}).then(
|
||||
(result) => {
|
||||
showDisplayMode(result?.hostContext);
|
||||
send({ method: 'ui/notifications/initialized' });
|
||||
notify('ui/notifications/size-changed', {
|
||||
width: document.body.clientWidth,
|
||||
height: config.panelHeight,
|
||||
});
|
||||
},
|
||||
(error) =>
|
||||
say(
|
||||
"This client did not accept the God's Eye View panel" +
|
||||
(error?.message ? `: ${error.message}` : '.'),
|
||||
),
|
||||
);
|
||||
}
|
||||
+1
-13
@@ -4,7 +4,6 @@ import { initDrawTool } from '../annotations/drawTool.js';
|
||||
import { initImageryBoxTool } from '../ui/imageryBoxTool.js';
|
||||
import { createRecentImageryPanel } from '../ui/recentImagery.js';
|
||||
import { initGevVoiceCommands } from '../voice/gevRealtime.js';
|
||||
import { installViews, isEmbeddedInline } from './embed.js';
|
||||
import { installScopeMask, destroyScopeMask } from '../scopeMask.js';
|
||||
import {
|
||||
installRenderGovernor,
|
||||
@@ -119,9 +118,7 @@ export function createApplicationTools({
|
||||
// GPU. Holder/data state is untouched, so return is seamless: restore
|
||||
// the loop, refresh the one DOM surface we gated, render a frame.
|
||||
const syncVisibilitySuspension = () => {
|
||||
// A panel's host may report it hidden while it is on screen; the panel
|
||||
// keeps drawing itself (see keepPanelRendering in embed.js).
|
||||
const hidden = document.hidden && !isEmbeddedInline();
|
||||
const hidden = document.hidden;
|
||||
viewer.useDefaultRenderLoop = !hidden;
|
||||
cockpitCloudEffects?.setSuspended?.(hidden);
|
||||
if (!hidden) {
|
||||
@@ -179,14 +176,5 @@ export function createApplicationTools({
|
||||
delete window.__gevVoiceCommands;
|
||||
});
|
||||
debug.voiceCommands = voiceCommands;
|
||||
defer(
|
||||
installViews({
|
||||
shell: styleManager,
|
||||
viewer,
|
||||
dataManager,
|
||||
run: (name, args) => voiceCommands.runner(name, args, { signal }),
|
||||
signal,
|
||||
}),
|
||||
);
|
||||
return { sceneDirector, annotations, voiceCommands };
|
||||
}
|
||||
+54
-13
@@ -165,14 +165,17 @@ export function polygonsContain(polygons, lat, lon) {
|
||||
/**
|
||||
* A unit's geometry, decoded on first use. Longitudes are unwrapped around
|
||||
* the largest part for the box so units cut at the antimeridian (Alaska,
|
||||
* Chukotka) get a box on the right side of the globe.
|
||||
* Chukotka) get a box on the right side of the globe. An entry from another
|
||||
* encoding supplies `decode()` returning the same `[outer, ...holes][]`.
|
||||
*/
|
||||
function geometryOf(entry) {
|
||||
export function geometryOf(entry) {
|
||||
if (entry.geometry) return entry.geometry;
|
||||
const decimals = entry.feature.d ?? entry.decimals;
|
||||
const polygons = entry.feature.polygons.map((poly) =>
|
||||
poly.map((ring) => decodeRing(ring, decimals)),
|
||||
);
|
||||
const polygons = entry.decode
|
||||
? entry.decode()
|
||||
: entry.feature.polygons.map((poly) =>
|
||||
poly.map((ring) => decodeRing(ring, decimals)),
|
||||
);
|
||||
const ref = polygons[0]?.[0]?.[0]?.[0] ?? 0;
|
||||
let west = Infinity;
|
||||
let south = Infinity;
|
||||
@@ -190,7 +193,7 @@ function geometryOf(entry) {
|
||||
if (lat > north) north = lat;
|
||||
}
|
||||
}
|
||||
const label = entry.feature.label
|
||||
const label = Array.isArray(entry.feature.label)
|
||||
? { lon: entry.feature.label[0], lat: entry.feature.label[1] }
|
||||
: ringCentroid(polygons[0][0]);
|
||||
entry.geometry = {
|
||||
@@ -220,7 +223,7 @@ function ringCentroid(ring) {
|
||||
}
|
||||
|
||||
/** Km from a point to a unit's box (0 inside it). */
|
||||
function boxDistanceKm(bbox, lat, lon) {
|
||||
export function boxDistanceKm(bbox, lat, lon) {
|
||||
const [west, south, east, north] = bbox;
|
||||
const mid = (west + east) / 2;
|
||||
const x = lon - 360 * Math.round((lon - mid) / 360);
|
||||
@@ -417,8 +420,8 @@ const COUNTY_WORDS = [
|
||||
];
|
||||
const COUNTY_WORD_RE = new RegExp(`\\b(${COUNTY_WORDS.join('|')})\\b`);
|
||||
const STATE_PREFIX_RE =
|
||||
/^(?:free state|state|commonwealth|province|territory|prefecture|region|land|canton|department) of (.+)$/;
|
||||
const STATE_SUFFIX_RE = /^(.+) (?:state|province|prefecture|territory)$/;
|
||||
/^(free state|state|commonwealth|province|territory|prefecture|region|land|canton|department) of (.+)$/;
|
||||
const STATE_SUFFIX_RE = /^(.+) (state|province|prefecture|territory)$/;
|
||||
|
||||
/**
|
||||
* Split an ask into what it names and how.
|
||||
@@ -479,14 +482,17 @@ export function parseAdminQuery(text) {
|
||||
if (prefix)
|
||||
return {
|
||||
kind: 'state',
|
||||
name: prefix[1],
|
||||
name: prefix[2],
|
||||
qualifiers,
|
||||
postal: null,
|
||||
countyWord: null,
|
||||
full: head,
|
||||
form: 'prefix',
|
||||
adminType: prefix[1],
|
||||
};
|
||||
const suffix = STATE_SUFFIX_RE.exec(head);
|
||||
// South Africa's canonical province name is literally "Free State"; do not
|
||||
// strip its final word as though the ask explicitly named an admin type.
|
||||
const suffix = head === 'free state' ? null : STATE_SUFFIX_RE.exec(head);
|
||||
if (suffix)
|
||||
return {
|
||||
kind: 'state',
|
||||
@@ -496,6 +502,7 @@ export function parseAdminQuery(text) {
|
||||
countyWord: null,
|
||||
full: head,
|
||||
form: 'suffix',
|
||||
adminType: suffix[2],
|
||||
};
|
||||
return {
|
||||
kind: 'any',
|
||||
@@ -507,6 +514,30 @@ export function parseAdminQuery(text) {
|
||||
};
|
||||
}
|
||||
|
||||
/** Explicit wording is stronger than a stale same-name geometry. */
|
||||
function explicitStateTypeFits(entry, parsed) {
|
||||
if (!parsed.adminType) return true;
|
||||
const words = new Set(
|
||||
normalizeAdminName(entry.feature?.type)
|
||||
.split(/[^a-z]+/)
|
||||
.filter(Boolean),
|
||||
);
|
||||
const accepted =
|
||||
{
|
||||
'free state': ['state'],
|
||||
state: ['state'],
|
||||
commonwealth: ['state'],
|
||||
province: ['province'],
|
||||
territory: ['territory'],
|
||||
prefecture: ['prefecture'],
|
||||
region: ['region'],
|
||||
land: ['state'],
|
||||
canton: ['canton'],
|
||||
department: ['department', 'departamento'],
|
||||
}[parsed.adminType] || [];
|
||||
return accepted.some((word) => words.has(word));
|
||||
}
|
||||
|
||||
/** A qualifier names the unit's country ("Bavaria, Germany") or the unit
|
||||
* itself ("Washington, DC"). */
|
||||
function qualifierMatches(entry, qualifier) {
|
||||
@@ -523,7 +554,7 @@ function qualifierMatches(entry, qualifier) {
|
||||
* Rank candidates: a unit containing `near` first, then the match tier,
|
||||
* then distance from `near`, then larger area. Returns the ranked list.
|
||||
*/
|
||||
function rank(candidates, near) {
|
||||
export function rank(candidates, near) {
|
||||
const hasNear = Number.isFinite(near?.lat) && Number.isFinite(near?.lon);
|
||||
const scored = candidates.map(({ entry, tier }) => {
|
||||
const geometry = geometryOf(entry);
|
||||
@@ -644,6 +675,10 @@ async function stateCandidates(parsed, { allowAmbiguous = false } = {}) {
|
||||
list = list.filter((i) =>
|
||||
parsed.qualifiers.every((q) => qualifierMatches(i.entry, q)),
|
||||
);
|
||||
// Preserve explicit identity at the selector seam. In particular,
|
||||
// "Bagmati Province" must not select Natural Earth's historical NP-BA
|
||||
// Bagmati Zone merely because the point and normalized name overlap.
|
||||
list = list.filter((i) => explicitStateTypeFits(i.entry, parsed));
|
||||
if (allowAmbiguous) return list;
|
||||
// A bare name answers alone only for a prominent unit — a US state, a
|
||||
// Canadian province, Bavaria — never a French département, an English
|
||||
@@ -756,6 +791,12 @@ export async function findAdminAreaAt(names, lat, lon, scope) {
|
||||
.map(parseAdminQuery)
|
||||
.filter((p) => p?.name);
|
||||
if (!parsedList.length) return null;
|
||||
// If the ask explicitly names an admin type, do not let the geocoder's
|
||||
// untyped alias reopen an incompatible historical same-name boundary.
|
||||
const explicitStates = parsedList.filter(
|
||||
(parsed) => parsed.kind === 'state' && parsed.adminType,
|
||||
);
|
||||
const stateParses = explicitStates.length ? explicitStates : parsedList;
|
||||
const containing = (items) => {
|
||||
const out = [];
|
||||
for (const item of items)
|
||||
@@ -800,7 +841,7 @@ export async function findAdminAreaAt(names, lat, lon, scope) {
|
||||
return toResult(rank(found, { lat, lon })[0], found.length);
|
||||
}
|
||||
const states = [];
|
||||
for (const parsed of parsedList)
|
||||
for (const parsed of stateParses)
|
||||
states.push(
|
||||
...(await stateCandidates(
|
||||
{ ...parsed, qualifiers: [] },
|
||||
|
||||
@@ -225,6 +225,28 @@ test('geocoder-confirmed lookup: the named unit that contains the point', async
|
||||
assert.equal(await findAdminAreaAt(['Georgia'], 41.7, 44.8, 'state'), null);
|
||||
const newYork = await findAdminAreaAt(['New York'], 42.9, -75.5, 'state');
|
||||
assert.equal(newYork?.id, 'US-NY');
|
||||
assert.equal(
|
||||
(
|
||||
await findAdminAreaAt(
|
||||
['Ontario Province', 'Ontario'],
|
||||
43.6532,
|
||||
-79.3832,
|
||||
'state',
|
||||
)
|
||||
)?.id,
|
||||
'CA-ON',
|
||||
);
|
||||
// Natural Earth still carries the historical NP-BA Bagmati Zone. An ask
|
||||
// for today's province must not silently select that incompatible geometry.
|
||||
assert.equal(
|
||||
await findAdminAreaAt(
|
||||
['Bagmati Province', 'Bagmati'],
|
||||
27.7172,
|
||||
85.324,
|
||||
'state',
|
||||
),
|
||||
null,
|
||||
);
|
||||
// A county the geocoder typed, named without the county word.
|
||||
const travis = await findAdminAreaAt(
|
||||
['Travis', 'Travis County'],
|
||||
|
||||
@@ -5,9 +5,6 @@ import { createAisStreamSource } from '../sources/live/standalone.js';
|
||||
const aisLiveVesselsLayer = createApplicationVessels({
|
||||
source: createAisStreamSource({
|
||||
apiUrl: import.meta.env?.VITE_AIS_LIVE_API_URL || '/api/ais-live',
|
||||
// Resolved against the document's address, which a panel host may serve
|
||||
// from its own scheme.
|
||||
origin: () => globalThis.document?.baseURI ?? 'http://localhost',
|
||||
}),
|
||||
options: {
|
||||
maxRows: import.meta.env?.VITE_AIS_LIVE_MAX_ROWS,
|
||||
|
||||
+872
-144
File diff suppressed because it is too large.
Load diff
@@ -174,11 +174,95 @@ test('analyst: nearest sorting attaches distanceKm ascending', async () => {
|
||||
|
||||
test('analyst: follow-up re-filters the remembered set without re-snapshot', async () => {
|
||||
const eng = makeEngine();
|
||||
await eng.query({ layers: ['flights'], scope: { kind: 'region', name: 'Texland' } });
|
||||
const r = await eng.query({ followUp: true, filters: [{ field: 'onGround', op: 'eq', value: true }] });
|
||||
const first = await eng.query({
|
||||
layers: ['flights'],
|
||||
scope: { kind: 'region', name: 'Texland' },
|
||||
});
|
||||
const r = await eng.query({
|
||||
followUp: true,
|
||||
filters: [{ field: 'onGround', op: 'eq', value: true }],
|
||||
});
|
||||
assert.equal(r.count, 1);
|
||||
assert.equal(r.items[0].id, 'GND1');
|
||||
assert.equal(r.coverage.followUp, true);
|
||||
assert.equal(r.scopeLabel, first.scopeLabel);
|
||||
assert.equal(r.display.scope, first.display.scope);
|
||||
assert.equal(r.coverage.scope, first.coverage.scope);
|
||||
});
|
||||
|
||||
test('analyst: follow-ups preserve partial and unanswered coverage', async () => {
|
||||
const eng = createAnalystEngine({
|
||||
getRecords: (key) => (key === 'flights' ? FLIGHTS : []),
|
||||
getLayerSnapshot: (key) => ({
|
||||
enabled: key === 'flights',
|
||||
feedState: key === 'flights' ? 'nominal' : 'off',
|
||||
}),
|
||||
getViewContext: () => ({ lat: 30.27, lon: -97.74, viewRadiusKm: 150 }),
|
||||
});
|
||||
const first = await eng.query({
|
||||
layers: ['flights', 'military'],
|
||||
scope: { kind: 'anywhere' },
|
||||
});
|
||||
assert.equal(first.partial, true);
|
||||
assert.deepEqual(first.unanswered, ['military']);
|
||||
|
||||
const followUp = await eng.query({ followUp: true, limit: 1 });
|
||||
assert.equal(followUp.partial, true);
|
||||
assert.deepEqual(followUp.unanswered, ['military']);
|
||||
assert.deepEqual(
|
||||
followUp.coverage.layersQueried.map(({ layerKey, status }) => [
|
||||
layerKey,
|
||||
status,
|
||||
]),
|
||||
first.coverage.layersQueried.map(({ layerKey, status }) => [
|
||||
layerKey,
|
||||
status,
|
||||
]),
|
||||
);
|
||||
});
|
||||
|
||||
test('analyst: cancelled pre-commit work cannot replace follow-up memory', async () => {
|
||||
const eng = makeEngine();
|
||||
await eng.query({
|
||||
layers: ['flights'],
|
||||
scope: { kind: 'anywhere' },
|
||||
filters: [{ field: 'id', op: 'eq', value: 'SWA1' }],
|
||||
});
|
||||
let current = true;
|
||||
const stale = await eng.query(
|
||||
{ layers: ['local-firms'], scope: { kind: 'anywhere' } },
|
||||
{
|
||||
isCurrent: () => current,
|
||||
beforeCommit: async () => {
|
||||
current = false;
|
||||
},
|
||||
},
|
||||
);
|
||||
assert.equal(stale.code, 'CANCELLED');
|
||||
const followUp = await eng.query({ followUp: true });
|
||||
assert.deepEqual(followUp.items.map((item) => item.id), ['SWA1']);
|
||||
});
|
||||
|
||||
test('analyst: failed pre-commit work cannot replace follow-up memory', async () => {
|
||||
const eng = makeEngine();
|
||||
await eng.query({
|
||||
layers: ['flights'],
|
||||
scope: { kind: 'anywhere' },
|
||||
filters: [{ field: 'id', op: 'eq', value: 'SWA1' }],
|
||||
});
|
||||
await assert.rejects(
|
||||
eng.query(
|
||||
{ layers: ['local-firms'], scope: { kind: 'anywhere' } },
|
||||
{
|
||||
beforeCommit: async () => {
|
||||
throw new Error('projection failed');
|
||||
},
|
||||
},
|
||||
),
|
||||
/projection failed/,
|
||||
);
|
||||
const followUp = await eng.query({ followUp: true });
|
||||
assert.deepEqual(followUp.items.map((item) => item.id), ['SWA1']);
|
||||
});
|
||||
|
||||
test('analyst: unresolved region is an honest failure, not empty success', async () => {
|
||||
@@ -200,6 +284,39 @@ test('analyst: a region lookup timeout is reported as region-timeout', async ()
|
||||
assert.equal(r.coverage.scope, 'region:Texas:timeout');
|
||||
});
|
||||
|
||||
test('analyst: region lookup receives and obeys the owning action signal', async () => {
|
||||
const controller = new AbortController();
|
||||
let receivedSignal = null;
|
||||
const eng = createAnalystEngine({
|
||||
getRecords: () => FLIGHTS,
|
||||
resolveRegionRing: async (_name, signal) => {
|
||||
receivedSignal = signal;
|
||||
await new Promise((resolve) =>
|
||||
signal.addEventListener('abort', resolve, { once: true }),
|
||||
);
|
||||
return null;
|
||||
},
|
||||
getViewContext: () => ({
|
||||
lat: 30.27,
|
||||
lon: -97.74,
|
||||
viewRadiusKm: 150,
|
||||
}),
|
||||
});
|
||||
const pending = eng.query(
|
||||
{ layers: ['flights'], scope: { kind: 'region', name: 'Texas' } },
|
||||
{
|
||||
signal: controller.signal,
|
||||
isCurrent: () => !controller.signal.aborted,
|
||||
},
|
||||
);
|
||||
await Promise.resolve();
|
||||
controller.abort();
|
||||
const result = await pending;
|
||||
assert.equal(receivedSignal, controller.signal);
|
||||
assert.equal(result.code, 'CANCELLED');
|
||||
assert.equal(result.cancelled, true);
|
||||
});
|
||||
|
||||
test('analyst: route fields queryable from cached enrichment only', async () => {
|
||||
const r = await makeEngine().query({
|
||||
layers: ['flights'], scope: { kind: 'anywhere' },
|
||||
@@ -262,20 +379,132 @@ test('helpers: haversine sanity + scope radius', () => {
|
||||
assert.deepEqual(scoped.map((f) => f.id).sort(), ['GND1', 'SWA1']);
|
||||
});
|
||||
|
||||
test('bounded loaded cohorts disclose truncation before filtering and retain it on follow-up', async () => {
|
||||
test('capped cohorts report returned/total/truncated and keep it on follow-up', async () => {
|
||||
const engine = createAnalystEngine({
|
||||
getRecords: () => [{ id: 'sample dam', lat: 0, lon: 0, name: 'sample' }],
|
||||
getRecordCoverage: () => ({ basis: 'bounded-loaded-records', recordsExamined: 1, loadedCount: 3000, sourceTruncated: true }),
|
||||
getRecordCoverage: () => ({ total: 3000, truncated: true }),
|
||||
getViewContext: () => ({ lat: 0, lon: 0, viewRadiusKm: 25 }),
|
||||
});
|
||||
const result = await engine.query({ layers: ['local-dams'], scope: { kind: 'anywhere' }, sortBy: 'distance' });
|
||||
assert.equal(result.count, 1);
|
||||
assert.equal(result.coverage.layersQueried[0].sourceTruncated, true);
|
||||
assert.match(result.coverage.note, /omitted records may change the nearest item or count/);
|
||||
const layer = result.coverage.layersQueried[0];
|
||||
assert.deepEqual({ returned: layer.returned, total: layer.total, truncated: layer.truncated }, { returned: 1, total: 3000, truncated: true });
|
||||
assert.deepEqual(result.coverage.records, { returned: 1, total: 3000, truncated: true });
|
||||
assert.match(result.display.caveat, /counted 1 of 3,000 loaded records/);
|
||||
const followUp = await engine.query({ followUp: true });
|
||||
assert.equal(followUp.coverage.layersQueried[0].loadedCount, 3000);
|
||||
assert.equal(followUp.coverage.layersQueried[0].total, 3000);
|
||||
});
|
||||
|
||||
test('an uncapped cohort is complete and carries no caveat', async () => {
|
||||
const result = await makeEngine().query({ layers: ['flights'], scope: { kind: 'anywhere' } });
|
||||
assert.deepEqual(result.coverage.records, { returned: 4, total: 4, truncated: false });
|
||||
assert.equal(result.display.caveat, undefined);
|
||||
assert.equal(result.display.scope, 'anywhere in the loaded data');
|
||||
});
|
||||
|
||||
test('unknown fields, operators, scopes and wrong-typed values are refused with the allowed values', async () => {
|
||||
const engine = makeEngine();
|
||||
const field = await engine.query({ layers: ['flights'], filters: [{ field: 'time', op: 'gte', value: 5 }] });
|
||||
assert.equal(field.ok, false);
|
||||
assert.equal(field.code, 'UNKNOWN_FIELD');
|
||||
assert.ok(field.allowed.includes('altitudeM'));
|
||||
const op = await engine.query({ layers: ['flights'], filters: [{ field: 'altitudeM', op: 'wat', value: 5 }] });
|
||||
assert.equal(op.code, 'BAD_OPERATOR');
|
||||
assert.deepEqual(op.allowed, ['gt', 'gte', 'lt', 'lte', 'eq', 'neq']);
|
||||
const flag = await engine.query({ layers: ['flights'], filters: [{ field: 'military', op: 'eq', value: 'false' }] });
|
||||
assert.equal(flag.code, 'BAD_VALUE', 'the string "false" must not match the TRUE record');
|
||||
const num = await engine.query({ layers: ['flights'], filters: [{ field: 'altitudeM', op: 'gt', value: 'high' }] });
|
||||
assert.equal(num.code, 'BAD_VALUE');
|
||||
const scope = await engine.query({ layers: ['flights'], scope: { kind: 'polygon' } });
|
||||
assert.equal(scope.code, 'UNKNOWN_SCOPE');
|
||||
assert.deepEqual(scope.allowed, ['view', 'region', 'radius', 'anywhere']);
|
||||
const sort = await engine.query({ layers: ['flights'], sortBy: 'loudness' });
|
||||
assert.equal(sort.code, 'UNKNOWN_FIELD');
|
||||
assert.equal(engine.hasMemory(), false, 'refusals never become follow-up memory');
|
||||
});
|
||||
|
||||
test('a typed boolean filter still matches exactly', async () => {
|
||||
const r = await makeEngine().query({ layers: ['flights'], scope: { kind: 'anywhere' }, filters: [{ field: 'military', op: 'eq', value: false }] });
|
||||
assert.deepEqual(r.items.map((i) => i.id).sort(), ['GND1', 'N123', 'SWA1']);
|
||||
});
|
||||
|
||||
test('distance honours sortDir: desc is farthest first', async () => {
|
||||
const far = await makeEngine().query({ layers: ['flights'], scope: { kind: 'anywhere' }, sortBy: 'distance', sortDir: 'desc', limit: 1 });
|
||||
assert.equal(far.items[0].id, 'N123');
|
||||
const near = await makeEngine().query({ layers: ['flights'], scope: { kind: 'anywhere' }, sortBy: 'distance', limit: 1 });
|
||||
assert.equal(near.items[0].id, 'SWA1');
|
||||
});
|
||||
|
||||
test('a follow-up without a previous answer is refused, not a fresh flights query', async () => {
|
||||
const r = await makeEngine().query({ followUp: true, filters: [] });
|
||||
assert.equal(r.ok, false);
|
||||
assert.equal(r.code, 'NO_RESULT_CONTEXT');
|
||||
});
|
||||
|
||||
test('follow-ups never rewrite the remembered rows', async () => {
|
||||
const engine = makeEngine();
|
||||
await engine.query({ layers: ['flights'], scope: { kind: 'anywhere' } });
|
||||
await engine.query({ followUp: true, sortBy: 'distance' });
|
||||
const again = await engine.query({ followUp: true });
|
||||
assert.equal(again.count, 4, 'a follow-up keeps the remembered set, not the view');
|
||||
assert.ok(again.items.every((item) => item.distanceKm === undefined));
|
||||
});
|
||||
|
||||
test('a superseded slow region query cannot overwrite the newer follow-up memory', async () => {
|
||||
let releaseSlow;
|
||||
const engine = createAnalystEngine({
|
||||
getRecords: (key) => ({ flights: FLIGHTS, 'ais-live-vessels': SHIPS }[key] || []),
|
||||
resolveRegionRing: () => new Promise((resolve) => { releaseSlow = () => resolve(TEXLAND); }),
|
||||
getViewContext: () => ({ lat: 30.27, lon: -97.74, viewRadiusKm: 150 }),
|
||||
});
|
||||
let generation = 0;
|
||||
const slowGeneration = ++generation;
|
||||
const slow = engine.query({ layers: ['flights'], scope: { kind: 'region', name: 'Texland' } }, { isCurrent: () => generation === slowGeneration });
|
||||
const newGeneration = ++generation;
|
||||
const fresh = await engine.query({ layers: ['ais-live-vessels'], scope: { kind: 'anywhere' } }, { isCurrent: () => generation === newGeneration });
|
||||
assert.equal(fresh.ok, true);
|
||||
releaseSlow();
|
||||
const stale = await slow;
|
||||
assert.equal(stale.code, 'CANCELLED');
|
||||
const followUp = await engine.query({ followUp: true });
|
||||
assert.deepEqual(followUp.items.map((i) => i.layerKey), ['ais-live-vessels', 'ais-live-vessels']);
|
||||
});
|
||||
|
||||
test('off, loading and unavailable layers are refused distinctly; a real zero stays ok', async () => {
|
||||
const engineFor = (snapshot, rows = [], warming = false) => createAnalystEngine({
|
||||
getRecords: () => rows,
|
||||
getLayerSnapshot: () => snapshot,
|
||||
isWarming: () => warming,
|
||||
getViewContext: () => ({ lat: 0, lon: 0, viewRadiusKm: 25 }),
|
||||
});
|
||||
assert.equal((await engineFor({ enabled: false, feedState: 'off' }).query({ layers: ['earthquakes'] })).code, 'LAYER_OFF');
|
||||
assert.equal((await engineFor({ enabled: true, feedState: 'loading' }).query({ layers: ['earthquakes'] })).code, 'NOT_READY');
|
||||
assert.equal((await engineFor({ enabled: true, feedState: 'nominal' }, [], true).query({ layers: ['earthquakes'] })).code, 'NOT_READY');
|
||||
assert.equal((await engineFor({ enabled: true, feedState: 'unavailable' }).query({ layers: ['earthquakes'] })).code, 'FEED_UNAVAILABLE');
|
||||
const zero = await engineFor({ enabled: true, feedState: 'nominal' }, [{ id: 'q', lat: 50, lon: 50, magnitude: 2 }]).query({ layers: ['earthquakes'], filters: [{ field: 'magnitude', op: 'gte', value: 5 }] });
|
||||
assert.equal(zero.ok, true);
|
||||
assert.equal(zero.count, 0);
|
||||
assert.equal(zero.coverage.layersQueried[0].status, 'ok');
|
||||
});
|
||||
|
||||
test('time fields filter by ISO and derive ageHours; the feed window is exposed', async () => {
|
||||
const now = Date.parse('2026-09-23T12:00:00Z');
|
||||
const engine = createAnalystEngine({
|
||||
now: () => now,
|
||||
getRecords: () => [
|
||||
{ id: 'recent', lat: 0, lon: 0, magnitude: 4, timeMs: now - 2 * 3_600_000 },
|
||||
{ id: 'older', lat: 0, lon: 0, magnitude: 5, timeMs: now - 20 * 3_600_000 },
|
||||
],
|
||||
getViewContext: () => ({ lat: 0, lon: 0, viewRadiusKm: 25 }),
|
||||
});
|
||||
const iso = await engine.query({ layers: ['earthquakes'], scope: { kind: 'anywhere' }, filters: [{ field: 'timeMs', op: 'gte', value: '2026-09-23T06:00:00Z' }] });
|
||||
assert.deepEqual(iso.items.map((i) => i.id), ['recent']);
|
||||
const age = await engine.query({ layers: ['earthquakes'], scope: { kind: 'anywhere' }, filters: [{ field: 'ageHours', op: 'lte', value: 6 }] });
|
||||
assert.deepEqual(age.items.map((i) => i.id), ['recent']);
|
||||
assert.equal(age.items[0].ageHours, 2);
|
||||
assert.equal(age.display.window, 'last 24 h');
|
||||
assert.equal(age.coverage.layersQueried[0].window, 'last 24 h');
|
||||
});
|
||||
|
||||
test('follow-up provenance stays attached to old rows after a feed recovers', async () => {
|
||||
let state = 'stale';
|
||||
@@ -293,3 +522,141 @@ test('follow-up provenance stays attached to old rows after a feed recovers', as
|
||||
assert.equal(engine.hasMemory(), false);
|
||||
assert.equal((await engine.query()).coverage.feedProvenance.overall, 'nominal');
|
||||
});
|
||||
|
||||
test('a large loaded set sorts and summarizes without overflowing the stack', async () => {
|
||||
const rows = Array.from({ length: 200_000 }, (_, i) => ({ id: `FIRE-${i}`, lat: 0, lon: 0, frp: i }));
|
||||
const engine = createAnalystEngine({
|
||||
getRecords: () => rows,
|
||||
getViewContext: () => ({ lat: 0, lon: 0, viewRadiusKm: 25 }),
|
||||
});
|
||||
const result = await engine.query({ layers: ['local-firms'], scope: { kind: 'anywhere' }, sortBy: 'frp', limit: 1 });
|
||||
assert.equal(result.count, 200_000);
|
||||
assert.equal(result.items[0].frp, 199_999);
|
||||
assert.equal(result.summary.frpMin, 0);
|
||||
});
|
||||
|
||||
test('an enabled but empty layer says why on screen', async () => {
|
||||
const engine = createAnalystEngine({
|
||||
getRecords: () => [],
|
||||
getLayerSnapshot: () => ({ enabled: true, feedState: 'nominal' }),
|
||||
getEmptyNote: () => 'Fly below 400 km to a covered region',
|
||||
getViewContext: () => ({ lat: 0, lon: 0, viewRadiusKm: 25 }),
|
||||
});
|
||||
const result = await engine.query({ layers: ['transit'] });
|
||||
assert.equal(result.ok, true);
|
||||
assert.equal(result.count, 0);
|
||||
assert.equal(result.coverage.layersQueried[0].status, 'empty');
|
||||
assert.match(result.display.caveat, /transit: Fly below 400 km/);
|
||||
});
|
||||
|
||||
test('a radius around a named place without a centre is refused, not measured around the camera', async () => {
|
||||
const r = await makeEngine().query({ layers: ['ais-live-vessels'], scope: { kind: 'radius', name: 'Singapore', km: 200 } });
|
||||
assert.equal(r.ok, false);
|
||||
assert.equal(r.code, 'BAD_SCOPE');
|
||||
const ok = await makeEngine().query({ layers: ['ais-live-vessels'], scope: { kind: 'radius', name: 'Houston', km: 200, center: { lat: 29.6, lon: -95.0 } } });
|
||||
assert.equal(ok.ok, true);
|
||||
});
|
||||
|
||||
test('a field copied with its hinted unit still resolves', async () => {
|
||||
const r = await makeEngine().query({ layers: ['local-firms'], scope: { kind: 'anywhere' }, sortBy: 'frp(MW)', filters: [{ field: 'frp (MW)', op: 'gt', value: 100 }] });
|
||||
assert.equal(r.ok, true);
|
||||
assert.deepEqual(r.items.map((i) => i.id), ['FIRE-3', 'FIRE-1']);
|
||||
});
|
||||
|
||||
test('a written unit converts to the field unit or is refused', async () => {
|
||||
const engine = makeEngine();
|
||||
const feet = await engine.query({ layers: ['flights'], scope: { kind: 'anywhere' }, filters: [{ field: 'altitudeM(ft)', op: 'gt', value: 40000 }] });
|
||||
assert.equal(feet.ok, true);
|
||||
assert.deepEqual(feet.items.map((i) => i.id), ['RCH01'], '40,000 ft is 12,192 m, not 40,000 m');
|
||||
const same = await engine.query({ layers: ['flights'], scope: { kind: 'anywhere' }, filters: [{ field: 'altitudeM (m)', op: 'gt', value: 12192 }] });
|
||||
assert.deepEqual(same.items.map((i) => i.id), ['RCH01']);
|
||||
const knots = await engine.query({ layers: ['flights'], scope: { kind: 'anywhere' }, filters: [{ field: 'speedMps(kts)', op: 'gt', value: 470 }] });
|
||||
assert.deepEqual(knots.items.map((i) => i.id), ['RCH01'], '470 kn is ~242 m/s');
|
||||
const wrong = await engine.query({ layers: ['flights'], filters: [{ field: 'altitudeM(kts)', op: 'gt', value: 1 }] });
|
||||
assert.equal(wrong.code, 'BAD_UNIT');
|
||||
const unitless = await engine.query({ layers: ['flights'], filters: [{ field: 'callsign(ft)', op: 'eq', value: 'X' }] });
|
||||
assert.equal(unitless.code, 'BAD_UNIT');
|
||||
const hinted = await engine.query({ layers: ['local-firms'], scope: { kind: 'anywhere' }, sortBy: 'frp(MW)' });
|
||||
assert.equal(hinted.ok, true);
|
||||
});
|
||||
|
||||
test('unambiguous numeric strings are numbers; blanks, booleans and words are not', async () => {
|
||||
const engine = makeEngine();
|
||||
const text = await engine.query({ layers: ['flights'], scope: { kind: 'anywhere' }, filters: [{ field: 'altitudeM', op: 'gt', value: '12192' }] });
|
||||
assert.equal(text.ok, true);
|
||||
assert.deepEqual(text.items.map((i) => i.id), ['RCH01']);
|
||||
for (const value of ['', ' ', true, 'high', '12k'])
|
||||
assert.equal((await engine.query({ layers: ['flights'], filters: [{ field: 'altitudeM', op: 'gt', value }] })).code, 'BAD_VALUE', JSON.stringify(value));
|
||||
});
|
||||
|
||||
test('any supplied centre must be a real coordinate', async () => {
|
||||
const engine = makeEngine();
|
||||
for (const center of [{ lat: 0 }, { lat: 999, lon: 0 }, { lat: 10, lon: 200 }, { lat: '10', lon: 20 }])
|
||||
assert.equal((await engine.query({ layers: ['flights'], scope: { kind: 'radius', km: 100, center } })).code, 'BAD_SCOPE', JSON.stringify(center));
|
||||
assert.equal((await engine.query({ layers: ['flights'], scope: { kind: 'radius', km: 100, center: { lat: 30.2, lon: -97.7 } } })).ok, true);
|
||||
});
|
||||
|
||||
test('mixed off/loading/unavailable layers refuse; a partly answerable query says so', async () => {
|
||||
const snapshots = { earthquakes: { enabled: true, feedState: 'unavailable' }, 'local-firms': { enabled: true, feedState: 'loading' }, flights: { enabled: false, feedState: 'off' } };
|
||||
const engine = createAnalystEngine({
|
||||
getRecords: (key) => (key === 'ais-live-vessels' ? SHIPS : []),
|
||||
getLayerSnapshot: (key) => snapshots[key] || { enabled: true, feedState: 'nominal' },
|
||||
getViewContext: () => ({ lat: 0, lon: 0, viewRadiusKm: 25 }),
|
||||
});
|
||||
const none = await engine.query({ layers: ['earthquakes', 'local-firms', 'flights'] });
|
||||
assert.equal(none.ok, false);
|
||||
assert.equal(none.code, 'NOT_READY');
|
||||
assert.deepEqual(none.layerStatus.map((l) => l.status), ['error', 'loading', 'off']);
|
||||
assert.equal(engine.hasMemory(), false, 'a refusal never becomes follow-up memory');
|
||||
const down = await engine.query({ layers: ['earthquakes', 'flights'] });
|
||||
assert.equal(down.code, 'FEED_UNAVAILABLE');
|
||||
const partial = await engine.query({ layers: ['ais-live-vessels', 'earthquakes'], scope: { kind: 'anywhere' } });
|
||||
assert.equal(partial.ok, true);
|
||||
assert.equal(partial.partial, true);
|
||||
assert.deepEqual(partial.unanswered, ['earthquakes']);
|
||||
assert.match(partial.display.caveat, /earthquakes error/);
|
||||
});
|
||||
|
||||
test('returned items are copies: mutating them cannot change the remembered answer', async () => {
|
||||
const engine = makeEngine();
|
||||
const first = await engine.query({ layers: ['flights'], scope: { kind: 'anywhere' }, sortBy: 'altitudeM' });
|
||||
for (const item of first.items) {
|
||||
item.altitudeM = -1;
|
||||
item.id = 'MUTATED';
|
||||
}
|
||||
const again = await engine.query({ followUp: true, sortBy: 'altitudeM' });
|
||||
assert.deepEqual(again.items.map((i) => i.id), ['RCH01', 'SWA1', 'N123', 'GND1']);
|
||||
assert.equal(FLIGHTS[1].altitudeM, 13500, 'the provider rows are untouched');
|
||||
});
|
||||
|
||||
test('ranking 250k rows stays within a budget and yields to the page', async () => {
|
||||
const rows = Array.from({ length: 250_000 }, (_, i) => ({ id: `F${i}`, lat: (i % 170) - 85, lon: (i % 350) - 175, frp: (i * 7919) % 100_003 }));
|
||||
const engine = createAnalystEngine({ getRecords: () => rows, getViewContext: () => ({ lat: 0, lon: 0, viewRadiusKm: 25 }) });
|
||||
let ticks = 0;
|
||||
const timer = setInterval(() => ticks++, 0);
|
||||
const started = performance.now();
|
||||
const result = await engine.query({ layers: ['local-firms'], scope: { kind: 'anywhere' }, sortBy: 'frp', limit: 1 });
|
||||
const elapsed = performance.now() - started;
|
||||
clearInterval(timer);
|
||||
assert.equal(result.count, 250_000);
|
||||
assert.equal(result.items[0].frp, 100_002);
|
||||
assert.ok(elapsed < 400, `took ${Math.round(elapsed)} ms`);
|
||||
assert.ok(ticks > 0, 'the scan yielded between slices');
|
||||
let current = true;
|
||||
const pending = engine.query({ layers: ['local-firms'], scope: { kind: 'anywhere' }, sortBy: 'distance' }, { isCurrent: () => current });
|
||||
current = false;
|
||||
assert.equal((await pending).code, 'CANCELLED', 'a superseded scan stops at the next slice');
|
||||
});
|
||||
|
||||
test('analyst: a follow-up naming other layers than the last answer is refused', async () => {
|
||||
const engine = createAnalystEngine({
|
||||
getRecords: (key) => ({ flights: FLIGHTS, 'local-firms': FIRES }[key] || []),
|
||||
resolveRegionRing: async () => null,
|
||||
getViewContext: () => ({ lat: 30.27, lon: -97.74, viewRadiusKm: 150 }),
|
||||
});
|
||||
await engine.query({ layers: ['flights'], scope: { kind: 'anywhere' } });
|
||||
const r = await engine.query({ layers: ['local-firms'], followUp: true });
|
||||
assert.equal(r.ok, false);
|
||||
assert.equal(r.code, 'FOLLOW_UP_MISMATCH');
|
||||
assert.equal(engine.hasMemory(), true, 'the earlier answer is still the last answer');
|
||||
});
|
||||
@@ -0,0 +1,162 @@
|
||||
// Outline names are drawn by the DOM annotation overlay, never as Cesium
|
||||
// text labels: the boundary data modules, the outline resolver and the
|
||||
// annotation renderers must not import or construct Cesium label primitives,
|
||||
// and drawing a bundled outline must create none at runtime.
|
||||
import { test } from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { readFileSync } from 'node:fs';
|
||||
import * as Cesium from 'cesium';
|
||||
import { FEATURE_SOURCE_METHODS } from '../sources/featureSource.js';
|
||||
import { createAnnotationResolver } from '../annotations/resolver.js';
|
||||
import { createWorldAnnotationRenderer } from '../annotations/worldAnnotationRenderer.js';
|
||||
|
||||
const read = (path) => readFileSync(new URL(path, import.meta.url), 'utf8');
|
||||
|
||||
/** Pure data modules: no Cesium at all. */
|
||||
const DATA_MODULES = [
|
||||
'./adminBoundaries.js',
|
||||
'./placeBoundaries.js',
|
||||
'./neighborhoodPolygons.js',
|
||||
'./local_data/us_census_places/files.js',
|
||||
'./local_data/wof_neighborhoods/files.js',
|
||||
];
|
||||
|
||||
/** Modules that use Cesium for outlines but must not draw text with it. */
|
||||
const CESIUM_MODULES = [
|
||||
'../annotations/resolver.js',
|
||||
'../annotations/worldAnnotationRenderer.js',
|
||||
'../annotations/hybridAnnotationRenderer.js',
|
||||
];
|
||||
|
||||
/** The DOM overlay: callout elements, never Cesium label primitives. */
|
||||
const OVERLAY_MODULES = ['../annotations/screenAnnotationRenderer.js'];
|
||||
|
||||
const LABEL_PRIMITIVES = [
|
||||
/\bLabelCollection\b/,
|
||||
/\bLabelGraphics\b/,
|
||||
/\bLabelStyle\b/,
|
||||
/\bLabelVisualizer\b/,
|
||||
];
|
||||
|
||||
/** Entity label options, inline or assigned later. */
|
||||
const ENTITY_LABELS = [
|
||||
// `label: { text: … }` or `label: new …`
|
||||
/\blabel\s*:\s*\{[^{}]*\btext\s*:/s,
|
||||
/\blabel\s*:\s*new\b/,
|
||||
// `label: labelOptions` (anything but `null`)
|
||||
/\blabel\s*:\s*(?!null\b)[A-Za-z_$][\w$]*\s*[,}\n]/,
|
||||
// `entity.label = …`
|
||||
/\.label\s*=(?!=)/,
|
||||
];
|
||||
|
||||
test('boundary data modules never import Cesium', () => {
|
||||
for (const path of DATA_MODULES) {
|
||||
const source = read(path);
|
||||
assert.doesNotMatch(source, /from\s+['"]cesium['"]/, path);
|
||||
assert.doesNotMatch(source, /import\(\s*['"]cesium['"]/, path);
|
||||
}
|
||||
});
|
||||
|
||||
test('outline modules and renderers construct no Cesium labels', () => {
|
||||
for (const path of [...DATA_MODULES, ...CESIUM_MODULES, ...OVERLAY_MODULES])
|
||||
for (const pattern of LABEL_PRIMITIVES)
|
||||
assert.doesNotMatch(read(path), pattern, `${path}: ${pattern}`);
|
||||
for (const path of [...DATA_MODULES, ...CESIUM_MODULES])
|
||||
for (const pattern of ENTITY_LABELS)
|
||||
assert.doesNotMatch(read(path), pattern, `${path}: ${pattern}`);
|
||||
});
|
||||
|
||||
test('the static check catches a Cesium label however it is written', () => {
|
||||
const offending = [
|
||||
'viewer.scene.primitives.add(new Cesium.LabelCollection());',
|
||||
"viewer.entities.add({ position, label: { text: 'Austin' } });",
|
||||
'const graphics = new LabelGraphics({ text });',
|
||||
'viewer.entities.add({ position, label: labelOptions });',
|
||||
'entity.label = new Cesium.LabelGraphics(options);',
|
||||
'entity.label = options;',
|
||||
];
|
||||
const all = [...LABEL_PRIMITIVES, ...ENTITY_LABELS];
|
||||
for (const source of offending)
|
||||
assert.ok(
|
||||
all.some((pattern) => pattern.test(source)),
|
||||
source,
|
||||
);
|
||||
// Data, not Cesium labels: a result's label point, a stripped proxy label.
|
||||
for (const source of [
|
||||
'label: geometry.label,',
|
||||
'liveProxy(anno, { label: null });',
|
||||
'if (entity.label === x) {}',
|
||||
])
|
||||
assert.ok(!ENTITY_LABELS.some((p) => p.test(source)), source);
|
||||
});
|
||||
|
||||
test('drawing bundled place and neighborhood outlines creates no Cesium label', async (t) => {
|
||||
// Cesium's material setup checks for these browser image types.
|
||||
for (const name of [
|
||||
'HTMLCanvasElement',
|
||||
'HTMLImageElement',
|
||||
'ImageBitmap',
|
||||
'OffscreenCanvas',
|
||||
]) {
|
||||
if (name in globalThis) continue;
|
||||
globalThis[name] = class {};
|
||||
t.after(() => delete globalThis[name]);
|
||||
}
|
||||
const featureSource = {};
|
||||
for (const method of FEATURE_SOURCE_METHODS)
|
||||
featureSource[method] = async () => null;
|
||||
const { resolveAnnotationTarget } = createAnnotationResolver({
|
||||
featureSource,
|
||||
});
|
||||
const primitives = [];
|
||||
const dataSources = [];
|
||||
const viewer = {
|
||||
camera: {
|
||||
positionCartographic: Cesium.Cartographic.fromDegrees(
|
||||
-97.7431,
|
||||
30.2672,
|
||||
3000,
|
||||
),
|
||||
},
|
||||
scene: { primitives: { add: (p) => primitives.push(p) } },
|
||||
dataSources: {
|
||||
add: (ds) => dataSources.push(ds),
|
||||
remove() {},
|
||||
},
|
||||
};
|
||||
const renderer = createWorldAnnotationRenderer(viewer);
|
||||
for (const [target, lon, lat] of [
|
||||
['Austin', -97.7431, 30.2672],
|
||||
['Notting Hill', -0.2, 51.511],
|
||||
]) {
|
||||
viewer.camera.positionCartographic = Cesium.Cartographic.fromDegrees(
|
||||
lon,
|
||||
lat,
|
||||
3000,
|
||||
);
|
||||
const resolved = await resolveAnnotationTarget({
|
||||
// The resolver reads only the camera (no scene to pick from).
|
||||
viewer: { camera: viewer.camera },
|
||||
target,
|
||||
footprint: true,
|
||||
deferFootprint: true,
|
||||
});
|
||||
assert.equal(resolved.source, 'bundled', target);
|
||||
renderer.add({
|
||||
id: target,
|
||||
type: 'area',
|
||||
label: resolved.label,
|
||||
ring: resolved.ring,
|
||||
polygons: resolved.polygons,
|
||||
footprintKind: resolved.footprintKind,
|
||||
synthesized: resolved.synthesized,
|
||||
anchor: { lon: resolved.lon, lat: resolved.lat, height: 0 },
|
||||
color: 'primary',
|
||||
});
|
||||
}
|
||||
assert.equal(primitives.length, 0, 'no primitive collections');
|
||||
const entities = dataSources.flatMap((ds) => ds.entities.values);
|
||||
assert.ok(entities.length >= 2, 'the outlines were drawn');
|
||||
for (const entity of entities) assert.equal(entity.label, undefined);
|
||||
renderer.destroy();
|
||||
});
|
||||
+12
-3
@@ -10,17 +10,26 @@
|
||||
* The attributed import cannot serve the browser: the Vite dev server answers
|
||||
* it with a JavaScript module, which the browser rejects for a JSON import.
|
||||
*
|
||||
* A pack split into many files passes no `importJson` (a literal import per
|
||||
* file would also bundle every file as a script chunk); under Node its file
|
||||
* is then read directly.
|
||||
*
|
||||
* @module data/bundledJson
|
||||
*/
|
||||
|
||||
/**
|
||||
* @param {URL} url - Resolved against the calling module's `import.meta.url`.
|
||||
* @param {() => Promise<{default: any}>} importJson - The caller's literal
|
||||
* `import('./….json', { with: { type: 'json' } })`, used only under Node.
|
||||
* @param {(() => Promise<{default: any}>)|null} [importJson] - The caller's
|
||||
* literal `import('./….json', { with: { type: 'json' } })`, used only under
|
||||
* Node; without it Node reads the file.
|
||||
* @returns {Promise<any>} The parsed JSON.
|
||||
*/
|
||||
export async function loadBundledJson(url, importJson, { signal } = {}) {
|
||||
if (url.protocol === 'file:') return (await importJson()).default;
|
||||
if (url.protocol === 'file:') {
|
||||
if (importJson) return (await importJson()).default;
|
||||
const fs = globalThis.process.getBuiltinModule('node:fs');
|
||||
return JSON.parse(fs.readFileSync(url, 'utf8'));
|
||||
}
|
||||
signal?.throwIfAborted();
|
||||
const response = await fetch(url, { signal });
|
||||
if (!response.ok) {
|
||||
|
||||
@@ -128,14 +128,6 @@ test('the catalog applies the fair cap to configured packs end to end', async ()
|
||||
sources.slice(0, 3).map((s) => s.id),
|
||||
['env-0', 'env-1', 'env-2'],
|
||||
);
|
||||
assert.equal(sources[0].pack, 'env');
|
||||
const [trimmed] = getSources.trimmedPacks();
|
||||
assert.deepEqual(
|
||||
{ ...trimmed, region: undefined },
|
||||
{ pack: 'env', available: 12, served: 9, region: undefined },
|
||||
);
|
||||
assert.equal(trimmed.region.south, 30);
|
||||
assert.ok(Math.abs(trimmed.region.north - 30.11) < 1e-9);
|
||||
} finally {
|
||||
for (const key of Object.keys(process.env)) {
|
||||
if (!(key in saved)) delete process.env[key];
|
||||
@@ -143,20 +135,3 @@ test('the catalog applies the fair cap to configured packs end to end', async ()
|
||||
Object.assign(process.env, saved);
|
||||
}
|
||||
});
|
||||
|
||||
test('a pack trimmed to its nearest cameras reports how many it offered', async () => {
|
||||
const { prioritizeSources } =
|
||||
await import('../../server/providers/cctv/normalize.js');
|
||||
const cameras = Array.from({ length: 5 }, (_, i) => ({
|
||||
id: `c${i}`,
|
||||
lat: i,
|
||||
lon: 0,
|
||||
}));
|
||||
const kept = prioritizeSources(cameras, 2, [{ lat: 0, lon: 0 }]);
|
||||
assert.deepEqual(
|
||||
kept.map((camera) => camera.id),
|
||||
['c0', 'c1'],
|
||||
);
|
||||
assert.equal(kept.available, 5);
|
||||
assert.deepEqual(kept.region, { west: 0, south: 0, east: 0, north: 4 });
|
||||
});
|
||||
+14
-3
@@ -377,15 +377,26 @@ export const NATURAL_EARTH_CREDIT = {
|
||||
'<a href="https://www.naturalearthdata.com" target="_blank" rel="noopener">Natural Earth</a> (public domain)',
|
||||
};
|
||||
|
||||
/** Registered when the first bundled US county outline resolves (public
|
||||
* domain — credited as a courtesy). */
|
||||
/** Registered when the first bundled US county or place outline resolves
|
||||
* (public domain — credited as a courtesy). */
|
||||
export const US_CENSUS_CREDIT = {
|
||||
key: 'us-census-counties',
|
||||
html:
|
||||
'US county boundaries from the ' +
|
||||
'US county and place boundaries from the ' +
|
||||
'<a href="https://www.census.gov/geographies/mapping-files/time-series/geo/cartographic-boundary.html" target="_blank" rel="noopener">U.S. Census Bureau</a> (public domain)',
|
||||
};
|
||||
|
||||
/** Registered when the first bundled Who's On First neighborhood outline
|
||||
* resolves. The link leads to the pack's per-source credits and licences and
|
||||
* says how the boundaries were modified. */
|
||||
export const WOF_CREDIT = {
|
||||
key: 'wof-neighborhoods',
|
||||
html:
|
||||
'Neighborhood boundaries: ' +
|
||||
'<a href="https://whosonfirst.org/docs/licenses/" target="_blank" rel="noopener">Who\'s On First</a> and contributors, simplified (' +
|
||||
'<a href="https://github.com/bilawalsidhu/gods-eye-view/blob/main/src/data/local_data/wof_neighborhoods/ATTRIBUTION.md" target="_blank" rel="noopener">sources and licences</a>)',
|
||||
};
|
||||
|
||||
/**
|
||||
* Per-feed transit credit, registered the first time that feed's vehicles
|
||||
* render (see `src/data/transitFeeds.js` for the license of each).
|
||||
|
||||
@@ -25,6 +25,7 @@ test('firms analyst record: full record maps every contract field', () => {
|
||||
frp: 1520.4,
|
||||
confidence: 0.9,
|
||||
satellite: 'N21',
|
||||
sensor: 'VIIRS',
|
||||
acqTime: 1_753_600_000_000,
|
||||
});
|
||||
});
|
||||
|
||||
@@ -41,3 +41,14 @@
|
||||
2026-09-25. Each record names its XYZ tile and expected traffic eligibility.
|
||||
Modified, test-only OpenStreetMap data; same ODbL attribution and database
|
||||
share-alike terms as the OpenFreeMap fixtures above.
|
||||
|
||||
- `ofm-outlines-austin-14-3743-{6745,6746}.pbf` — OpenFreeMap / OpenMapTiles
|
||||
z14 tiles from
|
||||
`https://tiles.openfreemap.org/planet/20260913_164504_pt/{z}/{x}/{y}.pbf`,
|
||||
retrieved 2026-09-28. Trimmed to the `transportation_name` features for
|
||||
Congress Avenue, South Congress Avenue and East 11th Street, and the
|
||||
`building` features within about 150 m of the Texas Capitol and the
|
||||
`landcover` park polygon around it, preserving
|
||||
original geometry and dictionaries. Used by the street and building outline
|
||||
tests. Same OpenFreeMap / OpenMapTiles / OpenStreetMap attribution and ODbL
|
||||
1.0 share-alike terms as the fixtures above. Test-only.
|
||||
Binary file not shown.
@@ -0,0 +1,175 @@
|
||||
¬
|
||||
transportation_name\œÝ”�——–———"> Ú €BŠ-3oxÇ:ÑNƒøçÀË‚£~—ˆ�ô© ~§Ð‡b—èó:©
|
||||
ref_lengthsubclassrefclassnamename_int
|
||||
name:latinname_dename_enname:es route_1_refroute_2_networkroute_1_network route_2_refnetworklayer"0"
|
||||
|
||||
junction"
|
||||
233"
|
||||
|
||||
motorway"
|
||||
232"0"
|
||||
234B"
|
||||
234A"
|
||||
South Interstate 35"
|
||||
secondary_construction"
|
||||
|
||||
The Circle"
|
||||
footway"
|
||||
path"$
|
||||
"Lady Bird Lake Hike and Bike Trail"
|
||||
|
||||
cycleway"
|
||||
Waller Creek Greenbelt Trail"&
|
||||
$Ann & Roy Butler Hike and Bike Trail"
|
||||
North Interstate 35"
|
||||
path_construction"
|
||||
Woodland Trail"
|
||||
Harper's Branch Connection"
|
||||
steps"
|
||||
|
||||
Snake Path"
|
||||
Sendero del Serpiente"
|
||||
Nellie Street Crossing"
|
||||
Woodland Avenue"
|
||||
Alameda Drive"
|
||||
minor"
|
||||
Forest Avenue"
|
||||
South 2nd Street"
|
||||
Post Road"
|
||||
Taylor Street"
|
||||
Pecan Grove Road"
|
||||
East Avenue"
|
||||
Fletcher Street"
|
||||
Lockhart Drive"
|
||||
Alta Vista Avenue"
|
||||
Retama Street"
|
||||
Haywood Avenue"
|
||||
Ramona Street"
|
||||
Ravine Drive"
|
||||
Betty Jo Drive"
|
||||
Spence Street"
|
||||
|
||||
Rebel Road"
|
||||
Bickler Road"
|
||||
Clifton Street"
|
||||
Columbus Street"
|
||||
Reagan Terrace"
|
||||
Daniel Drive"
|
||||
East Mary Street"
|
||||
Lindell Avenue"
|
||||
Drake Avenue"
|
||||
Mariposa Drive"
|
||||
Travis Heights Boulevard"
|
||||
Christopher Street"
|
||||
|
||||
Music Lane"
|
||||
Flores Street"
|
||||
Sunset Lane"
|
||||
Mission Ridge"
|
||||
Oak Heights Drive"
|
||||
Bonniview Street"
|
||||
Algarita Avenue"
|
||||
Edgecliff Terrace"
|
||||
Melissa Lane"
|
||||
|
||||
Sunny Lane"
|
||||
Le Grande Avenue"
|
||||
Avondale Road"
|
||||
Milam Place"
|
||||
Fairmount Avenue"
|
||||
River Street"
|
||||
Garden Street"
|
||||
Rosedale Terrace"
|
||||
Nash Hernandez Senior Road"
|
||||
West Monroe Street"
|
||||
Gillespie Place"
|
||||
Newning Avenue"
|
||||
West James Street"
|
||||
College Avenue"
|
||||
Sylvan Drive"
|
||||
Leland Street"
|
||||
Bonham Terrace"
|
||||
Sunny Vale Street"
|
||||
Bartlett Street"
|
||||
Ebony Street"
|
||||
Lupine Lane"
|
||||
Terrace Drive"
|
||||
Newton Street"
|
||||
Parkinson Drive"
|
||||
Inglewood Street"
|
||||
Manlove Street"
|
||||
Academy Drive"
|
||||
Clermont Avenue"
|
||||
Copeland Street"
|
||||
South 3rd Street"
|
||||
Crockett Street"
|
||||
East Gibson Street"
|
||||
West Gibson Street"
|
||||
Summit Street"
|
||||
Chelsea Lane"
|
||||
Willow Street"
|
||||
San Marcos Street"
|
||||
East Side Drive"
|
||||
Lone Oak Drive"
|
||||
East Elizabeth Street"
|
||||
West Elizabeth Street"
|
||||
Lambie Street"
|
||||
Wilson Street"
|
||||
|
||||
Eva Street"
|
||||
Roma Street"
|
||||
Waller Street"
|
||||
Elmhurst Drive"
|
||||
Kenwood Avenue"
|
||||
Brackenridge Street"
|
||||
Hillside Avenue"
|
||||
East Milton Street"
|
||||
West Milton Street"
|
||||
Fairlawn Lane"
|
||||
Euclid Avenue"
|
||||
Rutherford Place"
|
||||
Cummings Street"
|
||||
Nickerson Street"
|
||||
Nellie Street"
|
||||
Park Lane"
|
||||
Bellaire Drive"
|
||||
West Johanna Street"
|
||||
Post Oak Street"
|
||||
Rainey Street"
|
||||
Taylor Gaines Street"
|
||||
Harwood Place"
|
||||
Brooklyn Street"
|
||||
Jewell Street"
|
||||
LBJ Way"
|
||||
Bierce Street"
|
||||
Festival Beach Road"
|
||||
Art Dilly Drive"
|
||||
Haskell Street"
|
||||
Holly Street"
|
||||
|
||||
tertiary"
|
||||
West Mary Street"
|
||||
West Live Oak Street"
|
||||
East Live Oak Street"
|
||||
East Monroe Street"
|
||||
Bouldin Avenue"
|
||||
East Annie Street"
|
||||
West Annie Street"
|
||||
Dawson Road"
|
||||
secondary"
|
||||
South 1st Street"
|
||||
West Oltorf Street"
|
||||
East Riverside Drive"
|
||||
primary"
|
||||
South Congress Avenue"
|
||||
Barton Springs Road"
|
||||
35"
|
||||
Purple Heart Trail"
|
||||
US:US"
|
||||
US:I"
|
||||
290"
|
||||
us-interstate"0"
|
||||
Skyline Views"
|
||||
Blunn Creek Connection"
|
||||
Waterfront Promenade"
|
||||
Hartman Bridge"0(€ x
|
||||
@@ -1,8 +1,12 @@
|
||||
import { createLocalGeoJsonLayer } from './localGeojsonCore.js';
|
||||
import { INFRASTRUCTURE_DATA_URLS } from '../sources/infrastructureData.js';
|
||||
|
||||
const datacentersUrl = INFRASTRUCTURE_DATA_URLS['local-datacenters'];
|
||||
const damsUrl = INFRASTRUCTURE_DATA_URLS['local-dams'];
|
||||
// Resolved by Vite in builds and relative to this module in other consumers.
|
||||
const datacentersUrl = new URL(
|
||||
'./local_data/datacenters/datacenters.geojsonl',
|
||||
import.meta.url,
|
||||
).href;
|
||||
const damsUrl = new URL('./local_data/dams/dams.geojsonl', import.meta.url)
|
||||
.href;
|
||||
|
||||
/**
|
||||
* Create fresh datacenter and dam layers without starting or loading them.
|
||||
|
||||
@@ -192,7 +192,6 @@ test('consumer build includes only infrastructure code and resolves assets under
|
||||
const sources = Object.keys(entry.modules).filter((id) => id.endsWith('.js'));
|
||||
assert.deepEqual(sources.map((id) => id.split('/').at(-1)).sort(), [
|
||||
'infrastructure.js',
|
||||
'infrastructureData.js',
|
||||
'infrastructureOverlayEntry.js',
|
||||
'localGeojsonCore.js',
|
||||
'localGeojsonLod.js',
|
||||
|
||||
+723
-2
@@ -11,6 +11,8 @@ export const LAYER_STATE_TOKEN_ALPHABET =
|
||||
'0123456789abcdefghijklmnopqrstuvwxyz';
|
||||
const LAYER_STATE_TOKEN_PATTERN = /^[a-z0-9]{1,2}$/;
|
||||
const NEW_SINGLE_CHARACTER_TOKEN_PATTERN = /^[0-9]$/;
|
||||
/** Re-check cadence while a shared subject waits for its feed row to arrive. */
|
||||
const PENDING_TRACKING_POLL_MS = 1_000;
|
||||
/**
|
||||
* A tracking ID is a transponder address, not free text: 6 hex digits for an
|
||||
* ICAO24, with slack for TIS-B (`~abc123`) and similar prefixed forms. Bounding
|
||||
@@ -431,6 +433,12 @@ const OPTION_GROUPS = Object.freeze({
|
||||
]),
|
||||
});
|
||||
|
||||
const TRACKING_OPTION_KEY_BY_LAYER = Object.freeze({
|
||||
flights: 'selectedFlightsTrackingId',
|
||||
military: 'selectedMilitaryTrackingId',
|
||||
satellites: 'selectedSatTrackingId',
|
||||
});
|
||||
|
||||
export const SHARE_TRACKING_RESTORE_POLICIES = Object.freeze({
|
||||
flights: Object.freeze({
|
||||
optionOwner: 'flights',
|
||||
@@ -732,7 +740,7 @@ export function validateLayerStateAllocations(
|
||||
return true;
|
||||
}
|
||||
|
||||
export const REGISTRY_BY_ID = new Map(
|
||||
const REGISTRY_BY_ID = new Map(
|
||||
LAYER_STATE_REGISTRY.map((entry) => [entry.id, entry]),
|
||||
);
|
||||
const REGISTRY_BY_TOKEN = new Map(
|
||||
@@ -744,7 +752,7 @@ const OPTION_OWNER_IDS = Object.freeze([
|
||||
),
|
||||
]);
|
||||
|
||||
export function optionSpecs(ownerId) {
|
||||
function optionSpecs(ownerId) {
|
||||
return OPTION_GROUPS[ownerId] || [];
|
||||
}
|
||||
|
||||
@@ -1047,3 +1055,716 @@ export function layerOptionsForRestore(state, layerId) {
|
||||
if (!entry?.optionOwner) return null;
|
||||
return { ...normalizeLayerState(state).options[entry.optionOwner] };
|
||||
}
|
||||
|
||||
function safeStorage() {
|
||||
try {
|
||||
return globalThis.localStorage || null;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
function currentLayerOutcome(dataManager, layerId) {
|
||||
const state = dataManager.getLayerLifecycleState?.(layerId);
|
||||
return {
|
||||
settledEnabled: Boolean(state?.enabled),
|
||||
lifecycleState: state?.lifecycleState || 'missing',
|
||||
lifecycleUncertain: Boolean(state?.uncertain),
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Owns durable user layer preferences independently from transient runtime
|
||||
* choreography, and coordinates passive post-registration restoration.
|
||||
*/
|
||||
export class LayerStateCoordinator {
|
||||
constructor(
|
||||
dataManager,
|
||||
shareLinkManager,
|
||||
{
|
||||
storage = safeStorage(),
|
||||
restoreGate = null,
|
||||
onDurableStateChange = null,
|
||||
onTrackingRestoreStatus = null,
|
||||
now = () => Date.now(),
|
||||
// Injectable so the pending-window behavior is deterministically testable
|
||||
// without sleeping out a 90 s expiry.
|
||||
setTimer = (fn, ms) => setTimeout(fn, ms),
|
||||
clearTimer = (handle) => clearTimeout(handle),
|
||||
} = {},
|
||||
) {
|
||||
if (!dataManager?.registrationsFinalized) {
|
||||
throw new Error(
|
||||
'Layer state requires finalized data-layer registrations',
|
||||
);
|
||||
}
|
||||
this.dataManager = dataManager;
|
||||
this.shareLinkManager = shareLinkManager || null;
|
||||
this.storage = storage;
|
||||
this.restoreGate = restoreGate;
|
||||
this.onDurableStateChange = onDurableStateChange;
|
||||
this.onTrackingRestoreStatus = onTrackingRestoreStatus;
|
||||
this.now = now;
|
||||
this.setTimer = setTimer;
|
||||
this.clearTimer = clearTimer;
|
||||
this._durableState = createDefaultLayerState();
|
||||
this._source = 'defaults';
|
||||
this._destroyed = false;
|
||||
this._restoreControllers = new Map();
|
||||
this._shareCreatedAtMs = null;
|
||||
this._trackingRestoreController = null;
|
||||
this._trackingRestoreGeneration = 0;
|
||||
this._pendingTrackingTimer = null;
|
||||
this._pendingTrackingContext = null;
|
||||
this._unsubscribe = this.dataManager.subscribe((change) =>
|
||||
this._handleManagerChange(change),
|
||||
);
|
||||
this._unsubscribeVisibilityRequests =
|
||||
this.dataManager.subscribeVisibilityRequests((change) =>
|
||||
this._handleVisibilityRequest(change),
|
||||
);
|
||||
this.restorePromise = Promise.resolve([]);
|
||||
this.lastRestoreResults = [];
|
||||
}
|
||||
|
||||
start({
|
||||
shareLayerState = null,
|
||||
allowLocalState = true,
|
||||
shareCreatedAtMs = null,
|
||||
} = {}) {
|
||||
if (this._destroyed)
|
||||
throw new Error('Layer-state coordinator is destroyed');
|
||||
let selected = shareLayerState
|
||||
? normalizeLayerState(shareLayerState)
|
||||
: null;
|
||||
if (selected) {
|
||||
this._source = 'share';
|
||||
this._shareCreatedAtMs = Number.isFinite(shareCreatedAtMs)
|
||||
? shareCreatedAtMs
|
||||
: null;
|
||||
} else if (allowLocalState) {
|
||||
let stored = null;
|
||||
try {
|
||||
stored = parseStoredLayerState(
|
||||
this.storage?.getItem?.(LAYER_STATE_STORAGE_KEY),
|
||||
);
|
||||
} catch {
|
||||
/* best effort */
|
||||
}
|
||||
if (stored) {
|
||||
selected = stored;
|
||||
this._source = 'local';
|
||||
}
|
||||
} else {
|
||||
// A valid historical camera/style share with no v2 layer payload keeps
|
||||
// the exact legacy default-layer behavior. It must not inherit an
|
||||
// unrelated recipient's saved local layer preferences.
|
||||
this._source = 'legacy-share';
|
||||
}
|
||||
this._durableState = selected || createDefaultLayerState();
|
||||
this.shareLinkManager?.setLayerStateProvider?.(() =>
|
||||
this.getDurableState(),
|
||||
);
|
||||
this.shareLinkManager?.onLayerStateChange?.();
|
||||
this._notifyDurableState();
|
||||
if (!selected) return this.restorePromise;
|
||||
this.restorePromise = this._restoreSelectedState(
|
||||
this._source === 'share'
|
||||
? LAYER_RESTORE_ORIGINS.share
|
||||
: LAYER_RESTORE_ORIGINS.local,
|
||||
);
|
||||
return this.restorePromise;
|
||||
}
|
||||
|
||||
get source() {
|
||||
return this._source;
|
||||
}
|
||||
|
||||
getDurableState() {
|
||||
return cloneLayerState(this._durableState);
|
||||
}
|
||||
|
||||
_notifyDurableState() {
|
||||
try {
|
||||
this.onDurableStateChange?.(this.getDurableState());
|
||||
} catch {
|
||||
/* UI sync is best effort */
|
||||
}
|
||||
}
|
||||
|
||||
_handleVisibilityRequest(change) {
|
||||
if (!isExplicitLayerStateOrigin(change?.origin)) return;
|
||||
this._restoreControllers
|
||||
.get(change.layerId)
|
||||
?.abort('superseded-by-explicit-visibility');
|
||||
if (SHARE_TRACKING_RESTORE_POLICIES[change.layerId]) {
|
||||
this._revokePendingTrackingWatch('superseded-by-explicit-visibility');
|
||||
}
|
||||
}
|
||||
|
||||
/** Revoke every passive restore before explicit navigation can be reclaimed. */
|
||||
cancelPendingRestores(reason = 'superseded-by-explicit-navigation') {
|
||||
for (const controller of this._restoreControllers.values())
|
||||
controller.abort(reason);
|
||||
this._revokePendingTrackingWatch(reason);
|
||||
}
|
||||
|
||||
/**
|
||||
* Revoke a pending shared Follow. Physical navigation may also clear only
|
||||
* the exact passive selection, without writing recipient preferences.
|
||||
*/
|
||||
cancelPendingShareTracking(
|
||||
reason = 'superseded-by-explicit-navigation',
|
||||
{ clearSelection = false } = {},
|
||||
) {
|
||||
this._revokePendingTrackingWatch(reason);
|
||||
if (!clearSelection) return false;
|
||||
const selected = this._selectedShareTrackingTarget();
|
||||
return selected ? this._passivelyClearTrackingSelection(selected) : false;
|
||||
}
|
||||
|
||||
_handleManagerChange(change) {
|
||||
if (!change || this._destroyed) return;
|
||||
// Parameter and visibility ownership are independent. A newer explicit
|
||||
// option request may replace passive share options, but it must not abort
|
||||
// the same layer's visibility lifecycle.
|
||||
if (change.type === 'params-requested') {
|
||||
if (
|
||||
isExplicitLayerStateOrigin(change.origin) &&
|
||||
SHARE_TRACKING_RESTORE_POLICIES[change.layerId]
|
||||
) {
|
||||
this._revokePendingTrackingWatch('superseded-by-explicit-params');
|
||||
}
|
||||
return;
|
||||
}
|
||||
// A layer that goes away takes its latch with it, at ANY origin — a
|
||||
// programmatic disable or teardown never reaches the explicit-intent path
|
||||
// below, so revoke here before that early return.
|
||||
if (
|
||||
change.type === 'visibility' &&
|
||||
change.enabled === false &&
|
||||
SHARE_TRACKING_RESTORE_POLICIES[change.layerId]
|
||||
) {
|
||||
this._revokePendingTrackingWatch('owner-layer-disabled');
|
||||
}
|
||||
if (!isExplicitLayerStateOrigin(change.origin)) return;
|
||||
if (change.type === 'visibility') {
|
||||
this._restoreControllers
|
||||
.get(change.layerId)
|
||||
?.abort('superseded-by-explicit-visibility');
|
||||
if (SHARE_TRACKING_RESTORE_POLICIES[change.layerId]) {
|
||||
this._revokePendingTrackingWatch('superseded-by-explicit-visibility');
|
||||
}
|
||||
const enabled = new Set(this._durableState.enabledLayerIds);
|
||||
if (change.enabled) enabled.add(change.layerId);
|
||||
else enabled.delete(change.layerId);
|
||||
this._commitExplicit({
|
||||
...this._durableState,
|
||||
enabledLayerIds: [...enabled],
|
||||
});
|
||||
return;
|
||||
}
|
||||
if (change.type !== 'params') return;
|
||||
const entry = REGISTRY_BY_ID.get(change.layerId);
|
||||
if (!entry?.optionOwner) return;
|
||||
const ownerId = entry.optionOwner;
|
||||
const nextOwnerOptions = { ...this._durableState.options[ownerId] };
|
||||
const requestedParams = change.requestedParams || {};
|
||||
const trackingOptionKey =
|
||||
TRACKING_OPTION_KEY_BY_LAYER[change.layerId] || null;
|
||||
let changed = false;
|
||||
for (const spec of optionSpecs(ownerId)) {
|
||||
// Only persist keys present in this explicit request. getLayerParams()
|
||||
// can return a wider live snapshot containing transient or passively
|
||||
// changed values that this user action did not own. Tracking is the one
|
||||
// exception: an unrelated explicit option cancels a pending restoration
|
||||
// in that family, so its wider live value (active ID or null) must replace
|
||||
// the formerly durable pending ID instead of allowing reload resurrection.
|
||||
const explicitlyRequested = Object.hasOwn(requestedParams, spec.key);
|
||||
const implicitTrackingSync =
|
||||
!explicitlyRequested && spec.key === trackingOptionKey;
|
||||
if (!explicitlyRequested && !implicitTrackingSync) continue;
|
||||
const value = spec.normalize(change.params[spec.key]);
|
||||
if (value === null) {
|
||||
if (implicitTrackingSync) {
|
||||
nextOwnerOptions[spec.key] = null;
|
||||
changed = true;
|
||||
continue;
|
||||
}
|
||||
if (spec.defaultValue !== null || change.params[spec.key] !== null)
|
||||
continue;
|
||||
}
|
||||
nextOwnerOptions[spec.key] = value;
|
||||
changed = true;
|
||||
}
|
||||
if (!changed) return;
|
||||
this._commitExplicit({
|
||||
...this._durableState,
|
||||
options: { ...this._durableState.options, [ownerId]: nextOwnerOptions },
|
||||
});
|
||||
}
|
||||
|
||||
_commitExplicit(candidate) {
|
||||
this._durableState = normalizeLayerState(candidate);
|
||||
const serialized = serializeStoredLayerState(this._durableState);
|
||||
try {
|
||||
if (this.storage?.getItem?.(LAYER_STATE_STORAGE_KEY) !== serialized) {
|
||||
this.storage?.setItem?.(LAYER_STATE_STORAGE_KEY, serialized);
|
||||
}
|
||||
} catch {
|
||||
/* storage can be unavailable or quota-limited */
|
||||
}
|
||||
this.shareLinkManager?.onLayerStateChange?.();
|
||||
this._notifyDurableState();
|
||||
}
|
||||
|
||||
async _waitForRestoreGate() {
|
||||
if (!this.restoreGate) return;
|
||||
await (typeof this.restoreGate === 'function'
|
||||
? this.restoreGate()
|
||||
: this.restoreGate);
|
||||
}
|
||||
|
||||
async _restoreSelectedState(origin) {
|
||||
for (const entry of LAYER_STATE_REGISTRY) {
|
||||
this._restoreControllers.set(entry.id, new AbortController());
|
||||
}
|
||||
try {
|
||||
await this._waitForRestoreGate();
|
||||
const enabled = new Set(this._durableState.enabledLayerIds);
|
||||
const settled = await Promise.allSettled(
|
||||
LAYER_STATE_REGISTRY.map(async (entry) => {
|
||||
const controller = this._restoreControllers.get(entry.id);
|
||||
const targetEnabled = enabled.has(entry.id);
|
||||
const options = layerOptionsForRestore(this._durableState, entry.id);
|
||||
if (origin === LAYER_RESTORE_ORIGINS.share && options) {
|
||||
for (const trackingKey of Object.values(
|
||||
TRACKING_OPTION_KEY_BY_LAYER,
|
||||
)) {
|
||||
delete options[trackingKey];
|
||||
}
|
||||
}
|
||||
if (this._destroyed || controller?.signal.aborted) {
|
||||
return {
|
||||
layerId: entry.id,
|
||||
targetEnabled,
|
||||
origin,
|
||||
phase: 'reserved',
|
||||
...currentLayerOutcome(this.dataManager, entry.id),
|
||||
appliedOptions: {},
|
||||
cancellationReason: this._destroyed ? 'destroyed' : 'superseded',
|
||||
errorClass: 'cancelled',
|
||||
persistenceWrite: false,
|
||||
succeeded: false,
|
||||
};
|
||||
}
|
||||
// Reserve passive option state before any asynchronous lifecycle work.
|
||||
// A later explicit params intent then wins on its own lane without
|
||||
// cancelling or being overwritten by the visibility restore.
|
||||
const paramsSucceeded =
|
||||
!options ||
|
||||
Object.keys(options).length === 0 ||
|
||||
this.dataManager.setLayerParams(entry.id, options, { origin });
|
||||
return this.dataManager
|
||||
.restoreLayerState(
|
||||
entry.id,
|
||||
{
|
||||
enabled: targetEnabled,
|
||||
params: null,
|
||||
},
|
||||
{ origin, signal: controller.signal },
|
||||
)
|
||||
.then((result) => ({
|
||||
...result,
|
||||
appliedOptions: paramsSucceeded && options ? options : {},
|
||||
errorClass: paramsSucceeded
|
||||
? result.errorClass
|
||||
: 'ParamsRejected',
|
||||
succeeded: paramsSucceeded && result.succeeded,
|
||||
}));
|
||||
}),
|
||||
);
|
||||
this.lastRestoreResults = settled.map((result, index) => {
|
||||
if (result.status === 'fulfilled') return result.value;
|
||||
const entry = LAYER_STATE_REGISTRY[index];
|
||||
return {
|
||||
layerId: entry.id,
|
||||
targetEnabled: enabled.has(entry.id),
|
||||
origin,
|
||||
phase: 'coordinator',
|
||||
...currentLayerOutcome(this.dataManager, entry.id),
|
||||
appliedOptions: {},
|
||||
cancellationReason: null,
|
||||
errorClass: result.reason?.name || 'Error',
|
||||
error: String(result.reason?.message || result.reason),
|
||||
persistenceWrite: false,
|
||||
succeeded: false,
|
||||
};
|
||||
});
|
||||
return this.lastRestoreResults.map((result) => ({ ...result }));
|
||||
} finally {
|
||||
this._restoreControllers.clear();
|
||||
this._notifyDurableState();
|
||||
}
|
||||
}
|
||||
|
||||
_selectedShareTrackingTarget() {
|
||||
if (this._source !== 'share') return null;
|
||||
for (const [layerId, policy] of Object.entries(
|
||||
SHARE_TRACKING_RESTORE_POLICIES,
|
||||
)) {
|
||||
const targetId =
|
||||
this._durableState.options?.[policy.optionOwner]?.[policy.optionKey];
|
||||
if (targetId !== null && targetId !== undefined && targetId !== '') {
|
||||
return { layerId, targetId, ...policy };
|
||||
}
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
_passivelyClearTrackingSelection(selected) {
|
||||
const current =
|
||||
this._durableState.options?.[selected.optionOwner]?.[selected.optionKey];
|
||||
if (String(current) !== String(selected.targetId)) return false;
|
||||
const ownerOptions = {
|
||||
...this._durableState.options[selected.optionOwner],
|
||||
[selected.optionKey]: null,
|
||||
};
|
||||
this._durableState = normalizeLayerState({
|
||||
...this._durableState,
|
||||
options: {
|
||||
...this._durableState.options,
|
||||
[selected.optionOwner]: ownerOptions,
|
||||
},
|
||||
});
|
||||
this.dataManager.setLayerParams?.(
|
||||
selected.layerId,
|
||||
{ [selected.optionKey]: null },
|
||||
{ origin: LAYER_RESTORE_ORIGINS.share },
|
||||
);
|
||||
this.shareLinkManager?.onLayerStateChange?.();
|
||||
this._notifyDurableState();
|
||||
return true;
|
||||
}
|
||||
|
||||
/**
|
||||
* `atMs` is the moment the subject was first found ABSENT, not the moment the
|
||||
* verdict is delivered. Waiting out the pending window must not by itself
|
||||
* push a fresh link into the "expired" wording — that word describes the
|
||||
* SHARE's age, not how long this client watched for the subject.
|
||||
*/
|
||||
_classifyMissingTrackingTarget(selected, atMs = this.now()) {
|
||||
const copiedAt = this._shareCreatedAtMs;
|
||||
if (!Number.isFinite(copiedAt)) return 'unavailable';
|
||||
const ageMs = atMs - copiedAt;
|
||||
return ageMs > selected.expiryWindowMs ? 'expired' : 'unavailable';
|
||||
}
|
||||
|
||||
/** Whether the owning layer currently follows the shared subject. */
|
||||
_trackingTargetLatched(selected) {
|
||||
const params = this.dataManager.getLayerParams?.(selected.layerId);
|
||||
const active = params?.[selected.optionKey];
|
||||
return (
|
||||
active !== null &&
|
||||
active !== undefined &&
|
||||
String(active) === String(selected.targetId)
|
||||
);
|
||||
}
|
||||
|
||||
/** Stop watching a pending shared subject without deciding its fate. */
|
||||
_cancelPendingTrackingWatch() {
|
||||
if (this._pendingTrackingTimer !== null)
|
||||
this.clearTimer(this._pendingTrackingTimer);
|
||||
this._pendingTrackingTimer = null;
|
||||
const pending = this._pendingTrackingContext;
|
||||
if (pending?.signal && pending.abortHandler) {
|
||||
pending.signal.removeEventListener('abort', pending.abortHandler);
|
||||
pending.abortHandler = null;
|
||||
}
|
||||
}
|
||||
|
||||
/** Publish a share-follow lifecycle update without allowing UI errors to own state. */
|
||||
_publishTrackingRestoreStatus(status) {
|
||||
try {
|
||||
this.onTrackingRestoreStatus?.(status);
|
||||
} catch {
|
||||
/* status UI is best effort */
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Revoke a pending shared Follow wholesale.
|
||||
*
|
||||
* The watch and the LAYER's own deferred-restore latch are two halves of one
|
||||
* mechanism, so they must die together. Aborting only the restore controller
|
||||
* left the timer alive: the controller has already settled by the time the
|
||||
* watch exists, so the abort was a no-op and the orphaned timer went on to
|
||||
* announce "Shared … unavailable" for a subject whose latch had been
|
||||
* cancelled — a notice about work no longer being attempted.
|
||||
*/
|
||||
_revokePendingTrackingWatch(reason) {
|
||||
this._trackingRestoreController?.abort(reason);
|
||||
this._trackingRestoreGeneration += 1;
|
||||
this._cancelPendingTrackingWatch();
|
||||
const pending = this._pendingTrackingContext;
|
||||
this._pendingTrackingContext = null;
|
||||
if (pending) {
|
||||
this.dataManager.cancelPendingLayerRestore?.(pending.selected.layerId, {
|
||||
origin: LAYER_RESTORE_ORIGINS.share,
|
||||
reason: String(reason || 'cancelled'),
|
||||
});
|
||||
this._publishTrackingRestoreStatus({
|
||||
...pending.probe,
|
||||
...pending.selected,
|
||||
status: 'cancelled',
|
||||
classification: 'cancelled',
|
||||
reason: String(reason || 'cancelled'),
|
||||
cleared: false,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Hold a not-yet-arrived shared subject PENDING instead of declaring it gone.
|
||||
*
|
||||
* A recipient's first authoritative refresh routinely lands without a given
|
||||
* contact — the feed is polled, coverage is partial, and rendering trails the
|
||||
* snapshot by a poll. Reload-from-local already survives this: its restore
|
||||
* arms the layer's own deferred-restore latch, which re-attempts on every
|
||||
* later poll. The shared path used to decide on that single refresh, clear
|
||||
* the subject from durable state AND from the URL, then post a failure notice
|
||||
* seconds into startup — so the same link healed on reload but never on the
|
||||
* share.
|
||||
*
|
||||
* Arm the SAME latch, then watch it in the background: the caller is never
|
||||
* blocked (startup must not wait out a 90 s window before it may write the
|
||||
* URL again), and the terminal verdict is deferred until the source-specific
|
||||
* window has genuinely expired. The existing wordings are unchanged.
|
||||
*/
|
||||
async _beginPendingTrackingRestore(selected, probe, signal = null) {
|
||||
const generation = this._trackingRestoreGeneration;
|
||||
const absentAtMs = this.now();
|
||||
// Arm the layer's deferred-restore latch under the passive share origin, so
|
||||
// it re-attempts each poll and never rewrites recipient preferences.
|
||||
let armed = false;
|
||||
let armError = null;
|
||||
try {
|
||||
armed =
|
||||
(await this.dataManager.setLayerParams?.(
|
||||
selected.layerId,
|
||||
{ [selected.optionKey]: selected.targetId },
|
||||
{ origin: LAYER_RESTORE_ORIGINS.share },
|
||||
)) === true;
|
||||
} catch (error) {
|
||||
armError = error;
|
||||
}
|
||||
if (
|
||||
this._destroyed ||
|
||||
generation !== this._trackingRestoreGeneration ||
|
||||
signal?.aborted
|
||||
) {
|
||||
if (armed) {
|
||||
this.dataManager.cancelPendingLayerRestore?.(selected.layerId, {
|
||||
origin: LAYER_RESTORE_ORIGINS.share,
|
||||
reason: String(signal?.reason || 'superseded'),
|
||||
});
|
||||
}
|
||||
return {
|
||||
...probe,
|
||||
...selected,
|
||||
status: 'cancelled',
|
||||
classification: 'cancelled',
|
||||
reason: String(signal?.reason || 'superseded'),
|
||||
cleared: false,
|
||||
};
|
||||
}
|
||||
if (!armed) {
|
||||
const terminal = {
|
||||
...probe,
|
||||
...selected,
|
||||
status: 'source-unavailable',
|
||||
classification: 'source-unavailable',
|
||||
reason: String(
|
||||
armError?.message || armError || 'tracking restore latch rejected',
|
||||
),
|
||||
cleared: this._passivelyClearTrackingSelection(selected),
|
||||
};
|
||||
this._publishTrackingRestoreStatus(terminal);
|
||||
return terminal;
|
||||
}
|
||||
const deadline = this.now() + selected.expiryWindowMs;
|
||||
const settle = (terminal) => {
|
||||
if (this._pendingTrackingContext?.generation !== generation) return;
|
||||
this._cancelPendingTrackingWatch();
|
||||
this._pendingTrackingContext = null;
|
||||
this._publishTrackingRestoreStatus(terminal);
|
||||
};
|
||||
const poll = () => {
|
||||
this._pendingTrackingTimer = null;
|
||||
if (this._destroyed || generation !== this._trackingRestoreGeneration)
|
||||
return;
|
||||
// The layer that owns the latch may have gone away since the last tick
|
||||
// (disable, teardown, replacement). There is nothing left attempting this
|
||||
// restore, so abandon it silently rather than announcing a verdict.
|
||||
if (this.dataManager.isEffectivelyEnabled?.(selected.layerId) === false) {
|
||||
this._revokePendingTrackingWatch('owner-layer-disabled');
|
||||
return;
|
||||
}
|
||||
if (this._trackingTargetLatched(selected)) {
|
||||
settle({
|
||||
...probe,
|
||||
...selected,
|
||||
status: 'found',
|
||||
classification: 'followed',
|
||||
cleared: false,
|
||||
});
|
||||
return;
|
||||
}
|
||||
if (this.now() >= deadline) {
|
||||
// The window really has elapsed — only now does the verdict apply.
|
||||
const classification =
|
||||
probe.status === 'missing'
|
||||
? this._classifyMissingTrackingTarget(selected, absentAtMs)
|
||||
: 'source-unavailable';
|
||||
const cleared = this._passivelyClearTrackingSelection(selected);
|
||||
settle({ ...probe, ...selected, classification, cleared });
|
||||
return;
|
||||
}
|
||||
this._pendingTrackingTimer = this.setTimer(
|
||||
poll,
|
||||
PENDING_TRACKING_POLL_MS,
|
||||
);
|
||||
this._pendingTrackingTimer?.unref?.();
|
||||
};
|
||||
this._cancelPendingTrackingWatch();
|
||||
const pending = {
|
||||
...probe,
|
||||
...selected,
|
||||
status: 'pending',
|
||||
classification: 'pending',
|
||||
cleared: false,
|
||||
};
|
||||
const pendingContext = {
|
||||
generation,
|
||||
selected,
|
||||
probe,
|
||||
signal,
|
||||
abortHandler: null,
|
||||
};
|
||||
if (signal) {
|
||||
pendingContext.abortHandler = () => {
|
||||
if (this._pendingTrackingContext?.generation !== generation) return;
|
||||
this._revokePendingTrackingWatch(signal.reason || 'aborted');
|
||||
};
|
||||
signal.addEventListener('abort', pendingContext.abortHandler, {
|
||||
once: true,
|
||||
});
|
||||
}
|
||||
this._pendingTrackingContext = pendingContext;
|
||||
this._publishTrackingRestoreStatus(pending);
|
||||
this._pendingTrackingTimer = this.setTimer(poll, PENDING_TRACKING_POLL_MS);
|
||||
this._pendingTrackingTimer?.unref?.();
|
||||
return pending;
|
||||
}
|
||||
|
||||
/**
|
||||
* Refresh and restore the one shareable tracked target after destination
|
||||
* camera and ordinary layer restoration have settled.
|
||||
*/
|
||||
async restoreShareTrackingSelection({ signal = null } = {}) {
|
||||
const selected = this._selectedShareTrackingTarget();
|
||||
if (!selected || this._destroyed)
|
||||
return { status: 'skipped', reason: 'no-shared-target' };
|
||||
this._revokePendingTrackingWatch('superseded-by-newer-restore');
|
||||
const controller = new AbortController();
|
||||
const combinedSignal = signal
|
||||
? AbortSignal.any([signal, controller.signal])
|
||||
: controller.signal;
|
||||
this._trackingRestoreController = controller;
|
||||
const generation = ++this._trackingRestoreGeneration;
|
||||
let result;
|
||||
try {
|
||||
result = await this.dataManager.resolveLayerTrackingTarget(
|
||||
selected.layerId,
|
||||
selected.targetId,
|
||||
{ signal: combinedSignal, origin: LAYER_RESTORE_ORIGINS.share },
|
||||
);
|
||||
} catch (error) {
|
||||
result = combinedSignal.aborted
|
||||
? {
|
||||
status: 'cancelled',
|
||||
reason: String(combinedSignal.reason || 'aborted'),
|
||||
}
|
||||
: {
|
||||
status: 'source-unavailable',
|
||||
reason: String(error?.message || error),
|
||||
};
|
||||
}
|
||||
if (
|
||||
generation !== this._trackingRestoreGeneration ||
|
||||
this._destroyed ||
|
||||
combinedSignal.aborted
|
||||
) {
|
||||
if (this._trackingRestoreController === controller)
|
||||
this._trackingRestoreController = null;
|
||||
return {
|
||||
...result,
|
||||
status: 'cancelled',
|
||||
reason: String(combinedSignal.reason || 'superseded'),
|
||||
};
|
||||
}
|
||||
if (this._trackingRestoreController === controller)
|
||||
this._trackingRestoreController = null;
|
||||
|
||||
if (result.status === 'found') {
|
||||
const terminal = {
|
||||
...result,
|
||||
...selected,
|
||||
classification: 'followed',
|
||||
cleared: false,
|
||||
};
|
||||
this._publishTrackingRestoreStatus(terminal);
|
||||
return terminal;
|
||||
}
|
||||
if (['cancelled', 'superseded', 'destroyed'].includes(result.status))
|
||||
return result;
|
||||
|
||||
// A subject that is simply not here YET is not a subject that is gone. Hold
|
||||
// it on the layer's own deferred-restore latch for its source-specific
|
||||
// window before any verdict is reached or shown. `unsupported` layers have
|
||||
// no latch to arm, so they still decide immediately.
|
||||
if (result.status === 'missing' || result.status === 'source-unavailable') {
|
||||
return this._beginPendingTrackingRestore(
|
||||
selected,
|
||||
result,
|
||||
combinedSignal,
|
||||
);
|
||||
}
|
||||
|
||||
const classification =
|
||||
result.status === 'missing'
|
||||
? this._classifyMissingTrackingTarget(selected)
|
||||
: 'source-unavailable';
|
||||
const cleared = this._passivelyClearTrackingSelection(selected);
|
||||
const terminal = { ...result, ...selected, classification, cleared };
|
||||
this._publishTrackingRestoreStatus(terminal);
|
||||
return terminal;
|
||||
}
|
||||
|
||||
destroy() {
|
||||
if (this._destroyed) return;
|
||||
this._destroyed = true;
|
||||
for (const controller of this._restoreControllers.values())
|
||||
controller.abort('coordinator-destroyed');
|
||||
this._restoreControllers.clear();
|
||||
this._revokePendingTrackingWatch('coordinator-destroyed');
|
||||
this._trackingRestoreController = null;
|
||||
this._unsubscribe?.();
|
||||
this._unsubscribe = null;
|
||||
this._unsubscribeVisibilityRequests?.();
|
||||
this._unsubscribeVisibilityRequests = null;
|
||||
this.shareLinkManager?.setLayerStateProvider?.(null);
|
||||
this.onDurableStateChange = null;
|
||||
this.onTrackingRestoreStatus = null;
|
||||
}
|
||||
}
|
||||
@@ -11,6 +11,7 @@ import {
|
||||
LAYER_STATE_STORAGE_KEY,
|
||||
LAYER_STATE_TOKEN_ALPHABET,
|
||||
LAYER_STATE_TOKEN_RESERVATIONS,
|
||||
LayerStateCoordinator,
|
||||
REGISTERED_LAYER_IDS,
|
||||
SHARE_TRACKING_RESTORE_POLICIES,
|
||||
createDefaultLayerState,
|
||||
@@ -23,7 +24,6 @@ import {
|
||||
validateLayerStateAllocations,
|
||||
validateLayerStateRegistry,
|
||||
} from './layerState.js';
|
||||
import { LayerStateCoordinator } from './layerStateCoordinator.js';
|
||||
import radioLayer from './radio.js';
|
||||
import { stampInitialShareGesture } from '../navigationPolicy.js';
|
||||
|
||||
|
||||
@@ -1,742 +0,0 @@
|
||||
/**
|
||||
* Applies layer state to the running application: restores shared and stored
|
||||
* layer choices and persists changes to browser storage.
|
||||
*/
|
||||
|
||||
import {
|
||||
LAYER_RESTORE_ORIGINS,
|
||||
LAYER_STATE_REGISTRY,
|
||||
LAYER_STATE_STORAGE_KEY,
|
||||
REGISTRY_BY_ID,
|
||||
SHARE_TRACKING_RESTORE_POLICIES,
|
||||
cloneLayerState,
|
||||
createDefaultLayerState,
|
||||
isExplicitLayerStateOrigin,
|
||||
layerOptionsForRestore,
|
||||
normalizeLayerState,
|
||||
optionSpecs,
|
||||
parseStoredLayerState,
|
||||
serializeStoredLayerState,
|
||||
} from './layerState.js';
|
||||
|
||||
/** Re-check cadence while a shared subject waits for its feed row to arrive. */
|
||||
const PENDING_TRACKING_POLL_MS = 1_000;
|
||||
|
||||
const TRACKING_OPTION_KEY_BY_LAYER = Object.freeze({
|
||||
flights: 'selectedFlightsTrackingId',
|
||||
military: 'selectedMilitaryTrackingId',
|
||||
satellites: 'selectedSatTrackingId',
|
||||
});
|
||||
|
||||
function safeStorage() {
|
||||
try {
|
||||
return globalThis.localStorage || null;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
function currentLayerOutcome(dataManager, layerId) {
|
||||
const state = dataManager.getLayerLifecycleState?.(layerId);
|
||||
return {
|
||||
settledEnabled: Boolean(state?.enabled),
|
||||
lifecycleState: state?.lifecycleState || 'missing',
|
||||
lifecycleUncertain: Boolean(state?.uncertain),
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Owns durable user layer preferences independently from transient runtime
|
||||
* choreography, and coordinates passive post-registration restoration.
|
||||
*/
|
||||
export class LayerStateCoordinator {
|
||||
constructor(
|
||||
dataManager,
|
||||
shareLinkManager,
|
||||
{
|
||||
storage = safeStorage(),
|
||||
restoreGate = null,
|
||||
onDurableStateChange = null,
|
||||
onTrackingRestoreStatus = null,
|
||||
now = () => Date.now(),
|
||||
// Injectable so the pending-window behavior is deterministically testable
|
||||
// without sleeping out a 90 s expiry.
|
||||
setTimer = (fn, ms) => setTimeout(fn, ms),
|
||||
clearTimer = (handle) => clearTimeout(handle),
|
||||
} = {},
|
||||
) {
|
||||
if (!dataManager?.registrationsFinalized) {
|
||||
throw new Error(
|
||||
'Layer state requires finalized data-layer registrations',
|
||||
);
|
||||
}
|
||||
this.dataManager = dataManager;
|
||||
this.shareLinkManager = shareLinkManager || null;
|
||||
this.storage = storage;
|
||||
this.restoreGate = restoreGate;
|
||||
this.onDurableStateChange = onDurableStateChange;
|
||||
this.onTrackingRestoreStatus = onTrackingRestoreStatus;
|
||||
this.now = now;
|
||||
this.setTimer = setTimer;
|
||||
this.clearTimer = clearTimer;
|
||||
this._durableState = createDefaultLayerState();
|
||||
this._source = 'defaults';
|
||||
this._destroyed = false;
|
||||
this._restoreControllers = new Map();
|
||||
this._shareCreatedAtMs = null;
|
||||
this._trackingRestoreController = null;
|
||||
this._trackingRestoreGeneration = 0;
|
||||
this._pendingTrackingTimer = null;
|
||||
this._pendingTrackingContext = null;
|
||||
this._unsubscribe = this.dataManager.subscribe((change) =>
|
||||
this._handleManagerChange(change),
|
||||
);
|
||||
this._unsubscribeVisibilityRequests =
|
||||
this.dataManager.subscribeVisibilityRequests((change) =>
|
||||
this._handleVisibilityRequest(change),
|
||||
);
|
||||
this.restorePromise = Promise.resolve([]);
|
||||
this.lastRestoreResults = [];
|
||||
}
|
||||
|
||||
start({
|
||||
shareLayerState = null,
|
||||
allowLocalState = true,
|
||||
shareCreatedAtMs = null,
|
||||
} = {}) {
|
||||
if (this._destroyed)
|
||||
throw new Error('Layer-state coordinator is destroyed');
|
||||
let selected = shareLayerState
|
||||
? normalizeLayerState(shareLayerState)
|
||||
: null;
|
||||
if (selected) {
|
||||
this._source = 'share';
|
||||
this._shareCreatedAtMs = Number.isFinite(shareCreatedAtMs)
|
||||
? shareCreatedAtMs
|
||||
: null;
|
||||
} else if (allowLocalState) {
|
||||
let stored = null;
|
||||
try {
|
||||
stored = parseStoredLayerState(
|
||||
this.storage?.getItem?.(LAYER_STATE_STORAGE_KEY),
|
||||
);
|
||||
} catch {
|
||||
/* best effort */
|
||||
}
|
||||
if (stored) {
|
||||
selected = stored;
|
||||
this._source = 'local';
|
||||
}
|
||||
} else {
|
||||
// A valid historical camera/style share with no v2 layer payload keeps
|
||||
// the exact legacy default-layer behavior. It must not inherit an
|
||||
// unrelated recipient's saved local layer preferences.
|
||||
this._source = 'legacy-share';
|
||||
}
|
||||
this._durableState = selected || createDefaultLayerState();
|
||||
this.shareLinkManager?.setLayerStateProvider?.(() =>
|
||||
this.getDurableState(),
|
||||
);
|
||||
this.shareLinkManager?.onLayerStateChange?.();
|
||||
this._notifyDurableState();
|
||||
if (!selected) return this.restorePromise;
|
||||
this.restorePromise = this._restoreSelectedState(
|
||||
this._source === 'share'
|
||||
? LAYER_RESTORE_ORIGINS.share
|
||||
: LAYER_RESTORE_ORIGINS.local,
|
||||
);
|
||||
return this.restorePromise;
|
||||
}
|
||||
|
||||
get source() {
|
||||
return this._source;
|
||||
}
|
||||
|
||||
getDurableState() {
|
||||
return cloneLayerState(this._durableState);
|
||||
}
|
||||
|
||||
_notifyDurableState() {
|
||||
try {
|
||||
this.onDurableStateChange?.(this.getDurableState());
|
||||
} catch {
|
||||
/* UI sync is best effort */
|
||||
}
|
||||
}
|
||||
|
||||
_handleVisibilityRequest(change) {
|
||||
if (!isExplicitLayerStateOrigin(change?.origin)) return;
|
||||
this._restoreControllers
|
||||
.get(change.layerId)
|
||||
?.abort('superseded-by-explicit-visibility');
|
||||
if (SHARE_TRACKING_RESTORE_POLICIES[change.layerId]) {
|
||||
this._revokePendingTrackingWatch('superseded-by-explicit-visibility');
|
||||
}
|
||||
}
|
||||
|
||||
/** Revoke every passive restore before explicit navigation can be reclaimed. */
|
||||
cancelPendingRestores(reason = 'superseded-by-explicit-navigation') {
|
||||
for (const controller of this._restoreControllers.values())
|
||||
controller.abort(reason);
|
||||
this._revokePendingTrackingWatch(reason);
|
||||
}
|
||||
|
||||
/**
|
||||
* Revoke a pending shared Follow. Physical navigation may also clear only
|
||||
* the exact passive selection, without writing recipient preferences.
|
||||
*/
|
||||
cancelPendingShareTracking(
|
||||
reason = 'superseded-by-explicit-navigation',
|
||||
{ clearSelection = false } = {},
|
||||
) {
|
||||
this._revokePendingTrackingWatch(reason);
|
||||
if (!clearSelection) return false;
|
||||
const selected = this._selectedShareTrackingTarget();
|
||||
return selected ? this._passivelyClearTrackingSelection(selected) : false;
|
||||
}
|
||||
|
||||
_handleManagerChange(change) {
|
||||
if (!change || this._destroyed) return;
|
||||
// Parameter and visibility ownership are independent. A newer explicit
|
||||
// option request may replace passive share options, but it must not abort
|
||||
// the same layer's visibility lifecycle.
|
||||
if (change.type === 'params-requested') {
|
||||
if (
|
||||
isExplicitLayerStateOrigin(change.origin) &&
|
||||
SHARE_TRACKING_RESTORE_POLICIES[change.layerId]
|
||||
) {
|
||||
this._revokePendingTrackingWatch('superseded-by-explicit-params');
|
||||
}
|
||||
return;
|
||||
}
|
||||
// A layer that goes away takes its latch with it, at ANY origin — a
|
||||
// programmatic disable or teardown never reaches the explicit-intent path
|
||||
// below, so revoke here before that early return.
|
||||
if (
|
||||
change.type === 'visibility' &&
|
||||
change.enabled === false &&
|
||||
SHARE_TRACKING_RESTORE_POLICIES[change.layerId]
|
||||
) {
|
||||
this._revokePendingTrackingWatch('owner-layer-disabled');
|
||||
}
|
||||
if (!isExplicitLayerStateOrigin(change.origin)) return;
|
||||
if (change.type === 'visibility') {
|
||||
this._restoreControllers
|
||||
.get(change.layerId)
|
||||
?.abort('superseded-by-explicit-visibility');
|
||||
if (SHARE_TRACKING_RESTORE_POLICIES[change.layerId]) {
|
||||
this._revokePendingTrackingWatch('superseded-by-explicit-visibility');
|
||||
}
|
||||
const enabled = new Set(this._durableState.enabledLayerIds);
|
||||
if (change.enabled) enabled.add(change.layerId);
|
||||
else enabled.delete(change.layerId);
|
||||
this._commitExplicit({
|
||||
...this._durableState,
|
||||
enabledLayerIds: [...enabled],
|
||||
});
|
||||
return;
|
||||
}
|
||||
if (change.type !== 'params') return;
|
||||
const entry = REGISTRY_BY_ID.get(change.layerId);
|
||||
if (!entry?.optionOwner) return;
|
||||
const ownerId = entry.optionOwner;
|
||||
const nextOwnerOptions = { ...this._durableState.options[ownerId] };
|
||||
const requestedParams = change.requestedParams || {};
|
||||
const trackingOptionKey =
|
||||
TRACKING_OPTION_KEY_BY_LAYER[change.layerId] || null;
|
||||
let changed = false;
|
||||
for (const spec of optionSpecs(ownerId)) {
|
||||
// Only persist keys present in this explicit request. getLayerParams()
|
||||
// can return a wider live snapshot containing transient or passively
|
||||
// changed values that this user action did not own. Tracking is the one
|
||||
// exception: an unrelated explicit option cancels a pending restoration
|
||||
// in that family, so its wider live value (active ID or null) must replace
|
||||
// the formerly durable pending ID instead of allowing reload resurrection.
|
||||
const explicitlyRequested = Object.hasOwn(requestedParams, spec.key);
|
||||
const implicitTrackingSync =
|
||||
!explicitlyRequested && spec.key === trackingOptionKey;
|
||||
if (!explicitlyRequested && !implicitTrackingSync) continue;
|
||||
const value = spec.normalize(change.params[spec.key]);
|
||||
if (value === null) {
|
||||
if (implicitTrackingSync) {
|
||||
nextOwnerOptions[spec.key] = null;
|
||||
changed = true;
|
||||
continue;
|
||||
}
|
||||
if (spec.defaultValue !== null || change.params[spec.key] !== null)
|
||||
continue;
|
||||
}
|
||||
nextOwnerOptions[spec.key] = value;
|
||||
changed = true;
|
||||
}
|
||||
if (!changed) return;
|
||||
this._commitExplicit({
|
||||
...this._durableState,
|
||||
options: { ...this._durableState.options, [ownerId]: nextOwnerOptions },
|
||||
});
|
||||
}
|
||||
|
||||
_commitExplicit(candidate) {
|
||||
this._durableState = normalizeLayerState(candidate);
|
||||
const serialized = serializeStoredLayerState(this._durableState);
|
||||
try {
|
||||
if (this.storage?.getItem?.(LAYER_STATE_STORAGE_KEY) !== serialized) {
|
||||
this.storage?.setItem?.(LAYER_STATE_STORAGE_KEY, serialized);
|
||||
}
|
||||
} catch {
|
||||
/* storage can be unavailable or quota-limited */
|
||||
}
|
||||
this.shareLinkManager?.onLayerStateChange?.();
|
||||
this._notifyDurableState();
|
||||
}
|
||||
|
||||
async _waitForRestoreGate() {
|
||||
if (!this.restoreGate) return;
|
||||
await (typeof this.restoreGate === 'function'
|
||||
? this.restoreGate()
|
||||
: this.restoreGate);
|
||||
}
|
||||
|
||||
async _restoreSelectedState(origin) {
|
||||
for (const entry of LAYER_STATE_REGISTRY) {
|
||||
this._restoreControllers.set(entry.id, new AbortController());
|
||||
}
|
||||
try {
|
||||
await this._waitForRestoreGate();
|
||||
const enabled = new Set(this._durableState.enabledLayerIds);
|
||||
const settled = await Promise.allSettled(
|
||||
LAYER_STATE_REGISTRY.map(async (entry) => {
|
||||
const controller = this._restoreControllers.get(entry.id);
|
||||
const targetEnabled = enabled.has(entry.id);
|
||||
const options = layerOptionsForRestore(this._durableState, entry.id);
|
||||
if (origin === LAYER_RESTORE_ORIGINS.share && options) {
|
||||
for (const trackingKey of Object.values(
|
||||
TRACKING_OPTION_KEY_BY_LAYER,
|
||||
)) {
|
||||
delete options[trackingKey];
|
||||
}
|
||||
}
|
||||
if (this._destroyed || controller?.signal.aborted) {
|
||||
return {
|
||||
layerId: entry.id,
|
||||
targetEnabled,
|
||||
origin,
|
||||
phase: 'reserved',
|
||||
...currentLayerOutcome(this.dataManager, entry.id),
|
||||
appliedOptions: {},
|
||||
cancellationReason: this._destroyed ? 'destroyed' : 'superseded',
|
||||
errorClass: 'cancelled',
|
||||
persistenceWrite: false,
|
||||
succeeded: false,
|
||||
};
|
||||
}
|
||||
// Reserve passive option state before any asynchronous lifecycle work.
|
||||
// A later explicit params intent then wins on its own lane without
|
||||
// cancelling or being overwritten by the visibility restore.
|
||||
const paramsSucceeded =
|
||||
!options ||
|
||||
Object.keys(options).length === 0 ||
|
||||
this.dataManager.setLayerParams(entry.id, options, { origin });
|
||||
return this.dataManager
|
||||
.restoreLayerState(
|
||||
entry.id,
|
||||
{
|
||||
enabled: targetEnabled,
|
||||
params: null,
|
||||
},
|
||||
{ origin, signal: controller.signal },
|
||||
)
|
||||
.then((result) => ({
|
||||
...result,
|
||||
appliedOptions: paramsSucceeded && options ? options : {},
|
||||
errorClass: paramsSucceeded
|
||||
? result.errorClass
|
||||
: 'ParamsRejected',
|
||||
succeeded: paramsSucceeded && result.succeeded,
|
||||
}));
|
||||
}),
|
||||
);
|
||||
this.lastRestoreResults = settled.map((result, index) => {
|
||||
if (result.status === 'fulfilled') return result.value;
|
||||
const entry = LAYER_STATE_REGISTRY[index];
|
||||
return {
|
||||
layerId: entry.id,
|
||||
targetEnabled: enabled.has(entry.id),
|
||||
origin,
|
||||
phase: 'coordinator',
|
||||
...currentLayerOutcome(this.dataManager, entry.id),
|
||||
appliedOptions: {},
|
||||
cancellationReason: null,
|
||||
errorClass: result.reason?.name || 'Error',
|
||||
error: String(result.reason?.message || result.reason),
|
||||
persistenceWrite: false,
|
||||
succeeded: false,
|
||||
};
|
||||
});
|
||||
return this.lastRestoreResults.map((result) => ({ ...result }));
|
||||
} finally {
|
||||
this._restoreControllers.clear();
|
||||
this._notifyDurableState();
|
||||
}
|
||||
}
|
||||
|
||||
_selectedShareTrackingTarget() {
|
||||
if (this._source !== 'share') return null;
|
||||
for (const [layerId, policy] of Object.entries(
|
||||
SHARE_TRACKING_RESTORE_POLICIES,
|
||||
)) {
|
||||
const targetId =
|
||||
this._durableState.options?.[policy.optionOwner]?.[policy.optionKey];
|
||||
if (targetId !== null && targetId !== undefined && targetId !== '') {
|
||||
return { layerId, targetId, ...policy };
|
||||
}
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
_passivelyClearTrackingSelection(selected) {
|
||||
const current =
|
||||
this._durableState.options?.[selected.optionOwner]?.[selected.optionKey];
|
||||
if (String(current) !== String(selected.targetId)) return false;
|
||||
const ownerOptions = {
|
||||
...this._durableState.options[selected.optionOwner],
|
||||
[selected.optionKey]: null,
|
||||
};
|
||||
this._durableState = normalizeLayerState({
|
||||
...this._durableState,
|
||||
options: {
|
||||
...this._durableState.options,
|
||||
[selected.optionOwner]: ownerOptions,
|
||||
},
|
||||
});
|
||||
this.dataManager.setLayerParams?.(
|
||||
selected.layerId,
|
||||
{ [selected.optionKey]: null },
|
||||
{ origin: LAYER_RESTORE_ORIGINS.share },
|
||||
);
|
||||
this.shareLinkManager?.onLayerStateChange?.();
|
||||
this._notifyDurableState();
|
||||
return true;
|
||||
}
|
||||
|
||||
/**
|
||||
* `atMs` is the moment the subject was first found ABSENT, not the moment the
|
||||
* verdict is delivered. Waiting out the pending window must not by itself
|
||||
* push a fresh link into the "expired" wording — that word describes the
|
||||
* SHARE's age, not how long this client watched for the subject.
|
||||
*/
|
||||
_classifyMissingTrackingTarget(selected, atMs = this.now()) {
|
||||
const copiedAt = this._shareCreatedAtMs;
|
||||
if (!Number.isFinite(copiedAt)) return 'unavailable';
|
||||
const ageMs = atMs - copiedAt;
|
||||
return ageMs > selected.expiryWindowMs ? 'expired' : 'unavailable';
|
||||
}
|
||||
|
||||
/** Whether the owning layer currently follows the shared subject. */
|
||||
_trackingTargetLatched(selected) {
|
||||
const params = this.dataManager.getLayerParams?.(selected.layerId);
|
||||
const active = params?.[selected.optionKey];
|
||||
return (
|
||||
active !== null &&
|
||||
active !== undefined &&
|
||||
String(active) === String(selected.targetId)
|
||||
);
|
||||
}
|
||||
|
||||
/** Stop watching a pending shared subject without deciding its fate. */
|
||||
_cancelPendingTrackingWatch() {
|
||||
if (this._pendingTrackingTimer !== null)
|
||||
this.clearTimer(this._pendingTrackingTimer);
|
||||
this._pendingTrackingTimer = null;
|
||||
const pending = this._pendingTrackingContext;
|
||||
if (pending?.signal && pending.abortHandler) {
|
||||
pending.signal.removeEventListener('abort', pending.abortHandler);
|
||||
pending.abortHandler = null;
|
||||
}
|
||||
}
|
||||
|
||||
/** Publish a share-follow lifecycle update without allowing UI errors to own state. */
|
||||
_publishTrackingRestoreStatus(status) {
|
||||
try {
|
||||
this.onTrackingRestoreStatus?.(status);
|
||||
} catch {
|
||||
/* status UI is best effort */
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Revoke a pending shared Follow wholesale.
|
||||
*
|
||||
* The watch and the LAYER's own deferred-restore latch are two halves of one
|
||||
* mechanism, so they must die together. Aborting only the restore controller
|
||||
* left the timer alive: the controller has already settled by the time the
|
||||
* watch exists, so the abort was a no-op and the orphaned timer went on to
|
||||
* announce "Shared … unavailable" for a subject whose latch had been
|
||||
* cancelled — a notice about work no longer being attempted.
|
||||
*/
|
||||
_revokePendingTrackingWatch(reason) {
|
||||
this._trackingRestoreController?.abort(reason);
|
||||
this._trackingRestoreGeneration += 1;
|
||||
this._cancelPendingTrackingWatch();
|
||||
const pending = this._pendingTrackingContext;
|
||||
this._pendingTrackingContext = null;
|
||||
if (pending) {
|
||||
this.dataManager.cancelPendingLayerRestore?.(pending.selected.layerId, {
|
||||
origin: LAYER_RESTORE_ORIGINS.share,
|
||||
reason: String(reason || 'cancelled'),
|
||||
});
|
||||
this._publishTrackingRestoreStatus({
|
||||
...pending.probe,
|
||||
...pending.selected,
|
||||
status: 'cancelled',
|
||||
classification: 'cancelled',
|
||||
reason: String(reason || 'cancelled'),
|
||||
cleared: false,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Hold a not-yet-arrived shared subject PENDING instead of declaring it gone.
|
||||
*
|
||||
* A recipient's first authoritative refresh routinely lands without a given
|
||||
* contact — the feed is polled, coverage is partial, and rendering trails the
|
||||
* snapshot by a poll. Reload-from-local already survives this: its restore
|
||||
* arms the layer's own deferred-restore latch, which re-attempts on every
|
||||
* later poll. The shared path used to decide on that single refresh, clear
|
||||
* the subject from durable state AND from the URL, then post a failure notice
|
||||
* seconds into startup — so the same link healed on reload but never on the
|
||||
* share.
|
||||
*
|
||||
* Arm the SAME latch, then watch it in the background: the caller is never
|
||||
* blocked (startup must not wait out a 90 s window before it may write the
|
||||
* URL again), and the terminal verdict is deferred until the source-specific
|
||||
* window has genuinely expired. The existing wordings are unchanged.
|
||||
*/
|
||||
async _beginPendingTrackingRestore(selected, probe, signal = null) {
|
||||
const generation = this._trackingRestoreGeneration;
|
||||
const absentAtMs = this.now();
|
||||
// Arm the layer's deferred-restore latch under the passive share origin, so
|
||||
// it re-attempts each poll and never rewrites recipient preferences.
|
||||
let armed = false;
|
||||
let armError = null;
|
||||
try {
|
||||
armed =
|
||||
(await this.dataManager.setLayerParams?.(
|
||||
selected.layerId,
|
||||
{ [selected.optionKey]: selected.targetId },
|
||||
{ origin: LAYER_RESTORE_ORIGINS.share },
|
||||
)) === true;
|
||||
} catch (error) {
|
||||
armError = error;
|
||||
}
|
||||
if (
|
||||
this._destroyed ||
|
||||
generation !== this._trackingRestoreGeneration ||
|
||||
signal?.aborted
|
||||
) {
|
||||
if (armed) {
|
||||
this.dataManager.cancelPendingLayerRestore?.(selected.layerId, {
|
||||
origin: LAYER_RESTORE_ORIGINS.share,
|
||||
reason: String(signal?.reason || 'superseded'),
|
||||
});
|
||||
}
|
||||
return {
|
||||
...probe,
|
||||
...selected,
|
||||
status: 'cancelled',
|
||||
classification: 'cancelled',
|
||||
reason: String(signal?.reason || 'superseded'),
|
||||
cleared: false,
|
||||
};
|
||||
}
|
||||
if (!armed) {
|
||||
const terminal = {
|
||||
...probe,
|
||||
...selected,
|
||||
status: 'source-unavailable',
|
||||
classification: 'source-unavailable',
|
||||
reason: String(
|
||||
armError?.message || armError || 'tracking restore latch rejected',
|
||||
),
|
||||
cleared: this._passivelyClearTrackingSelection(selected),
|
||||
};
|
||||
this._publishTrackingRestoreStatus(terminal);
|
||||
return terminal;
|
||||
}
|
||||
const deadline = this.now() + selected.expiryWindowMs;
|
||||
const settle = (terminal) => {
|
||||
if (this._pendingTrackingContext?.generation !== generation) return;
|
||||
this._cancelPendingTrackingWatch();
|
||||
this._pendingTrackingContext = null;
|
||||
this._publishTrackingRestoreStatus(terminal);
|
||||
};
|
||||
const poll = () => {
|
||||
this._pendingTrackingTimer = null;
|
||||
if (this._destroyed || generation !== this._trackingRestoreGeneration)
|
||||
return;
|
||||
// The layer that owns the latch may have gone away since the last tick
|
||||
// (disable, teardown, replacement). There is nothing left attempting this
|
||||
// restore, so abandon it silently rather than announcing a verdict.
|
||||
if (this.dataManager.isEffectivelyEnabled?.(selected.layerId) === false) {
|
||||
this._revokePendingTrackingWatch('owner-layer-disabled');
|
||||
return;
|
||||
}
|
||||
if (this._trackingTargetLatched(selected)) {
|
||||
settle({
|
||||
...probe,
|
||||
...selected,
|
||||
status: 'found',
|
||||
classification: 'followed',
|
||||
cleared: false,
|
||||
});
|
||||
return;
|
||||
}
|
||||
if (this.now() >= deadline) {
|
||||
// The window really has elapsed — only now does the verdict apply.
|
||||
const classification =
|
||||
probe.status === 'missing'
|
||||
? this._classifyMissingTrackingTarget(selected, absentAtMs)
|
||||
: 'source-unavailable';
|
||||
const cleared = this._passivelyClearTrackingSelection(selected);
|
||||
settle({ ...probe, ...selected, classification, cleared });
|
||||
return;
|
||||
}
|
||||
this._pendingTrackingTimer = this.setTimer(
|
||||
poll,
|
||||
PENDING_TRACKING_POLL_MS,
|
||||
);
|
||||
this._pendingTrackingTimer?.unref?.();
|
||||
};
|
||||
this._cancelPendingTrackingWatch();
|
||||
const pending = {
|
||||
...probe,
|
||||
...selected,
|
||||
status: 'pending',
|
||||
classification: 'pending',
|
||||
cleared: false,
|
||||
};
|
||||
const pendingContext = {
|
||||
generation,
|
||||
selected,
|
||||
probe,
|
||||
signal,
|
||||
abortHandler: null,
|
||||
};
|
||||
if (signal) {
|
||||
pendingContext.abortHandler = () => {
|
||||
if (this._pendingTrackingContext?.generation !== generation) return;
|
||||
this._revokePendingTrackingWatch(signal.reason || 'aborted');
|
||||
};
|
||||
signal.addEventListener('abort', pendingContext.abortHandler, {
|
||||
once: true,
|
||||
});
|
||||
}
|
||||
this._pendingTrackingContext = pendingContext;
|
||||
this._publishTrackingRestoreStatus(pending);
|
||||
this._pendingTrackingTimer = this.setTimer(poll, PENDING_TRACKING_POLL_MS);
|
||||
this._pendingTrackingTimer?.unref?.();
|
||||
return pending;
|
||||
}
|
||||
|
||||
/**
|
||||
* Refresh and restore the one shareable tracked target after destination
|
||||
* camera and ordinary layer restoration have settled.
|
||||
*/
|
||||
async restoreShareTrackingSelection({ signal = null } = {}) {
|
||||
const selected = this._selectedShareTrackingTarget();
|
||||
if (!selected || this._destroyed)
|
||||
return { status: 'skipped', reason: 'no-shared-target' };
|
||||
this._revokePendingTrackingWatch('superseded-by-newer-restore');
|
||||
const controller = new AbortController();
|
||||
const combinedSignal = signal
|
||||
? AbortSignal.any([signal, controller.signal])
|
||||
: controller.signal;
|
||||
this._trackingRestoreController = controller;
|
||||
const generation = ++this._trackingRestoreGeneration;
|
||||
let result;
|
||||
try {
|
||||
result = await this.dataManager.resolveLayerTrackingTarget(
|
||||
selected.layerId,
|
||||
selected.targetId,
|
||||
{ signal: combinedSignal, origin: LAYER_RESTORE_ORIGINS.share },
|
||||
);
|
||||
} catch (error) {
|
||||
result = combinedSignal.aborted
|
||||
? {
|
||||
status: 'cancelled',
|
||||
reason: String(combinedSignal.reason || 'aborted'),
|
||||
}
|
||||
: {
|
||||
status: 'source-unavailable',
|
||||
reason: String(error?.message || error),
|
||||
};
|
||||
}
|
||||
if (
|
||||
generation !== this._trackingRestoreGeneration ||
|
||||
this._destroyed ||
|
||||
combinedSignal.aborted
|
||||
) {
|
||||
if (this._trackingRestoreController === controller)
|
||||
this._trackingRestoreController = null;
|
||||
return {
|
||||
...result,
|
||||
status: 'cancelled',
|
||||
reason: String(combinedSignal.reason || 'superseded'),
|
||||
};
|
||||
}
|
||||
if (this._trackingRestoreController === controller)
|
||||
this._trackingRestoreController = null;
|
||||
|
||||
if (result.status === 'found') {
|
||||
const terminal = {
|
||||
...result,
|
||||
...selected,
|
||||
classification: 'followed',
|
||||
cleared: false,
|
||||
};
|
||||
this._publishTrackingRestoreStatus(terminal);
|
||||
return terminal;
|
||||
}
|
||||
if (['cancelled', 'superseded', 'destroyed'].includes(result.status))
|
||||
return result;
|
||||
|
||||
// A subject that is simply not here YET is not a subject that is gone. Hold
|
||||
// it on the layer's own deferred-restore latch for its source-specific
|
||||
// window before any verdict is reached or shown. `unsupported` layers have
|
||||
// no latch to arm, so they still decide immediately.
|
||||
if (result.status === 'missing' || result.status === 'source-unavailable') {
|
||||
return this._beginPendingTrackingRestore(
|
||||
selected,
|
||||
result,
|
||||
combinedSignal,
|
||||
);
|
||||
}
|
||||
|
||||
const classification =
|
||||
result.status === 'missing'
|
||||
? this._classifyMissingTrackingTarget(selected)
|
||||
: 'source-unavailable';
|
||||
const cleared = this._passivelyClearTrackingSelection(selected);
|
||||
const terminal = { ...result, ...selected, classification, cleared };
|
||||
this._publishTrackingRestoreStatus(terminal);
|
||||
return terminal;
|
||||
}
|
||||
|
||||
destroy() {
|
||||
if (this._destroyed) return;
|
||||
this._destroyed = true;
|
||||
for (const controller of this._restoreControllers.values())
|
||||
controller.abort('coordinator-destroyed');
|
||||
this._restoreControllers.clear();
|
||||
this._revokePendingTrackingWatch('coordinator-destroyed');
|
||||
this._trackingRestoreController = null;
|
||||
this._unsubscribe?.();
|
||||
this._unsubscribe = null;
|
||||
this._unsubscribeVisibilityRequests?.();
|
||||
this._unsubscribeVisibilityRequests = null;
|
||||
this.shareLinkManager?.setLayerStateProvider?.(null);
|
||||
this.onDurableStateChange = null;
|
||||
this.onTrackingRestoreStatus = null;
|
||||
}
|
||||
}
|
||||
@@ -1,9 +1,4 @@
|
||||
import * as Cesium from 'cesium';
|
||||
import {
|
||||
layerTitle,
|
||||
mapAnalystRecord,
|
||||
parseGeojsonLines,
|
||||
} from '../sources/infrastructureData.js';
|
||||
import { createInfrastructureOverlayEntry } from './infrastructureOverlayEntry.js';
|
||||
import { isPointerFree } from './inputOwnership.js';
|
||||
import {
|
||||
@@ -91,6 +86,78 @@ export function localInfrastructureOverlayCopy(properties, layerId) {
|
||||
return { title, details };
|
||||
}
|
||||
|
||||
/**
|
||||
* Map one local infrastructure feature to a JSON-safe analyst record
|
||||
* (analyst query engine seam). Pure — no Cesium types. Missing/unknown
|
||||
* fields are null, never NaN/undefined. Layer-specific fields that do not
|
||||
* apply (river/output on datacenters, capacity on dams) stay null so a
|
||||
* shared compact payload can copy them without inventing values.
|
||||
* Names are unclamped — overlay cards shorten for paint; queries need the
|
||||
* full source string ("Usina Hidrelétrica de Itaipu").
|
||||
* @param {Object|null|undefined} raw - {id, lat, lon, properties}.
|
||||
* @param {string} [layerId] Local layer id (`local-datacenters` / `local-dams`).
|
||||
* @returns {{id: string, name: string|null, lat: number|null, lon: number|null,
|
||||
* operator: string|null, capacity: string|null, river: string|null,
|
||||
* output: string|null}}
|
||||
*/
|
||||
export function mapAnalystRecord(raw, layerId = '') {
|
||||
const num = (v) => (Number.isFinite(v) ? v : null);
|
||||
const text = (v) => {
|
||||
const t = String(v ?? '').trim();
|
||||
return t && t !== 'undefined' && t !== 'null' ? t : null;
|
||||
};
|
||||
const props =
|
||||
raw?.properties &&
|
||||
typeof raw.properties === 'object' &&
|
||||
!Array.isArray(raw.properties)
|
||||
? raw.properties
|
||||
: {};
|
||||
const tags =
|
||||
props.tags && typeof props.tags === 'object' && !Array.isArray(props.tags)
|
||||
? props.tags
|
||||
: {};
|
||||
const name =
|
||||
text(props.name) ||
|
||||
text(tags.name) ||
|
||||
text(tags['name:en']) ||
|
||||
text(tags.official_name) ||
|
||||
null;
|
||||
const operator =
|
||||
text(tags.operator) ||
|
||||
text(props.operator) ||
|
||||
text(tags['operator:short']) ||
|
||||
null;
|
||||
const capacity =
|
||||
layerId === 'local-datacenters'
|
||||
? text(tags['capacity:it_load']) ||
|
||||
text(tags.it_load) ||
|
||||
text(tags.capacity) ||
|
||||
text(props.capacity)
|
||||
: null;
|
||||
const river =
|
||||
layerId === 'local-dams'
|
||||
? text(tags.associated_river) ||
|
||||
text(props.associated_river) ||
|
||||
text(tags.river) ||
|
||||
text(props.river) ||
|
||||
text(tags['river:name'])
|
||||
: null;
|
||||
const output =
|
||||
layerId === 'local-dams'
|
||||
? text(props.output) || text(tags['plant:output:electricity'])
|
||||
: null;
|
||||
return {
|
||||
id: name || text(raw?.id) || layerTitle(layerId),
|
||||
name,
|
||||
lat: num(raw?.lat),
|
||||
lon: num(raw?.lon),
|
||||
operator,
|
||||
capacity,
|
||||
river,
|
||||
output,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Produce one normalized-contract input owned by a local infrastructure layer.
|
||||
* The host revalidates the authoritative `source` value while normalizing it.
|
||||
@@ -597,7 +664,11 @@ export function createLocalGeoJsonLayer(
|
||||
}
|
||||
const text = await response.text();
|
||||
if (_destroyed) return;
|
||||
features = parseGeojsonLines(text);
|
||||
const lines = text
|
||||
.split('\n')
|
||||
.filter((l) => l.trim().length > 0);
|
||||
|
||||
features = lines.map((line) => JSON.parse(line));
|
||||
_cachedFeatures = features;
|
||||
}
|
||||
|
||||
@@ -1289,4 +1360,8 @@ function clampCardLine(value) {
|
||||
return text.length > 48 ? `${text.slice(0, 45)}...` : text;
|
||||
}
|
||||
|
||||
export { mapAnalystRecord };
|
||||
function layerTitle(layerId) {
|
||||
if (layerId === 'local-datacenters') return 'Datacenter';
|
||||
if (layerId === 'local-dams') return 'Dam';
|
||||
return 'Feature';
|
||||
}
|
||||
@@ -0,0 +1 @@
|
||||
{"st":"AK","state":"Alaska","fips":"02","features":[{"geoid":"0200065","name":"Adak","full":"Adak city","lsad":"city","polygons":[[[-1766965,518536,115,-143,3,-288,815,0,533,339,-114,0,-127,-30,-39,-2,-35,-34,-50,-11,-6,-11,-45,-3,16,41,-38,-6,-28,11,5,56,19,6,4,14,31,24,-8,17,19,7,6,10,68,10,-65,38,-44,-15,8,-20,-25,-19,-25,3,-15,11,-24,-11,-32,0,-36,11,-96,3,-27,-11,-40,10,-49,0,-37,-11,-67,4,-16,-29,-59,-8,-48,9,-14,17,-8,19,10,42,123,22,23,13,94,8,18,19,22,44,-22,21,-8,65,12,20,0,51,76,163,23,15,157,49,40,4,37,-5,36,24,63,24,115,22,75,-7,31,-11,2,-41,16,-15,45,8,10,9,-23,16,9,29,14,14,52,24,-16,30,5,17,25,18,8,25,24,20,-32,22,-34,3,-96,71,-41,98,-10,0,-17,49,-70,21,-455,7,-56,44,-56,10,-78,-32,-176,-51,-96,1,-9,-21,24,-39,-7,-48,23,-17,37,2,61,-18,23,2,12,-20,35,-26,4,-23,28,-14,-1,-22,-22,-33,7,-12,26,-11,23,0,22,-19,-13,-14,-111,0,-468,-437]]]},{"geoid":"0200650","name":"Akhiok","full":"Akhiok city","lsad":"city","polygons":[[[-1542668,569259,517,0,528,72,0,114,39,18,-7,13,36,12,-11,22,-21,-1,-5,20,-19,-4,53,42,9,18,-25,25,16,10,-1111,0]]]},{"geoid":"0200760","name":"Akiachak","full":"Akiachak CDP","lsad":"CDP","cdp":true,"polygons":[[[-1614587,609108,106,-27,47,9,21,-11,45,0,45,-14,-58,1,-41,10,-48,-4,38,-30,36,-13,147,-22,67,11,80,48,61,16,64,-4,91,-37,107,-82,66,27,34,28,-16,17,-54,22,-68,8,-33,35,42,19,63,14,30,-10,18,-26,5,-29,17,-16,25,-3,13,16,-23,19,19,18,61,4,35,-6,53,12,-4,10,-54,36,-3,42,-121,158,-29,-1,-30,22,19,18,-24,-2,-18,10,3,37,-117,-10,-76,18,-22,-4,-17,-24,-27,-2,-38,11,-15,34,-49,6,-46,-6,-79,-34,53,-21,-9,-24,37,-17,-32,-23,20,-27,-34,-3,-40,16,-20,-17,-23,-1,-7,-15,25,-13,-27,-13,-41,-6,4,-23,-20,-27,-55,44,-57,19,-7,-22,-24,-17,-3,-58,-96,-14]]]},{"geoid":"0200870","name":"Akiak","full":"Akiak city","lsad":"city","polygons":[[[-1612344,608917,445,0,-1,289,-446,0]]]},{"geoid":"0201090","name":"Akutan","full":"Akutan city","lsad":"city","polygons":[[[-1659430,541813,247,0,0,-578,246,0,0,-145,731,0,0,-145,738,0,0,145,736,0,-5,19,15,18,128,53,24,19,-5,14,-43,14,-9,25,9,34,-102,26,-36,-25,-35,-4,-19,-23,-7,-26,-131,-19,-83,-42,-131,-7,-7,32,-17,18,-62,0,-33,-34,-5,-33,-73,-24,-169,45,-93,4,-81,25,-61,-1,-36,23,-64,7,-60,24,-52,0,-59,17,-171,-2,-19,26,7,30,38,21,78,13,195,-20,31,-18,45,1,38,18,57,-5,29,15,95,2,64,16,57,41,114,40,88,-5,26,6,-45,33,-43,-3,-16,14,-10,30,-90,85,-38,10,-45,-6,-105,43,-67,-5,-68,22,-100,57,-81,-22,-38,-20,-33,3,-27,21,0,24,29,32,45,22,41,4,21,14,2,15,-45,17,-59,-14,-81,-48,-202,-163,-45,-3,-233,51,-55,25,-17,21,5,28,-57,22,-59,38,-24,32,2,21,55,35,26,7,10,54,26,23,31,10,19,25,-12,31,5,25,-67,76,-159,39,-69,7,-57,-19,-67,-36,-109,-12,-81,9,-43,14,-7,65,-40,31,0,19,-22,17,-47,-17,-22,-31]],[[-1656791,542343,100,-53,316,4,114,49,19,19,2,21,38,18,24,29,38,18,140,-22,95,-35,57,-50,22,-40,0,-46,-34,-30,-42,-16,-41,-44,-78,-3,-188,-89,-71,-19,-36,-45,-45,-29,-10,-31,50,-27,62,-77,50,-42,74,-29,145,-11,26,-24,-2,-40,-17,-57,-33,-36,-81,-50,-36,-42,-93,-66,-45,-15,-69,10,-102,-21,-43,-22,86,-10,43,-13,43,-23,0,-24,23,-17,45,-8,41,-22,28,-63,48,-10,12,-48,40,-21,31,22,57,4,240,-78,112,-4,17,31,33,12,90,3,36,22,14,57,26,23,33,11,19,46,-19,21,-59,26,-50,8,-90,-1,-12,18,19,18,26,4,29,21,-29,16,-5,21,69,15,50,29,41,6,38,-7,90,-46,-1,663,-493,0,-1,289,-106,0,0,145,-494,0,0,-145,-489,0]]]},{"geoid":"0201200","name":"Alakanuk","full":"Alakanuk city","lsad":"city","polygons":[[[-1647500,627112,305,0,4,-722,1568,0,0,144,308,0,-3,433,315,0,-1,289,-1249,1,1,144,-1248,0]]]},{"geoid":"0201305","name":"Alatna","full":"Alatna CDP","lsad":"CDP","cdp":true,"polygons":[[[-1530015,665958,25,-62,23,-27,81,-60,52,-5,44,-17,21,-46,120,-63,7,-16,-17,-35,8,-10,1,-71,39,-23,-47,-33,0,-16,68,-18,86,0,56,-9,-1,-7,94,-14,91,-1,64,-15,79,8,30,-16,41,6,72,-8,68,10,51,-5,120,37,44,0,24,11,36,-4,22,9,106,-19,20,-19,18,-2,18,-39,38,-12,74,-7,16,-11,-25,-3,-25,-53,-58,-19,-17,-20,-72,-5,-40,-27,2,-16,-37,-6,27,-19,-43,-4,11,-25,-43,-17,20,-13,-42,-2,4,-17,-92,-88,210,-6,187,5,329,-33,53,2,68,21,16,32,45,15,14,34,-6,42,-15,37,-25,36,-14,7,10,33,62,37,118,54,57,45,35,12,34,-1,71,-15,-8,-11,74,-4,70,8,53,12,100,68,76,26,158,12,68,-19,-1,345,163,0,13,15,-97,98,-33,-8,-642,14,-2462,147,5,-35,-47,-30]]]},{"geoid":"0201390","name":"Alcan Border","full":"Alcan Border CDP","lsad":"CDP","cdp":true,"polygons":[[[-1413223,626355,17,-62,101,-91,134,-59,3,-37,-57,-55,72,-56,-24,-3,16,-19,-12,-17,28,-10,-36,-2,23,-8,18,-26,14,1,-17,-23,70,-32,4,18,21,-17,5,17,52,-22,19,10,14,-10,29,1,24,14,14,-9,-26,-13,25,0,-24,-14,24,-11,42,0,-42,-20,66,5,-1,-16,24,-5,3,12,68,-26,-28,-13,36,-22,-9,-8,-53,-1,-22,-18,51,8,33,-5,9,-22,-14,-12,57,-15,19,9,30,-2,-14,-11,58,0,31,-21,-1,25,17,1,3,-16,18,-10,46,4,17,-4,-23,-5,64,-19,-10,12,46,4,-1,-13,38,-9,8,16,-27,12,66,2,14,17,0,-22,20,-5,75,6,6,12,27,-20,-19,-7,19,-12,-31,6,-17,-11,19,4,29,-16,4,12,19,-15,69,-14,-23,-24,31,-1,-17,11,61,2,22,10,5,-14,55,-12,-13,-27,36,-6,-2,16,30,-17,-1,12,25,-4,1,-12,67,13,14,-20,33,-6,56,16,-4,11,-38,6,62,23,33,3,3,9,38,-3,39,24,130,-27,30,-14,37,Line truncated
|
||||
@@ -0,0 +1 @@
|
||||
{"st":"AL","state":"Alabama","fips":"01","features":[{"geoid":"0100100","name":"Abanda","full":"Abanda CDP","lsad":"CDP","cdp":true,"polygons":[[[-855420,331037,13,-55,47,-73,4,-26,72,-49,-9,-23,-40,-43,-9,-29,46,9,98,0,42,22,0,43,83,21,-1,29,-96,106,-35,19,-34,-12,25,44,47,5,30,17,-7,45,-271,-6,0,-27]]]},{"geoid":"0100124","name":"Abbeville","full":"Abbeville city","lsad":"city","polygons":[[[-852958,315472,246,0,0,31,86,0,31,-64,-7,-118,77,1,2,38,88,-6,4,68,134,-5,-4,-53,91,-5,2,26,27,-2,-2,-26,34,-3,3,23,31,-2,2,28,-22,1,-9,14,-32,3,16,281,-9,147,-81,4,-7,72,-71,3,-12,72,-38,0,10,-71,-506,12,2,-190,-45,-4,-2,-124,-38,1]]]},{"geoid":"0100460","name":"Adamsville","full":"Adamsville city","lsad":"city","polygons":[[[-870488,336355,26,-1,16,8,1,-19,44,0,1,-72,212,0,-3,12,12,24,21,-1,1,-36,-23,1,0,-38,-47,1,0,-36,17,-16,17,-59,-32,1,1,-37,46,-1,8,65,18,-29,19,0,0,9,93,23,1,-181,11,-11,16,0,6,11,40,0,0,-18,-16,0,-20,-27,-9,0,0,-18,-30,18,2,-27,87,-1,1,54,5,0,1,-23,36,12,1,-25,23,0,2,18,14,-4,-19,-28,23,-5,-1,-36,-32,0,-11,15,-22,-15,0,-36,-22,0,0,-37,36,-36,48,-1,-1,-62,7,-10,55,-1,20,-49,11,27,36,21,5,-13,19,0,8,-9,10,14,-2,-16,53,-23,0,-22,-20,-16,-18,-42,8,-6,118,-20,43,-60,18,-9,36,-1,10,19,2,28,15,7,32,-21,63,2,-6,-4,27,-20,18,0,34,17,-5,18,-25,8,-1,9,17,7,-26,1,0,7,25,-1,0,14,-40,4,-11,18,-4,66,27,-5,9,10,6,-14,38,-6,0,-158,24,-59,16,0,5,-85,-10,-19,12,6,9,-15,-16,-5,-1,-50,-13,-7,39,-18,36,14,-18,10,13,23,-13,19,1,175,97,-11,1,9,-22,0,0,9,32,0,1,27,-154,1,1,112,43,-1,0,37,-18,0,0,-21,-21,0,0,14,-19,-5,15,13,0,29,28,-4,4,10,-32,1,2,69,-38,-18,-20,21,55,37,4,-18,80,-5,-2,16,-77,15,0,11,17,9,-4,21,-23,-7,10,-5,-1,-36,-15,0,1,30,-12,13,0,-13,-17,-4,11,21,-42,15,-1,-24,32,-2,0,-9,-63,-6,-5,34,7,27,-28,15,4,51,-39,9,-4,64,-58,-3,-29,4,-16,10,-5,-8,-20,7,3,14,84,-9,-4,38,-87,2,-1,60,-14,-3,-15,10,-6,16,-5,-26,-8,7,-42,-1,-83,-19,44,-1,1,-21,-25,-1,-20,10,1,-29,11,0,0,-18,-45,0,0,18,-37,9,-3,-26,-13,0,0,35,43,0,0,35,-40,0,0,10,-33,27,-27,0,8,-8,-14,-15,-3,23,-108,4,0,37,85,-3,0,66,8,6,81,1,-2,207,11,11,12,0,22,71,-141,3,1,53,-86,22,-44,0,-4,22,-37,-34,-5,-19,3,62,-27,-36,-24,-13,-28,21,-47,64,-35,35,-19,11,-20,-3,-27,-22,-48,-90,8,-19,19,0,-7,-20,17,1,0,5,34,-5,3,20,41,-18,-52,-36,-100,-38,-24,-34,-19,-47,1,-55,-21,0,-40,-23],[-869779,335825,89,-1,-1,73,174,-2,-1,-36,-86,1,-3,-110,-174,2]],[[-869343,335929,44,-1,1,37,35,-1,8,35,13,0,1,-19,15,-1,0,-16,15,0,-1,-37,43,-1,0,-30,-54,4,10,-9,152,-3,-7,11,28,-1,1,79,43,-1,0,-38,25,0,0,14,17,-12,-1,75,-40,5,-1,47,22,-1,-22,17,6,21,-28,0,0,-37,-56,1,1,65,-52,9,0,37,-43,0,0,-18,-23,0,22,-5,-1,-50,43,-1,1,-36,-108,3,-22,14,0,-13,-45,0,1,-28,23,1,3,-14,-69,5]],[[-869707,336730,6,-82,8,-5,-12,-14,-21,-80,66,0,14,26,43,35,-21,50,42,20,-8,13,-55,2,4,-18,-35,-3,-20,41,0,42,36,0,-3,10,-33,6,0,20,-19,9,-12,26,8,13,-16,0]]]},{"geoid":"0100484","name":"Addison","full":"Addison town","lsad":"town","polygons":[[[-872034,342013,49,-1,0,-44,15,-15,14,-1,6,15,12,0,4,-28,-15,-10,-33,-2,6,-24,21,0,22,28,10,0,1,-35,-13,-25,11,-39,64,0,-9,-7,13,-26,7,23,-7,10,7,79,44,0,1,27,38,0,0,-76,14,14,-2,26,35,1,4,19,16,0,0,15,156,0,0,-33,45,0,0,30,10,-2,14,14,0,15,-24,9,-45,-8,-8,14,0,68,-121,-1,1,71,43,4,0,-12,24,0,0,37,26,20,0,-13,14,0,1,15,-12,0,32,23,26,-13,15,29,-32,14,-94,-73,0,42,-85,0,0,-36,-133,1,0,-32,-51,0,0,-41,-42,-1,0,-34,-79,7,-17,-8]]]},{"geoid":"0100676","name":"Akron","full":"Akron town","lsad":"town","polygons":[[[-877497,328726,160,-2,1,29,-107,134,-53,-58]]]},{"geoid":"0100820","name":"Alabaster","full":"Alabaster city","lsad":"city","d":5,"polygons":[[[-8687877,3323076,438,-6,-8,-594,434,-7,-1,-140,769,-20,-8,-144,-99,-323,-240,6,-1,-257,268,-1,39,-153,163,-219,-185,3,-4,-36,-523,5,36,-59,477,-9,-6,-30,-285,-60,-89,2,187,-155,114,-12,-1,-67,43,-146,-181,208,-493,8,1,374,-301,1,-133,-40,5,-1042,432,-20,13,-2548,431,-15,-8,366,492,-10,83,357,-321,7,-41,-268,-218,-14,-19,1032,489,-38,-37,-73,300,20,-16,100,-115,341,260,-2,-3,-378,434,-7,-2,-368,291,-6,4,379,133,-47,-9,-703,452,-10,2,-351,1141,-31,153,-117,-5,-247,434,-10,6,366,177,52,-34,1,35,60,220,69,-119,-121,241,101,-60,-177,1,-500,481,-5,-8,-62,282,-6,4,553,-328,8,-1,94,-320,0,-62,158,-11,80,1019,-77,-185,-273,41,-538,141,-270,115,-60,347,78,57,-161,131,-6,-282,-224,311,-5,1,245,243,134,-151,177,-46,-33,-126,-38,0,387,29,172,242,-10,-1,-143,208,-11,32,240,432,-22,9,-862,16,-3,2,129,422,-13,-4,178,438,-6,1,180,589,-12,-571,1125,757,-25,-2,47,43,-1,27,-642,167,-252,214,-92,81,3,-7,361,445,-16,0,381,59,-109,888,-46,-166,307,241,102,78,-119,663,-16,7,735,-895,17,-5,125,-537,24,-129,215,-145,126,-67,-163,-496,-59,65,133,-136,5,-84,362,237,48,-186,210,-250,521,536,-34,-3,333,-67,55,-263,-19,-1,75,-220,-11,1,-80,-309,-11,-9,-186,202,-229,-176,-160,-186,151,-419,9,8,130,-689,34,-623,769,462,-2,-1,194,-234,30,-66,-116,-176,-106,-126,205,7,250,287,-98,307,-12,2,184,68,46,-4,116,-65,-4,-4,390,1094,-22,25,336,-32,4,4,-337,-1094,21,3,-395,69,-107,-150,-44,-176,150,-48,-206,-34,262,-137,166,-257,425,90,123,632,-15,-1,723,425,3,-3,356,437,Line truncated
|
||||
@@ -0,0 +1 @@
|
||||
{"st":"AR","state":"Arkansas","fips":"05","features":[{"geoid":"0500190","name":"Acorn","full":"Acorn CDP","lsad":"CDP","cdp":true,"polygons":[[[-942258,346517,72,-69,24,-34,32,-78,-5,-80,-34,-42,37,-22,88,17,30,-2,25,-11,10,-28,15,-8,16,0,38,34,10,49,-9,13,0,35,44,12,25,19,15,45,-16,29,-17,3,-18,28,-4,29,18,4,1,47,-93,2,-18,39,-107,1,-1,-66,-15,5,-38,53,-36,14,-62,-2,-61,-19]]]},{"geoid":"0500250","name":"Adona","full":"Adona city","lsad":"city","polygons":[[[-929103,350346,221,-3,3,109,-222,4]]]},{"geoid":"0500580","name":"Alexander","full":"Alexander city","lsad":"city","polygons":[[[-924723,346094,46,-2,0,-37,277,-2,-4,34,7,16,-19,18,-13,-5,12,30,-21,0,3,28,44,22,1,36,-35,1,-7,15,45,2,6,19,-49,0,50,44,-47,38,-22,-15,2,44,-8,15,9,9,3,61,-52,-8,-2,-31,8,0,8,-47,-27,0,-27,-11,0,-9,43,-1,-1,-52,32,-12,-21,-23,-191,-143,-49,1]]]},{"geoid":"0500700","name":"Alicia","full":"Alicia town","lsad":"town","polygons":[[[-910886,358875,100,-5,2,110,-56,3,30,36,-10,6,4,-8,-28,-33,-39,2]]]},{"geoid":"0500730","name":"Alix","full":"Alix CDP","lsad":"CDP","cdp":true,"polygons":[[[-937366,354212,40,-1,5,-38,12,1,-1,-33,45,-2,0,34,45,0,-24,54,-17,10,1,18,43,23,-132,7,-1,-20,21,0,-1,-30,-34,-7]]]},{"geoid":"0500820","name":"Alleene","full":"Alleene CDP","lsad":"CDP","cdp":true,"polygons":[[[-942841,337672,290,-175,24,37,4,39,14,12,-10,21,41,19,16,32,77,27,-313,76,-18,-6,-18,-25,-24,-11,-19,4,-5,-21,-39,-12]]]},{"geoid":"0500940","name":"Allport","full":"Allport town","lsad":"town","polygons":[[[-917973,345415,110,-1,4,-39,17,7,-1,-58,-17,1,17,-6,-1,-46,-20,1,0,-10,23,0,2,119,45,0,1,51,10,8,-48,1,-6,19,-1,-33,-26,-3,0,-8,-104,2,-5,6,0,-6]]]},{"geoid":"0500970","name":"Alma","full":"Alma city","lsad":"city","polygons":[[[-942543,354939,9,-12,134,-2,-14,-6,-1,-12,24,-15,-1,-77,40,-1,8,-36,18,-12,24,6,-2,-67,134,-3,-1,-36,24,0,0,-14,-24,0,0,-23,45,0,1,54,-30,1,-11,18,86,-1,2,72,-27,1,5,22,-22,-1,1,20,135,9,24,57,66,-2,4,36,82,6,91,35,65,12,1,44,-49,76,-110,-2,0,-8,42,-2,0,-58,-67,1,-19,-25,37,-17,47,-8,-49,-16,9,-30,-71,-7,-155,15,0,21,-150,-23,-15,-16,-17,6,2,18,37,9,0,17,20,12,2,44,17,32,-22,17,1,17,26,16,15,1,9,21,47,-8,16,11,-31,15,17,21,-19,45,-13,-5,0,-23,-32,-17,-58,-51,-15,-18,-8,-28,-57,52,8,-31,13,-5,0,-9,-12,-6,-44,2,0,18,-33,0,-11,18,-2,-72,-40,1,-14,-26,-28,1,-7,19,6,32,-12,12,-32,0,2,-35,20,-1,0,-24,17,5,-2,-8,-39,1,-1,8,-20,1,0,18,-22,1,-2,-46,45,-1,-1,-28,-66,2]]]},{"geoid":"0501030","name":"Almyra","full":"Almyra town","lsad":"town","polygons":[[[-914176,344020,133,0,2,74,-133,0]]]},{"geoid":"0501060","name":"Alpena","full":"Alpena town","lsad":"town","polygons":[[[-933157,362819,84,0,6,-31,43,-1,1,31,44,0,2,74,92,0,2,80,-270,-5]]]},{"geoid":"0501090","name":"Alpine","full":"Alpine CDP","lsad":"CDP","cdp":true,"polygons":[[[-933999,342255,19,-8,17,2,35,-29,142,10,7,-52,115,27,41,79,3,24,-77,-7,1,70,-21,-19,-145,-26,-36,-15,-9,4,-14,-33,-23,-4,-9,-9,-18,5,-31,-10]]]},{"geoid":"0501150","name":"Altheimer","full":"Altheimer city","lsad":"city","polygons":[[[-918728,343168,161,-5,0,-26,155,-11,2,71,76,-3,3,118,-44,1,1,72,-14,0,23,41,-310,-229,-53,-18]]]},{"geoid":"0501210","name":"Altus","full":"Altus city","lsad":"city","polygons":[[[-937892,354759,58,-12,-1,-12,31,-15,-13,-18,15,-6,37,-72,-18,0,-1,-24,31,-1,-1,-11,-30,1,1,-11,41,-31,3,22,40,-62,-2,-50,-28,-16,-17,1,-1,-37,44,-1,-2,-72,223,-7,2,73,-44,1,4,109,-134,4,2,37,-37,1,-1,-28,-11,3,-22,44,27,-1,1,18,-41,1,-37,74,35,-1,1,19,-44,1,3,54,43,-1,2,36,45,-2,2,37,-90,3,-1,-37,-23,1,-1,-33,-12,1,9,18,-18,15,2,36,-12,1,-2,-53,-58,18]]]},{"geoid":"0501270","name":"Amagon","full":"Amagon town","lsad":"town","polygons":[[[-911119,355604,30,-3,8,12,18,2,1,41,-60,0]]]},{"geoid":"0501360","name":"Amity","full":"Amity city","lsad":"city","polygons":[[[-934804,342563,11,24,34,0,-1,-17,-18,-37,304,-5,0,32,-22,6,1,35,17,-1,26,-22,2,58,-44,1,1,36,-44,1,1,36,44,0,0,18,44,-1,1,55,-44,1,1,26,-44,1,-1,-27,-263,4,-4,-135]]]},{"geoid":"0501457","name":"Anthonyville","full":"Anthonyville town","lsad":"town","polygons":[[[-903425,350372,21,-1,0,12,25,0,1,36,-47,1]]]},{"geoid":"0501540","name":"Antoine","full":"Antoine town","lsad":"town","polygons":[[[-934283,340278,88,0,-1,61,61,7,-26,34,-30,20,-88,2]]]},{"geoid":"0501630","name":"Aplin","full":"Aplin CDP","lsad":"CDP","cdp":true,"polygons":[[[-930023,349657,38,-10,30,1,32,-30,38,-13,19,-31,43,1,14,8,8,26,14,2,-1,-25,16,-16,34,-7,-1,35,23,7,18,-6,25,-27,10,0,35,31,-26,14,7,33,30,55,-10,26,-79,9,-15,31,-34,-11,-1,15,-19,0,-35,-26,-211,1]]]},{"geoid":"0501660","name":"Appleton","full":"Appleton CDP","lsad":"CDP","cdp":true,"polygons":[[[-929068,354342,11,-23,32,-14,28,-32,26,-10,22,-52,33,-39,8,10,34,-23,2,-20,44,-21,41,-62,40,-35,113,-57,114,-24,6,336,-44,-8,-64,6,17,31,-9,21,11,19,-397,5,-59,10]]]},{"geoid":"0501870","name":"Arkadelphia","full":"Arkadelphia city","lsad":"city","polygons":[[[-931112,341076,49,-1,38,36,44,-2,0,22,17,0,5,14,18,-1,-7,-13,10,-9,-16,0,-28,-48,42,-2,1,36,64,-2,1,36,22,0,-1,-36,42,-1,0,-37,45,-1,1,18,88,-3,Line truncated
|
||||
@@ -0,0 +1 @@
|
||||
{"st":"AS","state":"American Samoa","fips":"60","features":[{"geoid":"6000100","name":"Aasu","full":"Aasu village","lsad":"village","polygons":[[[-1707739,-143135,28,-8,32,1,42,-13,32,7,-8,-11,14,-21,10,17,62,4,31,15,28,-47,26,31,3,48,40,69,7,43,12,14,59,10,-101,68,-12,-9,-30,-4,-21,9,-27,-24,-11,-24,-11,-1,-19,46,-14,4,-31,-11,-16,9,10,28,-10,11,-18,-4,-10,-20,-26,8,-10,-4,-57,-138,2,-86]]]},{"geoid":"6002500","name":"Afao","full":"Afao village","lsad":"village","polygons":[[[-1708043,-143324,28,5,21,-11,0,-9,27,178,-5,11,-68,2,-9,-21]]]},{"geoid":"6003300","name":"Afono","full":"Afono village","lsad":"village","polygons":[[[-1706592,-142641,29,-13,24,-60,78,15,13,79,63,77,-16,71,-6,-2,4,-42,-10,-13,-17,-1,0,25,-24,13,-5,-20,6,-20,-8,-22,-17,-5,-12,9,-12,-11,-3,-25,-19,-4,-14,12,-1,16,7,48,12,20,-8,8,-42,-93,-34,-33]]]},{"geoid":"6004500","name":"Agugulu","full":"Agugulu village","lsad":"village","polygons":[[[-1708215,-143310,20,-2,12,-12,10,4,-34,131,-13,-34]]]},{"geoid":"6004900","name":"Alao","full":"Alao village","lsad":"village","polygons":[[[-1705782,-142619,110,-61,12,-27,15,9,-5,12,30,88,2,17,-10,11,-15,-9,-22,5,-68,-7,-28,-21]]]},{"geoid":"6006500","name":"Alega","full":"Alega village","lsad":"village","polygons":[[[-1706461,-142699,10,-36,49,-130,11,19,0,33,9,11,41,-8,-66,109,-46,3]]]},{"geoid":"6007300","name":"Alofau","full":"Alofau village","lsad":"village","polygons":[[[-1706049,-142725,13,-48,-36,-31,49,-37,35,23,24,6,-1,105,-37,26,-71,-38]]]},{"geoid":"6008100","name":"Amaluia","full":"Amaluia village","lsad":"village","polygons":[[[-1707946,-143337,10,-14,6,12,13,0,14,-7,-1,-13,10,-10,62,179,27,39,-33,9,-37,31,-19,6,-8,-2,2,-42,-11,-29,9,-24,-4,-20,-21,-14,-17,-78]]]},{"geoid":"6008900","name":"Amanave","full":"Amanave village","lsad":"village","polygons":[[[-1708462,-143233,46,-11,23,7,33,-23,36,13,23,-20,8,-31,27,33,-13,6,21,54,-24,6,-119,-22,-19,10,-49,-13]]]},{"geoid":"6009700","name":"Amaua","full":"Amaua village","lsad":"village","polygons":[[[-1706316,-142664,31,-46,44,-33,1,11,30,20,-13,50,-21,39,-49,-32]]]},{"geoid":"6010500","name":"Amouli","full":"Amouli village","lsad":"village","polygons":[[[-1705964,-142812,8,10,45,5,13,21,30,10,-5,11,24,16,23,6,16,-3,1,-11,23,-1,20,8,-27,28,-30,61,-116,-69,-26,13,-5,-12]]]},{"geoid":"6011300","name":"Anua","full":"Anua village","lsad":"village","polygons":[[[-1706893,-142718,35,3,-24,61,-16,6]]]},{"geoid":"6012100","name":"Aoa","full":"Aoa village","lsad":"village","polygons":[[[-1705939,-142720,116,69,22,-1,19,33,21,17,-103,60,2,-29,14,-30,-26,-16,-31,-3,-11,43,-47,-131]]]},{"geoid":"6012800","name":"Aoloau","full":"Aoloau village","lsad":"village","polygons":[[[-1707946,-142928,29,-44,15,-135,27,-4,37,-31,145,-44,80,-11,0,38,8,11,-32,-7,-102,20,6,17,-2,86,57,138,-62,-37,-13,21,-27,-10,-6,-23,-21,-8,-22,77,-21,-2,-6,-25,-42,-3,-7,6,-12,-8,-29,-2]]]},{"geoid":"6013700","name":"Asili","full":"Asili village","lsad":"village","polygons":[[[-1707975,-143316,25,1,4,-22,19,101,21,14,4,20,-9,24,11,29,-2,42,-37,-9,-33,-34,5,-11,-27,-178]]]},{"geoid":"6014500","name":"Atu'u","full":"Atu'u village","lsad":"village","polygons":[[[-1706876,-142674,20,-37,14,14,-16,46,-19,-1]]]},{"geoid":"6015300","name":"Aua","full":"Aua village","lsad":"village","polygons":[[[-1706671,-142699,19,-10,10,-26,-14,-27,10,-26,-5,-14,4,-39,14,-37,43,56,51,108,-24,60,-58,18,-78,-5,-18,-54]]]},{"geoid":"6016100","name":"Auasi","full":"Auasi village","lsad":"village","polygons":[[[-1705823,-142651,30,-61,27,-28,2,11,30,18,9,-2,-5,22,-52,72,-19,-33]]]},{"geoid":"6016900","name":"Auma","full":"Auma village","lsad":"village","polygons":[[[-1707860,-143368,33,37,-15,41,-12,-6,-7,15,-32,-86]]]},{"geoid":"6017700","name":"Aumi","full":"Aumi village","lsad":"village","polygons":[[[-1706544,-142729,12,-40,31,-57,-12,-52,4,-17,40,21,17,-8,22,18,28,-1,-59,166,-78,-15]]]},{"geoid":"6018500","name":"Aunu'u","full":"Aunu'u village","lsad":"village","polygons":[[[-1705617,-142857,30,-24,26,-5,16,11,39,-17,24,9,23,26,2,25,-33,36,-16,-3,-29,16,-50,-14,-17,-13,-8,-28]]]},{"geoid":"6020100","name":"Auto","full":"Auto village","lsad":"village","polygons":[[[-1706324,-142776,41,-21,8,33,34,21,-44,33,-31,46,-38,-14]]]},{"geoid":"6020900","name":"Avaio","full":"Avaio village","lsad":"village","polygons":[[[-1706341,-142810,21,-3,37,16,-41,21,-30,98,-53,-23]]]},{"geoid":"6022500","name":"Faga'alu","full":"Faga'alu village","lsad":"village","polygons":[[[-1706952,-142918,44,-25,42,2,49,-19,46,10,-3,8,-54,28,18,25,16,1,38,31,-119,-17,-20,26,-32,-25,-14,-40]]]},{"geoid":"6023300","name":"Faga'itua","full":"Faga'itua village","lsad":"village","polygons":[[[-1706199,-142663,20,-45,32,27,33,-4,4,-17,8,14,69,41,-22,0,-41,25,-56,-6,-52,6]]]},{"geoid":"6024900","name":"Fagali'i","full":"Fagali'i village","lsad":"village","polygons":[[[-1708274,-143171,16,-34,17,1,62,32,-58,147,-34,-51,-22,-5,-30,-28]]]},{"geoid":"6025700","name":"Fagamalo","full":"Fagamalo village","lsad":"village","polygons":[[[-1708122,-1Line truncated
|
||||
@@ -0,0 +1 @@
|
||||
{"st":"AZ","state":"Arizona","fips":"04","features":[{"geoid":"0400730","name":"Aguila","full":"Aguila CDP","lsad":"CDP","cdp":true,"polygons":[[[-1131855,339431,59,-3,31,7,-39,-19,1,-59,38,0,0,-73,217,3,1,143,-43,0,-1,42,-84,-18,-178,0]]]},{"geoid":"0400870","name":"Ajo","full":"Ajo CDP","lsad":"CDP","cdp":true,"polygons":[[[-1129509,323966,33,-131,4,-72,19,-21,-4,-11,11,-24,30,-20,41,-46,29,-13,40,14,37,-35,29,-55,19,-12,65,0,41,-22,22,-1,21,-22,19,-7,36,-7,17,-12,68,-10,40,-21,60,-1,65,-16,53,6,23,-17,29,-5,55,10,111,-22,38,41,22,10,84,11,40,26,16,38,-7,61,-99,89,-122,24,-32,27,-39,0,30,113,74,126,86,307,2,24,-18,60,2,25,31,104,-37,45,-69,7,-4,54,-94,67,-153,2,19,-68,-1,-186,16,1,-1,-71,-303,-3,-28,-42,-10,-32,8,-25,-11,-51,-44,-49,-43,-29,-37,-46,-22,-4,-6,-17,-89,-28,-117,29,-56,-22,-18,-28,-12,-2]]]},{"geoid":"0400940","name":"Ak Chin","full":"Ak Chin CDP","lsad":"CDP","cdp":true,"polygons":[[[-1120184,322839,19,-9,53,9,18,-38,43,16,17,19,11,49,-4,12,-33,7,10,41,-64,9,2,-37,11,-19,-30,1,-18,-33,-19,6]]]},{"geoid":"0401090","name":"Ak-Chin Village","full":"Ak-Chin Village CDP","lsad":"CDP","cdp":true,"polygons":[[[-1121173,330003,692,-2,2,363,-86,0,0,35,-87,-1,0,71,-37,-12,15,-13,-6,-9,-244,-2,-32,-53,-12,-4,-19,-22,-29,-4,-4,-20,15,-40,-166,2,-1,-17]]]},{"geoid":"0401170","name":"Alamo Lake","full":"Alamo Lake CDP","lsad":"CDP","cdp":true,"polygons":[[[-1136127,342297,10,-11,48,-15,16,-18,47,4,30,-15,11,7,13,32,-21,9,8,28,37,9,5,-12,50,-22,40,1,-5,-9,8,-8,97,-5,410,-167,-2,-20,12,3,16,22,-7,25,-14,14,-7,38,18,28,212,-186,54,-12,71,14,60,-61,12,78,33,60,-3,75,14,37,-8,35,44,-73,107,-52,70,-10,-76,249,307,-145,-4,31,-51,122,-22,34,-1,17,-10,14,-67,33,45,23,-2,21,-11,12,7,20,12,6,46,-11,35,21,-19,40,49,53,7,50,-7,42,21,28,53,24,32,-3,26,18,39,-10,43,8,10,13,1,21,-30,22,-5,20,2,33,23,14,-24,39,16,8,-68,18,-40,-2,-27,12,-134,-22,-58,7,-135,39,-15,0,-32,-20,-19,1,-12,24,-24,10,-51,-22,-87,20,-47,-23,-86,0,-49,19,-25,-4,-62,60,-29,2,-8,-16,-40,-1,-25,17,-41,-5,-84,-36,-75,-91,1,-9,-69,-108,-16,-3,-63,-52,-43,-45,-27,-10,-4,-47,-7,-10,-37,-15,-27,-32,-59,-45,-13,-19,-34,-145,-105,-62,-22,-39,-56,-38,-11,-2,-13,24,-48,-7]]]},{"geoid":"0401560","name":"Ali Chuk","full":"Ali Chuk CDP","lsad":"CDP","cdp":true,"polygons":[[[-1125721,318126,139,-30,-25,-21,45,-10,15,22,36,-7,15,10,18,28,1,43,22,18,-1,40,-23,30,-45,12,-6,-15,-38,-33,3,24,-136,12]]]},{"geoid":"0401570","name":"Ali Chukson","full":"Ali Chukson CDP","lsad":"CDP","cdp":true,"polygons":[[[-1118201,319030,52,-35,94,-9,15,12,6,32,34,31,21,-3,19,-17,60,22,-18,60,56,-5,7,38,17,23,-75,85,-19,4,-8,-23,-32,-5,-42,-20,-30,-67,-12,-11,-64,-8,-45,11,-18,-12,12,-64,-22,-20]]]},{"geoid":"0401620","name":"Ali Molina","full":"Ali Molina CDP","lsad":"CDP","cdp":true,"polygons":[[[-1117875,318989,-19,-13,9,-25,69,18,86,11,-6,-14,48,15,-19,10,-22,44,-27,30,12,54,-43,-9,13,-23,-46,3,-48,-20,-26,-23]]]},{"geoid":"0401920","name":"Alpine","full":"Alpine CDP","lsad":"CDP","cdp":true,"polygons":[[[-1091539,338431,35,11,0,-27,18,12,41,5,50,-5,41,-22,29,-4,16,23,-61,31,16,21,-13,2,-6,26,12,27,-21,-3,-19,-28,-33,8,-17,18,-22,-18,-19,7,-1,-44,-12,16,0,-23,-35,-6]]]},{"geoid":"0401990","name":"Amado","full":"Amado CDP","lsad":"CDP","cdp":true,"polygons":[[[-1110769,316988,9,-138,24,6,25,24,65,-3,16,-169,36,-180,82,-2,-24,81,25,74,-16,138,-65,45,-6,22,25,20,2,38,-11,24,71,-2,56,-35,13,5,-1,181,43,-1,-30,65,10,3,9,24,-9,15,32,39,-192,0,-36,-71,-26,-77,-15,9,-14,-6,4,-10,-53,-35,-72,-14]]]},{"geoid":"0402270","name":"Anegam","full":"Anegam CDP","lsad":"CDP","cdp":true,"polygons":[[[-1120524,323637,329,1,-4,98,-47,7,-8,46,-22,-8,18,143,-37,-16,-20,-29,-19,-11,-112,-34]]]},{"geoid":"0402410","name":"Antares","full":"Antares CDP","lsad":"CDP","cdp":true,"polygons":[[[-1138169,354165,84,21,87,-2,4,97,-94,1,0,-10,-82,0]]]},{"geoid":"0402430","name":"Anthem","full":"Anthem CDP","lsad":"CDP","cdp":true,"polygons":[[[-1121434,338483,-13,-147,8,-62,89,1,0,147,522,5,1,254,-43,0,0,36,-478,-5,0,72,-120,0]]]},{"geoid":"0402830","name":"Apache Junction","full":"Apache Junction city","lsad":"city","polygons":[[[-1115849,334164,43,5,-1,-292,17,0,0,-12,24,-22,3,-55,-80,0,1,-63,164,1,0,-36,-44,0,0,-36,-119,0,7,-582,27,-2,0,-69,583,0,-17,33,-19,101,-44,140,-153,234,688,0,-3,316,-126,0,0,36,-43,0,0,218,-86,0,-1,217,86,0,-1,218,-28,0,-14,-10,-1,-63,-86,0,0,73,-734,-1,-1,-300,-22,0,0,-26,-20,0]]]},{"geoid":"0403320","name":"Arivaca","full":"Arivaca CDP","lsad":"CDP","cdp":true,"polygons":[[[-1113553,315889,38,-28,-5,-10,102,-75,-36,-7,-19,-17,4,-22,18,-17,12,-46,-1,-57,32,-3,8,-19,-3,-46,55,-72,21,-125,16,-13,8,-39,16,-1,31,25,52,14,24,23,57,8,35,-4,-4,24,46,-6,41,6,39,-32,6,-18,17,-4,48,-68,22,-6,36,20,54,1,59,19,-8,16,17,14,16,5,11,-12,-52,135,-30,54,-18,4,4,22,-10,12,-21,0,-22,34,0,75,49,-20,76,10,51,-17,79,18,105,-23,41,0,13,-8,95,-5,10,72,-9,21,0,171,-75,0,-2,13,30,17,-11,31,33,25,-10,9,23,57,-16,75,-42,-12,-18,10,-25,-3,-31,16,-690,-23,-91,-69,-6,-16,-24,-15,-4,5,-35,-25,-197,-24,-33,-23]]]},{"geoiLine truncated
|
||||
@@ -0,0 +1 @@
|
||||
{"st":"CA","state":"California","fips":"06","features":[{"geoid":"0600135","name":"Acalanes Ridge","full":"Acalanes Ridge CDP","lsad":"CDP","cdp":true,"polygons":[[[-1220847,379081,41,-17,-1,-32,78,1,-4,5,19,4,-18,28,36,23,-51,0,-21,21,-30,-10,-68,2]]]},{"geoid":"0600156","name":"Acampo","full":"Acampo CDP","lsad":"CDP","cdp":true,"polygons":[[[-1212882,381773,1,-26,-13,0,2,-72,63,-1,0,-25,13,-5,77,1,-21,58,50,0,-3,117,-91,0,7,-20,-13,-7,-86,0]]]},{"geoid":"0600212","name":"Acton","full":"Acton CDP","lsad":"CDP","cdp":true,"polygons":[[[-1182577,344818,117,-1,10,-36,-3,-108,83,1,0,-40,174,-7,-1,30,117,4,2,-32,12,0,3,-68,216,-4,0,-20,356,-17,6,141,298,2,2,141,8,2,210,2,0,-73,170,0,0,219,-411,-2,-19,20,-7,31,-81,76,-14,0,-214,199,-87,-2,1,-39,-133,1,2,76,-92,-2,0,36,-43,2,1,36,-176,4,0,37,-132,1,0,-37,-353,2,-6,-6,6,-17,0,-411,-44,0]]]},{"geoid":"0600296","name":"Adelanto","full":"Adelanto city","lsad":"city","polygons":[[[-1175068,345575,31,0,1,-287,393,-2,2,72,559,0,0,-73,-265,0,0,-110,-252,1,1,35,-155,3,0,-37,-110,2,0,-109,868,-3,1,364,352,-1,1,142,-100,2,-1,148,-243,-2,0,73,-22,0,-1,219,109,0,0,146,-18,0,-1,296,327,-2,-11,24,4,19,-17,103,-393,1,0,-368,-176,-1,-177,3,-2,370,-529,1,-6,-293,89,0,0,-147,89,0,5,-293,-136,-2,-1,-73,-222,-1]]]},{"geoid":"0600310","name":"Adin","full":"Adin CDP","lsad":"CDP","cdp":true,"polygons":[[[-1209714,411843,335,-1,4,63,58,-1,38,14,22,18,17,33,35,28,-41,2,-83,64,-4,12,-13,-7,-38,9,8,-41,-33,-30,-24,-10,-17,0,-123,52,-142,0]]]},{"geoid":"0600394","name":"Agoura Hills","full":"Agoura Hills city","lsad":"city","polygons":[[[-1188004,341394,130,-31,176,-17,132,62,182,-80,16,-16,20,14,63,0,0,-11,46,1,-2,-63,43,0,0,37,-13,24,13,7,0,42,-43,-1,0,19,-51,3,-79,46,0,114,44,0,40,135,-602,3,-39,-222]]]},{"geoid":"0600450","name":"Agua Dulce","full":"Agua Dulce CDP","lsad":"CDP","cdp":true,"polygons":[[[-1183432,344674,-1,-63,27,38,32,13,74,8,-10,31,21,10,26,-2,-1,143,17,35,162,-1,-2,-72,197,2,20,26,9,55,31,26,122,47,102,15,22,9,28,24,1,352,-6,17,8,44,-9,7,-18,-5,-79,16,-14,-4,-30,-34,2,-12,-50,-55,-42,-6,-20,26,1,10,-37,3,-47,-15,1,16,-24,-10,-10,3,-32,-20,-4,10,-57,-24,-28,17,-56,12,-43,-39,-22,-10,-10,-16,-55,-26,-15,-20,-243,-3,-63,-16,-28,21,-32,-32,12,-29,-12,-17,2,-213,-111,-4,-32,-16,-15,-21,-7,-35,5,-78,-14,-136]]]},{"geoid":"0600464","name":"Aguanga","full":"Aguanga CDP","lsad":"CDP","cdp":true,"polygons":[[[-1169035,334466,13,7,9,-15,25,-5,5,-17,36,10,44,-26,117,-15,0,-28,-47,2,-9,-23,-32,-7,-6,-37,9,-19,30,-11,3,-13,620,0,13,15,20,6,6,41,-37,12,-4,12,8,27,-19,0,-20,13,18,13,0,171,-19,4,0,40,-37,16,-3,21,24,3,15,-11,30,28,14,-4,6,12,15,4,-7,9,36,22,-10,38,-15,7,-22,41,-18,-17,-10,-34,-21,-11,-17,6,-13,17,-10,-7,-52,7,3,63,-7,11,-83,-19,-53,-40,-56,0,-45,-18,-1,19,-18,18,-17,-2,-37,-32,-20,8,-19,-13,-15,8,-5,-8,-29,-1,12,-13,-16,-19,14,-49,-3,-20,39,-1,37,-24,-58,-6,-19,-63,-18,52,-73,79,-99,2,5,-138,-53,-13,16,-34,-1,-16,-15,-9,-24,1,-19,16,-22,1,12,-18,-15,-5]]]},{"geoid":"0600478","name":"Ahwahnee","full":"Ahwahnee CDP","lsad":"CDP","cdp":true,"polygons":[[[-1197528,373759,53,-62,1,-32,17,-12,-3,-24,44,-49,-24,-68,-18,-10,41,-55,21,16,23,-16,20,-2,16,15,11,-18,41,-14,79,-49,8,-20,65,-26,20,5,-10,-9,1,-30,130,1,-3,71,31,12,15,-8,64,22,43,-37,61,34,36,7,-18,6,-76,-6,-58,14,-44,-10,-23,8,20,20,5,74,15,-4,9,14,24,9,14,22,17,9,2,29,28,35,-102,85,-52,120,-2,13,9,7,4,24,45,18,26,20,8,19,-30,26,-38,9,-3,8,-10,-6,-25,9,-35,-10,-46,19,-24,-11,-6,32,-29,-11,-10,-13,-22,16,-23,-9,-34,-39,-1,-39,-13,-11,-2,-36,-43,-19,-11,-40,-23,7,-19,15,-6,18,-24,11,1,55,-34,9,-50,-7,-6,-13,-20,0,-29,-45,8,-7,2,-60]]]},{"geoid":"0600535","name":"Airport","full":"Airport CDP","lsad":"CDP","cdp":true,"polygons":[[[-1209782,376347,35,0,0,-37,69,-2,15,9,0,23,-17,0,0,15,-35,0,0,7,-32,-4,-1,10,-34,10]]]},{"geoid":"0600562","name":"Alameda","full":"Alameda city","lsad":"city","polygons":[[[-1223309,377951,-7,-80,36,-57,102,-17,24,-31,90,-5,65,-38,111,-36,-2,-41,59,-7,37,42,44,-24,31,-42,280,-97,-10,-15,-168,-69,7,-38,50,-39,75,-78,6,5,117,-85,79,78,15,0,3,204,31,-1,-8,148,-24,41,-139,68,-30,26,-13,28,-4,39,-14,17,-47,22,-27,-1,-61,-23,-84,34,-97,52,-78,5,-150,-16,-275,55,-45,16]]]},{"geoid":"0600618","name":"Alamo","full":"Alamo CDP","lsad":"CDP","cdp":true,"polygons":[[[-1220555,378731,25,-74,-11,-14,9,-10,8,-208,90,-5,1,-45,55,2,21,-7,98,-91,35,13,8,-12,12,32,47,45,-33,34,27,12,22,30,-19,16,35,13,33,1,44,-15,78,4,1,-81,101,3,0,72,209,7,-5,-17,9,-1,-4,-5,13,12,41,13,0,116,-44,15,-17,-5,-46,19,-23,23,-156,-5,-8,66,-84,14,12,25,-47,14,13,33,-30,8,-29,20,-35,-19,-18,-33,14,-27,-97,16,-2,-70,-30,-1,0,56,-29,14,-1,-69,-24,-7,-3,21,-66,-1,-15,-21,-18,30,-19,11,-7,-5,-35,36,2,12,-32,14,-29,52,-24,-15,14,-45,-20,-16]]]},{"geoid":"0600674","name":"Albany","full":"Albany city","lsad":"city","polygons":[[[-1223269,378887,107,-38,34,-36,159,22,140,-10,11,25,-9,4,10,29,-7,29,-46,-3,3,76,-32,-9,-63,13,-169,-16,36,-43,-16,-26,-43,18,-115,3,0,-29]]]},{"geoid":"0600716","name":"Albion","full":"Albion CDP","lsad":"CDP","cdpLine truncated
|
||||
@@ -0,0 +1 @@
|
||||
{"st":"CO","state":"Colorado","fips":"08","features":[{"geoid":"0800320","name":"Acres Green","full":"Acres Green CDP","lsad":"CDP","cdp":true,"polygons":[[[-1049042,395510,154,-2,15,30,13,3,4,9,-8,1,1,31,-32,1,-30,32,-42,6,-2,-9,-30,-1,-23,-10,0,-19,-26,0]]]},{"geoid":"0800620","name":"Aetna Estates","full":"Aetna Estates CDP","lsad":"CDP","cdp":true,"polygons":[[[-1046783,397364,94,0,0,-15,5,0,0,50,-99,-4]]]},{"geoid":"0800760","name":"Aguilar","full":"Aguilar town","lsad":"town","polygons":[[[-1046626,373994,26,3,32,-18,9,17,92,1,-34,28,0,22,-14,0,17,26,-3,22,-76,0,0,-49,-20,1,-5,-16,-24,0]]]},{"geoid":"0800870","name":"Air Force Academy","full":"Air Force Academy CDP","lsad":"CDP","cdp":true,"polygons":[[[-1048985,390014,9,-13,50,-9,9,-15,-5,-20,14,-17,40,6,4,-18,-7,-18,-24,-23,-2,-25,34,-28,8,-26,52,-11,67,-33,5,-43,38,-25,93,-17,24,-21,26,-7,5,-9,36,3,11,11,82,-30,4,-9,16,0,28,-30,13,90,22,71,-21,6,15,54,-15,125,-169,279,-136,-26,-141,16,-101,28,-61,-13,-4,-51,12,-22,-13,-57,-17,-24,-5,-21]]]},{"geoid":"0800925","name":"Akron","full":"Akron town","lsad":"town","polygons":[[[-1032256,401721,122,-20,-96,-5,3,-187,39,0,7,41,44,0,-7,-40,50,-1,-9,-66,-51,1,0,27,-35,0,-2,-75,90,-1,14,177,84,-11,5,64,40,0,-1,-69,29,-1,1,17,15,-6,2,93,-132,1,1,35,-45,0,0,36,-46,-1,0,-15,-119,19]],[[-1032463,401624,189,0,-3,144,-185,1]]]},{"geoid":"0801090","name":"Alamosa","full":"Alamosa city","lsad":"city","polygons":[[[-1059136,374832,357,-156,-7,-11,-42,18,7,10,-208,89,2,-143,90,0,0,47,51,0,0,-19,-32,0,0,-9,26,0,0,-11,18,0,0,-24,52,0,8,-38,14,0,0,17,45,0,-1,-8,-33,0,0,-9,36,-2,-14,-27,14,0,-9,-19,-15,0,-13,-26,38,0,0,-49,-17,2,-10,16,-28,0,-20,-37,49,10,0,-10,28,0,0,-28,187,-4,-1,28,38,-2,0,73,-45,0,-1,27,-45,10,0,26,60,14,37,-30,8,7,31,1,5,-11,30,6,-28,63,15,14,-3,19,-36,-8,-30,10,0,-37,-45,-7,0,61,-35,15,-11,18,0,55,43,4,0,36,-14,0,14,11,-2,65,-88,1,0,71,-44,-1,-1,73,-46,0,-1,73,-183,0,1,-39,27,-21,1,-51,-74,-1,-1,24,-24,-3,-2,-15,-24,-7,0,-20,36,-14,0,-13,-31,-6,0,-14,27,6,9,-6,-9,-20,2,-22,-26,0,0,-37,-27,1,-18,16,-46,-6,-16,7]]]},{"geoid":"0801145","name":"Alamosa East","full":"Alamosa East CDP","lsad":"CDP","cdp":true,"polygons":[[[-1058619,374708,11,-18,35,-15,0,-9,106,-1,258,-25,0,241,-369,-2,2,-65,-14,-11,14,0,0,-36,-43,-4,0,-32]]]},{"geoid":"0801420","name":"Allenspark","full":"Allenspark CDP","lsad":"CDP","cdp":true,"polygons":[[[-1055426,402250,100,1,-3,-56,-24,-1,-4,-21,31,-8,1,-51,-102,-8,-1,-201,11,4,-1,-20,40,-2,3,23,48,20,-6,-220,72,24,37,79,286,-9,0,35,95,-4,0,-35,47,-1,-2,-100,60,32,37,-1,-7,-72,94,-1,8,13,4,45,37,25,-16,8,118,-5,0,90,-50,2,-3,38,-139,2,-1,35,-46,-3,-1,35,-142,-7,-188,29,-195,-4,-2,35,80,1,8,10,-50,91,11,11,45,0,-2,248,101,-3,-7,10,2,138,-93,2,0,34,-46,-1,0,72,-247,5]]]},{"geoid":"0801530","name":"Alma","full":"Alma town","lsad":"town","polygons":[[[-1060774,392893,31,-29,5,-28,24,0,21,-9,4,-9,46,0,0,-18,24,0,0,-9,16,10,19,-13,4,19,-16,15,9,13,-18,9,-13,22,-10,43,-91,-13,10,-42,-30,9,-40,35,42,0,1,27,-46,0,-1,-25,-55,9]]]},{"geoid":"0801640","name":"Alpine","full":"Alpine CDP","lsad":"CDP","cdp":true,"polygons":[[[-1066001,376839,43,0,0,-20,17,-12,32,0,65,31,116,2,2,22,40,-2,-5,46,-37,-24,-16,5,6,25,17,0,-23,10,1,7,-217,-1,1,-33,-22,23,-22,-2]]]},{"geoid":"0801740","name":"Altona","full":"Altona CDP","lsad":"CDP","cdp":true,"polygons":[[[-1053079,401147,11,-22,17,0,1,-29,19,-6,10,5,-23,11,0,69,59,-15,76,5,20,-8,25,-29,43,-8,-2,298,-40,0,-24,-9,-2,-11,-23,-12,2,-36,-7,-24,-21,2,-32,21,-57,-76,-18,-6,-16,-31,-7,-67]]]},{"geoid":"0801915","name":"Amherst","full":"Amherst CDP","lsad":"CDP","cdp":true,"polygons":[[[-1021835,406825,122,-49,126,86,-247,0]]]},{"geoid":"0802355","name":"Antonito","full":"Antonito town","lsad":"town","polygons":[[[-1060158,370752,44,0,0,-32,10,-1,-44,-27,21,-25,50,27,-1,23,18,1,2,109,-94,0]]]},{"geoid":"0802575","name":"Applewood","full":"Applewood CDP","lsad":"CDP","cdp":true,"polygons":[[[-1051829,397573,37,-15,-46,-43,-5,-114,177,1,56,11,-23,36,-21,-11,0,34,89,1,-4,-17,13,-10,26,25,19,2,-12,-14,23,-17,-15,-12,2,-43,-23,-3,7,-41,58,13,1,-28,-29,6,-31,-6,-11,-19,1,-18,163,0,0,72,48,5,0,33,-65,-1,0,-18,-21,-8,0,23,-56,-6,22,6,26,20,-2,165,-6,0,6,37,-28,3,0,18,-20,15,-47,-1,0,62,-33,9,-195,-10,0,-103,-50,6,-7,-13,-45,10]]]},{"geoid":"0802850","name":"Arapahoe","full":"Arapahoe CDP","lsad":"CDP","cdp":true,"polygons":[[[-1021839,388500,43,-2,13,-21,14,10,36,-7,-14,27,1,63,-93,0]]]},{"geoid":"0802905","name":"Arboles","full":"Arboles CDP","lsad":"CDP","cdp":true,"polygons":[[[-1074424,370000,316,0,121,111,10,43,-6,119,-11,22,-48,41,-83,-11,-42,97,9,38,-28,-6,-8,9,-10,-11,-51,-17,-27,-40,-143,2]]]},{"geoid":"0803015","name":"Aristocrat Ranchettes","full":"Aristocrat Ranchettes CDP","lsad":"CDP","cdp":true,"polygons":[[[-1047727,401019,282,2,1,71,-98,0,1,73,-188,-1]]]},{"geoid":"0803235","name":"Arriba","full":"Arriba town","lsad":"town","polygons":[[[-1032812,392808,47,0,-1,-20,91,-1,0,106,-137,0]]]},{"geoid":"0803455","name":"Arvada","full":"Arvada city","lsad":"ciLine truncated
|
||||
@@ -0,0 +1 @@
|
||||
{"st":"CT","state":"Connecticut","fips":"09","features":[{"geoid":"0901150","name":"Ansonia","full":"Ansonia city","lsad":"city","polygons":[[[-731024,413529,69,-137,35,-1,18,-10,21,-30,108,-48,9,-13,43,-21,59,-5,12,-12,93,-13,166,28,-9,57,-19,52,-10,12,-8,35,-26,38,-9,144,-251,1,-84,23,-52,0,-53,-24,-36,2,-57,-67]]]},{"geoid":"0902480","name":"Ball Pond","full":"Ball Pond CDP","lsad":"CDP","cdp":true,"polygons":[[[-735362,414496,76,15,0,17,23,-10,28,10,-2,26,33,4,51,-8,-3,29,-11,13,-7,74,-7,1,1,12,-21,12,-10,37,-63,6,-43,-27,-31,0,-13,-194]]]},{"geoid":"0902550","name":"Baltic","full":"Baltic CDP","lsad":"CDP","cdp":true,"polygons":[[[-721076,416144,147,6,0,-33,33,0,6,-7,10,-72,241,45,-59,17,-27,21,-20,31,-42,17,-1,13,-42,19,-22,45,-33,-5,5,-42,-12,-9,-55,7,-29,36,-38,-28,-4,8,-28,-20,-42,-3]]]},{"geoid":"0902690","name":"Bantam","full":"Bantam borough","lsad":"borough","polygons":[[[-732545,417189,248,-1,-7,117,-237,5,-4,-101]]]},{"geoid":"0904790","name":"Bethel","full":"Bethel CDP","lsad":"CDP","cdp":true,"polygons":[[[-734350,413724,107,-371,182,30,31,110,104,152,24,20,50,-32,-5,38,-48,48,3,15,-10,46,16,12,-8,47,-27,18,-22,-7,-60,16,-62,31,-8,21,10,39,-166,-134,-87,-38,-12,-26]]]},{"geoid":"0904945","name":"Bethlehem Village","full":"Bethlehem Village CDP","lsad":"CDP","cdp":true,"polygons":[[[-732276,416389,36,-85,-38,-76,16,-68,19,-12,10,-23,81,-4,42,27,31,6,13,27,9,3,83,-12,62,-49,165,3,-3,14,-20,15,-33,52,-29,22,-17,24,0,104,-19,43,10,33,15,13,13,30,-7,20,60,2,-6,73,-60,115,-404,-21,24,-22,31,-10,1,-38,-14,-35,6,-71,8,-10,-76,7,-20,-6]]]},{"geoid":"0904965","name":"Bigelow Corners","full":"Bigelow Corners CDP","lsad":"CDP","cdp":true,"polygons":[[[-735161,414654,27,-21,23,-5,86,11,70,24,-8,16,11,19,-10,7,-27,2,-27,26,7,25,-40,38,-15,-4,3,-16,-42,-23,8,-15,-2,-30,-58,-29]]]},{"geoid":"0906050","name":"Blue Hills","full":"Blue Hills CDP","lsad":"CDP","cdp":true,"polygons":[[[-727078,418070,164,6,13,-68,41,-6,-10,264,-16,16,-16,-9,12,-21,-17,-18,-54,-9,7,-43,-136,-8]]]},{"geoid":"0906155","name":"Bogus Hill","full":"Bogus Hill CDP","lsad":"CDP","cdp":true,"polygons":[[[-734672,415081,61,-61,19,16,-1,43,-22,45,-12,4,-20,-26]]]},{"geoid":"0906680","name":"Botsford","full":"Botsford CDP","lsad":"CDP","cdp":true,"polygons":[[[-732786,413571,27,-52,29,0,22,19,4,-11,21,-12,121,59,7,71,20,-3,-39,22,-90,-47,-39,40,-18,1,-27,-37,-19,0,-19,-24]]]},{"geoid":"0907100","name":"Branchville","full":"Branchville CDP","lsad":"CDP","cdp":true,"polygons":[[[-734467,412687,15,-33,-3,-33,73,31,-68,101,-14,-46]]]},{"geoid":"0907345","name":"Branford Center","full":"Branford Center CDP","lsad":"CDP","cdp":true,"polygons":[[[-728310,412659,22,-16,12,6,14,-14,14,2,1,-17,34,-4,5,-19,31,17,65,15,2,10,-18,12,-28,5,-27,22,11,28,35,12,16,27,18,11,31,3,27,-11,11,11,4,10,-11,11,13,17,-16,21,16,2,7,14,-14,6,22,17,22,-7,39,49,12,-4,0,13,14,18,-3,6,-26,3,-129,-21,-112,-31,16,-26,-85,-83,-37,-22,5,-25,-20,-38]]]},{"geoid":"0908000","name":"Bridgeport","full":"Bridgeport city","lsad":"city","polygons":[[[-732191,411978,-13,-99,24,-46,-16,-18,3,-21,-38,-30,-8,-24,-16,-1,-21,-45,39,8,14,-13,-51,-48,-7,-20,-91,-70,0,-14,11,-13,24,-3,8,-45,-19,-36,72,-10,49,53,-11,18,25,12,16,20,26,-5,6,-41,10,-3,61,64,67,33,165,12,18,51,66,23,71,-18,-15,-58,74,-22,60,174,32,34,14,53,-9,70,13,25,-9,9,-81,268,17,44,20,13,15,25,-71,-28,-48,-28,-78,-5,-60,-16,-40,-2,-34,37,75,-5,-13,22,31,12,-69,4,-26,-25,-55,27,-47,6,-3,12,-59,-17,-51,7,-318,-19]]]},{"geoid":"0908140","name":"Bridgewater","full":"Bridgewater CDP","lsad":"CDP","cdp":true,"polygons":[[[-733692,415343,-4,-15,7,-12,35,7,1,-22,22,-12,27,8,-13,25,16,10,4,15,19,20,-40,-6,-8,23,5,10,-14,7,-52,-15,-1,-22,-21,-11]]]},{"geoid":"0908420","name":"Bristol","full":"Bristol city","lsad":"city","polygons":[[[-729831,416395,545,60,8,-48,58,4,-19,44,405,40,-149,737,-1002,-107]]]},{"geoid":"0908770","name":"Broad Brook","full":"Broad Brook CDP","lsad":"CDP","cdp":true,"polygons":[[[-725737,418798,16,23,12,-1,26,19,26,3,0,16,45,7,34,15,60,-21,-18,89,9,24,107,47,34,9,106,-24,128,-9,11,331,-42,-13,-15,14,-22,-14,-3,16,-29,11,-14,-25,15,60,-22,1,-18,-24,-116,-91,-79,7,-16,-8,9,-22,-11,-7,-6,-21,-25,0,7,-14,-33,-22,0,-26,-30,6,9,-29,-44,11,3,-14,-42,-22,-12,-16,-4,-24,-34,-18,-10,-16,19,-7,-23,-28,13,-10,-15,-18,3,-21,-13,-76,-38,2,-9,-9,-1,-26,11,-5,-27,-19]]]},{"geoid":"0909050","name":"Brookfield Center","full":"Brookfield Center CDP","lsad":"CDP","cdp":true,"polygons":[[[-733906,414669,9,-26,-25,-7,16,-48,-22,-25,8,-46,79,49,37,-4,-24,33,-9,68,17,22,-45,35,-65,-31]]]},{"geoid":"0909120","name":"Brooklyn","full":"Brooklyn CDP","lsad":"CDP","cdp":true,"polygons":[[[-719714,417866,21,-32,0,-12,62,1,6,-7,13,7,13,-6,92,-2,19,-9,13,35,51,19,-10,16,-16,2,7,32,18,18,-23,-1,0,9,-35,41,-14,33,-82,1,4,-19,-21,-4,-12,-29,12,-40,-6,-45,-17,-17,-58,31,-30,-21]]]},{"geoid":"0910660","name":"Byram","full":"Byram CDP","lsad":"CDP","cdp":true,"polygons":[[[-736593,409972,-3,-29,24,-34,-18,-20,12,-13,10,1,56,75,24,16,26,34,-2,16,41Line truncated
|
||||
@@ -0,0 +1 @@
|
||||
{"st":"DC","state":"District of Columbia","fips":"11","features":[{"geoid":"1150000","name":"Washington","full":"Washington city","lsad":"city","polygons":[[[-771179,389324,13,-35,32,-37,71,-61,29,-62,22,-18,110,-69,80,-23,121,-12,19,-9,9,-6,38,-106,52,-85,31,-1,11,-9,30,-39,-2,-20,22,-25,28,6,7,17,-10,19,12,3,48,-40,15,-31,-2,-38,53,-72,23,-66,-28,-100,-40,-12,-57,9,-8,-17,1,-38,14,-15,23,5,20,-15,11,-36,-10,-72,17,-72,-15,-134,-1,-92,1296,1012,-1316,1023,-788,-608]]]}]}
|
||||
Loaded 100 of 568 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user